2024-09-12T00:36:03.377 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T00:36:03.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T00:36:03.522 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901507' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG9lEskuxgpID5IsVjI0QO95l2LoxKdZ+o93DCuqBpZQ4QKdEIXr2emHEzlIYbBGx4pB38FKwfJc67fjUaUWKa0= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAAkFXZDUfmnC26De29iPTyJl3Bv4pis4inLqMIwSW/mT67PemXxF2kBAebtUJ80Dv7EK9QT6z5pWj793q8dpeM= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2zJnQSg+J5V7vYxBYllhbaYaV57lJ08gmgTFVzGuU5DDp4nt/DAMpu2mBdUrKxy63hSCNfFtq7bhHMCWR4By8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T00:36:03.522 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T00:36:03.523 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T00:36:03.523 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T00:36:03.524 INFO:teuthology.task.internal:Checking packages... 2024-09-12T00:36:03.546 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T00:36:03.546 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T00:36:03.546 INFO:teuthology.packaging:ref: None 2024-09-12T00:36:03.546 INFO:teuthology.packaging:tag: None 2024-09-12T00:36:03.547 INFO:teuthology.packaging:branch: squid 2024-09-12T00:36:03.547 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:36:03.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T00:36:03.791 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T00:36:03.793 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T00:36:03.799 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T00:36:03.799 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T00:36:03.836 INFO:teuthology.task.internal:Saving configuration 2024-09-12T00:36:03.856 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T00:36:03.889 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T00:36:03.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 00:31:02.404232', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG9lEskuxgpID5IsVjI0QO95l2LoxKdZ+o93DCuqBpZQ4QKdEIXr2emHEzlIYbBGx4pB38FKwfJc67fjUaUWKa0='} 2024-09-12T00:36:03.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 00:31:02.407574', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAAkFXZDUfmnC26De29iPTyJl3Bv4pis4inLqMIwSW/mT67PemXxF2kBAebtUJ80Dv7EK9QT6z5pWj793q8dpeM='} 2024-09-12T00:36:03.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 00:31:02.406025', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2zJnQSg+J5V7vYxBYllhbaYaV57lJ08gmgTFVzGuU5DDp4nt/DAMpu2mBdUrKxy63hSCNfFtq7bhHMCWR4By8='} 2024-09-12T00:36:03.989 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T00:36:03.994 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T00:36:03.994 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T00:36:03.994 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T00:36:03.994 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T00:36:04.133 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2c9f073be0>, signals=[15]) 2024-09-12T00:36:04.134 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T00:36:04.139 INFO:teuthology.task.internal:Opening connections... 2024-09-12T00:36:04.140 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:36:04.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:36:04.227 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-12T00:36:04.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:36:04.310 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-09-12T00:36:04.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:36:04.393 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T00:36:04.400 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-12T00:36:04.418 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-12T00:36:04.419 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-12T00:36:04.478 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T00:36:04.479 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T00:36:04.479 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T00:36:04.479 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-12T00:36:04.513 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-12T00:36:04.531 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-12T00:36:04.532 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-12T00:36:04.591 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-12T00:36:04.592 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T00:36:04.592 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-12T00:36:04.592 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T00:36:04.592 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T00:36:04.592 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T00:36:04.592 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-12T00:36:04.629 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-09-12T00:36:04.647 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-09-12T00:36:04.648 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-09-12T00:36:04.705 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-09-12T00:36:04.706 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-09-12T00:36:04.707 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T00:36:04.707 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-09-12T00:36:04.707 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T00:36:04.707 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T00:36:04.707 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T00:36:04.708 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-09-12T00:36:04.737 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T00:36:04.747 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T00:36:04.777 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T00:36:04.777 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-12T00:36:04.780 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-12T00:36:04.782 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-09-12T00:36:04.799 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T00:36:04.805 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T00:36:04.805 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-12T00:36:04.837 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-12T00:36:04.840 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-09-12T00:36:04.875 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T00:36:04.961 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T00:36:04.974 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T00:36:04.975 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T00:36:04.975 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T00:36:04.975 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T00:36:04.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:36:04.993 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T00:36:05.051 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-12T00:36:05.051 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-12T00:36:05.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-12T00:36:08.858 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-12T00:36:13.376 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 20 MB 00:03 2024-09-12T00:36:17.718 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-12T00:36:19.310 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-12T00:36:20.156 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T00:36:25.812 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T00:36:27.919 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T00:36:27.919 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T00:36:28.020 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T00:36:28.024 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-12T00:36:28.025 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:36:28.026 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-12T00:36:28.026 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-12T00:36:28.026 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T00:36:28.422 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 160 kB/s | 23 kB 00:00 2024-09-12T00:36:28.764 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-12T00:36:29.231 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-12T00:36:29.498 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-12T00:36:29.498 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T00:36:29.498 INFO:teuthology.orchestra.run.smithi016.stdout:Total 53 MB/s | 79 MB 00:01 2024-09-12T00:36:30.211 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T00:36:30.607 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T00:36:30.607 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T00:36:31.562 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T00:36:31.562 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T00:36:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T00:36:34.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T00:36:34.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T00:36:37.707 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T00:36:37.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T00:36:42.970 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T00:36:43.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:36:48.151 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:37:26.915 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:37:26.955 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:37:27.358 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:37:27.358 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T00:37:27.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T00:37:27.359 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T00:37:28.209 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T00:37:28.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:37:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T00:37:28.561 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T00:37:28.651 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.652 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T00:37:28.653 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T00:37:28.653 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T00:37:28.653 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T00:37:28.653 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T00:37:28.653 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T00:37:28.653 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T00:37:28.712 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T00:37:28.712 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T00:37:28.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-12T00:37:28.754 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-12T00:37:28.754 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-12T00:37:28.754 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-12T00:37:28.754 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-12T00:37:28.754 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-12T00:37:28.755 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:37:28.756 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T00:37:28.757 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T00:37:28.757 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-505.el9.x86_64 2024-09-12T00:37:28.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T00:37:29.462 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-12T00:37:30.715 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T00:37:30.756 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-12T00:37:30.758 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-12T00:37:30.773 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.KNTmD1ceF0 2024-09-12T00:37:30.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KNTmD1ceF0 2024-09-12T00:37:30.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.KNTmD1ceF0 2024-09-12T00:37:30.979 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.KNTmD1ceF0 is 6KB 2024-09-12T00:37:31.028 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.KNTmD1ceF0 2024-09-12T00:37:31.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T00:37:31.104 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T00:37:31.104 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T00:37:31.105 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T00:37:31.105 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T00:37:31.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T00:37:31.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-12T00:38:01.276 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T00:38:01.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T00:38:01.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:39:01.305 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T00:39:10.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T00:39:10.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:39:10.710 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T00:39:11.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T00:39:11.294 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T00:39:11.294 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-12T00:39:11.339 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-505.el9.x86_64 2024-09-12T00:39:11.340 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T00:39:11.340 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T00:39:11.340 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T00:39:12.341 DEBUG:teuthology.parallel:result is None 2024-09-12T00:39:12.341 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T00:39:12.349 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T00:39:12.349 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T00:39:12.352 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T00:39:12.354 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T00:39:12.373 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T00:39:12.437 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T00:39:12.517 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T00:39:12.518 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T00:39:12.520 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T00:39:12.522 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T00:39:12.649 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T00:39:12.655 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T00:39:12.655 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T00:39:12.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:39:12.672 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T00:39:12.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:39:12.689 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T00:39:12.692 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T00:39:12.715 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T00:39:12.776 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.777 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.797 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.799 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.872 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.899 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T00:39:12.901 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T00:39:12.908 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T00:39:12.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T00:39:12.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T00:39:12.913 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T00:39:12.940 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T00:39:13.062 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T00:39:13.062 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T00:39:13.064 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T00:39:13.066 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T00:39:13.083 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.226 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.357 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:39:13.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T00:39:13.425 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.478 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.530 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.656 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:39:13.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T00:39:13.723 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T00:39:13.824 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.917 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T00:39:13.952 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:39:13.952 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T00:39:14.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-12T00:39:14.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-12T00:39:14.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-09-12T00:39:14.057 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:39:14.092 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T00:39:14.571 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T00:39:14.579 INFO:teuthology.task.internal:Starting timer... 2024-09-12T00:39:14.579 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T00:39:14.634 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T00:39:14.671 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T00:39:14.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-12T00:39:14.735 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-12T00:39:14.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-12T00:39:14.805 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-12T00:39:14.808 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-09-12T00:39:14.883 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-09-12T00:39:14.885 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T00:39:14.885 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-12T00:39:14.922 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-12T00:39:14.922 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-12T00:39:14.942 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-12T00:39:14.943 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-09-12T00:39:14.969 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-09-12T00:39:14.969 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2024-09-12T00:39:14.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:39:14.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:39:14.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:39:15.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:39:15.031 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T00:39:15.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:39:15.060 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T00:39:15.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-12T00:39:15.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-12T00:39:15.130 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-09-12T00:39:15.167 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T00:39:15.178 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T00:39:15.181 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T00:39:15.302 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T00:39:15.313 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T00:39:15.314 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T00:43:17.917 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2024-09-12T00:43:17.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T00:43:17.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:43:18.020 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-12T00:43:18.101 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-12T00:43:18.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-12T00:43:18.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:43:18.201 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-12T00:43:18.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-12T00:43:18.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-12T00:43:18.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T00:43:18.364 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-12T00:43:18.441 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-12T00:43:18.442 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T00:43:18.451 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T00:43:18.452 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T00:43:18.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:43:18.454 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T00:43:18.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:43:18.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T00:43:18.461 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T00:43:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T00:43:18.488 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T00:43:18.497 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T00:43:18.504 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T00:43:18.512 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T00:43:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-12T00:43:18.526 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T00:43:18.529 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-12T00:43:18.534 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-12T00:43:18.541 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-12T00:43:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T00:43:18.552 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-09-12T00:43:18.556 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T00:43:18.563 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-09-12T00:43:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T00:43:18.571 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T00:43:18.577 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T00:43:18.591 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T00:43:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:43:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.629 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.632 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.648 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.652 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T00:43:18.653 INFO:teuthology.run_tasks:Running task install... 2024-09-12T00:43:18.665 DEBUG:teuthology.task.install:project ceph 2024-09-12T00:43:18.665 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T00:43:18.665 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T00:43:18.665 INFO:teuthology.task.install:Using flavor: default 2024-09-12T00:43:18.670 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T00:43:18.671 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T00:43:18.671 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T00:43:18.672 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:43:18.674 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T00:43:18.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:43:18.675 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T00:43:18.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:43:18.826 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T00:43:18.826 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T00:43:18.843 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T00:43:18.843 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T00:43:18.854 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T00:43:18.855 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T00:43:18.968 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T00:43:18.968 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:43:18.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T00:43:18.977 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T00:43:18.977 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:43:18.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T00:43:18.986 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T00:43:18.987 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:43:18.987 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T00:43:18.996 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T00:43:18.996 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T00:43:19.003 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T00:43:19.003 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T00:43:19.013 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T00:43:19.013 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T00:43:19.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T00:43:19.068 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T00:43:19.080 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T00:43:19.142 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T00:43:19.144 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T00:43:19.154 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T00:43:19.208 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-12T00:43:19.208 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-12T00:43:19.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-12T00:43:19.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-12T00:43:19.219 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-09-12T00:43:19.220 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-09-12T00:43:19.436 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-12T00:43:19.445 INFO:teuthology.orchestra.run.smithi159.stdout:45 files removed 2024-09-12T00:43:19.458 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-12T00:43:19.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T00:43:19.472 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T00:43:19.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T00:43:20.012 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-12T00:43:20.055 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-12T00:43:20.131 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 218 kB/s | 84 kB 00:00 2024-09-12T00:43:20.321 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-09-12T00:43:20.339 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T00:43:20.422 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T00:43:20.597 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T00:43:20.606 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-12T00:43:20.689 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T00:43:22.255 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-12T00:43:22.406 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-12T00:43:22.697 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-12T00:43:24.239 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-12T00:43:25.645 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 20 MB 00:02 2024-09-12T00:43:26.758 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 20 MB 00:02 2024-09-12T00:43:29.965 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-12T00:43:30.277 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-12T00:43:31.845 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-12T00:43:32.323 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-12T00:43:32.458 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-12T00:43:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T00:43:33.477 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-12T00:43:34.000 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-12T00:43:34.821 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T00:43:38.247 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-12T00:43:38.891 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-12T00:43:40.477 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T00:43:40.733 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:40.734 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:40.746 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T00:43:40.747 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T00:43:40.748 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T00:43:40.748 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T00:43:40.750 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T00:43:40.753 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T00:43:40.754 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T00:43:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-12T00:43:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T00:43:40.838 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T00:43:40.839 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T00:43:40.840 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T00:43:40.841 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T00:43:40.842 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T00:43:40.843 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T00:43:40.844 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T00:43:40.845 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T00:43:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T00:43:40.847 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T00:43:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T00:43:40.849 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T00:43:40.850 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-12T00:43:40.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:43:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-12T00:43:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-12T00:43:41.201 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:41.202 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:41.213 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T00:43:41.214 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T00:43:41.215 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T00:43:41.216 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T00:43:41.217 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T00:43:41.220 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T00:43:41.221 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T00:43:41.284 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T00:43:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T00:43:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T00:43:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-12T00:43:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T00:43:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T00:43:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T00:43:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T00:43:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T00:43:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T00:43:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T00:43:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T00:43:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-09-12T00:43:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 8 Packages 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:43:41.312 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 226 M 2024-09-12T00:43:41.313 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-12T00:43:41.382 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T00:43:41.632 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.5 MB/s | 897 kB 00:00 2024-09-12T00:43:41.715 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-12T00:43:41.741 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.3 MB/s | 145 kB 00:00 2024-09-12T00:43:41.858 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 2.4 MB 00:00 2024-09-12T00:43:41.967 INFO:teuthology.orchestra.run.smithi099.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2024-09-12T00:43:41.974 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-12T00:43:42.067 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 23 MB/s | 4.7 MB 00:00 2024-09-12T00:43:42.592 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 20 MB/s | 11 MB 00:00 2024-09-12T00:43:42.625 INFO:teuthology.orchestra.run.smithi099.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.3 MB/s | 897 kB 00:00 2024-09-12T00:43:42.700 INFO:teuthology.orchestra.run.smithi099.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-12T00:43:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:42.757 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T00:43:42.771 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T00:43:42.771 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T00:43:42.772 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T00:43:42.773 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T00:43:42.775 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T00:43:42.778 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T00:43:42.779 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T00:43:42.830 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 21 MB 00:01 2024-09-12T00:43:42.842 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-12T00:43:42.842 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 100 kB/s | 25 kB 00:00 2024-09-12T00:43:42.863 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T00:43:42.864 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T00:43:42.865 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T00:43:42.866 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T00:43:42.867 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T00:43:42.868 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T00:43:42.869 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T00:43:42.870 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T00:43:42.871 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T00:43:42.872 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T00:43:42.873 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T00:43:42.874 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T00:43:42.875 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-12T00:43:42.876 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-12T00:43:42.877 INFO:teuthology.orchestra.run.smithi159.stdout:Install 148 Packages 2024-09-12T00:43:42.877 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 8 Packages 2024-09-12T00:43:42.877 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:43:42.877 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 226 M 2024-09-12T00:43:42.877 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-12T00:43:42.893 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.4 MB/s | 228 kB 00:00 2024-09-12T00:43:42.951 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 465 kB/s | 27 kB 00:00 2024-09-12T00:43:43.010 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 544 kB/s | 32 kB 00:00 2024-09-12T00:43:43.102 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T00:43:43.177 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T00:43:43.253 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T00:43:43.317 INFO:teuthology.orchestra.run.smithi099.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-09-12T00:43:43.336 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-12T00:43:43.659 INFO:teuthology.orchestra.run.smithi099.stdout:(5/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 21 MB 00:01 2024-09-12T00:43:43.705 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T00:43:43.743 INFO:teuthology.orchestra.run.smithi099.stdout:(6/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.7 MB/s | 5.4 MB 00:01 2024-09-12T00:43:43.754 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 933 kB/s | 45 kB 00:00 2024-09-12T00:43:43.785 INFO:teuthology.orchestra.run.smithi099.stdout:(7/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-09-12T00:43:43.821 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T00:43:43.905 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-12T00:43:43.945 INFO:teuthology.orchestra.run.smithi159.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-12T00:43:43.996 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-12T00:43:44.019 INFO:teuthology.orchestra.run.smithi099.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-12T00:43:44.069 INFO:teuthology.orchestra.run.smithi099.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 507 kB/s | 25 kB 00:00 2024-09-12T00:43:44.197 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 17 MB 00:02 2024-09-12T00:43:44.222 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 301 kB 00:00 2024-09-12T00:43:44.289 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.1 MB/s | 100 kB 00:00 2024-09-12T00:43:44.312 INFO:teuthology.orchestra.run.smithi159.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.4 MB/s | 897 kB 00:00 2024-09-12T00:43:44.314 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 928 kB/s | 85 kB 00:00 2024-09-12T00:43:44.420 INFO:teuthology.orchestra.run.smithi159.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.3 MB/s | 145 kB 00:00 2024-09-12T00:43:44.865 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.4 MB/s | 3.1 MB 00:00 2024-09-12T00:43:44.982 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 273 kB/s | 32 kB 00:00 2024-09-12T00:43:45.055 INFO:teuthology.orchestra.run.smithi159.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.8 MB/s | 2.4 MB 00:00 2024-09-12T00:43:45.082 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.4 MB/s | 146 kB 00:00 2024-09-12T00:43:45.107 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 216 kB/s | 171 kB 00:00 2024-09-12T00:43:45.397 INFO:teuthology.orchestra.run.smithi159.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 4.5 MB/s | 1.5 MB 00:00 2024-09-12T00:43:45.716 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.6 MB/s | 3.6 MB 00:00 2024-09-12T00:43:45.816 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T00:43:45.916 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 493 kB/s | 49 kB 00:00 2024-09-12T00:43:46.010 INFO:teuthology.orchestra.run.smithi099.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 4.8 MB/s | 11 MB 00:02 2024-09-12T00:43:46.094 INFO:teuthology.orchestra.run.smithi099.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 228 kB 00:00 2024-09-12T00:43:46.144 INFO:teuthology.orchestra.run.smithi099.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 542 kB/s | 27 kB 00:00 2024-09-12T00:43:46.211 INFO:teuthology.orchestra.run.smithi099.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 476 kB/s | 32 kB 00:00 2024-09-12T00:43:46.288 INFO:teuthology.orchestra.run.smithi159.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 5.3 MB/s | 4.7 MB 00:00 2024-09-12T00:43:46.353 INFO:teuthology.orchestra.run.smithi099.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 752 kB 00:00 2024-09-12T00:43:46.400 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 14 MB/s | 49 MB 00:03 2024-09-12T00:43:46.412 INFO:teuthology.orchestra.run.smithi099.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T00:43:46.434 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 32 kB/s | 16 kB 00:00 2024-09-12T00:43:46.467 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-12T00:43:46.470 INFO:teuthology.orchestra.run.smithi099.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T00:43:46.522 INFO:teuthology.orchestra.run.smithi159.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 7.6 MB/s | 21 MB 00:02 2024-09-12T00:43:46.560 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 5.1 MB/s | 7.4 MB 00:01 2024-09-12T00:43:46.562 INFO:teuthology.orchestra.run.smithi099.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T00:43:46.576 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 168 kB/s | 18 kB 00:00 2024-09-12T00:43:46.589 INFO:teuthology.orchestra.run.smithi159.stdout:(8/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 1.9 MB/s | 5.4 MB 00:02 2024-09-12T00:43:46.610 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.3 MB/s | 765 kB 00:00 2024-09-12T00:43:46.656 INFO:teuthology.orchestra.run.smithi159.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-12T00:43:46.718 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-12T00:43:46.802 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 996 kB/s | 191 kB 00:00 2024-09-12T00:43:46.836 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-12T00:43:46.870 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-12T00:43:46.895 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T00:43:46.920 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T00:43:46.938 INFO:teuthology.orchestra.run.smithi099.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.3 MB/s | 17 MB 00:03 2024-09-12T00:43:46.945 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T00:43:46.971 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T00:43:47.021 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-12T00:43:47.138 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-12T00:43:47.163 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-09-12T00:43:47.189 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-12T00:43:47.214 INFO:teuthology.orchestra.run.smithi159.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 15 MB/s | 11 MB 00:00 2024-09-12T00:43:47.231 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-09-12T00:43:47.256 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-12T00:43:47.264 INFO:teuthology.orchestra.run.smithi159.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T00:43:47.281 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T00:43:47.306 INFO:teuthology.orchestra.run.smithi159.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 654 kB/s | 27 kB 00:00 2024-09-12T00:43:47.307 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T00:43:47.332 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T00:43:47.348 INFO:teuthology.orchestra.run.smithi159.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 762 kB/s | 32 kB 00:00 2024-09-12T00:43:47.357 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T00:43:47.382 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-12T00:43:47.388 INFO:teuthology.orchestra.run.smithi099.stdout:(19/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 15 MB/s | 49 MB 00:03 2024-09-12T00:43:47.413 INFO:teuthology.orchestra.run.smithi099.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 95 kB/s | 45 kB 00:00 2024-09-12T00:43:47.423 INFO:teuthology.orchestra.run.smithi159.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-12T00:43:47.424 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-12T00:43:47.438 INFO:teuthology.orchestra.run.smithi099.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-12T00:43:47.466 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T00:43:47.474 INFO:teuthology.orchestra.run.smithi159.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T00:43:47.480 INFO:teuthology.orchestra.run.smithi099.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T00:43:47.491 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-12T00:43:47.517 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 938 kB/s | 47 kB 00:00 2024-09-12T00:43:47.524 INFO:teuthology.orchestra.run.smithi159.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T00:43:47.547 INFO:teuthology.orchestra.run.smithi099.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 5.3 MB 00:00 2024-09-12T00:43:47.567 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.9 MB/s | 3.0 MB 00:00 2024-09-12T00:43:47.572 INFO:teuthology.orchestra.run.smithi099.stdout:(24/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.3 MB/s | 321 kB 00:00 2024-09-12T00:43:47.591 INFO:teuthology.orchestra.run.smithi159.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T00:43:47.593 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-12T00:43:47.606 INFO:teuthology.orchestra.run.smithi099.stdout:(25/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.3 MB/s | 301 kB 00:00 2024-09-12T00:43:47.626 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T00:43:47.631 INFO:teuthology.orchestra.run.smithi099.stdout:(26/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-12T00:43:47.652 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 458 kB/s | 38 kB 00:00 2024-09-12T00:43:47.656 INFO:teuthology.orchestra.run.smithi099.stdout:(27/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-12T00:43:47.690 INFO:teuthology.orchestra.run.smithi099.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T00:43:47.715 INFO:teuthology.orchestra.run.smithi099.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 545 kB/s | 32 kB 00:00 2024-09-12T00:43:47.718 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-12T00:43:47.740 INFO:teuthology.orchestra.run.smithi099.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T00:43:47.786 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-12T00:43:47.852 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-12T00:43:47.904 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T00:43:47.936 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-12T00:43:47.942 INFO:teuthology.orchestra.run.smithi159.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 17 MB 00:01 2024-09-12T00:43:47.949 INFO:teuthology.orchestra.run.smithi099.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-12T00:43:47.970 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T00:43:47.996 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T00:43:48.024 INFO:teuthology.orchestra.run.smithi099.stdout:(32/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 7.4 MB/s | 3.1 MB 00:00 2024-09-12T00:43:48.025 INFO:teuthology.orchestra.run.smithi159.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-12T00:43:48.037 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T00:43:48.050 INFO:teuthology.orchestra.run.smithi099.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T00:43:48.050 INFO:teuthology.orchestra.run.smithi159.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 416 kB/s | 45 kB 00:00 2024-09-12T00:43:48.075 INFO:teuthology.orchestra.run.smithi099.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 986 kB/s | 49 kB 00:00 2024-09-12T00:43:48.075 INFO:teuthology.orchestra.run.smithi159.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.6 MB/s | 136 kB 00:00 2024-09-12T00:43:48.109 INFO:teuthology.orchestra.run.smithi099.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 282 kB/s | 16 kB 00:00 2024-09-12T00:43:48.109 INFO:teuthology.orchestra.run.smithi159.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.7 MB/s | 162 kB 00:00 2024-09-12T00:43:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T00:43:48.137 INFO:teuthology.orchestra.run.smithi159.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T00:43:48.137 INFO:teuthology.orchestra.run.smithi099.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T00:43:48.146 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T00:43:48.168 INFO:teuthology.orchestra.run.smithi159.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T00:43:48.171 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T00:43:48.176 INFO:teuthology.orchestra.run.smithi099.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 11 MB/s | 765 kB 00:00 2024-09-12T00:43:48.188 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-09-12T00:43:48.193 INFO:teuthology.orchestra.run.smithi159.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T00:43:48.201 INFO:teuthology.orchestra.run.smithi099.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 273 kB/s | 18 kB 00:00 2024-09-12T00:43:48.205 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 568 kB/s | 19 kB 00:00 2024-09-12T00:43:48.220 INFO:teuthology.orchestra.run.smithi159.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-12T00:43:48.230 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-12T00:43:48.248 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-09-12T00:43:48.287 INFO:teuthology.orchestra.run.smithi159.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T00:43:48.323 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-12T00:43:48.340 INFO:teuthology.orchestra.run.smithi159.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 636 kB/s | 32 kB 00:00 2024-09-12T00:43:48.343 INFO:teuthology.orchestra.run.smithi099.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-09-12T00:43:48.348 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-12T00:43:48.374 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T00:43:48.396 INFO:teuthology.orchestra.run.smithi159.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-12T00:43:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T00:43:48.421 INFO:teuthology.orchestra.run.smithi159.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-12T00:43:48.424 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-12T00:43:48.427 INFO:teuthology.orchestra.run.smithi099.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T00:43:48.449 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T00:43:48.475 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T00:43:48.509 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-12T00:43:48.519 INFO:teuthology.orchestra.run.smithi099.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-12T00:43:48.525 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T00:43:48.544 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-12T00:43:48.544 INFO:teuthology.orchestra.run.smithi099.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-12T00:43:48.576 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-12T00:43:48.605 INFO:teuthology.orchestra.run.smithi159.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 17 MB/s | 3.6 MB 00:00 2024-09-12T00:43:48.636 INFO:teuthology.orchestra.run.smithi099.stdout:(43/156): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-12T00:43:48.652 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T00:43:48.655 INFO:teuthology.orchestra.run.smithi159.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T00:43:48.661 INFO:teuthology.orchestra.run.smithi099.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-09-12T00:43:48.677 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-12T00:43:48.702 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T00:43:48.706 INFO:teuthology.orchestra.run.smithi159.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 984 kB/s | 49 kB 00:00 2024-09-12T00:43:48.711 INFO:teuthology.orchestra.run.smithi099.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T00:43:48.727 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-12T00:43:48.753 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-12T00:43:48.756 INFO:teuthology.orchestra.run.smithi159.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 329 kB/s | 16 kB 00:00 2024-09-12T00:43:48.770 INFO:teuthology.orchestra.run.smithi099.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-12T00:43:48.778 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T00:43:48.804 INFO:teuthology.orchestra.run.smithi099.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 809 kB/s | 135 kB 00:00 2024-09-12T00:43:48.812 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-12T00:43:48.815 INFO:teuthology.orchestra.run.smithi159.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T00:43:48.837 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T00:43:48.845 INFO:teuthology.orchestra.run.smithi099.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2024-09-12T00:43:48.862 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T00:43:48.887 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-12T00:43:48.899 INFO:teuthology.orchestra.run.smithi159.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-12T00:43:48.949 INFO:teuthology.orchestra.run.smithi159.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 366 kB/s | 18 kB 00:00 2024-09-12T00:43:48.988 INFO:teuthology.orchestra.run.smithi099.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-12T00:43:49.013 INFO:teuthology.orchestra.run.smithi099.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 640 kB/s | 107 kB 00:00 2024-09-12T00:43:49.038 INFO:teuthology.orchestra.run.smithi099.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-12T00:43:49.041 INFO:teuthology.orchestra.run.smithi159.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-12T00:43:49.080 INFO:teuthology.orchestra.run.smithi099.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-12T00:43:49.105 INFO:teuthology.orchestra.run.smithi099.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-12T00:43:49.113 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-12T00:43:49.138 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-09-12T00:43:49.156 INFO:teuthology.orchestra.run.smithi099.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-12T00:43:49.181 INFO:teuthology.orchestra.run.smithi099.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-12T00:43:49.191 INFO:teuthology.orchestra.run.smithi159.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 330 kB/s | 80 kB 00:00 2024-09-12T00:43:49.197 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T00:43:49.206 INFO:teuthology.orchestra.run.smithi099.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T00:43:49.222 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-12T00:43:49.231 INFO:teuthology.orchestra.run.smithi099.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T00:43:49.233 INFO:teuthology.orchestra.run.smithi159.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-12T00:43:49.247 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-12T00:43:49.257 INFO:teuthology.orchestra.run.smithi099.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-12T00:43:49.273 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T00:43:49.283 INFO:teuthology.orchestra.run.smithi159.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-12T00:43:49.298 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T00:43:49.309 INFO:teuthology.orchestra.run.smithi159.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-12T00:43:49.323 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-12T00:43:49.323 INFO:teuthology.orchestra.run.smithi099.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-12T00:43:49.349 INFO:teuthology.orchestra.run.smithi099.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-12T00:43:49.357 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-12T00:43:49.374 INFO:teuthology.orchestra.run.smithi099.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-12T00:43:49.390 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-12T00:43:49.392 INFO:teuthology.orchestra.run.smithi159.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-09-12T00:43:49.399 INFO:teuthology.orchestra.run.smithi099.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T00:43:49.425 INFO:teuthology.orchestra.run.smithi099.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T00:43:49.432 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-12T00:43:49.434 INFO:teuthology.orchestra.run.smithi159.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-12T00:43:49.458 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T00:43:49.458 INFO:teuthology.orchestra.run.smithi099.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-12T00:43:49.468 INFO:teuthology.orchestra.run.smithi159.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-12T00:43:49.483 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T00:43:49.483 INFO:teuthology.orchestra.run.smithi099.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-12T00:43:49.508 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-12T00:43:49.509 INFO:teuthology.orchestra.run.smithi099.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-12T00:43:49.510 INFO:teuthology.orchestra.run.smithi159.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-12T00:43:49.542 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-12T00:43:49.567 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T00:43:49.568 INFO:teuthology.orchestra.run.smithi159.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-12T00:43:49.576 INFO:teuthology.orchestra.run.smithi099.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-12T00:43:49.601 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-12T00:43:49.635 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-12T00:43:49.651 INFO:teuthology.orchestra.run.smithi099.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-12T00:43:49.710 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-12T00:43:49.718 INFO:teuthology.orchestra.run.smithi099.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-12T00:43:49.735 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-12T00:43:49.743 INFO:teuthology.orchestra.run.smithi099.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-12T00:43:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T00:43:49.785 INFO:teuthology.orchestra.run.smithi099.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.2 MB/s | 910 kB 00:00 2024-09-12T00:43:49.786 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-12T00:43:49.811 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-12T00:43:49.827 INFO:teuthology.orchestra.run.smithi099.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-12T00:43:49.836 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-12T00:43:49.853 INFO:teuthology.orchestra.run.smithi099.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-12T00:43:49.862 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-12T00:43:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-09-12T00:43:49.895 INFO:teuthology.orchestra.run.smithi099.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T00:43:49.895 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-12T00:43:49.921 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T00:43:49.938 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-12T00:43:49.978 INFO:teuthology.orchestra.run.smithi099.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-12T00:43:49.980 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-12T00:43:49.997 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 992 kB/s | 16 kB 00:00 2024-09-12T00:43:50.003 INFO:teuthology.orchestra.run.smithi099.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-12T00:43:50.022 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T00:43:50.037 INFO:teuthology.orchestra.run.smithi099.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-12T00:43:50.047 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T00:43:50.054 INFO:teuthology.orchestra.run.smithi099.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-12T00:43:50.073 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T00:43:50.098 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-12T00:43:50.129 INFO:teuthology.orchestra.run.smithi099.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-12T00:43:50.146 INFO:teuthology.orchestra.run.smithi099.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-12T00:43:50.148 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-12T00:43:50.172 INFO:teuthology.orchestra.run.smithi099.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-12T00:43:50.173 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-09-12T00:43:50.188 INFO:teuthology.orchestra.run.smithi099.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-09-12T00:43:50.199 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T00:43:50.222 INFO:teuthology.orchestra.run.smithi099.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-12T00:43:50.233 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-12T00:43:50.247 INFO:teuthology.orchestra.run.smithi099.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T00:43:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 888 kB/s | 74 kB 00:00 2024-09-12T00:43:50.272 INFO:teuthology.orchestra.run.smithi099.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T00:43:50.283 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T00:43:50.298 INFO:teuthology.orchestra.run.smithi099.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-12T00:43:50.308 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-12T00:43:50.323 INFO:teuthology.orchestra.run.smithi099.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T00:43:50.333 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-12T00:43:50.348 INFO:teuthology.orchestra.run.smithi099.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T00:43:50.359 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T00:43:50.373 INFO:teuthology.orchestra.run.smithi099.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T00:43:50.384 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T00:43:50.399 INFO:teuthology.orchestra.run.smithi099.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T00:43:50.424 INFO:teuthology.orchestra.run.smithi099.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T00:43:50.468 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-12T00:43:50.494 INFO:teuthology.orchestra.run.smithi159.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 701 kB/s | 689 kB 00:00 2024-09-12T00:43:50.508 INFO:teuthology.orchestra.run.smithi099.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-12T00:43:50.533 INFO:teuthology.orchestra.run.smithi099.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-12T00:43:50.552 INFO:teuthology.orchestra.run.smithi159.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 959 kB/s | 943 kB 00:00 2024-09-12T00:43:50.558 INFO:teuthology.orchestra.run.smithi099.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-12T00:43:50.583 INFO:teuthology.orchestra.run.smithi099.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T00:43:50.609 INFO:teuthology.orchestra.run.smithi099.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-12T00:43:50.634 INFO:teuthology.orchestra.run.smithi099.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-12T00:43:50.659 INFO:teuthology.orchestra.run.smithi099.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T00:43:50.676 INFO:teuthology.orchestra.run.smithi099.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-09-12T00:43:50.689 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-12T00:43:50.701 INFO:teuthology.orchestra.run.smithi099.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-12T00:43:50.719 INFO:teuthology.orchestra.run.smithi099.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-12T00:43:50.735 INFO:teuthology.orchestra.run.smithi099.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-12T00:43:50.796 INFO:teuthology.orchestra.run.smithi099.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-12T00:43:50.797 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.7 MB/s | 3.2 MB 00:00 2024-09-12T00:43:50.844 INFO:teuthology.orchestra.run.smithi099.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T00:43:50.878 INFO:teuthology.orchestra.run.smithi099.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T00:43:50.889 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.5 MB/s | 2.3 MB 00:00 2024-09-12T00:43:50.906 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 376 kB/s | 41 kB 00:00 2024-09-12T00:43:50.973 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-12T00:43:51.015 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 5.5 MB/s | 1.8 MB 00:00 2024-09-12T00:43:51.061 INFO:teuthology.orchestra.run.smithi099.stdout:(106/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-12T00:43:51.087 INFO:teuthology.orchestra.run.smithi099.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 93 kB/s | 19 kB 00:00 2024-09-12T00:43:51.119 INFO:teuthology.orchestra.run.smithi159.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-12T00:43:51.120 INFO:teuthology.orchestra.run.smithi099.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T00:43:51.153 INFO:teuthology.orchestra.run.smithi159.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 162 kB/s | 107 kB 00:00 2024-09-12T00:43:51.162 INFO:teuthology.orchestra.run.smithi099.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-12T00:43:51.188 INFO:teuthology.orchestra.run.smithi099.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 747 kB/s | 49 kB 00:00 2024-09-12T00:43:51.215 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-12T00:43:51.221 INFO:teuthology.orchestra.run.smithi099.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.8 MB/s | 151 kB 00:00 2024-09-12T00:43:51.255 INFO:teuthology.orchestra.run.smithi099.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-09-12T00:43:51.297 INFO:teuthology.orchestra.run.smithi099.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-12T00:43:51.322 INFO:teuthology.orchestra.run.smithi099.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T00:43:51.347 INFO:teuthology.orchestra.run.smithi099.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T00:43:51.373 INFO:teuthology.orchestra.run.smithi099.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-12T00:43:51.398 INFO:teuthology.orchestra.run.smithi099.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-12T00:43:51.440 INFO:teuthology.orchestra.run.smithi099.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-12T00:43:51.465 INFO:teuthology.orchestra.run.smithi099.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-12T00:43:51.549 INFO:teuthology.orchestra.run.smithi099.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-12T00:43:51.574 INFO:teuthology.orchestra.run.smithi099.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-09-12T00:43:51.582 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-09-12T00:43:51.588 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T00:43:51.588 INFO:teuthology.orchestra.run.smithi016.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-12T00:43:51.608 INFO:teuthology.orchestra.run.smithi099.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T00:43:51.633 INFO:teuthology.orchestra.run.smithi099.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-12T00:43:51.658 INFO:teuthology.orchestra.run.smithi099.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-12T00:43:51.684 INFO:teuthology.orchestra.run.smithi099.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-12T00:43:51.695 INFO:teuthology.orchestra.run.smithi159.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-12T00:43:51.709 INFO:teuthology.orchestra.run.smithi099.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-12T00:43:51.734 INFO:teuthology.orchestra.run.smithi099.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-12T00:43:51.809 INFO:teuthology.orchestra.run.smithi099.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-09-12T00:43:51.835 INFO:teuthology.orchestra.run.smithi099.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T00:43:51.860 INFO:teuthology.orchestra.run.smithi099.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T00:43:51.885 INFO:teuthology.orchestra.run.smithi099.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T00:43:51.927 INFO:teuthology.orchestra.run.smithi099.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-12T00:43:51.953 INFO:teuthology.orchestra.run.smithi099.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T00:43:51.978 INFO:teuthology.orchestra.run.smithi099.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-12T00:43:51.995 INFO:teuthology.orchestra.run.smithi159.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 358 kB/s | 313 kB 00:00 2024-09-12T00:43:52.003 INFO:teuthology.orchestra.run.smithi099.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T00:43:52.045 INFO:teuthology.orchestra.run.smithi099.stdout:(136/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-09-12T00:43:52.062 INFO:teuthology.orchestra.run.smithi099.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-09-12T00:43:52.079 INFO:teuthology.orchestra.run.smithi099.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-12T00:43:52.096 INFO:teuthology.orchestra.run.smithi099.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-12T00:43:52.113 INFO:teuthology.orchestra.run.smithi099.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-12T00:43:52.138 INFO:teuthology.orchestra.run.smithi099.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-12T00:43:52.164 INFO:teuthology.orchestra.run.smithi099.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-12T00:43:52.189 INFO:teuthology.orchestra.run.smithi099.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T00:43:52.214 INFO:teuthology.orchestra.run.smithi099.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T00:43:52.239 INFO:teuthology.orchestra.run.smithi099.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-12T00:43:52.264 INFO:teuthology.orchestra.run.smithi099.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T00:43:52.290 INFO:teuthology.orchestra.run.smithi099.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T00:43:52.340 INFO:teuthology.orchestra.run.smithi099.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T00:43:52.624 INFO:teuthology.orchestra.run.smithi099.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-09-12T00:43:52.691 INFO:teuthology.orchestra.run.smithi099.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.9 MB/s | 3.2 MB 00:00 2024-09-12T00:43:52.749 INFO:teuthology.orchestra.run.smithi099.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-12T00:43:52.791 INFO:teuthology.orchestra.run.smithi099.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-09-12T00:43:52.817 INFO:teuthology.orchestra.run.smithi099.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 325 kB/s | 41 kB 00:00 2024-09-12T00:43:52.837 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-12T00:43:52.846 INFO:teuthology.orchestra.run.smithi159.stdout:(54/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 7.9 MB/s | 49 MB 00:06 2024-09-12T00:43:52.867 INFO:teuthology.orchestra.run.smithi099.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-12T00:43:52.871 INFO:teuthology.orchestra.run.smithi159.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 21 kB/s | 18 kB 00:00 2024-09-12T00:43:52.975 INFO:teuthology.orchestra.run.smithi099.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.1 MB/s | 249 kB 00:00 2024-09-12T00:43:53.006 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-12T00:43:53.006 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-12T00:43:53.100 INFO:teuthology.orchestra.run.smithi099.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-09-12T00:43:53.106 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-12T00:43:53.107 INFO:teuthology.orchestra.run.smithi099.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T00:43:53.338 INFO:teuthology.orchestra.run.smithi159.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.8 MB/s | 3.0 MB 00:01 2024-09-12T00:43:53.363 INFO:teuthology.orchestra.run.smithi159.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-12T00:43:53.646 INFO:teuthology.orchestra.run.smithi159.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 207 kB/s | 160 kB 00:00 2024-09-12T00:43:53.963 INFO:teuthology.orchestra.run.smithi159.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-12T00:43:54.322 INFO:teuthology.orchestra.run.smithi159.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 691 kB/s | 662 kB 00:00 2024-09-12T00:43:54.351 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-12T00:43:54.498 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-12T00:43:54.498 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-12T00:43:54.581 INFO:teuthology.orchestra.run.smithi159.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 623 kB/s | 581 kB 00:00 2024-09-12T00:43:54.606 INFO:teuthology.orchestra.run.smithi159.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 38 kB/s | 24 kB 00:00 2024-09-12T00:43:54.906 INFO:teuthology.orchestra.run.smithi159.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-12T00:43:55.333 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-12T00:43:55.333 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-12T00:43:55.340 INFO:teuthology.orchestra.run.smithi159.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-12T00:43:55.390 INFO:teuthology.orchestra.run.smithi159.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 362 kB/s | 292 kB 00:00 2024-09-12T00:43:55.532 INFO:teuthology.orchestra.run.smithi159.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 62 kB/s | 38 kB 00:00 2024-09-12T00:43:56.299 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-12T00:43:56.300 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-12T00:43:56.466 INFO:teuthology.orchestra.run.smithi159.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-12T00:43:56.516 INFO:teuthology.orchestra.run.smithi159.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-12T00:43:56.574 INFO:teuthology.orchestra.run.smithi159.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-12T00:43:57.233 INFO:teuthology.orchestra.run.smithi159.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-12T00:43:57.492 INFO:teuthology.orchestra.run.smithi159.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 887 kB/s | 910 kB 00:01 2024-09-12T00:43:57.867 INFO:teuthology.orchestra.run.smithi159.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-09-12T00:43:58.192 INFO:teuthology.orchestra.run.smithi159.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-12T00:43:58.267 INFO:teuthology.orchestra.run.smithi159.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 113 kB/s | 87 kB 00:00 2024-09-12T00:43:58.876 INFO:teuthology.orchestra.run.smithi159.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 90 kB/s | 61 kB 00:00 2024-09-12T00:43:59.101 INFO:teuthology.orchestra.run.smithi159.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 294 kB/s | 245 kB 00:00 2024-09-12T00:43:59.526 INFO:teuthology.orchestra.run.smithi159.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-09-12T00:43:59.718 INFO:teuthology.orchestra.run.smithi159.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-09-12T00:43:59.768 INFO:teuthology.orchestra.run.smithi159.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 278 kB/s | 248 kB 00:00 2024-09-12T00:44:00.077 INFO:teuthology.orchestra.run.smithi159.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-09-12T00:44:00.402 INFO:teuthology.orchestra.run.smithi159.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-09-12T00:44:00.527 INFO:teuthology.orchestra.run.smithi159.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 213 kB/s | 172 kB 00:00 2024-09-12T00:44:01.253 INFO:teuthology.orchestra.run.smithi159.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 295 kB/s | 250 kB 00:00 2024-09-12T00:44:01.328 INFO:teuthology.orchestra.run.smithi159.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-12T00:44:01.370 INFO:teuthology.orchestra.run.smithi159.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 264 kB/s | 222 kB 00:00 2024-09-12T00:44:01.921 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-12T00:44:01.962 INFO:teuthology.orchestra.run.smithi159.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 109 kB/s | 77 kB 00:00 2024-09-12T00:44:01.987 INFO:teuthology.orchestra.run.smithi159.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-09-12T00:44:02.044 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T00:44:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T00:44:02.355 INFO:teuthology.orchestra.run.smithi159.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 311 kB/s | 269 kB 00:00 2024-09-12T00:44:02.355 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T00:44:02.380 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T00:44:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T00:44:02.455 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T00:44:02.575 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-12T00:44:02.653 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T00:44:02.692 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T00:44:02.753 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T00:44:02.788 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T00:44:02.788 INFO:teuthology.orchestra.run.smithi159.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 192 kB/s | 159 kB 00:00 2024-09-12T00:44:02.813 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T00:44:02.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T00:44:02.855 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T00:44:02.880 INFO:teuthology.orchestra.run.smithi159.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 313 kB/s | 279 kB 00:00 2024-09-12T00:44:02.913 INFO:teuthology.orchestra.run.smithi159.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-09-12T00:44:02.939 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:03.064 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:03.090 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:03.183 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:03.228 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:44:03.257 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T00:44:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:03.421 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T00:44:03.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T00:44:03.480 INFO:teuthology.orchestra.run.smithi159.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 78 kB/s | 54 kB 00:00 2024-09-12T00:44:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:03.528 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:03.539 INFO:teuthology.orchestra.run.smithi159.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-12T00:44:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:03.600 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:03.629 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:03.694 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:44:03.695 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:03.722 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:03.745 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:03.842 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T00:44:03.891 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T00:44:03.919 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:03.940 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:04.037 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:04.129 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:04.139 INFO:teuthology.orchestra.run.smithi159.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 67 kB/s | 44 kB 00:00 2024-09-12T00:44:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:04.312 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T00:44:04.323 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:04.413 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:04.464 INFO:teuthology.orchestra.run.smithi159.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T00:44:04.500 INFO:teuthology.orchestra.run.smithi159.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 230 kB/s | 221 kB 00:00 2024-09-12T00:44:04.522 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:04.561 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T00:44:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:04.737 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:04.874 INFO:teuthology.orchestra.run.smithi159.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 49 kB/s | 36 kB 00:00 2024-09-12T00:44:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T00:44:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T00:44:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T00:44:05.091 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T00:44:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T00:44:05.140 INFO:teuthology.orchestra.run.smithi159.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 71 kB/s | 47 kB 00:00 2024-09-12T00:44:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T00:44:05.185 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T00:44:05.221 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T00:44:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T00:44:05.255 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T00:44:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T00:44:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T00:44:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T00:44:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T00:44:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T00:44:05.365 INFO:teuthology.orchestra.run.smithi159.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 351 kB/s | 304 kB 00:00 2024-09-12T00:44:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T00:44:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T00:44:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T00:44:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T00:44:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T00:44:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T00:44:05.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T00:44:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T00:44:05.582 INFO:teuthology.orchestra.run.smithi159.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 82 kB/s | 58 kB 00:00 2024-09-12T00:44:05.586 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T00:44:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T00:44:05.639 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T00:44:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T00:44:05.670 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T00:44:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T00:44:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T00:44:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T00:44:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T00:44:05.790 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T00:44:05.824 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T00:44:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T00:44:05.868 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T00:44:05.874 INFO:teuthology.orchestra.run.smithi159.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 132 kB/s | 96 kB 00:00 2024-09-12T00:44:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T00:44:05.905 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T00:44:05.958 INFO:teuthology.orchestra.run.smithi159.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-09-12T00:44:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T00:44:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T00:44:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T00:44:06.033 INFO:teuthology.orchestra.run.smithi159.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-09-12T00:44:06.083 INFO:teuthology.orchestra.run.smithi159.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T00:44:06.101 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T00:44:06.117 INFO:teuthology.orchestra.run.smithi159.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.2 MB/s | 309 kB 00:00 2024-09-12T00:44:06.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T00:44:06.145 INFO:teuthology.orchestra.run.smithi159.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-12T00:44:06.181 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T00:44:06.201 INFO:teuthology.orchestra.run.smithi159.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T00:44:06.226 INFO:teuthology.orchestra.run.smithi159.stdout:(108/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-12T00:44:06.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T00:44:06.288 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T00:44:06.290 INFO:teuthology.orchestra.run.smithi159.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 295 kB/s | 26 kB 00:00 2024-09-12T00:44:06.318 INFO:teuthology.orchestra.run.smithi159.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T00:44:06.345 INFO:teuthology.orchestra.run.smithi159.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 443 kB/s | 49 kB 00:00 2024-09-12T00:44:06.369 INFO:teuthology.orchestra.run.smithi159.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-12T00:44:06.453 INFO:teuthology.orchestra.run.smithi159.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-12T00:44:06.478 INFO:teuthology.orchestra.run.smithi159.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T00:44:06.503 INFO:teuthology.orchestra.run.smithi159.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T00:44:06.529 INFO:teuthology.orchestra.run.smithi159.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-12T00:44:06.554 INFO:teuthology.orchestra.run.smithi159.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-12T00:44:06.579 INFO:teuthology.orchestra.run.smithi159.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-12T00:44:06.613 INFO:teuthology.orchestra.run.smithi159.stdout:(119/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 839 kB 00:00 2024-09-12T00:44:06.638 INFO:teuthology.orchestra.run.smithi159.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-12T00:44:06.663 INFO:teuthology.orchestra.run.smithi159.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T00:44:06.689 INFO:teuthology.orchestra.run.smithi159.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-12T00:44:06.714 INFO:teuthology.orchestra.run.smithi159.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-12T00:44:06.734 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T00:44:06.739 INFO:teuthology.orchestra.run.smithi159.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-12T00:44:06.756 INFO:teuthology.orchestra.run.smithi159.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-09-12T00:44:06.773 INFO:teuthology.orchestra.run.smithi159.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 693 kB/s | 23 kB 00:00 2024-09-12T00:44:06.790 INFO:teuthology.orchestra.run.smithi159.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-09-12T00:44:06.807 INFO:teuthology.orchestra.run.smithi159.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 587 kB/s | 20 kB 00:00 2024-09-12T00:44:06.820 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T00:44:06.824 INFO:teuthology.orchestra.run.smithi159.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-12T00:44:06.836 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T00:44:06.849 INFO:teuthology.orchestra.run.smithi159.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T00:44:06.874 INFO:teuthology.orchestra.run.smithi159.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T00:44:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T00:44:06.901 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T00:44:06.908 INFO:teuthology.orchestra.run.smithi159.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T00:44:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T00:44:06.937 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T00:44:06.944 INFO:teuthology.orchestra.run.smithi159.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-12T00:44:06.971 INFO:teuthology.orchestra.run.smithi159.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-12T00:44:06.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T00:44:06.993 INFO:teuthology.orchestra.run.smithi159.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T00:44:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T00:44:07.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T00:44:07.018 INFO:teuthology.orchestra.run.smithi159.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T00:44:07.044 INFO:teuthology.orchestra.run.smithi159.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-12T00:44:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T00:44:07.069 INFO:teuthology.orchestra.run.smithi159.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T00:44:07.087 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T00:44:07.095 INFO:teuthology.orchestra.run.smithi159.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T00:44:07.118 INFO:teuthology.orchestra.run.smithi159.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-12T00:44:07.144 INFO:teuthology.orchestra.run.smithi159.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T00:44:07.170 INFO:teuthology.orchestra.run.smithi159.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T00:44:07.195 INFO:teuthology.orchestra.run.smithi159.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T00:44:07.197 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T00:44:07.237 INFO:teuthology.orchestra.run.smithi159.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-12T00:44:07.243 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T00:44:07.248 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T00:44:07.262 INFO:teuthology.orchestra.run.smithi159.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-09-12T00:44:07.304 INFO:teuthology.orchestra.run.smithi159.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T00:44:07.311 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T00:44:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T00:44:07.329 INFO:teuthology.orchestra.run.smithi159.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-12T00:44:07.381 INFO:teuthology.orchestra.run.smithi159.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T00:44:07.385 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T00:44:07.386 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T00:44:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T00:44:07.449 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T00:44:07.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T00:44:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T00:44:07.499 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T00:44:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T00:44:07.534 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T00:44:07.578 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T00:44:07.580 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T00:44:07.616 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T00:44:07.621 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T00:44:07.630 INFO:teuthology.orchestra.run.smithi159.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T00:44:07.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T00:44:07.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T00:44:07.713 INFO:teuthology.orchestra.run.smithi159.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.2 MB/s | 3.2 MB 00:00 2024-09-12T00:44:07.756 INFO:teuthology.orchestra.run.smithi159.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 6.2 MB/s | 2.3 MB 00:00 2024-09-12T00:44:07.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T00:44:07.845 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T00:44:07.847 INFO:teuthology.orchestra.run.smithi159.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 306 kB/s | 41 kB 00:00 2024-09-12T00:44:07.873 INFO:teuthology.orchestra.run.smithi159.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-12T00:44:07.887 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T00:44:07.915 INFO:teuthology.orchestra.run.smithi159.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 6.4 MB/s | 1.8 MB 00:00 2024-09-12T00:44:07.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T00:44:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T00:44:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T00:44:07.973 INFO:teuthology.orchestra.run.smithi159.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-12T00:44:08.023 INFO:teuthology.orchestra.run.smithi159.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-12T00:44:08.030 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-12T00:44:08.031 INFO:teuthology.orchestra.run.smithi159.stdout:Total 9.0 MB/s | 226 MB 00:25 2024-09-12T00:44:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T00:44:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T00:44:08.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T00:44:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T00:44:08.109 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T00:44:08.135 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T00:44:08.154 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T00:44:08.160 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T00:44:08.187 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T00:44:08.212 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T00:44:08.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T00:44:08.318 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T00:44:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T00:44:08.455 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T00:44:08.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T00:44:08.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T00:44:08.567 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T00:44:08.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T00:44:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T00:44:08.646 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T00:44:08.657 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T00:44:08.680 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T00:44:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T00:44:08.720 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T00:44:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T00:44:08.759 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T00:44:08.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T00:44:08.789 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T00:44:08.804 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T00:44:08.823 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T00:44:08.847 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T00:44:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T00:44:08.899 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T00:44:08.935 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T00:44:08.936 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T00:44:08.987 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T00:44:09.017 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T00:44:09.023 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T00:44:09.081 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T00:44:09.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T00:44:09.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T00:44:09.185 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T00:44:09.215 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T00:44:09.222 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-12T00:44:09.263 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T00:44:09.375 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-12T00:44:09.375 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-12T00:44:09.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T00:44:09.979 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T00:44:09.990 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T00:44:10.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T00:44:10.027 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T00:44:10.061 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T00:44:10.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T00:44:10.157 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T00:44:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T00:44:10.215 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T00:44:10.220 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T00:44:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T00:44:10.310 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T00:44:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T00:44:10.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T00:44:10.379 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T00:44:10.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T00:44:10.562 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T00:44:10.586 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T00:44:10.594 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T00:44:10.671 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T00:44:10.675 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T00:44:11.193 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-12T00:44:11.193 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-12T00:44:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T00:44:11.357 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T00:44:11.416 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T00:44:11.480 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T00:44:12.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T00:44:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T00:44:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T00:44:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T00:44:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T00:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T00:44:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T00:44:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T00:44:13.065 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T00:44:13.113 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:13.203 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:13.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T00:44:13.270 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T00:44:13.384 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T00:44:13.431 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:13.523 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T00:44:13.599 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T00:44:13.745 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T00:44:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T00:44:14.103 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T00:44:14.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:14.231 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:14.261 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T00:44:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:16.631 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:16.656 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:16.691 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:17.173 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:17.209 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:17.276 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:17.290 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T00:44:17.291 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T00:44:17.291 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:17.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:17.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:17.409 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T00:44:17.409 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T00:44:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:17.422 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-12T00:44:17.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:17.535 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T00:44:17.647 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T00:44:17.719 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T00:44:17.753 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T00:44:17.779 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T00:44:17.822 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T00:44:18.211 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T00:44:18.271 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:18.394 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:18.422 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:18.543 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:44:18.700 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T00:44:18.757 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T00:44:18.785 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:18.872 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:18.894 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:18.956 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T00:44:18.978 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:18.999 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T00:44:19.162 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:19.304 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T00:44:19.419 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T00:44:19.526 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:19.604 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T00:44:19.937 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T00:44:20.032 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T00:44:20.062 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T00:44:20.096 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T00:44:20.170 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T00:44:20.214 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T00:44:20.246 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T00:44:20.290 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T00:44:20.331 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T00:44:20.384 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T00:44:20.453 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T00:44:20.506 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T00:44:20.536 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T00:44:20.572 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T00:44:20.648 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T00:44:20.683 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T00:44:20.726 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T00:44:20.755 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T00:44:20.833 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T00:44:20.886 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T00:44:20.961 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T00:44:21.092 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T00:44:21.147 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T00:44:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T00:44:21.751 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T00:44:21.780 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T00:44:21.833 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T00:44:21.871 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T00:44:21.913 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T00:44:22.075 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T00:44:22.150 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T00:44:22.213 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T00:44:22.294 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T00:44:22.325 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T00:44:22.362 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T00:44:22.408 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T00:44:22.445 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T00:44:22.489 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T00:44:22.530 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T00:44:22.667 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T00:44:22.715 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T00:44:22.756 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T00:44:22.860 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T00:44:22.899 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T00:44:22.938 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T00:44:22.964 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T00:44:23.006 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T00:44:23.080 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T00:44:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T00:44:23.246 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T00:44:23.282 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T00:44:23.325 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T00:44:23.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T00:44:23.396 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T00:44:23.426 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T00:44:23.460 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T00:44:23.494 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T00:44:23.535 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T00:44:23.570 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T00:44:23.655 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T00:44:23.708 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T00:44:23.748 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T00:44:23.850 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T00:44:23.924 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T00:44:24.647 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T00:44:24.696 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T00:44:24.741 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T00:44:24.844 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T00:44:24.893 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T00:44:24.939 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T00:44:25.025 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T00:44:25.066 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T00:44:25.251 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T00:44:25.280 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T00:44:25.365 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T00:44:26.017 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T00:44:26.182 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T00:44:27.626 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T00:44:27.703 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T00:44:27.730 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T00:44:27.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T00:44:28.072 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T00:44:28.118 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T00:44:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T00:44:28.259 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T00:44:28.735 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T00:44:28.763 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:28.834 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T00:44:28.857 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-12T00:44:28.967 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-12T00:44:28.968 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-12T00:44:28.968 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:29.161 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:29.178 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:29.178 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:29.178 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T00:44:29.178 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:29.757 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:29.757 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-12T00:44:29.757 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-12T00:44:29.758 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:29.936 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:29.953 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:29.953 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:29.953 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T00:44:29.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T00:44:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:30.661 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:30.914 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:31.144 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:31.178 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:31.349 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:31.373 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:31.486 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T00:44:31.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:31.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T00:44:31.590 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T00:44:31.718 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T00:44:31.757 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:31.774 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:31.792 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T00:44:31.799 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:31.799 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:31.799 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T00:44:31.800 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:31.800 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:31.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:31.841 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T00:44:31.849 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T00:44:31.849 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T00:44:31.849 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T00:44:31.850 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:31.883 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:31.910 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T00:44:32.006 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T00:44:32.162 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T00:44:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T00:44:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T00:44:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T00:44:32.268 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:32.292 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:32.292 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:32.292 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T00:44:32.292 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:32.292 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:32.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:32.398 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T00:44:32.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T00:44:32.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T00:44:32.676 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:32.712 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:32.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T00:44:32.739 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T00:44:32.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T00:44:32.807 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T00:44:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:32.872 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T00:44:32.957 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T00:44:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T00:44:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T00:44:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T00:44:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T00:44:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T00:44:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T00:44:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:34.898 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:34.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:35.010 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:35.169 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T00:44:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:35.234 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:35.234 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T00:44:35.234 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:35.234 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:35.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T00:44:35.442 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T00:44:35.956 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:35.986 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:36.097 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:36.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T00:44:36.313 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:36.339 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:36.339 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:36.339 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T00:44:36.339 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:36.339 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:36.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T00:44:36.389 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:36.401 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:36.404 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:36.429 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:36.429 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T00:44:36.429 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:36.429 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:36.495 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:36.495 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T00:44:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:36.503 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T00:44:36.547 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:36.558 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:36.571 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:37.533 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T00:44:37.567 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:37.591 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:37.592 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:37.592 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T00:44:37.592 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:37.592 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:37.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:37.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:37.658 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T00:44:37.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:37.709 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:37.734 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:37.734 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:37.735 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T00:44:37.735 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:37.735 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:37.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:38.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:38.077 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:41.443 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T00:44:41.484 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T00:44:41.524 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T00:44:41.557 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T00:44:41.584 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T00:44:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T00:44:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T00:44:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T00:44:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T00:44:41.912 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T00:44:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:42.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T00:44:42.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T00:44:43.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T00:44:43.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T00:44:43.065 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T00:44:43.280 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T00:44:43.319 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T00:44:43.350 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T00:44:43.380 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T00:44:43.407 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T00:44:43.434 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T00:44:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T00:44:43.565 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T00:44:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T00:44:43.700 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T00:44:43.726 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T00:44:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T00:44:43.760 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-09-12T00:44:43.835 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-09-12T00:44:43.836 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:43.928 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:43.953 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T00:44:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:43.959 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:44.010 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:44.026 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T00:44:44.026 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:44.026 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T00:44:44.026 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:44.086 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:44.132 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:44.138 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:44.153 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:44.231 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:44.295 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:45.067 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T00:44:45.154 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T00:44:45.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T00:44:45.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T00:44:45.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T00:44:45.358 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T00:44:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T00:44:45.392 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:45.485 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:45.504 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T00:44:45.532 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:45.569 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T00:44:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:45.600 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:45.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:45.788 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T00:44:45.793 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:45.863 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:45.933 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:46.241 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T00:44:46.264 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:46.371 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T00:44:46.436 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T00:44:46.499 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T00:44:46.683 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T00:44:46.733 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T00:44:46.812 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T00:44:46.908 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T00:44:47.055 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T00:44:47.100 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T00:44:47.132 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T00:44:47.160 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T00:44:47.576 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:47.608 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T00:44:47.649 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T00:44:47.718 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T00:44:47.748 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:48.794 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T00:44:48.833 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:49.801 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T00:44:49.825 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:49.928 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:50.046 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T00:44:50.047 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T00:44:50.048 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T00:44:50.060 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T00:44:50.061 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T00:44:50.062 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T00:44:50.063 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T00:44:50.064 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T00:44:50.065 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T00:44:50.066 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T00:44:50.067 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T00:44:50.068 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T00:44:50.069 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T00:44:50.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T00:44:50.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T00:44:50.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T00:44:50.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T00:44:50.070 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T00:44:50.095 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T00:44:50.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:50.151 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T00:44:50.151 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:50.152 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T00:44:50.152 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:50.152 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T00:44:50.152 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:50.195 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:50.206 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T00:44:50.304 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T00:44:50.359 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T00:44:51.282 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T00:44:51.314 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:51.337 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T00:44:51.338 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:51.338 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T00:44:51.338 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:51.338 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T00:44:51.338 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:51.381 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:51.404 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T00:44:51.404 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:51.404 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T00:44:51.404 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:51.449 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T00:44:51.472 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:51.783 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:51.806 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T00:44:51.806 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T00:44:51.806 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T00:44:51.806 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:51.806 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T00:44:51.807 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:44:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T00:44:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T00:44:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T00:44:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T00:44:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T00:44:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T00:44:51.935 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T00:44:51.936 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T00:44:51.937 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T00:44:51.938 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T00:44:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T00:44:51.940 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T00:44:51.941 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T00:44:51.942 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T00:44:51.943 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T00:44:51.944 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T00:44:51.944 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:52.617 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.618 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.619 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T00:44:52.620 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T00:44:52.621 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T00:44:52.622 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T00:44:52.623 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T00:44:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T00:44:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T00:44:52.626 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T00:44:52.627 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.628 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T00:44:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:44:52.630 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-12T00:44:53.095 DEBUG:teuthology.parallel:result is None 2024-09-12T00:44:54.861 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:44:54.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:54.861 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-12T00:44:54.861 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:54.861 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-12T00:44:54.862 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.863 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.865 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T00:44:54.866 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T00:44:54.867 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T00:44:54.868 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T00:44:54.869 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.870 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T00:44:54.871 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T00:44:54.872 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T00:44:54.873 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.874 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T00:44:54.875 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.876 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:44:54.877 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-12T00:44:55.298 DEBUG:teuthology.parallel:result is None 2024-09-12T00:44:56.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T00:44:56.369 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T00:44:56.410 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T00:44:56.451 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T00:44:56.478 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T00:44:56.657 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T00:44:56.708 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T00:44:56.747 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T00:44:56.776 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T00:44:56.831 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T00:44:57.471 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:57.512 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:57.532 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T00:44:58.377 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T00:44:58.467 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T00:44:58.507 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T00:44:58.589 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T00:44:58.628 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T00:44:58.662 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T00:44:58.688 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T00:44:58.688 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:58.772 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T00:44:58.772 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:58.856 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T00:44:58.881 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T00:44:58.886 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:58.886 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:59.066 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T00:44:59.067 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:59.159 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T00:44:59.221 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:45:05.086 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:45:05.086 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T00:45:05.087 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T00:45:05.088 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T00:45:05.089 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T00:45:05.090 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T00:45:05.091 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T00:45:05.092 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T00:45:05.093 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T00:45:05.094 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T00:45:05.095 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T00:45:05.096 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T00:45:05.097 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T00:45:05.098 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T00:45:05.099 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T00:45:05.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T00:45:07.799 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.800 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.801 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.802 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T00:45:07.803 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T00:45:07.804 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T00:45:07.805 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T00:45:07.806 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T00:45:07.807 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T00:45:07.808 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T00:45:07.809 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T00:45:07.810 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T00:45:07.811 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T00:45:07.812 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:45:07.813 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-12T00:45:08.195 DEBUG:teuthology.parallel:result is None 2024-09-12T00:45:08.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:08.355 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T00:45:08.382 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.383 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.383 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T00:45:08.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:08.537 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T00:45:08.563 INFO:teuthology.orchestra.run.smithi099.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.564 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.564 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T00:45:08.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:08.734 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T00:45:08.758 INFO:teuthology.orchestra.run.smithi159.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.758 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T00:45:08.758 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T00:45:08.760 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T00:45:08.761 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:45:08.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T00:45:08.794 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:45:08.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T00:45:08.828 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:45:08.828 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T00:45:08.858 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T00:45:08.858 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:45:08.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T00:45:08.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T00:45:08.960 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:45:08.961 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T00:45:08.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T00:45:09.050 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:45:09.051 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T00:45:09.078 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T00:45:09.145 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T00:45:09.146 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:45:09.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T00:45:09.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T00:45:09.245 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:45:09.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T00:45:09.271 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T00:45:09.337 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:45:09.337 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T00:45:09.365 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T00:45:09.432 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T00:45:09.433 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:45:09.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T00:45:09.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T00:45:09.523 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:45:09.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T00:45:09.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T00:45:09.613 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:45:09.613 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T00:45:09.642 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T00:45:09.709 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T00:45:09.807 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T00:45:09.807 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:09.808 INFO:tasks.cephadm:Cluster fsid is 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:45:09.808 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T00:45:09.808 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.159'} 2024-09-12T00:45:09.808 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-12T00:45:09.808 INFO:tasks.cephadm:First mgr is x 2024-09-12T00:45:09.809 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T00:45:09.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-12T00:45:09.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-09-12T00:45:09.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-09-12T00:45:09.889 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T00:45:09.889 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:10.047 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T00:45:10.200 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:45:10.202 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T00:45:10.202 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T00:45:10.203 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:10.561 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 00:45 /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:10.562 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:10.939 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 00:45 /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:10.939 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:11.308 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 00:45 /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:11.308 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:11.329 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:11.349 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T00:45:11.384 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T00:45:11.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T00:45:11.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T00:45:11.391 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T00:45:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T00:45:13.006 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T00:45:13.097 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T00:45:45.249 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:45:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T00:45:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T00:45:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-12T00:45:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T00:45:45.250 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T00:45:45.251 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:45:58.589 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-12T00:45:58.590 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-12T00:46:01.840 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-12T00:46:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T00:46:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T00:46:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-09-12T00:46:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T00:46:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-12T00:46:01.841 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-12T00:46:01.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-12T00:46:01.899 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-09-12T00:46:01.926 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-09-12T00:46:01.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-12T00:46:01.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-09-12T00:46:02.016 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-09-12T00:46:02.046 INFO:tasks.cephadm:Writing seed config... 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T00:46:02.047 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T00:46:02.048 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T00:46:02.049 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T00:46:02.050 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T00:46:02.051 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T00:46:02.051 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T00:46:02.051 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T00:46:02.051 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:46:02.051 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T00:46:02.071 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 434eab66-70a0-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T00:46:02.071 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T00:46:02.114 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T00:46:02.157 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T00:46:02.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:46:02.373 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-12T00:46:02.374 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '434eab66-70a0-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-12T00:46:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T00:46:02.374 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-12T00:46:02.398 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-12T00:46:02.399 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-12T00:46:02.399 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-12T00:46:02.407 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T00:46:02.407 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T00:46:02.413 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T00:46:02.413 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:46:02.420 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-12T00:46:02.427 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-12T00:46:02.427 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-12T00:46:02.427 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-12T00:46:02.450 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-12T00:46:02.450 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T00:46:02.450 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-12T00:46:02.450 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-12T00:46:02.457 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T00:46:02.457 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T00:46:02.463 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T00:46:02.463 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:46:02.470 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-12T00:46:02.477 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-12T00:46:02.477 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-12T00:46:02.477 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-12T00:46:02.477 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:02.478 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140624613975040 on /run/cephadm/434eab66-70a0-11ef-bcea-c7b262605968.lock 2024-09-12T00:46:02.478 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140624613975040 acquired on /run/cephadm/434eab66-70a0-11ef-bcea-c7b262605968.lock 2024-09-12T00:46:02.478 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-12T00:46:02.478 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-12T00:46:02.478 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T00:46:02.482 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-12T00:46:02.482 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-12T00:46:02.484 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T00:46:02.485 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T00:46:02.485 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T00:46:02.487 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T00:46:02.488 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-12T00:46:02.488 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-12T00:46:02.488 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T00:46:02.489 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T00:46:02.489 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T00:46:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T00:46:03.928 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T00:46:03.928 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T00:46:03.929 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T00:46:04.910 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-12T00:46:04.910 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-12T00:46:05.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQDNOeJmvgC0DRAAvEYE/5HxUk4++Ez21g98hw== 2024-09-12T00:46:06.954 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQDOOeJmJX7lDxAAAqRZlV6Ujg9EIXNQWoWuhQ== 2024-09-12T00:46:07.909 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQDPOeJmFJi6EhAA5NQaHRDorf7o5eiC9sNTyQ== 2024-09-12T00:46:07.910 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-12T00:46:08.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:08.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T00:46:08.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:46:08.833 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-12T00:46:16.731 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-12T00:46:17.290 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T00:46:17.819 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T00:46:18.407 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target. 2024-09-12T00:46:18.408 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target. 2024-09-12T00:46:19.200 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a 2024-09-12T00:46:19.200 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T00:46:19.513 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target.wants/ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968@.service. 2024-09-12T00:46:19.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:19 smithi016 podman[30585]: 2024-09-12 00:46:19.663555673 +0000 UTC m=+0.016647925 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:46:20.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 podman[30585]: 2024-09-12 00:46:20.663340262 +0000 UTC m=+1.016432506 container create ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T00:46:20.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 podman[30585]: 2024-09-12 00:46:20.882771682 +0000 UTC m=+1.235863927 container init ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T00:46:20.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 podman[30585]: 2024-09-12 00:46:20.886418279 +0000 UTC m=+1.239510517 container start ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T00:46:20.971 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T00:46:20.971 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T00:46:20.979 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T00:46:20.980 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:46:20.980 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T00:46:20.980 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T00:46:20.980 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-12T00:46:20.980 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-12T00:46:21.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 ceph-mon[30603]: mkfs 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:21.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 ceph-mon[30603]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T00:46:21.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 bash[30585]: ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 2024-09-12T00:46:21.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:20 smithi016 systemd[1]: Started Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:46:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T00:46:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: monmap epoch 1 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: last_changed 2024-09-12T00:46:08.235575+0000 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: min_mon_release 19 (squid) 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: election_strategy: 1 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: fsmap 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: mgrmap e1: no daemons active 2024-09-12T00:46:22.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:21 smithi016 ceph-mon[30603]: from='client.? 172.21.15.16:0/1716035115' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T00:46:22.174 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.552541s) 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T00:46:22.182 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.407+0000 7f3bb9c97640 1 Processor -- start 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.407+0000 7f3bb9c97640 1 -- start start 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.407+0000 7f3bb9c97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.407+0000 7f3bb9c97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4108170 con 0x7f3bb41077a0 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.408+0000 7f3bb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.408+0000 7f3bb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41700/0 (socket says 172.21.15.16:41700) 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.408+0000 7f3bb37fe640 1 -- 172.21.15.16:0/3757330463 learned_addr learned my addr 172.21.15.16:0/3757330463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.408+0000 7f3bb37fe640 1 -- 172.21.15.16:0/3757330463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb4108950 con 0x7f3bb41077a0 2024-09-12T00:46:22.183 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.410+0000 7f3bb37fe640 1 --2- 172.21.15.16:0/3757330463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009820 tx=0x7f3ba402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52c4cfb2a360c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.410+0000 7f3bb27fc640 1 -- 172.21.15.16:0/3757330463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ba4037070 con 0x7f3bb41077a0 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.410+0000 7f3bb27fc640 1 -- 172.21.15.16:0/3757330463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba4032440 con 0x7f3bb41077a0 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.410+0000 7f3bb27fc640 1 -- 172.21.15.16:0/3757330463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ba4032740 con 0x7f3bb41077a0 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 -- 172.21.15.16:0/3757330463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 msgr2=0x7f3bb4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 --2- 172.21.15.16:0/3757330463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009820 tx=0x7f3ba402a270 comp rx=0 tx=0).stop 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 -- 172.21.15.16:0/3757330463 shutdown_connections 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 --2- 172.21.15.16:0/3757330463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb4107ba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 -- 172.21.15.16:0/3757330463 >> 172.21.15.16:0/3757330463 conn(0x7f3bb4102fd0 msgr2=0x7f3bb41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:22.184 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 -- 172.21.15.16:0/3757330463 shutdown_connections 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 -- 172.21.15.16:0/3757330463 wait complete. 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.411+0000 7f3bb9c97640 1 Processor -- start 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb9c97640 1 -- start start 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb9c97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb9c97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4108ef0 con 0x7f3bb41077a0 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41706/0 (socket says 172.21.15.16:41706) 2024-09-12T00:46:22.185 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb37fe640 1 -- 172.21.15.16:0/1716035115 learned_addr learned my addr 172.21.15.16:0/1716035115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb37fe640 1 -- 172.21.15.16:0/1716035115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb419d670 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.412+0000 7f3bb37fe640 1 --2- 172.21.15.16:0/1716035115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40090b0 tx=0x7f3ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ba4041070 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba403c7a0 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb419d900 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ba4037040 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41a05f0 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3ba4044070 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.413+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3ba403d3f0 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.414+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb4109540 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.416+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3ba4046450 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.478+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3bb41a0b40 con 0x7f3bb41077a0 2024-09-12T00:46:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.478+0000 7f3bb0ff9640 1 -- 172.21.15.16:0/1716035115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f3ba4046630 con 0x7f3bb41077a0 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 msgr2=0x7f3bb419d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 --2- 172.21.15.16:0/1716035115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40090b0 tx=0x7f3ba40046c0 comp rx=0 tx=0).stop 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 shutdown_connections 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 --2- 172.21.15.16:0/1716035115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb41077a0 0x7f3bb419d130 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 >> 172.21.15.16:0/1716035115 conn(0x7f3bb4102fd0 msgr2=0x7f3bb407c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 shutdown_connections 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:21.479+0000 7f3bb9c97640 1 -- 172.21.15.16:0/1716035115 wait complete. 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-12T00:46:22.187 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T00:46:23.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:22 smithi016 ceph-mon[30603]: from='client.? 172.21.15.16:0/3669669490' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T00:46:23.236 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:22 smithi016 ceph-mon[30603]: from='client.? 172.21.15.16:0/3669669490' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T00:46:23.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:23.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T00:46:23.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:23.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T00:46:23.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.611+0000 7fc5019d5640 1 Processor -- start 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.611+0000 7fc5019d5640 1 -- start start 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc5019d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc5019d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc108170 con 0x7fc4fc1077a0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41720/0 (socket says 172.21.15.16:41720) 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc4faffd640 1 -- 172.21.15.16:0/2266941591 learned_addr learned my addr 172.21.15.16:0/2266941591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.612+0000 7fc4faffd640 1 -- 172.21.15.16:0/2266941591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc108950 con 0x7fc4fc1077a0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.613+0000 7fc4faffd640 1 --2- 172.21.15.16:0/2266941591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009820 tx=0x7fc4ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c728f5b78acfbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.613+0000 7fc4f9ffb640 1 -- 172.21.15.16:0/2266941591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ec037070 con 0x7fc4fc1077a0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.613+0000 7fc4f9ffb640 1 -- 172.21.15.16:0/2266941591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec032440 con 0x7fc4fc1077a0 2024-09-12T00:46:23.248 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.613+0000 7fc4f9ffb640 1 -- 172.21.15.16:0/2266941591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ec032740 con 0x7fc4fc1077a0 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 -- 172.21.15.16:0/2266941591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 msgr2=0x7fc4fc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 --2- 172.21.15.16:0/2266941591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009820 tx=0x7fc4ec02a270 comp rx=0 tx=0).stop 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 -- 172.21.15.16:0/2266941591 shutdown_connections 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 --2- 172.21.15.16:0/2266941591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc107ba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 -- 172.21.15.16:0/2266941591 >> 172.21.15.16:0/2266941591 conn(0x7fc4fc102fd0 msgr2=0x7fc4fc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 -- 172.21.15.16:0/2266941591 shutdown_connections 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 -- 172.21.15.16:0/2266941591 wait complete. 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.614+0000 7fc5019d5640 1 Processor -- start 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc5019d5640 1 -- start start 2024-09-12T00:46:23.249 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc5019d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc5019d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc108ef0 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41726/0 (socket says 172.21.15.16:41726) 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc4faffd640 1 -- 172.21.15.16:0/3669669490 learned_addr learned my addr 172.21.15.16:0/3669669490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc4faffd640 1 -- 172.21.15.16:0/3669669490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc07e910 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.615+0000 7fc4faffd640 1 --2- 172.21.15.16:0/3669669490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0090b0 tx=0x7fc4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ec041070 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec03c7a0 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc07cb00 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4ec037040 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc07d040 con 0x7fc4fc1077a0 2024-09-12T00:46:23.250 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4ec044070 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.616+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4ec03d3f0 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.617+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc109540 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.619+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc4ec046450 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.679+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc4fc07d340 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.682+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fc4ec046630 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.683+0000 7fc5009d3640 1 -- 172.21.15.16:0/3669669490 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4ec056070 con 0x7fc4fc1077a0 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 msgr2=0x7fc4fc07e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:23.251 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 --2- 172.21.15.16:0/3669669490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0090b0 tx=0x7fc4ec0046c0 comp rx=0 tx=0).stop 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 shutdown_connections 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 --2- 172.21.15.16:0/3669669490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4fc1077a0 0x7fc4fc07e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 >> 172.21.15.16:0/3669669490 conn(0x7fc4fc102fd0 msgr2=0x7fc4fc18f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 shutdown_connections 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:22.684+0000 7fc5019d5640 1 -- 172.21.15.16:0/3669669490 wait complete. 2024-09-12T00:46:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-12T00:46:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:23 smithi016 ceph-mon[30603]: from='client.? 172.21.15.16:0/3873943797' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:46:24.400 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8062da0640 1 Processor -- start 2024-09-12T00:46:24.400 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8062da0640 1 -- start start 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8062da0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8062da0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f805c108170 con 0x7f805c1077a0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8060b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8060b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41730/0 (socket says 172.21.15.16:41730) 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8060b3c640 1 -- 172.21.15.16:0/578853721 learned_addr learned my addr 172.21.15.16:0/578853721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.639+0000 7f8060b3c640 1 -- 172.21.15.16:0/578853721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c108950 con 0x7f805c1077a0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.640+0000 7f8060b3c640 1 --2- 172.21.15.16:0/578853721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8044009820 tx=0x7f804402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56b4d03d9701670 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.640+0000 7f805b7fe640 1 -- 172.21.15.16:0/578853721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8044037070 con 0x7f805c1077a0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.640+0000 7f805b7fe640 1 -- 172.21.15.16:0/578853721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8044032900 con 0x7f805c1077a0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.640+0000 7f805b7fe640 1 -- 172.21.15.16:0/578853721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8044030850 con 0x7f805c1077a0 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- 172.21.15.16:0/578853721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 msgr2=0x7f805c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:24.401 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 --2- 172.21.15.16:0/578853721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8044009820 tx=0x7f804402a270 comp rx=0 tx=0).stop 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- 172.21.15.16:0/578853721 shutdown_connections 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 --2- 172.21.15.16:0/578853721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c107ba0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- 172.21.15.16:0/578853721 >> 172.21.15.16:0/578853721 conn(0x7f805c102fd0 msgr2=0x7f805c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- 172.21.15.16:0/578853721 shutdown_connections 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- 172.21.15.16:0/578853721 wait complete. 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 Processor -- start 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.641+0000 7f8062da0640 1 -- start start 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8062da0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8062da0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f805c10bd00 con 0x7f805c1077a0 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8060b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8060b3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41734/0 (socket says 172.21.15.16:41734) 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8060b3c640 1 -- 172.21.15.16:0/3873943797 learned_addr learned my addr 172.21.15.16:0/3873943797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:24.402 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8060b3c640 1 -- 172.21.15.16:0/3873943797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c19bc80 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8060b3c640 1 --2- 172.21.15.16:0/3873943797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80440097f0 tx=0x7f8044030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8044037040 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8044040070 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c19bf10 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.642+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804403a390 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.643+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c19ec00 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.643+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f804403a630 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.643+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80440459a0 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.644+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c107c20 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.646+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8044045460 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.706+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f805c19edd0 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.706+0000 7f8059ffb640 1 -- 172.21.15.16:0/3873943797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f804403b430 con 0x7f805c1077a0 2024-09-12T00:46:24.403 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 msgr2=0x7f805c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 --2- 172.21.15.16:0/3873943797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f80440097f0 tx=0x7f8044030d80 comp rx=0 tx=0).stop 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 shutdown_connections 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 --2- 172.21.15.16:0/3873943797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c1077a0 0x7f805c19b740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 >> 172.21.15.16:0/3873943797 conn(0x7f805c102fd0 msgr2=0x7f805c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 shutdown_connections 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:23.707+0000 7f8062da0640 1 -- 172.21.15.16:0/3873943797 wait complete. 2024-09-12T00:46:24.404 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-12T00:46:24.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:24 smithi016 systemd[1]: Stopping Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:46:25.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:24 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a[30599]: 2024-09-12T00:46:24.681+0000 7fb1b21f4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T00:46:25.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:24 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a[30599]: 2024-09-12T00:46:24.681+0000 7fb1b21f4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T00:46:25.046 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:24 smithi016 podman[30847]: 2024-09-12 00:46:24.701959336 +0000 UTC m=+0.116136543 container died ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T00:46:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 podman[30847]: 2024-09-12 00:46:25.597786292 +0000 UTC m=+1.011963491 container cleanup ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T00:46:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 bash[30847]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a 2024-09-12T00:46:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 podman[30861]: 2024-09-12 00:46:25.807180028 +0000 UTC m=+1.100078665 container remove ddbd6e65020acdbfd0eff654e1266b9098489e091d2200f6e6f19816398aedd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T00:46:26.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 systemd[1]: ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T00:46:26.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 systemd[1]: Stopped Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:46:26.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:25 smithi016 systemd[1]: Starting Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:46:26.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 podman[30953]: 2024-09-12 00:46:26.071489069 +0000 UTC m=+0.024295109 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:46:26.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 podman[30953]: 2024-09-12 00:46:26.208003449 +0000 UTC m=+0.160809479 container create 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T00:46:26.458 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 podman[30953]: 2024-09-12 00:46:26.435301659 +0000 UTC m=+0.388107701 container init 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:46:26.458 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 podman[30953]: 2024-09-12 00:46:26.438423895 +0000 UTC m=+0.391229930 container start 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T00:46:26.513 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T00:46:26.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T00:46:26.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: pidfile_write: ignore empty --pid-file 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: load: jerasure load: lrc 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: RocksDB version: 7.9.2 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Git sha 0 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: DB SUMMARY 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: DB Session ID: CM69SEDSAAZLSSKU1J8J 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: CURRENT file: CURRENT 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106418 ; 2024-09-12T00:46:26.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.error_if_exists: 0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.create_if_missing: 0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.paranoid_checks: 1 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.env: 0x5623bb263940 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.info_log: 0x5623bd1b36e0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.statistics: (nil) 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.use_fsync: 0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_log_file_size: 0 2024-09-12T00:46:26.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_fallocate: 1 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.use_direct_reads: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.db_log_dir: 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.wal_dir: 2024-09-12T00:46:26.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.write_buffer_manager: 0x5623bd1c2dc0 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T00:46:26.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.unordered_write: 0 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.row_cache: None 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.wal_filter: None 2024-09-12T00:46:26.743 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.two_write_queues: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.wal_compression: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.atomic_flush: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T00:46:26.744 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.log_readahead_size: 0 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_background_jobs: 2 2024-09-12T00:46:26.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_background_compactions: -1 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_subcompactions: 1 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T00:46:26.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_open_files: -1 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_background_flushes: -1 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Compression algorithms supported: 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kZSTD supported: 0 2024-09-12T00:46:26.747 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kXpressCompression supported: 0 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kBZip2Compression supported: 0 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kLZ4Compression supported: 1 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kZlibCompression supported: 1 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: kSnappyCompression supported: 1 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T00:46:26.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.merge_operator: 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_filter: None 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623bd1b3320) 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-12T00:46:26.749 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x5623bd1a71f0 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-12T00:46:26.750 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-12T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-12T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression: NoCompression 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.num_levels: 7 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T00:46:26.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T00:46:26.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T00:46:26.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T00:46:26.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T00:46:26.757 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.inplace_update_support: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.bloom_locality: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.max_successive_merges: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T00:46:26.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.ttl: 2592000 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enable_blob_files: false 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.min_blob_size: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T00:46:26.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e868fddb-338a-4b52-99d5-f6d485958e53 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726101986465433, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726101986466550, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99314, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14564, "raw_average_key_size": 50, "raw_value_size": 91389, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726101986, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e868fddb-338a-4b52-99d5-f6d485958e53", "db_session_id": "CM69SEDSAAZLSSKU1J8J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726101986466648, "job": 1, "event": "recovery_finished"} 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5623bd1d4e00 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: DB pointer 0x5623bd2d0000 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T00:46:26.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T00:46:26.761 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 100.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 100.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T00:46:26.762 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 29.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 29.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x5623bd1a71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T00:46:26.763 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???) e1 preinit fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).mds e1 new map 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).mds e1 print_map 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-12T00:46:26.764 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-12T00:46:20:927044+0000 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T00:46:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 bash[30953]: 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 systemd[1]: Started Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: monmap epoch 1 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: last_changed 2024-09-12T00:46:08.235575+0000 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: election_strategy: 1 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: fsmap 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T00:46:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:26 smithi016 ceph-mon[30972]: mgrmap e1: no daemons active 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.929+0000 7f533d953640 1 Processor -- start 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.929+0000 7f533d953640 1 -- start start 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f533d953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f533d953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f533807be30 con 0x7f53381081b0 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f5336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:27.441 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f5336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41750/0 (socket says 172.21.15.16:41750) 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f5336ffd640 1 -- 172.21.15.16:0/502133725 learned_addr learned my addr 172.21.15.16:0/502133725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.930+0000 7f5336ffd640 1 -- 172.21.15.16:0/502133725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533810aae0 con 0x7f53381081b0 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.932+0000 7f5336ffd640 1 --2- 172.21.15.16:0/502133725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e80 tx=0x7f532802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc59b4e29981e522 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.932+0000 7f53367fc640 1 -- 172.21.15.16:0/502133725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5328036070 con 0x7f53381081b0 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.932+0000 7f53367fc640 1 -- 172.21.15.16:0/502133725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53280318c0 con 0x7f53381081b0 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.932+0000 7f53367fc640 1 -- 172.21.15.16:0/502133725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5328007ab0 con 0x7f53381081b0 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 -- 172.21.15.16:0/502133725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 msgr2=0x7f533810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 --2- 172.21.15.16:0/502133725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5328009e80 tx=0x7f532802a7a0 comp rx=0 tx=0).stop 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 -- 172.21.15.16:0/502133725 shutdown_connections 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 --2- 172.21.15.16:0/502133725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533810a5a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 -- 172.21.15.16:0/502133725 >> 172.21.15.16:0/502133725 conn(0x7f5338101db0 msgr2=0x7f5338104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:27.442 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 -- 172.21.15.16:0/502133725 shutdown_connections 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 -- 172.21.15.16:0/502133725 wait complete. 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.933+0000 7f533d953640 1 Processor -- start 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f533d953640 1 -- start start 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f533d953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f533d953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f533810bb20 con 0x7f53381081b0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f5336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f5336ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41752/0 (socket says 172.21.15.16:41752) 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f5336ffd640 1 -- 172.21.15.16:0/1041836016 learned_addr learned my addr 172.21.15.16:0/1041836016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f5336ffd640 1 -- 172.21.15.16:0/1041836016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533819bc90 con 0x7f53381081b0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.934+0000 7f5336ffd640 1 --2- 172.21.15.16:0/1041836016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5328031ca0 tx=0x7f5328031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5328036050 con 0x7f53381081b0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533819bf20 con 0x7f53381081b0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5328042070 con 0x7f53381081b0 2024-09-12T00:46:27.443 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f532803d940 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533819ec10 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f532803dc40 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.935+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5328007e00 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.936+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5338108610 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.938+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f53280073b0 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:26.998+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f533819f200 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.000+0000 7f533c951640 1 -- 172.21.15.16:0/1041836016 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f5328047460 con 0x7f53381081b0 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 msgr2=0x7f533819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 --2- 172.21.15.16:0/1041836016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5328031ca0 tx=0x7f5328031cd0 comp rx=0 tx=0).stop 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 shutdown_connections 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 --2- 172.21.15.16:0/1041836016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53381081b0 0x7f533819b750 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 >> 172.21.15.16:0/1041836016 conn(0x7f5338101db0 msgr2=0x7f5338102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 shutdown_connections 2024-09-12T00:46:27.445 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:27.001+0000 7f533d953640 1 -- 172.21.15.16:0/1041836016 wait complete. 2024-09-12T00:46:27.445 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T00:46:27.465 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:46:27.465 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-12T00:46:27.465 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T00:46:27.465 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T00:46:27.740 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x 2024-09-12T00:46:27.740 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T00:46:27.897 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968.target.wants/ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-434eab66-70a0-11ef-bcea-c7b262605968@.service. 2024-09-12T00:46:27.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:27 smithi016 systemd[1]: Starting Ceph mgr.x for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:46:28.253 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1041836016' entity='client.admin' 2024-09-12T00:46:28.254 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 podman[31182]: 2024-09-12 00:46:28.127547684 +0000 UTC m=+0.112513618 container create ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T00:46:28.254 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 podman[31182]: 2024-09-12 00:46:28.031819293 +0000 UTC m=+0.016785197 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T00:46:28.434 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T00:46:28.434 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T00:46:28.442 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T00:46:28.442 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:46:28.442 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T00:46:28.442 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T00:46:28.451 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T00:46:28.451 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T00:46:28.459 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T00:46:28.459 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:46:28.459 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T00:46:28.460 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T00:46:28.460 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-12T00:46:28.460 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 podman[31182]: 2024-09-12 00:46:28.33805775 +0000 UTC m=+0.323023642 container init ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 podman[31182]: 2024-09-12 00:46:28.34090143 +0000 UTC m=+0.325867322 container start ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 bash[31182]: ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 systemd[1]: Started Ceph mgr.x for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:28.466+0000 7feabc22b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:46:28.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:28.528+0000 7feabc22b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:46:28.900 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:28.687+0000 7feabc22b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:46:28.900 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:28.757+0000 7feabc22b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:46:29.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:29 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4243601101' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T00:46:29.209 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:28.918+0000 7feabc22b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:46:29.209 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:29.069+0000 7feabc22b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:46:29.469 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:29.407+0000 7feabc22b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:46:29.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:29.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:46:29.521 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:29.522 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T00:46:29.523 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:29.524 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T00:46:20:927044+0000", 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:29.525 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T00:46:20.927744+0000", 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.830+0000 7f742777c640 1 Processor -- start 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.830+0000 7f742777c640 1 -- start start 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.830+0000 7f742777c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.830+0000 7f742777c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f742007bd90 con 0x7f7420108250 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.831+0000 7f7425518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.831+0000 7f7425518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43988/0 (socket says 172.21.15.16:43988) 2024-09-12T00:46:29.526 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.831+0000 7f7425518640 1 -- 172.21.15.16:0/3958389722 learned_addr learned my addr 172.21.15.16:0/3958389722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.831+0000 7f7425518640 1 -- 172.21.15.16:0/3958389722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742010ab80 con 0x7f7420108250 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.832+0000 7f7425518640 1 --2- 172.21.15.16:0/3958389722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f741c009820 tx=0x7f741c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45777529357d26b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.832+0000 7f7417fff640 1 -- 172.21.15.16:0/3958389722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f741c037070 con 0x7f7420108250 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.832+0000 7f7417fff640 1 -- 172.21.15.16:0/3958389722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f741c032900 con 0x7f7420108250 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f7417fff640 1 -- 172.21.15.16:0/3958389722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f741c030850 con 0x7f7420108250 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 -- 172.21.15.16:0/3958389722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 msgr2=0x7f742010a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 --2- 172.21.15.16:0/3958389722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f741c009820 tx=0x7f741c02a270 comp rx=0 tx=0).stop 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 -- 172.21.15.16:0/3958389722 shutdown_connections 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 --2- 172.21.15.16:0/3958389722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f742010a640 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 -- 172.21.15.16:0/3958389722 >> 172.21.15.16:0/3958389722 conn(0x7f7420101db0 msgr2=0x7f7420104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:29.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 -- 172.21.15.16:0/3958389722 shutdown_connections 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.833+0000 7f742777c640 1 -- 172.21.15.16:0/3958389722 wait complete. 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f742777c640 1 Processor -- start 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f742777c640 1 -- start start 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f742777c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f742777c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f742010bb00 con 0x7f7420108250 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f7425518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f7425518640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43996/0 (socket says 172.21.15.16:43996) 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.834+0000 7f7425518640 1 -- 172.21.15.16:0/4243601101 learned_addr learned my addr 172.21.15.16:0/4243601101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f7425518640 1 -- 172.21.15.16:0/4243601101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7420108010 con 0x7f7420108250 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f7425518640 1 --2- 172.21.15.16:0/4243601101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f741c031000 tx=0x7f741c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f741c042070 con 0x7f7420108250 2024-09-12T00:46:29.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f741c032be0 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7420106200 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f741c037040 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7420106740 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f741c045070 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.835+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f741c047b20 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.836+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74201086b0 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.838+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f741c04c030 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.901+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7420106c70 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.901+0000 7f74167fc640 1 -- 172.21.15.16:0/4243601101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f741c0328c0 con 0x7f7420108250 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 msgr2=0x7f7420107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 --2- 172.21.15.16:0/4243601101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f741c031000 tx=0x7f741c0326e0 comp rx=0 tx=0).stop 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 shutdown_connections 2024-09-12T00:46:29.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 --2- 172.21.15.16:0/4243601101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7420108250 0x7f7420107ad0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:29.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 >> 172.21.15.16:0/4243601101 conn(0x7f7420101db0 msgr2=0x7f7420102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:29.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 shutdown_connections 2024-09-12T00:46:29.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:28.903+0000 7f742777c640 1 -- 172.21.15.16:0/4243601101 wait complete. 2024-09-12T00:46:29.530 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-12T00:46:29.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:29.470+0000 7feabc22b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:46:29.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:29.665+0000 7feabc22b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:46:30.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:30.589+0000 7feabc22b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:46:30.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:30.659+0000 7feabc22b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:46:31.683 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.411+0000 7feabc22b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:46:31.684 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.533+0000 7feabc22b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:46:31.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.685+0000 7feabc22b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:46:31.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.751+0000 7feabc22b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:46:31.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.909+0000 7feabc22b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:46:31.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:31.983+0000 7feabc22b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:46:32.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2003554695' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T00:46:32.286 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.049+0000 7feabc22b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:46:32.286 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.219+0000 7feabc22b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:46:32.287 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.287+0000 7feabc22b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:46:32.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.358+0000 7feabc22b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:46:32.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.444+0000 7feabc22b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:46:32.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.507+0000 7feabc22b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:46:32.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:32.570+0000 7feabc22b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:46:32.623 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:32.623 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T00:46:32.624 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T00:46:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T00:46:32.626 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T00:46:20:927044+0000", 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T00:46:32.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T00:46:20.927744+0000", 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.926+0000 7f89dc62b640 1 Processor -- start 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.926+0000 7f89dc62b640 1 -- start start 2024-09-12T00:46:32.628 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.926+0000 7f89dc62b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.927+0000 7f89dc62b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89d407bdf0 con 0x7f89d41081f0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.927+0000 7f89da3c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.927+0000 7f89da3c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44004/0 (socket says 172.21.15.16:44004) 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.927+0000 7f89da3c7640 1 -- 172.21.15.16:0/1466631302 learned_addr learned my addr 172.21.15.16:0/1466631302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.927+0000 7f89da3c7640 1 -- 172.21.15.16:0/1466631302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d410ab20 con 0x7f89d41081f0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.928+0000 7f89da3c7640 1 --2- 172.21.15.16:0/1466631302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009820 tx=0x7f89d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3b9073382549bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.928+0000 7f89d93c5640 1 -- 172.21.15.16:0/1466631302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0037070 con 0x7f89d41081f0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.928+0000 7f89d93c5640 1 -- 172.21.15.16:0/1466631302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89d0032900 con 0x7f89d41081f0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.928+0000 7f89d93c5640 1 -- 172.21.15.16:0/1466631302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0030850 con 0x7f89d41081f0 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 -- 172.21.15.16:0/1466631302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 msgr2=0x7f89d410a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 --2- 172.21.15.16:0/1466631302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009820 tx=0x7f89d002a270 comp rx=0 tx=0).stop 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 -- 172.21.15.16:0/1466631302 shutdown_connections 2024-09-12T00:46:32.629 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 --2- 172.21.15.16:0/1466631302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d410a5e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 -- 172.21.15.16:0/1466631302 >> 172.21.15.16:0/1466631302 conn(0x7f89d4101db0 msgr2=0x7f89d4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 -- 172.21.15.16:0/1466631302 shutdown_connections 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.929+0000 7f89dc62b640 1 -- 172.21.15.16:0/1466631302 wait complete. 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89dc62b640 1 Processor -- start 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89dc62b640 1 -- start start 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89dc62b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89dc62b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410bb00 con 0x7f89d41081f0 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89da3c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89da3c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44014/0 (socket says 172.21.15.16:44014) 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89da3c7640 1 -- 172.21.15.16:0/2003554695 learned_addr learned my addr 172.21.15.16:0/2003554695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:32.630 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.930+0000 7f89da3c7640 1 -- 172.21.15.16:0/2003554695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4107fb0 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89da3c7640 1 --2- 172.21.15.16:0/2003554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031000 tx=0x7f89d00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0042070 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41061a0 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89d0032be0 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0037040 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41066e0 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89d0045070 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.931+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89d0047b20 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.932+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d4108650 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.934+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f89d004c030 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.997+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f89d4106e10 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.998+0000 7f89cb7fe640 1 -- 172.21.15.16:0/2003554695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f89d0047dc0 con 0x7f89d41081f0 2024-09-12T00:46:32.631 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 msgr2=0x7f89d4107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 --2- 172.21.15.16:0/2003554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031000 tx=0x7f89d00326e0 comp rx=0 tx=0).stop 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 shutdown_connections 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 --2- 172.21.15.16:0/2003554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89d41081f0 0x7f89d4107a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 >> 172.21.15.16:0/2003554695 conn(0x7f89d4101db0 msgr2=0x7f89d4102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 shutdown_connections 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:31.999+0000 7f89dc62b640 1 -- 172.21.15.16:0/2003554695 wait complete. 2024-09-12T00:46:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-12T00:46:33.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:33.411+0000 7feabc22b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:46:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: Activating manager daemon x 2024-09-12T00:46:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: mgrmap e2: x(active, starting, since 0.003045s) 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: Manager daemon x is now available 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' 2024-09-12T00:46:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:33 smithi016 ceph-mon[30972]: from='mgr.14100 172.21.15.16:0/3109426639' entity='mgr.x' 2024-09-12T00:46:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:35 smithi016 ceph-mon[30972]: mgrmap e3: x(active, since 1.00731s) 2024-09-12T00:46:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1440404064' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T00:46:35.852 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T00:46:35.853 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T00:46:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T00:46:35.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T00:46:35.862 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T00:46:35.863 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T00:46:20:927044+0000", 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T00:46:35.864 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T00:46:20.927744+0000", 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d24b96640 1 Processor -- start 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d24b96640 1 -- start start 2024-09-12T00:46:35.865 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d24b96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d24b96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20108170 con 0x7f9d201077a0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44094/0 (socket says 172.21.15.16:44094) 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.044+0000 7f9d1e59c640 1 -- 172.21.15.16:0/2138259418 learned_addr learned my addr 172.21.15.16:0/2138259418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.045+0000 7f9d1e59c640 1 -- 172.21.15.16:0/2138259418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d20108950 con 0x7f9d201077a0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.045+0000 7f9d1e59c640 1 --2- 172.21.15.16:0/2138259418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009820 tx=0x7f9d1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf4f531644b4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d1d59a640 1 -- 172.21.15.16:0/2138259418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d10037070 con 0x7f9d201077a0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d1d59a640 1 -- 172.21.15.16:0/2138259418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d10032900 con 0x7f9d201077a0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d1d59a640 1 -- 172.21.15.16:0/2138259418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d10030850 con 0x7f9d201077a0 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 -- 172.21.15.16:0/2138259418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 msgr2=0x7f9d20107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 --2- 172.21.15.16:0/2138259418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009820 tx=0x7f9d1002a270 comp rx=0 tx=0).stop 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 -- 172.21.15.16:0/2138259418 shutdown_connections 2024-09-12T00:46:35.866 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 --2- 172.21.15.16:0/2138259418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d20107ba0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 -- 172.21.15.16:0/2138259418 >> 172.21.15.16:0/2138259418 conn(0x7f9d20102fd0 msgr2=0x7f9d201053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 -- 172.21.15.16:0/2138259418 shutdown_connections 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.046+0000 7f9d24b96640 1 -- 172.21.15.16:0/2138259418 wait complete. 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.047+0000 7f9d24b96640 1 Processor -- start 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.047+0000 7f9d24b96640 1 -- start start 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.047+0000 7f9d24b96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.047+0000 7f9d24b96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2010bd00 con 0x7f9d201077a0 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44104/0 (socket says 172.21.15.16:44104) 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d1e59c640 1 -- 172.21.15.16:0/1440404064 learned_addr learned my addr 172.21.15.16:0/1440404064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d1e59c640 1 -- 172.21.15.16:0/1440404064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2019bc80 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d1e59c640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9d100097f0 tx=0x7f9d10030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d10037040 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2019bf10 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d10040070 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d1003a390 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.048+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2019ec00 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.049+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f9d1003a650 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.049+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2010be80 con 0x7f9d201077a0 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.050+0000 7f9d0f7fe640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 0x7f9d0803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:35.868 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.050+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d1006bfe0 con 0x7f9d201077a0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.050+0000 7f9d1dd9b640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 0x7f9d0803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.050+0000 7f9d1dd9b640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 0x7f9d0803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d140098c0 tx=0x7f9d14006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.055+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d1003a9f0 con 0x7f9d201077a0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.300+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9d2019edd0 con 0x7f9d201077a0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.300+0000 7f9d0f7fe640 1 -- 172.21.15.16:0/1440404064 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f9d10043520 con 0x7f9d201077a0 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 msgr2=0x7f9d0803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 0x7f9d0803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d140098c0 tx=0x7f9d14006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 msgr2=0x7f9d2019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9d100097f0 tx=0x7f9d10030d80 comp rx=0 tx=0).stop 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 shutdown_connections 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9d08038e10 0x7f9d0803b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 --2- 172.21.15.16:0/1440404064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d201077a0 0x7f9d2019b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:35.869 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.302+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 >> 172.21.15.16:0/1440404064 conn(0x7f9d20102fd0 msgr2=0x7f9d20103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:35.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.303+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 shutdown_connections 2024-09-12T00:46:35.870 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:35.303+0000 7f9d24b96640 1 -- 172.21.15.16:0/1440404064 wait complete. 2024-09-12T00:46:35.870 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-12T00:46:36.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:36 smithi016 ceph-mon[30972]: mgrmap e4: x(active, since 2s) 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T00:46:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.252+0000 7ff084657640 1 Processor -- start 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.252+0000 7ff084657640 1 -- start start 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.252+0000 7ff084657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:37.031 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.252+0000 7ff084657640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c07c660 con 0x7ff07c106050 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.253+0000 7ff0823f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.253+0000 7ff0823f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44106/0 (socket says 172.21.15.16:44106) 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.253+0000 7ff0823f3640 1 -- 172.21.15.16:0/3653503911 learned_addr learned my addr 172.21.15.16:0/3653503911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.253+0000 7ff0823f3640 1 -- 172.21.15.16:0/3653503911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c106990 con 0x7ff07c106050 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff0823f3640 1 --2- 172.21.15.16:0/3653503911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff078009820 tx=0x7ff07802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4085943701ab195 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff0813f1640 1 -- 172.21.15.16:0/3653503911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078037070 con 0x7ff07c106050 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff0813f1640 1 -- 172.21.15.16:0/3653503911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff078032900 con 0x7ff07c106050 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff0813f1640 1 -- 172.21.15.16:0/3653503911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078030850 con 0x7ff07c106050 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff084657640 1 -- 172.21.15.16:0/3653503911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 msgr2=0x7ff07c106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.254+0000 7ff084657640 1 --2- 172.21.15.16:0/3653503911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff078009820 tx=0x7ff07802a270 comp rx=0 tx=0).stop 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- 172.21.15.16:0/3653503911 shutdown_connections 2024-09-12T00:46:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 --2- 172.21.15.16:0/3653503911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c106450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- 172.21.15.16:0/3653503911 >> 172.21.15.16:0/3653503911 conn(0x7ff07c101ea0 msgr2=0x7ff07c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- 172.21.15.16:0/3653503911 shutdown_connections 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- 172.21.15.16:0/3653503911 wait complete. 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 Processor -- start 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- start start 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.255+0000 7ff084657640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c107a90 con 0x7ff07c106050 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0823f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0823f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44114/0 (socket says 172.21.15.16:44114) 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0823f3640 1 -- 172.21.15.16:0/3173742835 learned_addr learned my addr 172.21.15.16:0/3173742835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0823f3640 1 -- 172.21.15.16:0/3173742835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c19de40 con 0x7ff07c106050 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0823f3640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff078032ce0 tx=0x7ff078032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:37.033 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078042070 con 0x7ff07c106050 2024-09-12T00:46:37.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0780308c0 con 0x7ff07c106050 2024-09-12T00:46:37.034 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c19e0d0 con 0x7ff07c106050 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff078037040 con 0x7ff07c106050 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.256+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c1a0dc0 con 0x7ff07c106050 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.257+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff078045070 con 0x7ff07c106050 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.257+0000 7ff0737fe640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 0x7ff06803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.257+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff07806bfd0 con 0x7ff07c106050 2024-09-12T00:46:37.036 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.257+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07c106450 con 0x7ff07c106050 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.257+0000 7ff081bf2640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 0x7ff06803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.258+0000 7ff081bf2640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 0x7ff06803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098c0 tx=0x7ff06c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.262+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff07806b9f0 con 0x7ff07c106050 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.441+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff07c1a0f90 con 0x7ff07c106050 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.442+0000 7ff0737fe640 1 -- 172.21.15.16:0/3173742835 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7ff07c1a0f90 con 0x7ff07c106050 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 msgr2=0x7ff06803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 0x7ff06803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098c0 tx=0x7ff06c006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 msgr2=0x7ff07c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff078032ce0 tx=0x7ff078032d10 comp rx=0 tx=0).stop 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 shutdown_connections 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7ff068038f70 0x7ff06803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 --2- 172.21.15.16:0/3173742835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07c106050 0x7ff07c19d900 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:37.037 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 >> 172.21.15.16:0/3173742835 conn(0x7ff07c101ea0 msgr2=0x7ff07c191c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:37.038 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 shutdown_connections 2024-09-12T00:46:37.038 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:36.444+0000 7ff084657640 1 -- 172.21.15.16:0/3173742835 wait complete. 2024-09-12T00:46:37.038 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-12T00:46:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:37 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3173742835' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T00:46:38.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1919940552' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T00:46:38.736 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setuser ceph since I am not root 2024-09-12T00:46:38.737 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setgroup ceph since I am not root 2024-09-12T00:46:38.737 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:38.576+0000 7f261f22f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:46:38.737 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:38.635+0000 7f261f22f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:46:38.994 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:38.793+0000 7f261f22f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:46:38.995 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:38.862+0000 7f261f22f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.433+0000 7f5734885640 1 Processor -- start 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f5734885640 1 -- start start 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f5734885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f5734885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5730108170 con 0x7f57301077a0 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f572e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f572e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44122/0 (socket says 172.21.15.16:44122) 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f572e59c640 1 -- 172.21.15.16:0/1560481873 learned_addr learned my addr 172.21.15.16:0/1560481873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.434+0000 7f572e59c640 1 -- 172.21.15.16:0/1560481873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730108950 con 0x7f57301077a0 2024-09-12T00:46:39.002 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.435+0000 7f572e59c640 1 --2- 172.21.15.16:0/1560481873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5720009820 tx=0x7f572002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9975841f7ec043a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.435+0000 7f572d59a640 1 -- 172.21.15.16:0/1560481873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720037070 con 0x7f57301077a0 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.435+0000 7f572d59a640 1 -- 172.21.15.16:0/1560481873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5720032900 con 0x7f57301077a0 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.435+0000 7f572d59a640 1 -- 172.21.15.16:0/1560481873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720030850 con 0x7f57301077a0 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 -- 172.21.15.16:0/1560481873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 msgr2=0x7f5730107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 --2- 172.21.15.16:0/1560481873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5720009820 tx=0x7f572002a270 comp rx=0 tx=0).stop 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 -- 172.21.15.16:0/1560481873 shutdown_connections 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 --2- 172.21.15.16:0/1560481873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f5730107ba0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 -- 172.21.15.16:0/1560481873 >> 172.21.15.16:0/1560481873 conn(0x7f5730102fd0 msgr2=0x7f57301053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 -- 172.21.15.16:0/1560481873 shutdown_connections 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.436+0000 7f5734885640 1 -- 172.21.15.16:0/1560481873 wait complete. 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f5734885640 1 Processor -- start 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f5734885640 1 -- start start 2024-09-12T00:46:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f5734885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f5734885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f573010bd00 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f572e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f572e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44134/0 (socket says 172.21.15.16:44134) 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f572e59c640 1 -- 172.21.15.16:0/1919940552 learned_addr learned my addr 172.21.15.16:0/1919940552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.437+0000 7f572e59c640 1 -- 172.21.15.16:0/1919940552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573019bc80 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f572e59c640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5720009940 tx=0x7f5720032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720042070 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573019bf10 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57200308c0 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5720037040 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.438+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573019ec00 con 0x7f57301077a0 2024-09-12T00:46:39.004 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5720045070 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f571f7fe640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 0x7f571803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f572006ccd0 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f572dd9b640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 0x7f571803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573010be80 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.439+0000 7f572dd9b640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 0x7f571803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57240098c0 tx=0x7f5724006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.444+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f572004b170 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:37.665+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f573019f150 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.423+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f572006ca90 con 0x7f57301077a0 2024-09-12T00:46:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.425+0000 7f571f7fe640 1 -- 172.21.15.16:0/1919940552 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5720042d00 con 0x7f57301077a0 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.426+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 msgr2=0x7f571803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.426+0000 7f5734885640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 0x7f571803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f57240098c0 tx=0x7f5724006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 msgr2=0x7f573019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5720009940 tx=0x7f5720032ce0 comp rx=0 tx=0).stop 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 shutdown_connections 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f5718038f20 0x7f571803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 --2- 172.21.15.16:0/1919940552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57301077a0 0x7f573019b740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 >> 172.21.15.16:0/1919940552 conn(0x7f5730102fd0 msgr2=0x7f5730103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 shutdown_connections 2024-09-12T00:46:39.006 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:38.427+0000 7f5734885640 1 -- 172.21.15.16:0/1919940552 wait complete. 2024-09-12T00:46:39.265 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:39.017+0000 7f261f22f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:46:39.265 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:39.167+0000 7f261f22f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:46:39.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1919940552' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T00:46:39.568 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-mon[30972]: mgrmap e5: x(active, since 5s) 2024-09-12T00:46:39.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:39.505+0000 7f261f22f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:46:39.569 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:39.570+0000 7f261f22f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:46:39.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:39.766+0000 7f261f22f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.453+0000 7f9c9db7d640 1 Processor -- start 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.453+0000 7f9c9db7d640 1 -- start start 2024-09-12T00:46:40.243 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c9db7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c9db7d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9807bdd0 con 0x7f9c98108210 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38656/0 (socket says 172.21.15.16:38656) 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c977fe640 1 -- 172.21.15.16:0/1886692423 learned_addr learned my addr 172.21.15.16:0/1886692423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.454+0000 7f9c977fe640 1 -- 172.21.15.16:0/1886692423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9810ab40 con 0x7f9c98108210 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.455+0000 7f9c977fe640 1 --2- 172.21.15.16:0/1886692423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009820 tx=0x7f9c8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa7c9d750cb64da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.455+0000 7f9c967fc640 1 -- 172.21.15.16:0/1886692423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c88037070 con 0x7f9c98108210 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.455+0000 7f9c967fc640 1 -- 172.21.15.16:0/1886692423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c88032900 con 0x7f9c98108210 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.455+0000 7f9c967fc640 1 -- 172.21.15.16:0/1886692423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c88030850 con 0x7f9c98108210 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1886692423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 msgr2=0x7f9c9810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1886692423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009820 tx=0x7f9c8802a270 comp rx=0 tx=0).stop 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1886692423 shutdown_connections 2024-09-12T00:46:40.244 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1886692423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9810a600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1886692423 >> 172.21.15.16:0/1886692423 conn(0x7f9c98101db0 msgr2=0x7f9c98104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1886692423 shutdown_connections 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1886692423 wait complete. 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.456+0000 7f9c9db7d640 1 Processor -- start 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c9db7d640 1 -- start start 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c9db7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c9db7d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9810bb20 con 0x7f9c98108210 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38666/0 (socket says 172.21.15.16:38666) 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c977fe640 1 -- 172.21.15.16:0/1830052018 learned_addr learned my addr 172.21.15.16:0/1830052018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c977fe640 1 -- 172.21.15.16:0/1830052018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9819bc90 con 0x7f9c98108210 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.457+0000 7f9c977fe640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9c880097f0 tx=0x7f9c88030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.458+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c88042070 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.458+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c88032ca0 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.458+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9819bf20 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.458+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c88037040 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.458+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9819ec10 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f9c88045070 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98108670 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c94ff9640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 0x7f9c8003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c8806c130 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c96ffd640 1 -- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 msgr2=0x7f9c8003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.459+0000 7f9c96ffd640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 0x7f9c8003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.468+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c88045520 con 0x7f9c98108210 2024-09-12T00:46:40.246 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.660+0000 7f9c96ffd640 1 -- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 msgr2=0x7f9c8003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.660+0000 7f9c96ffd640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 0x7f9c8003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.692+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9c9819f290 con 0x7f9c98108210 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.692+0000 7f9c94ff9640 1 -- 172.21.15.16:0/1830052018 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f9c8803cd10 con 0x7f9c98108210 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 msgr2=0x7f9c8003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 0x7f9c8003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 msgr2=0x7f9c9819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9c880097f0 tx=0x7f9c88030ec0 comp rx=0 tx=0).stop 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 shutdown_connections 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f9c80038f70 0x7f9c8003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 --2- 172.21.15.16:0/1830052018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c98108210 0x7f9c9819b750 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 >> 172.21.15.16:0/1830052018 conn(0x7f9c98101db0 msgr2=0x7f9c98102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 shutdown_connections 2024-09-12T00:46:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:39.695+0000 7f9c9db7d640 1 -- 172.21.15.16:0/1830052018 wait complete. 2024-09-12T00:46:40.248 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-12T00:46:40.248 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-12T00:46:40.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1830052018' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T00:46:41.143 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:40.703+0000 7f261f22f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:46:41.143 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:40.774+0000 7f261f22f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:46:41.786 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:41.518+0000 7f261f22f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:46:41.787 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:41.639+0000 7f261f22f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:46:42.052 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:41.788+0000 7f261f22f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:46:42.052 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:41.851+0000 7f261f22f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:46:42.052 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.008+0000 7f261f22f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:46:42.306 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.079+0000 7f261f22f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:46:42.306 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.141+0000 7f261f22f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:46:42.306 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.307+0000 7f261f22f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:46:42.591 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.375+0000 7f261f22f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:46:42.591 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.443+0000 7f261f22f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:46:42.591 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.530+0000 7f261f22f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:46:42.592 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.592+0000 7f261f22f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:46:42.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:42.653+0000 7f261f22f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:43.481+0000 7f261f22f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: Active manager daemon x restarted 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: Activating manager daemon x 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: mgrmap e6: x(active, starting, since 0.00355846s) 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:46:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: Manager daemon x is now available 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:46:43.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:43 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.644+0000 7f002cfaa640 1 Processor -- start 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.644+0000 7f002cfaa640 1 -- start start 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002cfaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002cfaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0028108130 con 0x7f0028107760 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:45.234 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38670/0 (socket says 172.21.15.16:38670) 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002659c640 1 -- 172.21.15.16:0/378329384 learned_addr learned my addr 172.21.15.16:0/378329384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.645+0000 7f002659c640 1 -- 172.21.15.16:0/378329384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0028108910 con 0x7f0028107760 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.646+0000 7f002659c640 1 --2- 172.21.15.16:0/378329384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0018009820 tx=0x7f001802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4ff490ad4552d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.646+0000 7f0025d9b640 1 -- 172.21.15.16:0/378329384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0018037070 con 0x7f0028107760 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.646+0000 7f0025d9b640 1 -- 172.21.15.16:0/378329384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0018032900 con 0x7f0028107760 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.646+0000 7f0025d9b640 1 -- 172.21.15.16:0/378329384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0018030850 con 0x7f0028107760 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 -- 172.21.15.16:0/378329384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 msgr2=0x7f0028107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 --2- 172.21.15.16:0/378329384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0018009820 tx=0x7f001802a270 comp rx=0 tx=0).stop 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 -- 172.21.15.16:0/378329384 shutdown_connections 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 --2- 172.21.15.16:0/378329384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f0028107b60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 -- 172.21.15.16:0/378329384 >> 172.21.15.16:0/378329384 conn(0x7f0028102f30 msgr2=0x7f0028105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 -- 172.21.15.16:0/378329384 shutdown_connections 2024-09-12T00:46:45.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.647+0000 7f002cfaa640 1 -- 172.21.15.16:0/378329384 wait complete. 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002cfaa640 1 Processor -- start 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002cfaa640 1 -- start start 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002cfaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002cfaa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f002810bcc0 con 0x7f0028107760 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38686/0 (socket says 172.21.15.16:38686) 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.648+0000 7f002659c640 1 -- 172.21.15.16:0/628612601 learned_addr learned my addr 172.21.15.16:0/628612601 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f002659c640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002819bcc0 con 0x7f0028107760 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f002659c640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0018009940 tx=0x7f00180326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0018042070 con 0x7f0028107760 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0018032be0 con 0x7f0028107760 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0018037040 con 0x7f0028107760 2024-09-12T00:46:45.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002819bf50 con 0x7f0028107760 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.649+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002819ec40 con 0x7f0028107760 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f0018045070 con 0x7f0028107760 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001f7fe640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f001403bb90 con 0x7f0014038fc0 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f001806bed0 con 0x7f0028107760 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001ffff640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 msgr2=0x7f001403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.651+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.852+0000 7f001ffff640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 msgr2=0x7f001403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:40.852+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:41.253+0000 7f001ffff640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 msgr2=0x7f001403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:41.253+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:42.054+0000 7f001ffff640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 msgr2=0x7f001403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/7504959 2024-09-12T00:46:45.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:42.054+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:43.484+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f001806a990 con 0x7f0028107760 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:43.484+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 msgr2=0x7f001403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:43.484+0000 7f001f7fe640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.485+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f0018039d80 con 0x7f0028107760 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.485+0000 7f001f7fe640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 0x7f001403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.485+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0018001d30 con 0x7f001403c8f0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.486+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 0x7f001403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.487+0000 7f001ffff640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 0x7f001403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0010003980 tx=0x7f00100092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.488+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f0018001d30 con 0x7f001403c8f0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.500+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f002810be40 con 0x7f001403c8f0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.500+0000 7f001f7fe640 1 -- 172.21.15.16:0/628612601 <== mgr.14118 v2:172.21.15.16:6800/1538480573 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f002810be40 con 0x7f001403c8f0 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 msgr2=0x7f001403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:45.238 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 0x7f001403ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0010003980 tx=0x7f00100092b0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 msgr2=0x7f002819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0018009940 tx=0x7f00180326e0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 shutdown_connections 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f001403c8f0 0x7f001403ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:6800/7504959,v1:172.21.15.16:6801/7504959] conn(0x7f0014038fc0 0x7f001403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 --2- 172.21.15.16:0/628612601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0028107760 0x7f002819b780 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 >> 172.21.15.16:0/628612601 conn(0x7f0028102f30 msgr2=0x7f0028103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 shutdown_connections 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:44.501+0000 7f002cfaa640 1 -- 172.21.15.16:0/628612601 wait complete. 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-12T00:46:45.239 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-12T00:46:45.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-mon[30972]: Found migration_current of "None". Setting to last migration. 2024-09-12T00:46:45.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:45.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:45.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-mon[30972]: mgrmap e7: x(active, since 1.00552s) 2024-09-12T00:46:45.636 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:45.455+0000 7f258bd94640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:00:46:45] ENGINE Error in HTTPServer.serve 2024-09-12T00:46:45.636 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Traceback (most recent call last): 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._connections.run(self.expiration_interval) 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._run(expiration_interval) 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s = self.context.wrap_socket( 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: return self.sslsocket_class._create( 2024-09-12T00:46:45.637 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:46:45.638 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self.do_handshake() 2024-09-12T00:46:45.638 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:46:45.638 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._sslobj.do_handshake() 2024-09-12T00:46:45.638 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:46:45.638 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: [12/Sep/2024:00:46:45] ENGINE Bus STARTING 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: [12/Sep/2024:00:46:45] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: [12/Sep/2024:00:46:45] ENGINE Error in HTTPServer.serve 2024-09-12T00:46:46.393 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-12T00:46:46.394 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:46.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:46 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.643+0000 7fad539eb640 1 Processor -- start 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.643+0000 7fad539eb640 1 -- start start 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.643+0000 7fad539eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.643+0000 7fad539eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c108160 con 0x7fad4c107790 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.643+0000 7fad51787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.644+0000 7fad51787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38766/0 (socket says 172.21.15.16:38766) 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.644+0000 7fad51787640 1 -- 172.21.15.16:0/2448217819 learned_addr learned my addr 172.21.15.16:0/2448217819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.644+0000 7fad51787640 1 -- 172.21.15.16:0/2448217819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c108940 con 0x7fad4c107790 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.644+0000 7fad51787640 1 --2- 172.21.15.16:0/2448217819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009820 tx=0x7fad3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038e237ac351304 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:46.427 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad4bfff640 1 -- 172.21.15.16:0/2448217819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c037070 con 0x7fad4c107790 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad4bfff640 1 -- 172.21.15.16:0/2448217819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad3c032900 con 0x7fad4c107790 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad4bfff640 1 -- 172.21.15.16:0/2448217819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c030850 con 0x7fad4c107790 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 -- 172.21.15.16:0/2448217819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 msgr2=0x7fad4c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 --2- 172.21.15.16:0/2448217819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009820 tx=0x7fad3c02a270 comp rx=0 tx=0).stop 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 -- 172.21.15.16:0/2448217819 shutdown_connections 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 --2- 172.21.15.16:0/2448217819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c107b90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 -- 172.21.15.16:0/2448217819 >> 172.21.15.16:0/2448217819 conn(0x7fad4c102fa0 msgr2=0x7fad4c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 -- 172.21.15.16:0/2448217819 shutdown_connections 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.645+0000 7fad539eb640 1 -- 172.21.15.16:0/2448217819 wait complete. 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad539eb640 1 Processor -- start 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad539eb640 1 -- start start 2024-09-12T00:46:46.428 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad539eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad539eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10bcf0 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad51787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad51787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38776/0 (socket says 172.21.15.16:38776) 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.646+0000 7fad51787640 1 -- 172.21.15.16:0/2996780380 learned_addr learned my addr 172.21.15.16:0/2996780380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad51787640 1 -- 172.21.15.16:0/2996780380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c19deb0 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad51787640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009940 tx=0x7fad3c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c042070 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad3c0308c0 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c19e140 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad3c037040 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.647+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c1a0e30 con 0x7fad4c107790 2024-09-12T00:46:46.429 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.648+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fad3c045070 con 0x7fad4c107790 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.648+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c10be70 con 0x7fad4c107790 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.648+0000 7fad4a7fc640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 0x7fad3403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.648+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad3c06be80 con 0x7fad4c107790 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.648+0000 7fad50f86640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 0x7fad3403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.649+0000 7fad50f86640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 0x7fad3403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad400098c0 tx=0x7fad40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.653+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad3c03cd90 con 0x7fad4c107790 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.847+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fad4c1041a0 con 0x7fad34038e60 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.852+0000 7fad4a7fc640 1 -- 172.21.15.16:0/2996780380 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad4c1041a0 con 0x7fad34038e60 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 msgr2=0x7fad3403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 0x7fad3403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad400098c0 tx=0x7fad40006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 msgr2=0x7fad4c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:46.430 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009940 tx=0x7fad3c032ce0 comp rx=0 tx=0).stop 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 shutdown_connections 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fad34038e60 0x7fad3403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 --2- 172.21.15.16:0/2996780380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad4c107790 0x7fad4c19d970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 >> 172.21.15.16:0/2996780380 conn(0x7fad4c102fa0 msgr2=0x7fad4c103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 shutdown_connections 2024-09-12T00:46:46.431 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:45.854+0000 7fad539eb640 1 -- 172.21.15.16:0/2996780380 wait complete. 2024-09-12T00:46:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:47 smithi016 ceph-mon[30972]: [12/Sep/2024:00:46:45] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T00:46:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:47 smithi016 ceph-mon[30972]: [12/Sep/2024:00:46:45] ENGINE Bus STARTED 2024-09-12T00:46:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:47 smithi016 ceph-mon[30972]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:47 smithi016 ceph-mon[30972]: mgrmap e8: x(active, since 2s) 2024-09-12T00:46:47.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.848+0000 7f9f7b30a640 1 Processor -- start 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.848+0000 7f9f7b30a640 1 -- start start 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f7b30a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f7b30a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74107070 con 0x7f9f741066a0 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f790a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f790a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38788/0 (socket says 172.21.15.16:38788) 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f790a6640 1 -- 172.21.15.16:0/4234738169 learned_addr learned my addr 172.21.15.16:0/4234738169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:47.696 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.849+0000 7f9f790a6640 1 -- 172.21.15.16:0/4234738169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74107800 con 0x7f9f741066a0 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.850+0000 7f9f790a6640 1 --2- 172.21.15.16:0/4234738169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bac0 tx=0x7f9f7002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb4191054e4058c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.850+0000 7f9f6bfff640 1 -- 172.21.15.16:0/4234738169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f700326d0 con 0x7f9f741066a0 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.850+0000 7f9f6bfff640 1 -- 172.21.15.16:0/4234738169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f70038070 con 0x7f9f741066a0 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.850+0000 7f9f6bfff640 1 -- 172.21.15.16:0/4234738169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70033d70 con 0x7f9f741066a0 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 -- 172.21.15.16:0/4234738169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 msgr2=0x7f9f74106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/4234738169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000bac0 tx=0x7f9f7002c920 comp rx=0 tx=0).stop 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 -- 172.21.15.16:0/4234738169 shutdown_connections 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/4234738169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f74106aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 -- 172.21.15.16:0/4234738169 >> 172.21.15.16:0/4234738169 conn(0x7f9f74101e70 msgr2=0x7f9f74104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 -- 172.21.15.16:0/4234738169 shutdown_connections 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.851+0000 7f9f7b30a640 1 -- 172.21.15.16:0/4234738169 wait complete. 2024-09-12T00:46:47.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f7b30a640 1 Processor -- start 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f7b30a640 1 -- start start 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f7b30a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f7b30a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7407b940 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f790a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f790a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38798/0 (socket says 172.21.15.16:38798) 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f790a6640 1 -- 172.21.15.16:0/3385355034 learned_addr learned my addr 172.21.15.16:0/3385355034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.852+0000 7f9f790a6640 1 -- 172.21.15.16:0/3385355034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f74199a20 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f790a6640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000b2c0 tx=0x7f9f70032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f70032c40 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f74199cb0 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f7003c610 con 0x7f9f741066a0 2024-09-12T00:46:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.853+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7419c9a0 con 0x7f9f741066a0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.854+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9f7003c7b0 con 0x7f9f741066a0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.854+0000 7f9f6a7fc640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 0x7f9f6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.854+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f70030050 con 0x7f9f741066a0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.854+0000 7f9f788a5640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 0x7f9f6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.854+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f74106aa0 con 0x7f9f741066a0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.855+0000 7f9f788a5640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 0x7f9f6003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f640098c0 tx=0x7f9f64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:46.859+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f70054080 con 0x7f9f741066a0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.055+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9f74191240 con 0x7f9f60038fc0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.055+0000 7f9f6a7fc640 1 -- 172.21.15.16:0/3385355034 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9f74191240 con 0x7f9f60038fc0 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 msgr2=0x7f9f6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 0x7f9f6003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f640098c0 tx=0x7f9f64006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 msgr2=0x7f9f741994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000b2c0 tx=0x7f9f70032820 comp rx=0 tx=0).stop 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 shutdown_connections 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f9f60038fc0 0x7f9f6003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 --2- 172.21.15.16:0/3385355034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f741066a0 0x7f9f741994e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 >> 172.21.15.16:0/3385355034 conn(0x7f9f74101e70 msgr2=0x7f9f7418faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 shutdown_connections 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:47.058+0000 7f9f7b30a640 1 -- 172.21.15.16:0/3385355034 wait complete. 2024-09-12T00:46:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-12T00:46:48.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-mon[30972]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Generating public/private rsa key pair. 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Your identification has been saved in /tmp/tmp1bvvimya/key 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Your public key has been saved in /tmp/tmp1bvvimya/key.pub 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: The key fingerprint is: 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: SHA256:Glpx0LJTpNnbTyL6jXWso0ObxJp6pG1R37smntE6RFc ceph-434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:48.644 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: The key's randomart image is: 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: +---[RSA 3072]----+ 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | .o. | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | .=o E | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | ++o . | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | oo.o. . | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | o+Sooo. | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | oo++.o*. | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | .+o* +o =. | 2024-09-12T00:46:48.645 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | . *.+++*o | 2024-09-12T00:46:48.646 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: | .+ ++*=.. | 2024-09-12T00:46:48.646 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:46:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: +----[SHA256]-----+ 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.115+0000 7fd11b84f640 1 Processor -- start 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.115+0000 7fd11b84f640 1 -- start start 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.115+0000 7fd11b84f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.116+0000 7fd11b84f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd114108170 con 0x7fd1141077a0 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.116+0000 7fd1195eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.116+0000 7fd1195eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38804/0 (socket says 172.21.15.16:38804) 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.116+0000 7fd1195eb640 1 -- 172.21.15.16:0/1337176371 learned_addr learned my addr 172.21.15.16:0/1337176371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.116+0000 7fd1195eb640 1 -- 172.21.15.16:0/1337176371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd114108950 con 0x7fd1141077a0 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd1195eb640 1 --2- 172.21.15.16:0/1337176371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009820 tx=0x7fd10c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0bcba5649f4f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd113fff640 1 -- 172.21.15.16:0/1337176371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd10c037070 con 0x7fd1141077a0 2024-09-12T00:46:49.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd113fff640 1 -- 172.21.15.16:0/1337176371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd10c032900 con 0x7fd1141077a0 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd113fff640 1 -- 172.21.15.16:0/1337176371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd10c030850 con 0x7fd1141077a0 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd11b84f640 1 -- 172.21.15.16:0/1337176371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 msgr2=0x7fd114107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.117+0000 7fd11b84f640 1 --2- 172.21.15.16:0/1337176371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009820 tx=0x7fd10c02a270 comp rx=0 tx=0).stop 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- 172.21.15.16:0/1337176371 shutdown_connections 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 --2- 172.21.15.16:0/1337176371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd114107ba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- 172.21.15.16:0/1337176371 >> 172.21.15.16:0/1337176371 conn(0x7fd114102fd0 msgr2=0x7fd1141053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- 172.21.15.16:0/1337176371 shutdown_connections 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- 172.21.15.16:0/1337176371 wait complete. 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 Processor -- start 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- start start 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.118+0000 7fd11b84f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd11410bd00 con 0x7fd1141077a0 2024-09-12T00:46:49.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1195eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1195eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38806/0 (socket says 172.21.15.16:38806) 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1195eb640 1 -- 172.21.15.16:0/3952642051 learned_addr learned my addr 172.21.15.16:0/3952642051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1195eb640 1 -- 172.21.15.16:0/3952642051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11419bc80 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1195eb640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009940 tx=0x7fd10c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd10c042070 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd10c0308c0 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11419bf10 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd10c037040 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.119+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11419ec00 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.120+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd10c045070 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.120+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11410be80 con 0x7fd1141077a0 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.120+0000 7fd1127fc640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 0x7fd0fc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:49.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.120+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd10c06c030 con 0x7fd1141077a0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.120+0000 7fd118dea640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 0x7fd0fc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.121+0000 7fd118dea640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 0x7fd0fc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1080098c0 tx=0x7fd108006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.125+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd10c045520 con 0x7fd1141077a0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.305+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd114104240 con 0x7fd0fc038fc0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.506+0000 7fd1127fc640 1 -- 172.21.15.16:0/3952642051 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd114104240 con 0x7fd0fc038fc0 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 msgr2=0x7fd0fc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 0x7fd0fc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1080098c0 tx=0x7fd108006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 msgr2=0x7fd11419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009940 tx=0x7fd10c032ce0 comp rx=0 tx=0).stop 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 shutdown_connections 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fd0fc038fc0 0x7fd0fc03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 --2- 172.21.15.16:0/3952642051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1141077a0 0x7fd11419b740 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:49.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 >> 172.21.15.16:0/3952642051 conn(0x7fd114102fd0 msgr2=0x7fd114103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:49.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 shutdown_connections 2024-09-12T00:46:49.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:48.508+0000 7fd11b84f640 1 -- 172.21.15.16:0/3952642051 wait complete. 2024-09-12T00:46:49.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:49 smithi016 ceph-mon[30972]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:49.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:49 smithi016 ceph-mon[30972]: Generating ssh key... 2024-09-12T00:46:49.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:49 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:49.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:49 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3456e4c640 1 Processor -- start 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3456e4c640 1 -- start start 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3456e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3456e4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34501060d0 con 0x7f3450105700 2024-09-12T00:46:50.193 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3454be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3454be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59982/0 (socket says 172.21.15.16:59982) 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.386+0000 7f3454be8640 1 -- 172.21.15.16:0/723740562 learned_addr learned my addr 172.21.15.16:0/723740562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.387+0000 7f3454be8640 1 -- 172.21.15.16:0/723740562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450106780 con 0x7f3450105700 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.387+0000 7f3454be8640 1 --2- 172.21.15.16:0/723740562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f344c009820 tx=0x7f344c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87d0e0652562bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.387+0000 7f34477fe640 1 -- 172.21.15.16:0/723740562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c037070 con 0x7f3450105700 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.387+0000 7f34477fe640 1 -- 172.21.15.16:0/723740562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f344c032900 con 0x7f3450105700 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.387+0000 7f34477fe640 1 -- 172.21.15.16:0/723740562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c030850 con 0x7f3450105700 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 -- 172.21.15.16:0/723740562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 msgr2=0x7f3450105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 --2- 172.21.15.16:0/723740562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f344c009820 tx=0x7f344c02a270 comp rx=0 tx=0).stop 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 -- 172.21.15.16:0/723740562 shutdown_connections 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 --2- 172.21.15.16:0/723740562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f3450105b00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:50.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 -- 172.21.15.16:0/723740562 >> 172.21.15.16:0/723740562 conn(0x7f3450101180 msgr2=0x7f34501035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 -- 172.21.15.16:0/723740562 shutdown_connections 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.388+0000 7f3456e4c640 1 -- 172.21.15.16:0/723740562 wait complete. 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3456e4c640 1 Processor -- start 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3456e4c640 1 -- start start 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3456e4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3456e4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3450107b40 con 0x7f3450105700 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3454be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3454be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59994/0 (socket says 172.21.15.16:59994) 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3454be8640 1 -- 172.21.15.16:0/4278846 learned_addr learned my addr 172.21.15.16:0/4278846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.389+0000 7f3454be8640 1 -- 172.21.15.16:0/4278846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345019de80 con 0x7f3450105700 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3454be8640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f344c032ce0 tx=0x7f344c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c042070 con 0x7f3450105700 2024-09-12T00:46:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345019e110 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f344c0308c0 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f344c037040 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.390+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34501a0e00 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.391+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f344c045070 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.391+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3450105b00 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.391+0000 7f3445ffb640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 0x7f343803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.391+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f344c06c2c0 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.391+0000 7f3447fff640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 0x7f343803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.392+0000 7f3447fff640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 0x7f343803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34400098c0 tx=0x7f3440006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.396+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f344c03cd00 con 0x7f3450105700 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.572+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f345002dab0 con 0x7f3438038f70 2024-09-12T00:46:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.573+0000 7f3445ffb640 1 -- 172.21.15.16:0/4278846 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f345002dab0 con 0x7f3438038f70 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.574+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 msgr2=0x7f343803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.574+0000 7f3456e4c640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 0x7f343803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f34400098c0 tx=0x7f3440006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 msgr2=0x7f345019d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f344c032ce0 tx=0x7f344c032d10 comp rx=0 tx=0).stop 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 shutdown_connections 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f3438038f70 0x7f343803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 --2- 172.21.15.16:0/4278846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3450105700 0x7f345019d940 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 >> 172.21.15.16:0/4278846 conn(0x7f3450101180 msgr2=0x7f3450194050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 shutdown_connections 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:49.575+0000 7f3456e4c640 1 -- 172.21.15.16:0/4278846 wait complete. 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T00:46:50.197 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T00:46:50.198 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-12T00:46:50.781 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:50 smithi016 ceph-mon[30972]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:51 smithi016 ceph-mon[30972]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:52.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:52 smithi016 ceph-mon[30972]: Deploying cephadm binary to smithi016 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.592+0000 7f58c35ac640 1 Processor -- start 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c35ac640 1 -- start start 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c35ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c35ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc108170 con 0x7f58bc1077a0 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c1348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c1348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60010/0 (socket says 172.21.15.16:60010) 2024-09-12T00:46:55.252 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.593+0000 7f58c1348640 1 -- 172.21.15.16:0/1440590341 learned_addr learned my addr 172.21.15.16:0/1440590341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.594+0000 7f58c1348640 1 -- 172.21.15.16:0/1440590341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc108950 con 0x7f58bc1077a0 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.594+0000 7f58c1348640 1 --2- 172.21.15.16:0/1440590341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009820 tx=0x7f58b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8890b09a04343224 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.594+0000 7f58bbfff640 1 -- 172.21.15.16:0/1440590341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58b4037070 con 0x7f58bc1077a0 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58bbfff640 1 -- 172.21.15.16:0/1440590341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58b4032900 con 0x7f58bc1077a0 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58bbfff640 1 -- 172.21.15.16:0/1440590341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58b4030850 con 0x7f58bc1077a0 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 -- 172.21.15.16:0/1440590341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 msgr2=0x7f58bc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 --2- 172.21.15.16:0/1440590341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009820 tx=0x7f58b402a270 comp rx=0 tx=0).stop 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 -- 172.21.15.16:0/1440590341 shutdown_connections 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 --2- 172.21.15.16:0/1440590341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc107ba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 -- 172.21.15.16:0/1440590341 >> 172.21.15.16:0/1440590341 conn(0x7f58bc102fd0 msgr2=0x7f58bc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 -- 172.21.15.16:0/1440590341 shutdown_connections 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.595+0000 7f58c35ac640 1 -- 172.21.15.16:0/1440590341 wait complete. 2024-09-12T00:46:55.253 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c35ac640 1 Processor -- start 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c35ac640 1 -- start start 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c35ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c35ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10bd00 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c1348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c1348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60012/0 (socket says 172.21.15.16:60012) 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c1348640 1 -- 172.21.15.16:0/2017536401 learned_addr learned my addr 172.21.15.16:0/2017536401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.596+0000 7f58c1348640 1 -- 172.21.15.16:0/2017536401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc07e910 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58c1348640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009940 tx=0x7f58b4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58b4042070 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58b40308c0 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc07cb00 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58b4037040 con 0x7f58bc1077a0 2024-09-12T00:46:55.254 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.597+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc07d040 con 0x7f58bc1077a0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f58b4045070 con 0x7f58bc1077a0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58bc107c20 con 0x7f58bc1077a0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58ba7fc640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 0x7f58a403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58b406c140 con 0x7f58bc1077a0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58c0b47640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 0x7f58a403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.598+0000 7f58c0b47640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 0x7f58a403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f58b00098c0 tx=0x7f58b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.603+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58b4045360 con 0x7f58bc1077a0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:50.782+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7f58bc104240 con 0x7f58a4038fc0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.711+0000 7f58ba7fc640 1 -- 172.21.15.16:0/2017536401 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f58bc104240 con 0x7f58a4038fc0 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 msgr2=0x7f58a403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 0x7f58a403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f58b00098c0 tx=0x7f58b0006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 msgr2=0x7f58bc07e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:55.255 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009940 tx=0x7f58b4032ce0 comp rx=0 tx=0).stop 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 shutdown_connections 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f58a4038fc0 0x7f58a403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 --2- 172.21.15.16:0/2017536401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58bc1077a0 0x7f58bc07e3d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.713+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 >> 172.21.15.16:0/2017536401 conn(0x7f58bc102fd0 msgr2=0x7f58bc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.714+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 shutdown_connections 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:54.714+0000 7f58c35ac640 1 -- 172.21.15.16:0/2017536401 wait complete. 2024-09-12T00:46:55.256 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-12T00:46:56.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:55 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:56.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:55 smithi016 ceph-mon[30972]: Added host smithi016 2024-09-12T00:46:56.070 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:55 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:46:57.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:56 smithi016 ceph-mon[30972]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:57.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:56 smithi016 ceph-mon[30972]: Saving service mon spec with placement count:5 2024-09-12T00:46:57.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:56 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T00:46:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.671+0000 7fcbfe929640 1 Processor -- start 2024-09-12T00:46:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.671+0000 7fcbfe929640 1 -- start start 2024-09-12T00:46:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbfe929640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:57.527 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbfe929640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf8108130 con 0x7fcbf8107760 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60024/0 (socket says 172.21.15.16:60024) 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2233078712 learned_addr learned my addr 172.21.15.16:0/2233078712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.672+0000 7fcbf7fff640 1 -- 172.21.15.16:0/2233078712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf8108910 con 0x7fcbf8107760 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.673+0000 7fcbf7fff640 1 --2- 172.21.15.16:0/2233078712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009820 tx=0x7fcbe802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d72b34154e61b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.673+0000 7fcbf77fe640 1 -- 172.21.15.16:0/2233078712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe8037070 con 0x7fcbf8107760 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.673+0000 7fcbf77fe640 1 -- 172.21.15.16:0/2233078712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcbe8032900 con 0x7fcbf8107760 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.673+0000 7fcbf77fe640 1 -- 172.21.15.16:0/2233078712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe8030850 con 0x7fcbf8107760 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- 172.21.15.16:0/2233078712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 msgr2=0x7fcbf8107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 --2- 172.21.15.16:0/2233078712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009820 tx=0x7fcbe802a270 comp rx=0 tx=0).stop 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- 172.21.15.16:0/2233078712 shutdown_connections 2024-09-12T00:46:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 --2- 172.21.15.16:0/2233078712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf8107b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- 172.21.15.16:0/2233078712 >> 172.21.15.16:0/2233078712 conn(0x7fcbf8102f30 msgr2=0x7fcbf8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- 172.21.15.16:0/2233078712 shutdown_connections 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- 172.21.15.16:0/2233078712 wait complete. 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 Processor -- start 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.674+0000 7fcbfe929640 1 -- start start 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbfe929640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbfe929640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf810bcc0 con 0x7fcbf8107760 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60030/0 (socket says 172.21.15.16:60030) 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf7fff640 1 -- 172.21.15.16:0/191459556 learned_addr learned my addr 172.21.15.16:0/191459556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf7fff640 1 -- 172.21.15.16:0/191459556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf807e8a0 con 0x7fcbf8107760 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf7fff640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80097f0 tx=0x7fcbe8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe8037040 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcbe8040070 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf807ca90 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbe803a390 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.675+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf807cf70 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.676+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcbe8032900 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.676+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf810be40 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.676+0000 7fcbf5ffb640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 0x7fcbe403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.676+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbe806c460 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.677+0000 7fcbeffff640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 0x7fcbe403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.677+0000 7fcbeffff640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 0x7fcbe403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098c0 tx=0x7fcbe0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.681+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbe8042310 con 0x7fcbf8107760 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.872+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcbf807d820 con 0x7fcbe4038f70 2024-09-12T00:46:57.530 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.877+0000 7fcbf5ffb640 1 -- 172.21.15.16:0/191459556 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcbf807d820 con 0x7fcbe4038f70 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 msgr2=0x7fcbe403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 0x7fcbe403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098c0 tx=0x7fcbe0006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 msgr2=0x7fcbf807e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80097f0 tx=0x7fcbe8030d80 comp rx=0 tx=0).stop 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 shutdown_connections 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fcbe4038f70 0x7fcbe403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 --2- 172.21.15.16:0/191459556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbf8107760 0x7fcbf807e360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 >> 172.21.15.16:0/191459556 conn(0x7fcbf8102f30 msgr2=0x7fcbf8103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 shutdown_connections 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:55.879+0000 7fcbfe929640 1 -- 172.21.15.16:0/191459556 wait complete. 2024-09-12T00:46:57.531 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-12T00:46:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:58 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:58 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:58 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f1445eb5640 1 Processor -- start 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f1445eb5640 1 -- start start 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f1445eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f1445eb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f144007d150 con 0x7f144007e6d0 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60044/0 (socket says 172.21.15.16:60044) 2024-09-12T00:46:59.286 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.887+0000 7f143f7fe640 1 -- 172.21.15.16:0/3440737048 learned_addr learned my addr 172.21.15.16:0/3440737048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.888+0000 7f143f7fe640 1 -- 172.21.15.16:0/3440737048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144007d2d0 con 0x7f144007e6d0 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.888+0000 7f143f7fe640 1 --2- 172.21.15.16:0/3440737048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1430009820 tx=0x7f143002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1170b3a7b77406 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f143effd640 1 -- 172.21.15.16:0/3440737048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1430037070 con 0x7f144007e6d0 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f143effd640 1 -- 172.21.15.16:0/3440737048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1430032920 con 0x7f144007e6d0 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f143effd640 1 -- 172.21.15.16:0/3440737048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1430030810 con 0x7f144007e6d0 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 -- 172.21.15.16:0/3440737048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 msgr2=0x7f144007cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 --2- 172.21.15.16:0/3440737048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1430009820 tx=0x7f143002a270 comp rx=0 tx=0).stop 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 -- 172.21.15.16:0/3440737048 shutdown_connections 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 --2- 172.21.15.16:0/3440737048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144007cb80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 -- 172.21.15.16:0/3440737048 >> 172.21.15.16:0/3440737048 conn(0x7f1440102fd0 msgr2=0x7f14401053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 -- 172.21.15.16:0/3440737048 shutdown_connections 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.889+0000 7f1445eb5640 1 -- 172.21.15.16:0/3440737048 wait complete. 2024-09-12T00:46:59.287 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f1445eb5640 1 Processor -- start 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f1445eb5640 1 -- start start 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f1445eb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f1445eb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f144010ddf0 con 0x7f144007e6d0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f143f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60046/0 (socket says 172.21.15.16:60046) 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.890+0000 7f143f7fe640 1 -- 172.21.15.16:0/1021368007 learned_addr learned my addr 172.21.15.16:0/1021368007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f143f7fe640 1 -- 172.21.15.16:0/1021368007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144019de70 con 0x7f144007e6d0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f143f7fe640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1430030380 tx=0x7f1430032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1430037040 con 0x7f144007e6d0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1430040070 con 0x7f144007e6d0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144019e100 con 0x7f144007e6d0 2024-09-12T00:46:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f143003a7b0 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.891+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a0df0 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.892+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f143003b420 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.892+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144007cc00 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.892+0000 7f143d7fa640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 0x7f142c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.892+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f143006c6c0 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.892+0000 7f1437fff640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 0x7f142c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.893+0000 7f1437fff640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 0x7f142c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f14280098c0 tx=0x7f1428006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:57.897+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f143004a200 con 0x7f144007e6d0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.149+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1440104230 con 0x7f142c038fc0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.153+0000 7f143d7fa640 1 -- 172.21.15.16:0/1021368007 <== mgr.14118 v2:172.21.15.16:6800/1538480573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1440104230 con 0x7f142c038fc0 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 msgr2=0x7f142c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 0x7f142c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f14280098c0 tx=0x7f1428006db0 comp rx=0 tx=0).stop 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 msgr2=0x7f144019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:46:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1430030380 tx=0x7f1430032670 comp rx=0 tx=0).stop 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 shutdown_connections 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f142c038fc0 0x7f142c03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 --2- 172.21.15.16:0/1021368007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f144007e6d0 0x7f144019d930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 >> 172.21.15.16:0/1021368007 conn(0x7f1440102fd0 msgr2=0x7f1440103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 shutdown_connections 2024-09-12T00:46:59.290 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:58.155+0000 7f1445eb5640 1 -- 172.21.15.16:0/1021368007 wait complete. 2024-09-12T00:46:59.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:59 smithi016 ceph-mon[30972]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:46:59.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:59 smithi016 ceph-mon[30972]: Saving service mgr spec with placement count:2 2024-09-12T00:46:59.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:46:59 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:47:00.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.710+0000 7f85f8483640 1 Processor -- start 2024-09-12T00:47:00.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f8483640 1 -- start start 2024-09-12T00:47:00.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f8483640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:00.468 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f8483640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85f007d010 con 0x7f85f007e620 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39796/0 (socket says 172.21.15.16:39796) 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.711+0000 7f85f621f640 1 -- 172.21.15.16:0/3942486520 learned_addr learned my addr 172.21.15.16:0/3942486520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.712+0000 7f85f621f640 1 -- 172.21.15.16:0/3942486520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f007d190 con 0x7f85f007e620 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.712+0000 7f85f621f640 1 --2- 172.21.15.16:0/3942486520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009820 tx=0x7f85ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda7f2f103e10084 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.712+0000 7f85f521d640 1 -- 172.21.15.16:0/3942486520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ec037070 con 0x7f85f007e620 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.712+0000 7f85f521d640 1 -- 172.21.15.16:0/3942486520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec032920 con 0x7f85f007e620 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f521d640 1 -- 172.21.15.16:0/3942486520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ec030810 con 0x7f85f007e620 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 -- 172.21.15.16:0/3942486520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 msgr2=0x7f85f007cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 --2- 172.21.15.16:0/3942486520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009820 tx=0x7f85ec02a270 comp rx=0 tx=0).stop 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 -- 172.21.15.16:0/3942486520 shutdown_connections 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 --2- 172.21.15.16:0/3942486520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f007cad0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:00.469 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 -- 172.21.15.16:0/3942486520 >> 172.21.15.16:0/3942486520 conn(0x7f85f0102f30 msgr2=0x7f85f0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 -- 172.21.15.16:0/3942486520 shutdown_connections 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.713+0000 7f85f8483640 1 -- 172.21.15.16:0/3942486520 wait complete. 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f8483640 1 Processor -- start 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f8483640 1 -- start start 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f8483640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f8483640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010dd90 con 0x7f85f007e620 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.714+0000 7f85f621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39806/0 (socket says 172.21.15.16:39806) 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85f621f640 1 -- 172.21.15.16:0/1847040980 learned_addr learned my addr 172.21.15.16:0/1847040980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85f621f640 1 -- 172.21.15.16:0/1847040980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f019df30 con 0x7f85f007e620 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85f621f640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0097f0 tx=0x7f85ec0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:00.470 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ec037040 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ec030c40 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f019e1c0 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85ec03a5d0 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.715+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a0eb0 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.716+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85ec03a7c0 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.716+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f007cb50 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.717+0000 7f85e77fe640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 0x7f85dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.717+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85ec06c540 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.717+0000 7f85f5a1e640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 0x7f85dc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.717+0000 7f85f5a1e640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 0x7f85dc03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85e00098c0 tx=0x7f85e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.722+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85ec0371e0 con 0x7f85f007e620 2024-09-12T00:47:00.471 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.898+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f85f01a1430 con 0x7f85f007e620 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.900+0000 7f85e77fe640 1 -- 172.21.15.16:0/1847040980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f85ec030df0 con 0x7f85f007e620 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 msgr2=0x7f85dc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 0x7f85dc03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f85e00098c0 tx=0x7f85e0006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 msgr2=0x7f85f019d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0097f0 tx=0x7f85ec0327b0 comp rx=0 tx=0).stop 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 shutdown_connections 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f85dc038fc0 0x7f85dc03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 --2- 172.21.15.16:0/1847040980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f007e620 0x7f85f019d9f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 >> 172.21.15.16:0/1847040980 conn(0x7f85f0102f30 msgr2=0x7f85f0103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 shutdown_connections 2024-09-12T00:47:00.472 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:46:59.903+0000 7f85f8483640 1 -- 172.21.15.16:0/1847040980 wait complete. 2024-09-12T00:47:00.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:00 smithi016 ceph-mon[30972]: from='mgr.14118 172.21.15.16:0/898721863' entity='mgr.x' 2024-09-12T00:47:00.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:00 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1847040980' entity='client.admin' 2024-09-12T00:47:02.160 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e93485640 1 Processor -- start 2024-09-12T00:47:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e93485640 1 -- start start 2024-09-12T00:47:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e93485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e93485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1080f0 con 0x7f5e8c107720 2024-09-12T00:47:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e91221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:02.161 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e91221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39818/0 (socket says 172.21.15.16:39818) 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e91221640 1 -- 172.21.15.16:0/2738462630 learned_addr learned my addr 172.21.15.16:0/2738462630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.856+0000 7f5e91221640 1 -- 172.21.15.16:0/2738462630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c108920 con 0x7f5e8c107720 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.857+0000 7f5e91221640 1 --2- 172.21.15.16:0/2738462630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009820 tx=0x7f5e8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d96c2b7b3445b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.857+0000 7f5e83fff640 1 -- 172.21.15.16:0/2738462630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88037070 con 0x7f5e8c107720 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.857+0000 7f5e83fff640 1 -- 172.21.15.16:0/2738462630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e88032920 con 0x7f5e8c107720 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.857+0000 7f5e83fff640 1 -- 172.21.15.16:0/2738462630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88030810 con 0x7f5e8c107720 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 -- 172.21.15.16:0/2738462630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 msgr2=0x7f5e8c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 --2- 172.21.15.16:0/2738462630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009820 tx=0x7f5e8802a270 comp rx=0 tx=0).stop 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 -- 172.21.15.16:0/2738462630 shutdown_connections 2024-09-12T00:47:02.162 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 --2- 172.21.15.16:0/2738462630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c107b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 -- 172.21.15.16:0/2738462630 >> 172.21.15.16:0/2738462630 conn(0x7f5e8c102ed0 msgr2=0x7f5e8c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 -- 172.21.15.16:0/2738462630 shutdown_connections 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.858+0000 7f5e93485640 1 -- 172.21.15.16:0/2738462630 wait complete. 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e93485640 1 Processor -- start 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e93485640 1 -- start start 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e93485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e93485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10bcd0 con 0x7f5e8c107720 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e91221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e91221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39822/0 (socket says 172.21.15.16:39822) 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e91221640 1 -- 172.21.15.16:0/3373634125 learned_addr learned my addr 172.21.15.16:0/3373634125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e91221640 1 -- 172.21.15.16:0/3373634125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19bcb0 con 0x7f5e8c107720 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.859+0000 7f5e91221640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e880097f0 tx=0x7f5e880327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:02.163 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e88037040 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e88030cc0 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19bf40 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e8803a640 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c19ec30 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.860+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5e8803b3f0 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.861+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c10be50 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.861+0000 7f5e827fc640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 0x7f5e7803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.861+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e8806c4e0 con 0x7f5e8c107720 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.861+0000 7f5e90a20640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 0x7f5e7803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.861+0000 7f5e90a20640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 0x7f5e7803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0098c0 tx=0x7f5e7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:02.164 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:00.866+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e88042e10 con 0x7f5e8c107720 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.042+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5e8c107ba0 con 0x7f5e8c107720 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.044+0000 7f5e827fc640 1 -- 172.21.15.16:0/3373634125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5e8803ba40 con 0x7f5e8c107720 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.046+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 msgr2=0x7f5e7803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.046+0000 7f5e93485640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 0x7f5e7803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0098c0 tx=0x7f5e7c006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.046+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 msgr2=0x7f5e8c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.046+0000 7f5e93485640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e880097f0 tx=0x7f5e880327b0 comp rx=0 tx=0).stop 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 shutdown_connections 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f5e78038f70 0x7f5e7803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 --2- 172.21.15.16:0/3373634125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c107720 0x7f5e8c19b770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 >> 172.21.15.16:0/3373634125 conn(0x7f5e8c102ed0 msgr2=0x7f5e8c103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 shutdown_connections 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:01.047+0000 7f5e93485640 1 -- 172.21.15.16:0/3373634125 wait complete. 2024-09-12T00:47:02.165 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-12T00:47:02.324 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:02 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3373634125' entity='client.admin' 2024-09-12T00:47:03.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1892988623' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T00:47:03.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setuser ceph since I am not root 2024-09-12T00:47:03.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setgroup ceph since I am not root 2024-09-12T00:47:03.199 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.199+0000 7fa6bbe73040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:47:03.488 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.259+0000 7fa6bbe73040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:47:03.488 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.419+0000 7fa6bbe73040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:47:03.488 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.488+0000 7fa6bbe73040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:47:03.705 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.575+0000 7f0a0515d640 1 Processor -- start 2024-09-12T00:47:03.705 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.575+0000 7f0a0515d640 1 -- start start 2024-09-12T00:47:03.705 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.575+0000 7f0a0515d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:03.705 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.575+0000 7f0a0515d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00108160 con 0x7f0a00107790 2024-09-12T00:47:03.705 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.575+0000 7f09fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.576+0000 7f09fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39826/0 (socket says 172.21.15.16:39826) 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.576+0000 7f09fed9d640 1 -- 172.21.15.16:0/1223521401 learned_addr learned my addr 172.21.15.16:0/1223521401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.576+0000 7f09fed9d640 1 -- 172.21.15.16:0/1223521401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a00108940 con 0x7f0a00107790 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.577+0000 7f09fed9d640 1 --2- 172.21.15.16:0/1223521401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009820 tx=0x7f09f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922eab4aa3270486 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.577+0000 7f09fdd9b640 1 -- 172.21.15.16:0/1223521401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09f0037070 con 0x7f0a00107790 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.577+0000 7f09fdd9b640 1 -- 172.21.15.16:0/1223521401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f0032920 con 0x7f0a00107790 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.577+0000 7f09fdd9b640 1 -- 172.21.15.16:0/1223521401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09f0030810 con 0x7f0a00107790 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 -- 172.21.15.16:0/1223521401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 msgr2=0x7f0a00107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1223521401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009820 tx=0x7f09f002a270 comp rx=0 tx=0).stop 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 -- 172.21.15.16:0/1223521401 shutdown_connections 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1223521401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a00107b90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:03.706 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 -- 172.21.15.16:0/1223521401 >> 172.21.15.16:0/1223521401 conn(0x7f0a00102fa0 msgr2=0x7f0a001053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 -- 172.21.15.16:0/1223521401 shutdown_connections 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 -- 172.21.15.16:0/1223521401 wait complete. 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.578+0000 7f0a0515d640 1 Processor -- start 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f0a0515d640 1 -- start start 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f0a0515d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f0a0515d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0010bcf0 con 0x7f0a00107790 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f09fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f09fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39838/0 (socket says 172.21.15.16:39838) 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f09fed9d640 1 -- 172.21.15.16:0/1892988623 learned_addr learned my addr 172.21.15.16:0/1892988623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.579+0000 7f09fed9d640 1 -- 172.21.15.16:0/1892988623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0019dee0 con 0x7f0a00107790 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f09fed9d640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09f00097f0 tx=0x7f09f00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09f0037040 con 0x7f0a00107790 2024-09-12T00:47:03.707 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f0030cc0 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09f003a640 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0019e170 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.580+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a001a0e60 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.581+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f09f003b3f0 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.581+0000 7f09e7fff640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 0x7f09e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.581+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0010be70 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.581+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09f006c4a0 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.581+0000 7f09fe59c640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 0x7f09e803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.582+0000 7f09fe59c640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 0x7f09e803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09f40098c0 tx=0x7f09f4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.586+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09f003b6e0 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:02.813+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0a001a1030 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.046+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f09f00328c0 con 0x7f0a00107790 2024-09-12T00:47:03.708 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.048+0000 7f09e7fff640 1 -- 172.21.15.16:0/1892988623 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f09f0049ca0 con 0x7f0a00107790 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 msgr2=0x7f09e803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 0x7f09e803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f09f40098c0 tx=0x7f09f4006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 msgr2=0x7f0a0019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09f00097f0 tx=0x7f09f00327b0 comp rx=0 tx=0).stop 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 shutdown_connections 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f09e8038fc0 0x7f09e803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 --2- 172.21.15.16:0/1892988623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a00107790 0x7f0a0019d9a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 >> 172.21.15.16:0/1892988623 conn(0x7f0a00102fa0 msgr2=0x7f0a00103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.050+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 shutdown_connections 2024-09-12T00:47:03.709 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:03.051+0000 7f0a0515d640 1 -- 172.21.15.16:0/1892988623 wait complete. 2024-09-12T00:47:03.794 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.644+0000 7fa6bbe73040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:47:03.794 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:03.795+0000 7fa6bbe73040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:47:04.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1892988623' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T00:47:04.330 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:04 smithi016 ceph-mon[30972]: mgrmap e9: x(active, since 19s) 2024-09-12T00:47:04.330 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:04.134+0000 7fa6bbe73040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:47:04.330 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:04.198+0000 7fa6bbe73040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:47:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:04.396+0000 7fa6bbe73040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:47:04.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe531d6c640 1 Processor -- start 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe531d6c640 1 -- start start 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe531d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe531d6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c07bd90 con 0x7fe52c106040 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39874/0 (socket says 172.21.15.16:39874) 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.123+0000 7fe52b7fe640 1 -- 172.21.15.16:0/88483913 learned_addr learned my addr 172.21.15.16:0/88483913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.124+0000 7fe52b7fe640 1 -- 172.21.15.16:0/88483913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c106980 con 0x7fe52c106040 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.125+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/88483913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009820 tx=0x7fe51c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0baa32b1775f47e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.125+0000 7fe52a7fc640 1 -- 172.21.15.16:0/88483913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c037070 con 0x7fe52c106040 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.125+0000 7fe52a7fc640 1 -- 172.21.15.16:0/88483913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c032920 con 0x7fe52c106040 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.125+0000 7fe52a7fc640 1 -- 172.21.15.16:0/88483913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c030810 con 0x7fe52c106040 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- 172.21.15.16:0/88483913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 msgr2=0x7fe52c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:04.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 --2- 172.21.15.16:0/88483913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009820 tx=0x7fe51c02a270 comp rx=0 tx=0).stop 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- 172.21.15.16:0/88483913 shutdown_connections 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 --2- 172.21.15.16:0/88483913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c106440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- 172.21.15.16:0/88483913 >> 172.21.15.16:0/88483913 conn(0x7fe52c101db0 msgr2=0x7fe52c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- 172.21.15.16:0/88483913 shutdown_connections 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- 172.21.15.16:0/88483913 wait complete. 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 Processor -- start 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.126+0000 7fe531d6c640 1 -- start start 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe531d6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe531d6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe52c107ac0 con 0x7fe52c106040 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe52b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39890/0 (socket says 172.21.15.16:39890) 2024-09-12T00:47:04.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe52b7fe640 1 -- 172.21.15.16:0/172296159 learned_addr learned my addr 172.21.15.16:0/172296159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe52b7fe640 1 -- 172.21.15.16:0/172296159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52c19dec0 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe52b7fe640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe51c030420 tx=0x7fe51c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c037040 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c040070 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52c19e150 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.127+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe51c03a6e0 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.128+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52c1a0e40 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.128+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fe51c03a990 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.129+0000 7fe528ff9640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 0x7fe51403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.129+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe51c06c790 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.129+0000 7fe52affd640 1 -- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 msgr2=0x7fe51403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.129+0000 7fe52affd640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 0x7fe51403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.129+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52c106440 con 0x7fe52c106040 2024-09-12T00:47:04.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.134+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe51c03ad70 con 0x7fe52c106040 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.330+0000 7fe52affd640 1 -- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 msgr2=0x7fe51403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.330+0000 7fe52affd640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 0x7fe51403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.361+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe52c02d860 con 0x7fe52c106040 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.361+0000 7fe528ff9640 1 -- 172.21.15.16:0/172296159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe51c045980 con 0x7fe52c106040 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 msgr2=0x7fe51403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 0x7fe51403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 msgr2=0x7fe52c19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe51c030420 tx=0x7fe51c032670 comp rx=0 tx=0).stop 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 shutdown_connections 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7fe514039020 0x7fe51403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 --2- 172.21.15.16:0/172296159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52c106040 0x7fe52c19d980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.363+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 >> 172.21.15.16:0/172296159 conn(0x7fe52c101db0 msgr2=0x7fe52c194640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:04.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.364+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 shutdown_connections 2024-09-12T00:47:04.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:04.364+0000 7fe531d6c640 1 -- 172.21.15.16:0/172296159 wait complete. 2024-09-12T00:47:04.993 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-12T00:47:04.993 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-12T00:47:05.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/172296159' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T00:47:05.602 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:05.338+0000 7fa6bbe73040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:47:05.602 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:05.411+0000 7fa6bbe73040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:47:06.281 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.160+0000 7fa6bbe73040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:47:06.281 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.281+0000 7fa6bbe73040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:47:06.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.430+0000 7fa6bbe73040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:47:06.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.494+0000 7fa6bbe73040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:47:06.948 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.649+0000 7fa6bbe73040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:47:06.948 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.720+0000 7fa6bbe73040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:47:06.948 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.783+0000 7fa6bbe73040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:47:06.948 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:06.948+0000 7fa6bbe73040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:47:07.234 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:07 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:07.016+0000 7fa6bbe73040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:47:07.234 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:07 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:07.086+0000 7fa6bbe73040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:47:07.234 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:07 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:07.172+0000 7fa6bbe73040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:47:07.234 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:07 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:07.234+0000 7fa6bbe73040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:47:07.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:07 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:07.296+0000 7fa6bbe73040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:47:08.385 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:08.123+0000 7fa6bbe73040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: Active manager daemon x restarted 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: Activating manager daemon x 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: mgrmap e10: x(active, starting, since 0.00343174s) 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:47:08.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:47:08.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: Manager daemon x is now available 2024-09-12T00:47:08.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:08.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:47:08.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:47:09.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-12T00:47:09.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.396+0000 7f2eebb44640 1 Processor -- start 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.396+0000 7f2eebb44640 1 -- start start 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.396+0000 7f2eebb44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.396+0000 7f2eebb44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4108130 con 0x7f2ee4107760 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.397+0000 7f2ee98e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.397+0000 7f2ee98e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39892/0 (socket says 172.21.15.16:39892) 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.397+0000 7f2ee98e0640 1 -- 172.21.15.16:0/755739781 learned_addr learned my addr 172.21.15.16:0/755739781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.397+0000 7f2ee98e0640 1 -- 172.21.15.16:0/755739781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee4108910 con 0x7f2ee4107760 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.398+0000 7f2ee98e0640 1 --2- 172.21.15.16:0/755739781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009820 tx=0x7f2ee002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6c5e6ed5a76f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.398+0000 7f2ee88de640 1 -- 172.21.15.16:0/755739781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee0037070 con 0x7f2ee4107760 2024-09-12T00:47:09.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.398+0000 7f2ee88de640 1 -- 172.21.15.16:0/755739781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee0032920 con 0x7f2ee4107760 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.398+0000 7f2ee88de640 1 -- 172.21.15.16:0/755739781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee0030810 con 0x7f2ee4107760 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 -- 172.21.15.16:0/755739781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 msgr2=0x7f2ee4107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 --2- 172.21.15.16:0/755739781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009820 tx=0x7f2ee002a270 comp rx=0 tx=0).stop 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 -- 172.21.15.16:0/755739781 shutdown_connections 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 --2- 172.21.15.16:0/755739781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee4107b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 -- 172.21.15.16:0/755739781 >> 172.21.15.16:0/755739781 conn(0x7f2ee4102f30 msgr2=0x7f2ee4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 -- 172.21.15.16:0/755739781 shutdown_connections 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 -- 172.21.15.16:0/755739781 wait complete. 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.399+0000 7f2eebb44640 1 Processor -- start 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2eebb44640 1 -- start start 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2eebb44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:09.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2eebb44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee410bcc0 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ee98e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ee98e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39894/0 (socket says 172.21.15.16:39894) 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ee98e0640 1 -- 172.21.15.16:0/444310520 learned_addr learned my addr 172.21.15.16:0/444310520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ee98e0640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee419bce0 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ee98e0640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0030560 tx=0x7f2ee0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee0037040 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee419bf70 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.400+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee0040070 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.401+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee003a6d0 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.401+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee419ec60 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.401+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2ee003a940 con 0x7f2ee4107760 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.402+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:09.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.402+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ee4107be0 con 0x7f2ed0038fc0 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.402+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ee006c690 con 0x7f2ee4107760 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.402+0000 7f2ee90df640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 msgr2=0x7f2ed003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.402+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.603+0000 7f2ee90df640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 msgr2=0x7f2ed003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:05.603+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:06.004+0000 7f2ee90df640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 msgr2=0x7f2ed003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:06.004+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:06.805+0000 7f2ee90df640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 msgr2=0x7f2ed003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1538480573 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:06.805+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:08.127+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f2ee006b140 con 0x7f2ee4107760 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:08.127+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 msgr2=0x7f2ed003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:08.127+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:09.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.128+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f2ee0045460 con 0x7f2ee4107760 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.129+0000 7f2ed6ffd640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 0x7f2ed003ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.129+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ee000af30 con 0x7f2ed003c890 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.129+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 0x7f2ed003ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.130+0000 7f2ee90df640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 0x7f2ed003ec80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003980 tx=0x7f2ed8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.131+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f2ee000af30 con 0x7f2ed003c890 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.143+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2ee410be40 con 0x7f2ed003c890 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.143+0000 7f2ed6ffd640 1 -- 172.21.15.16:0/444310520 <== mgr.14150 v2:172.21.15.16:6800/1272021982 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f2ee410be40 con 0x7f2ed003c890 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 msgr2=0x7f2ed003ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 0x7f2ed003ec80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003980 tx=0x7f2ed8006e10 comp rx=0 tx=0).stop 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 msgr2=0x7f2ee419b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0030560 tx=0x7f2ee0032670 comp rx=0 tx=0).stop 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 shutdown_connections 2024-09-12T00:47:09.944 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2ed003c890 0x7f2ed003ec80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:6800/1538480573,v1:172.21.15.16:6801/1538480573] conn(0x7f2ed0038fc0 0x7f2ed003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 --2- 172.21.15.16:0/444310520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee4107760 0x7f2ee419b7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 >> 172.21.15.16:0/444310520 conn(0x7f2ee4102f30 msgr2=0x7f2ee4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 shutdown_connections 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:09.144+0000 7f2eebb44640 1 -- 172.21.15.16:0/444310520 wait complete. 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-12T00:47:09.945 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:09.790+0000 7fa627187640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:00:47:09] ENGINE Error in HTTPServer.serve 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Traceback (most recent call last): 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._connections.run(self.expiration_interval) 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._run(expiration_interval) 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:47:10.048 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s = self.context.wrap_socket( 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: return self.sslsocket_class._create( 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self.do_handshake() 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._sslobj.do_handshake() 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:47:10.049 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:47:09 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:47:10.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: mgrmap e11: x(active, since 1.00608s) 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: [12/Sep/2024:00:47:09] ENGINE Bus STARTING 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: [12/Sep/2024:00:47:09] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: [12/Sep/2024:00:47:09] ENGINE Error in HTTPServer.serve 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:47:10.309 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: [12/Sep/2024:00:47:09] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T00:47:10.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:10 smithi016 ceph-mon[30972]: [12/Sep/2024:00:47:09] ENGINE Bus STARTED 2024-09-12T00:47:11.219 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T00:47:11.219 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.390+0000 7f2801f01640 1 Processor -- start 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.390+0000 7f2801f01640 1 -- start start 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.390+0000 7f2801f01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.390+0000 7f2801f01640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc108170 con 0x7f27fc1077a0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.391+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.391+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46192/0 (socket says 172.21.15.16:46192) 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.391+0000 7f27fb7fe640 1 -- 172.21.15.16:0/3934184960 learned_addr learned my addr 172.21.15.16:0/3934184960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.391+0000 7f27fb7fe640 1 -- 172.21.15.16:0/3934184960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc108950 con 0x7f27fc1077a0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.391+0000 7f27fb7fe640 1 --2- 172.21.15.16:0/3934184960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009820 tx=0x7f27ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33f7cb94d2fbf2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f27faffd640 1 -- 172.21.15.16:0/3934184960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ec037070 con 0x7f27fc1077a0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f27faffd640 1 -- 172.21.15.16:0/3934184960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ec032920 con 0x7f27fc1077a0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f27faffd640 1 -- 172.21.15.16:0/3934184960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ec030810 con 0x7f27fc1077a0 2024-09-12T00:47:11.220 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 -- 172.21.15.16:0/3934184960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 msgr2=0x7f27fc107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 --2- 172.21.15.16:0/3934184960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009820 tx=0x7f27ec02a270 comp rx=0 tx=0).stop 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 -- 172.21.15.16:0/3934184960 shutdown_connections 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 --2- 172.21.15.16:0/3934184960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc107ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 -- 172.21.15.16:0/3934184960 >> 172.21.15.16:0/3934184960 conn(0x7f27fc102fd0 msgr2=0x7f27fc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 -- 172.21.15.16:0/3934184960 shutdown_connections 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.392+0000 7f2801f01640 1 -- 172.21.15.16:0/3934184960 wait complete. 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f2801f01640 1 Processor -- start 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f2801f01640 1 -- start start 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f2801f01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f2801f01640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10bd00 con 0x7f27fc1077a0 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46200/0 (socket says 172.21.15.16:46200) 2024-09-12T00:47:11.221 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.393+0000 7f27fb7fe640 1 -- 172.21.15.16:0/742100714 learned_addr learned my addr 172.21.15.16:0/742100714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f27fb7fe640 1 -- 172.21.15.16:0/742100714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc19bcb0 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f27fb7fe640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27ec030560 tx=0x7f27ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ec037040 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc19bf40 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ec040070 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ec03a740 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.394+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc19ec30 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.395+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f27ec03a9b0 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.395+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc10be80 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.395+0000 7f27f97fa640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 0x7f27e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.395+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27ec06c890 con 0x7f27fc1077a0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.395+0000 7f27f2ffd640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 0x7f27e803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:11.222 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.396+0000 7f27f2ffd640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 0x7f27e803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27e40098c0 tx=0x7f27e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.400+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27ec03adf0 con 0x7f27fc1077a0 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.597+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f27fc104260 con 0x7f27e8038fc0 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.688+0000 7f27f97fa640 1 -- 172.21.15.16:0/742100714 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f27fc104260 con 0x7f27e8038fc0 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 msgr2=0x7f27e803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 0x7f27e803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f27e40098c0 tx=0x7f27e4006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 msgr2=0x7f27fc19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27ec030560 tx=0x7f27ec032670 comp rx=0 tx=0).stop 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 shutdown_connections 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27e8038fc0 0x7f27e803b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 --2- 172.21.15.16:0/742100714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27fc1077a0 0x7f27fc19b770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 >> 172.21.15.16:0/742100714 conn(0x7f27fc102fd0 msgr2=0x7f27fc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:11.223 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 shutdown_connections 2024-09-12T00:47:11.224 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:10.691+0000 7f2801f01640 1 -- 172.21.15.16:0/742100714 wait complete. 2024-09-12T00:47:11.224 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-12T00:47:11.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:11 smithi016 ceph-mon[30972]: mgrmap e12: x(active, since 2s) 2024-09-12T00:47:11.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:11.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:11 smithi016 ceph-mon[30972]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:11.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:11.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:12.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$hSoOwau3jIQcTjNXhC6EmuNcVakODWgdTC3y.LBVFWMLAUN.u.TT6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726102032, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T00:47:12.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.617+0000 7f55bdebf640 1 Processor -- start 2024-09-12T00:47:12.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.617+0000 7f55bdebf640 1 -- start start 2024-09-12T00:47:12.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.617+0000 7f55bdebf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:12.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.617+0000 7f55bdebf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8108100 con 0x7f55b8107730 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.618+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.618+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46214/0 (socket says 172.21.15.16:46214) 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.618+0000 7f55b77fe640 1 -- 172.21.15.16:0/3890828799 learned_addr learned my addr 172.21.15.16:0/3890828799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.618+0000 7f55b77fe640 1 -- 172.21.15.16:0/3890828799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b8108930 con 0x7f55b8107730 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55b77fe640 1 --2- 172.21.15.16:0/3890828799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009e80 tx=0x7f55a802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdd474aa974160b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55b67fc640 1 -- 172.21.15.16:0/3890828799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a8036070 con 0x7f55b8107730 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55b67fc640 1 -- 172.21.15.16:0/3890828799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a80318e0 con 0x7f55b8107730 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55b67fc640 1 -- 172.21.15.16:0/3890828799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a8007a00 con 0x7f55b8107730 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55bdebf640 1 -- 172.21.15.16:0/3890828799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 msgr2=0x7f55b8107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3890828799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009e80 tx=0x7f55a802a7a0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55bdebf640 1 -- 172.21.15.16:0/3890828799 shutdown_connections 2024-09-12T00:47:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3890828799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b8107b30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.619+0000 7f55bdebf640 1 -- 172.21.15.16:0/3890828799 >> 172.21.15.16:0/3890828799 conn(0x7f55b8102ec0 msgr2=0x7f55b8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.620+0000 7f55bdebf640 1 -- 172.21.15.16:0/3890828799 shutdown_connections 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.620+0000 7f55bdebf640 1 -- 172.21.15.16:0/3890828799 wait complete. 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.620+0000 7f55bdebf640 1 Processor -- start 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.620+0000 7f55bdebf640 1 -- start start 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55bdebf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55bdebf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810bce0 con 0x7f55b8107730 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46218/0 (socket says 172.21.15.16:46218) 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b77fe640 1 -- 172.21.15.16:0/3978237341 learned_addr learned my addr 172.21.15.16:0/3978237341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b77fe640 1 -- 172.21.15.16:0/3978237341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b819bce0 con 0x7f55b8107730 2024-09-12T00:47:12.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b77fe640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55a8031cc0 tx=0x7f55a8031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a8036050 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a8042070 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b819bf70 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55a803d8e0 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.621+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b819ec60 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.622+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f55a803db90 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.622+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b810be60 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.623+0000 7f55b4ff9640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 0x7f55a403b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.623+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f55a806d5a0 con 0x7f55b8107730 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.623+0000 7f55b6ffd640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 0x7f55a403b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.623+0000 7f55b6ffd640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 0x7f55a403b530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098c0 tx=0x7f55a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.628+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55a8044310 con 0x7f55b8107730 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:11.820+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f55b819eef0 con 0x7f55a4039070 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.071+0000 7f55b4ff9640 1 -- 172.21.15.16:0/3978237341 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f55b819eef0 con 0x7f55a4039070 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 msgr2=0x7f55a403b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 0x7f55a403b530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55a00098c0 tx=0x7f55a0006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 msgr2=0x7f55b819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f55a8031cc0 tx=0x7f55a8031cf0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 shutdown_connections 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.073+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f55a4039070 0x7f55a403b530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.074+0000 7f55bdebf640 1 --2- 172.21.15.16:0/3978237341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b8107730 0x7f55b819b7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.074+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 >> 172.21.15.16:0/3978237341 conn(0x7f55b8102ec0 msgr2=0x7f55b8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.074+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 shutdown_connections 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:12.074+0000 7f55bdebf640 1 -- 172.21.15.16:0/3978237341 wait complete. 2024-09-12T00:47:12.691 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-12T00:47:12.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:12.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:12 smithi016 ceph-mon[30972]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:12.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:13.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3059998969' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf9b55640 1 Processor -- start 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf9b55640 1 -- start start 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf9b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf9b55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4108160 con 0x7f3bf4107790 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:14.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46222/0 (socket says 172.21.15.16:46222) 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf37fe640 1 -- 172.21.15.16:0/4056743966 learned_addr learned my addr 172.21.15.16:0/4056743966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.065+0000 7f3bf37fe640 1 -- 172.21.15.16:0/4056743966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf4108940 con 0x7f3bf4107790 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.066+0000 7f3bf37fe640 1 --2- 172.21.15.16:0/4056743966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009820 tx=0x7f3be402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2084ac794ac6790 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.066+0000 7f3bf2ffd640 1 -- 172.21.15.16:0/4056743966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4037070 con 0x7f3bf4107790 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.066+0000 7f3bf2ffd640 1 -- 172.21.15.16:0/4056743966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be4032920 con 0x7f3bf4107790 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.066+0000 7f3bf2ffd640 1 -- 172.21.15.16:0/4056743966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4030810 con 0x7f3bf4107790 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 -- 172.21.15.16:0/4056743966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 msgr2=0x7f3bf4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/4056743966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009820 tx=0x7f3be402a270 comp rx=0 tx=0).stop 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 -- 172.21.15.16:0/4056743966 shutdown_connections 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/4056743966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf4107b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 -- 172.21.15.16:0/4056743966 >> 172.21.15.16:0/4056743966 conn(0x7f3bf4102fa0 msgr2=0x7f3bf41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 -- 172.21.15.16:0/4056743966 shutdown_connections 2024-09-12T00:47:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.067+0000 7f3bf9b55640 1 -- 172.21.15.16:0/4056743966 wait complete. 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf9b55640 1 Processor -- start 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf9b55640 1 -- start start 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf9b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf9b55640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf410bcf0 con 0x7f3bf4107790 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3059998969 learned_addr learned my addr 172.21.15.16:0/3059998969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf37fe640 1 -- 172.21.15.16:0/3059998969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf419bca0 con 0x7f3bf4107790 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf37fe640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3be4030560 tx=0x7f3be4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be4037040 con 0x7f3bf4107790 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be4040070 con 0x7f3bf4107790 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.068+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf419bf30 con 0x7f3bf4107790 2024-09-12T00:47:14.576 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.069+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be403a740 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.069+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf419ec20 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.069+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3be403b420 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.069+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf410be70 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.070+0000 7f3bf17fa640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 0x7f3be003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.070+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3be406c6d0 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.070+0000 7f3bebfff640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 0x7f3be003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.070+0000 7f3bebfff640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 0x7f3be003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0098c0 tx=0x7f3bdc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.074+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3be4031e80 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.252+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3bf419edf0 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.252+0000 7f3bf17fa640 1 -- 172.21.15.16:0/3059998969 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3be404a200 con 0x7f3bf4107790 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 msgr2=0x7f3be003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 0x7f3be003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0098c0 tx=0x7f3bdc006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:14.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 msgr2=0x7f3bf419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3be4030560 tx=0x7f3be4032670 comp rx=0 tx=0).stop 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 shutdown_connections 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3be0038fc0 0x7f3be003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 --2- 172.21.15.16:0/3059998969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bf4107790 0x7f3bf419b760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 >> 172.21.15.16:0/3059998969 conn(0x7f3bf4102fa0 msgr2=0x7f3bf4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 shutdown_connections 2024-09-12T00:47:14.578 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:13.255+0000 7f3bf9b55640 1 -- 172.21.15.16:0/3059998969 wait complete. 2024-09-12T00:47:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T00:47:14.584 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-12T00:47:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T00:47:14.593 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-12T00:47:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-12T00:47:14.594 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T00:47:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-12T00:47:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-12T00:47:14.595 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-12T00:47:14.596 INFO:teuthology.orchestra.run.smithi016.stdout: Password: 3r3gv7t7af 2024-09-12T00:47:14.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:14.596 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config directory 2024-09-12T00:47:15.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:15 smithi016 ceph-mon[30972]: mgrmap e13: x(active, since 6s) 2024-09-12T00:47:15.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:15 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3114322391' entity='client.admin' 2024-09-12T00:47:15.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33cba5d640 1 Processor -- start 2024-09-12T00:47:15.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33cba5d640 1 -- start start 2024-09-12T00:47:15.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33cba5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33cba5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33c407d170 con 0x7f33c407e740 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33c97f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33c97f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.034+0000 7f33c97f9640 1 -- 172.21.15.16:0/3413450074 learned_addr learned my addr 172.21.15.16:0/3413450074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.035+0000 7f33c97f9640 1 -- 172.21.15.16:0/3413450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c407d2f0 con 0x7f33c407e740 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.035+0000 7f33c97f9640 1 --2- 172.21.15.16:0/3413450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009820 tx=0x7f33c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc2447fc337163e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.035+0000 7f33bbfff640 1 -- 172.21.15.16:0/3413450074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c0037070 con 0x7f33c407e740 2024-09-12T00:47:15.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.035+0000 7f33bbfff640 1 -- 172.21.15.16:0/3413450074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c0032920 con 0x7f33c407e740 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.035+0000 7f33bbfff640 1 -- 172.21.15.16:0/3413450074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c0030810 con 0x7f33c407e740 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 -- 172.21.15.16:0/3413450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 msgr2=0x7f33c407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3413450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009820 tx=0x7f33c002a270 comp rx=0 tx=0).stop 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 -- 172.21.15.16:0/3413450074 shutdown_connections 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3413450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c407cba0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 -- 172.21.15.16:0/3413450074 >> 172.21.15.16:0/3413450074 conn(0x7f33c4102f40 msgr2=0x7f33c4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 -- 172.21.15.16:0/3413450074 shutdown_connections 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.036+0000 7f33cba5d640 1 -- 172.21.15.16:0/3413450074 wait complete. 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33cba5d640 1 Processor -- start 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33cba5d640 1 -- start start 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33cba5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:15.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33cba5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33c41099d0 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33c97f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33c97f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46234/0 (socket says 172.21.15.16:46234) 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.037+0000 7f33c97f9640 1 -- 172.21.15.16:0/3114322391 learned_addr learned my addr 172.21.15.16:0/3114322391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33c97f9640 1 -- 172.21.15.16:0/3114322391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c41a2300 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33c97f9640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33c00097f0 tx=0x7f33c00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c0037040 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c41a2590 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c0030cc0 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33c003a640 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.038+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41a5280 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.039+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c407cc20 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.039+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f33c003b3f0 con 0x7f33c407e740 2024-09-12T00:47:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.039+0000 7f33ba7fc640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 0x7f33b003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.039+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f33c006c6e0 con 0x7f33c407e740 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.040+0000 7f33c8ff8640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 0x7f33b003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.040+0000 7f33c8ff8640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 0x7f33b003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098c0 tx=0x7f33b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.044+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33c0043e10 con 0x7f33c407e740 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.311+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f33c402d9a0 con 0x7f33c407e740 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.314+0000 7f33ba7fc640 1 -- 172.21.15.16:0/3114322391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f33c003ba40 con 0x7f33c407e740 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 msgr2=0x7f33b003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 0x7f33b003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098c0 tx=0x7f33b4006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 msgr2=0x7f33c41a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33c00097f0 tx=0x7f33c00327b0 comp rx=0 tx=0).stop 2024-09-12T00:47:15.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 shutdown_connections 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f33b0039060 0x7f33b003b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 --2- 172.21.15.16:0/3114322391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c407e740 0x7f33c41a1dc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 >> 172.21.15.16:0/3114322391 conn(0x7f33c4102f40 msgr2=0x7f33c4198a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 shutdown_connections 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-12T00:47:15.316+0000 7f33cba5d640 1 -- 172.21.15.16:0/3114322391 wait complete. 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.993 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:15.994 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-12T00:47:16.027 INFO:tasks.cephadm:Fetching config... 2024-09-12T00:47:16.027 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:16.027 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T00:47:16.043 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T00:47:16.044 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:16.044 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T00:47:16.101 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T00:47:16.102 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:16.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T00:47:16.167 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T00:47:16.167 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:16.167 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T00:47:16.224 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T00:47:16.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T00:47:16.299 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:16.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T00:47:16.355 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:16.367 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T00:47:16.410 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCLdjl945URie+xmzRRDevKlCzu1cb9ZzPDEgtcjYc8SUZEgf1LtH06dwSOPq3mq9apOrCnXbQ7bWUMpg/s7ez+mUZjps8yYCqXLW05Nh67fADrIqpBoQvaIQL9g5zwhZyxUFc4htwhg24oL6ZlHOfHVC5ltZCT+U28t2gmcwc+MF7NAe6CHRc5KFiHwK1AEJIhSbHxjnZXyHfEuTh44aEYiYcM+ZDbd/rijiSF7g7odHlZbKwV+/NAggc2kBITtIt/NispqB7B+ifO67EdX7hz640OOl83kx2H+kt+Mkq2zMA2g88gdrGL1l2AUkJhJgYXX85oWc2xZYNdtD1PJ2jL7a4JJJMOdZaFvbDGc5UzWg+yvMBfo7JUycQuB0ALC2boL3QVhWdf7Qhs0hLpjziYC66MK94ZaM1UQWlHxg5ze7OgxuOyqq2gmXUIDYFGfQ0I6+sp4XT0MotuZBsy/ZggKjxPtlzOzMMkY+u9vqL0LzsC0LKgfVeiHvh6viRV/Xk= ceph-434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:16.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T00:47:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- 172.21.15.16:0/4007852957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 msgr2=0x7fc144104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/4007852957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144104580 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc138009880 tx=0x7fc13802a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- 172.21.15.16:0/4007852957 shutdown_connections 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/4007852957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144104580 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- 172.21.15.16:0/4007852957 >> 172.21.15.16:0/4007852957 conn(0x7fc1440fbcd0 msgr2=0x7fc1440fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- 172.21.15.16:0/4007852957 shutdown_connections 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- 172.21.15.16:0/4007852957 wait complete. 2024-09-12T00:47:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 Processor -- start 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.171+0000 7fc14c3b8640 1 -- start start 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14c3b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14c3b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441076d0 con 0x7fc144102190 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14a154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14a154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46254/0 (socket says 172.21.15.16:46254) 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14a154640 1 -- 172.21.15.16:0/1802724638 learned_addr learned my addr 172.21.15.16:0/1802724638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14a154640 1 -- 172.21.15.16:0/1802724638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144197290 con 0x7fc144102190 2024-09-12T00:47:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14a154640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc138004f60 tx=0x7fc1380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc13803f070 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138003ad0 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc144197520 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc138035030 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.172+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14419a210 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc138042070 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144107e50 con 0x7fc144102190 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc1377fe640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 0x7fc13003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc13806ce10 con 0x7fc144102190 2024-09-12T00:47:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc149953640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 0x7fc13003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.174+0000 7fc149953640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 0x7fc13003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc1400098c0 tx=0x7fc140006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.179+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc13803a3c0 con 0x7fc144102190 2024-09-12T00:47:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.354+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc14402d830 con 0x7fc144102190 2024-09-12T00:47:17.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.357+0000 7fc1377fe640 1 -- 172.21.15.16:0/1802724638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc138030d90 con 0x7fc144102190 2024-09-12T00:47:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 msgr2=0x7fc13003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 0x7fc13003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc1400098c0 tx=0x7fc140006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 msgr2=0x7fc144196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc138004f60 tx=0x7fc1380048f0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 shutdown_connections 2024-09-12T00:47:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1300390c0 0x7fc13003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 --2- 172.21.15.16:0/1802724638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc144102190 0x7fc144196d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 >> 172.21.15.16:0/1802724638 conn(0x7fc1440fbcd0 msgr2=0x7fc1440fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 shutdown_connections 2024-09-12T00:47:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:17.359+0000 7fc14c3b8640 1 -- 172.21.15.16:0/1802724638 wait complete. 2024-09-12T00:47:18.449 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T00:47:18.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1802724638' entity='client.admin' 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:18.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:18.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 -- 172.21.15.16:0/1517034761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 msgr2=0x7f15fc100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 --2- 172.21.15.16:0/1517034761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc100320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009880 tx=0x7f15f802a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 -- 172.21.15.16:0/1517034761 shutdown_connections 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 --2- 172.21.15.16:0/1517034761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc100320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 -- 172.21.15.16:0/1517034761 >> 172.21.15.16:0/1517034761 conn(0x7f15fc0fbcd0 msgr2=0x7f15fc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 -- 172.21.15.16:0/1517034761 shutdown_connections 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.382+0000 7f16047a1640 1 -- 172.21.15.16:0/1517034761 wait complete. 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f16047a1640 1 Processor -- start 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f16047a1640 1 -- start start 2024-09-12T00:47:19.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f16047a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f16047a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc1098d0 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f160253d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f160253d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57106/0 (socket says 172.21.15.16:57106) 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f160253d640 1 -- 172.21.15.16:0/3887840920 learned_addr learned my addr 172.21.15.16:0/3887840920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.383+0000 7f160253d640 1 -- 172.21.15.16:0/3887840920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1994b0 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f160253d640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15f8004f60 tx=0x7f15f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f803f070 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15f8003ad0 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc199740 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15f8035030 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.384+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc19c430 con 0x7f15fc0fff20 2024-09-12T00:47:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.385+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f15f8042070 con 0x7f15fc0fff20 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.385+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc10a050 con 0x7f15fc0fff20 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.385+0000 7f15f37fe640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 0x7f15e803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.385+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f15f806cda0 con 0x7f15fc0fff20 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.385+0000 7f1601d3c640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 0x7f15e803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.386+0000 7f1601d3c640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 0x7f15e803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098c0 tx=0x7f15ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.390+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15f8039350 con 0x7f15fc0fff20 2024-09-12T00:47:19.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.571+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f15fc02d830 con 0x7f15e80390c0 2024-09-12T00:47:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.574+0000 7f15f37fe640 1 -- 172.21.15.16:0/3887840920 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15fc02d830 con 0x7f15e80390c0 2024-09-12T00:47:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 msgr2=0x7f15e803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:19.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 0x7f15e803b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0098c0 tx=0x7f15ec006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 msgr2=0x7f15fc198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15f8004f60 tx=0x7f15f80048f0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 shutdown_connections 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f15e80390c0 0x7f15e803b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 --2- 172.21.15.16:0/3887840920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15fc0fff20 0x7f15fc198f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 >> 172.21.15.16:0/3887840920 conn(0x7f15fc0fbcd0 msgr2=0x7f15fc0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 shutdown_connections 2024-09-12T00:47:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:19.577+0000 7f16047a1640 1 -- 172.21.15.16:0/3887840920 wait complete. 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:20.739 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:20.768 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2024-09-12T00:47:20.768 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:47:20.768 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2024-09-12T00:47:20.791 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:47:20.791 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:20.849 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2024-09-12T00:47:20.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch host add smithi099 2024-09-12T00:47:21.100 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3361782513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 msgr2=0x7fc9c8106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3361782513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009c10 tx=0x7fc9b802aa10 comp rx=0 tx=0).stop 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3361782513 shutdown_connections 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3361782513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8106920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3361782513 >> 172.21.15.16:0/3361782513 conn(0x7fc9c8077310 msgr2=0x7fc9c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3361782513 shutdown_connections 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3361782513 wait complete. 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 Processor -- start 2024-09-12T00:47:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.570+0000 7fc9cf7e0640 1 -- start start 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cf7e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cf7e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c810ab70 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cd57c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cd57c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57114/0 (socket says 172.21.15.16:57114) 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cd57c640 1 -- 172.21.15.16:0/3378711054 learned_addr learned my addr 172.21.15.16:0/3378711054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cd57c640 1 -- 172.21.15.16:0/3378711054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c81976c0 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cd57c640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d30 tx=0x7fc9b8030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b8034070 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c8197950 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b802f8c0 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b80044d0 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.571+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c8197d50 con 0x7fc9c8106540 2024-09-12T00:47:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.572+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc9b8037020 con 0x7fc9c8106540 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.572+0000 7fc9c67fc640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 0x7fc9b003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.572+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9c8101bb0 con 0x7fc9c8106540 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.572+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc9b806cea0 con 0x7fc9c8106540 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.573+0000 7fc9ccd7b640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 0x7fc9b003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.573+0000 7fc9ccd7b640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 0x7fc9b003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0098c0 tx=0x7fc9bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.577+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9b803f070 con 0x7fc9c8106540 2024-09-12T00:47:21.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:21.756+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}) -- 0x7fc9c8198540 con 0x7fc9b0039060 2024-09-12T00:47:21.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:21 smithi016 ceph-mon[30972]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:47:22.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:22 smithi016 ceph-mon[30972]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:23.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:23 smithi016 ceph-mon[30972]: Deploying cephadm binary to smithi099 2024-09-12T00:47:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.587+0000 7fc9c67fc640 1 -- 172.21.15.16:0/3378711054 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc9c8198540 con 0x7fc9b0039060 2024-09-12T00:47:25.586 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi099' with addr '172.21.15.99' 2024-09-12T00:47:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.589+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 msgr2=0x7fc9b003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.589+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 0x7fc9b003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0098c0 tx=0x7fc9bc006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.589+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 msgr2=0x7fc9c8197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.589+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d30 tx=0x7fc9b8030c30 comp rx=0 tx=0).stop 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 shutdown_connections 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9b0039060 0x7fc9b003b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 --2- 172.21.15.16:0/3378711054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c8106540 0x7fc9c8197180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 >> 172.21.15.16:0/3378711054 conn(0x7fc9c8077310 msgr2=0x7fc9c80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 shutdown_connections 2024-09-12T00:47:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:25.590+0000 7fc9cf7e0640 1 -- 172.21.15.16:0/3378711054 wait complete. 2024-09-12T00:47:26.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T00:47:26.443 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 -- 172.21.15.16:0/2249861240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 msgr2=0x7f7e40107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2249861240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009880 tx=0x7f7e3002a670 comp rx=0 tx=0).stop 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 -- 172.21.15.16:0/2249861240 shutdown_connections 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2249861240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 -- 172.21.15.16:0/2249861240 >> 172.21.15.16:0/2249861240 conn(0x7f7e400fb930 msgr2=0x7f7e400fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 -- 172.21.15.16:0/2249861240 shutdown_connections 2024-09-12T00:47:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.874+0000 7f7e468a4640 1 -- 172.21.15.16:0/2249861240 wait complete. 2024-09-12T00:47:26.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:26.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:26 smithi016 ceph-mon[30972]: Added host smithi099 2024-09-12T00:47:26.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:26.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:26.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.875+0000 7f7e468a4640 1 Processor -- start 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.875+0000 7f7e468a4640 1 -- start start 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e468a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e468a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4010bc50 con 0x7f7e40107660 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57122/0 (socket says 172.21.15.16:57122) 2024-09-12T00:47:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3ffff640 1 -- 172.21.15.16:0/2264804601 learned_addr learned my addr 172.21.15.16:0/2264804601 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3ffff640 1 -- 172.21.15.16:0/2264804601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e40197680 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3ffff640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009230 tx=0x7f7e30003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.876+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e3003f070 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.877+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3003bc80 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.877+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e40197970 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.877+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e30035030 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.877+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e4019ba10 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7e30042070 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e3d7fa640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 0x7f7e2803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e3006ce10 con 0x7f7e40107660 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e3f7fe640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 0x7f7e2803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e40102d00 con 0x7f7e40107660 2024-09-12T00:47:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.878+0000 7f7e3f7fe640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 0x7f7e2803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7e340098c0 tx=0x7f7e34006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:26.883+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e30031dd0 con 0x7f7e40107660 2024-09-12T00:47:27.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.065+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e40107af0 con 0x7f7e280390c0 2024-09-12T00:47:27.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.066+0000 7f7e3d7fa640 1 -- 172.21.15.16:0/2264804601 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f7e40107af0 con 0x7f7e280390c0 2024-09-12T00:47:27.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:27.068 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.068+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 msgr2=0x7f7e2803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.068+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 0x7f7e2803b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7e340098c0 tx=0x7f7e34006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 msgr2=0x7f7e40197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009230 tx=0x7f7e30003d40 comp rx=0 tx=0).stop 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 shutdown_connections 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f7e280390c0 0x7f7e2803b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 --2- 172.21.15.16:0/2264804601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e40107660 0x7f7e40197140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 >> 172.21.15.16:0/2264804601 conn(0x7f7e400fb930 msgr2=0x7f7e400fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 shutdown_connections 2024-09-12T00:47:27.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:27.069+0000 7f7e468a4640 1 -- 172.21.15.16:0/2264804601 wait complete. 2024-09-12T00:47:27.720 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-09-12T00:47:27.720 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:47:27.720 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-09-12T00:47:27.740 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:47:27.740 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:27.798 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-09-12T00:47:27.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch host add smithi159 2024-09-12T00:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:28.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:28.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:27 smithi016 ceph-mon[30972]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:47:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 -- 172.21.15.16:0/1423182222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 msgr2=0x7f2788107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 --2- 172.21.15.16:0/1423182222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f2788107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2784009880 tx=0x7f278402a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 -- 172.21.15.16:0/1423182222 shutdown_connections 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 --2- 172.21.15.16:0/1423182222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f2788107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 -- 172.21.15.16:0/1423182222 >> 172.21.15.16:0/1423182222 conn(0x7f27880fd290 msgr2=0x7f27880ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 -- 172.21.15.16:0/1423182222 shutdown_connections 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.541+0000 7f278f2ab640 1 -- 172.21.15.16:0/1423182222 wait complete. 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278f2ab640 1 Processor -- start 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278f2ab640 1 -- start start 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278f2ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278f2ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f278810bc90 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278d047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278d047640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34846/0 (socket says 172.21.15.16:34846) 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.542+0000 7f278d047640 1 -- 172.21.15.16:0/3904294493 learned_addr learned my addr 172.21.15.16:0/3904294493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f278d047640 1 -- 172.21.15.16:0/3904294493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27880763e0 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f278d047640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2784009230 tx=0x7f2784003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278403f070 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278403bc80 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2784035030 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2788074a30 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.543+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2788074ef0 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.544+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2784042070 con 0x7f2788107660 2024-09-12T00:47:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.544+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2788102cd0 con 0x7f2788107660 2024-09-12T00:47:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.544+0000 7f277e7fc640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 0x7f277403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.544+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f278406ccc0 con 0x7f2788107660 2024-09-12T00:47:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.544+0000 7f278c846640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 0x7f277403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.545+0000 7f278c846640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 0x7f277403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27780098c0 tx=0x7f2778006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.549+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27840372d0 con 0x7f2788107660 2024-09-12T00:47:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:28.729+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) -- 0x7f2788075a90 con 0x7f27740390c0 2024-09-12T00:47:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:29 smithi016 ceph-mon[30972]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:29 smithi016 ceph-mon[30972]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:47:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: Deploying cephadm binary to smithi159 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:31.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:31 smithi016 ceph-mon[30972]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:31 smithi016 ceph-mon[30972]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:31 smithi016 ceph-mon[30972]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:47:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.579+0000 7f277e7fc640 1 -- 172.21.15.16:0/3904294493 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2788075a90 con 0x7f27740390c0 2024-09-12T00:47:32.578 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-09-12T00:47:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 msgr2=0x7f277403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 0x7f277403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f27780098c0 tx=0x7f2778006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 msgr2=0x7f27880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2784009230 tx=0x7f2784003d40 comp rx=0 tx=0).stop 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 shutdown_connections 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f27740390c0 0x7f277403b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 --2- 172.21.15.16:0/3904294493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2788107660 0x7f27880744f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 >> 172.21.15.16:0/3904294493 conn(0x7f27880fd290 msgr2=0x7f27880fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 shutdown_connections 2024-09-12T00:47:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:32.582+0000 7f278f2ab640 1 -- 172.21.15.16:0/3904294493 wait complete. 2024-09-12T00:47:33.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T00:47:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: Added host smithi159 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 -- 172.21.15.16:0/3940929750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 msgr2=0x7f5d64106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 --2- 172.21.15.16:0/3940929750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d64106920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009c10 tx=0x7f5d5402aa10 comp rx=0 tx=0).stop 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 -- 172.21.15.16:0/3940929750 shutdown_connections 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 --2- 172.21.15.16:0/3940929750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d64106920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 -- 172.21.15.16:0/3940929750 >> 172.21.15.16:0/3940929750 conn(0x7f5d64077310 msgr2=0x7f5d640fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 -- 172.21.15.16:0/3940929750 shutdown_connections 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 -- 172.21.15.16:0/3940929750 wait complete. 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.896+0000 7f5d69390640 1 Processor -- start 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d69390640 1 -- start start 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d69390640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d69390640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410ab70 con 0x7f5d64106540 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34852/0 (socket says 172.21.15.16:34852) 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d62ffd640 1 -- 172.21.15.16:0/349313811 learned_addr learned my addr 172.21.15.16:0/349313811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d62ffd640 1 -- 172.21.15.16:0/349313811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d64197610 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.897+0000 7f5d62ffd640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d54005dd0 tx=0x7f5d54030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.898+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d54034070 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.898+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5402f8c0 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.898+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d641978a0 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.898+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d54004520 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.898+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d64197cc0 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5d54037020 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d64101bb0 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 0x7f5d4c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d5406d020 con 0x7f5d64106540 2024-09-12T00:47:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d627fc640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 0x7f5d4c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.899+0000 7f5d627fc640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 0x7f5d4c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098c0 tx=0x7f5d58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:33.904+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d5403f070 con 0x7f5d64106540 2024-09-12T00:47:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.086+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d641984d0 con 0x7f5d4c039060 2024-09-12T00:47:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.087+0000 7f5d4bfff640 1 -- 172.21.15.16:0/349313811 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f5d641984d0 con 0x7f5d4c039060 2024-09-12T00:47:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:47:34.085 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-09-12T00:47:34.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 msgr2=0x7f5d4c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:34.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 0x7f5d4c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098c0 tx=0x7f5d58006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 msgr2=0x7f5d641970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d54005dd0 tx=0x7f5d54030c30 comp rx=0 tx=0).stop 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 shutdown_connections 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5d4c039060 0x7f5d4c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 --2- 172.21.15.16:0/349313811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d64106540 0x7f5d641970d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 >> 172.21.15.16:0/349313811 conn(0x7f5d64077310 msgr2=0x7f5d640696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 shutdown_connections 2024-09-12T00:47:34.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:34.089+0000 7f5d69390640 1 -- 172.21.15.16:0/349313811 wait complete. 2024-09-12T00:47:34.697 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T00:47:34.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T00:47:34.948 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:35.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:35.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:34 smithi016 ceph-mon[30972]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:47:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 -- 172.21.15.16:0/2552648496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 msgr2=0x7fa8cc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/2552648496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc106920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009810 tx=0x7fa8bc02a540 comp rx=0 tx=0).stop 2024-09-12T00:47:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 -- 172.21.15.16:0/2552648496 shutdown_connections 2024-09-12T00:47:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/2552648496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc106920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 -- 172.21.15.16:0/2552648496 >> 172.21.15.16:0/2552648496 conn(0x7fa8cc077310 msgr2=0x7fa8cc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 -- 172.21.15.16:0/2552648496 shutdown_connections 2024-09-12T00:47:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.404+0000 7fa8d2a21640 1 -- 172.21.15.16:0/2552648496 wait complete. 2024-09-12T00:47:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8d2a21640 1 Processor -- start 2024-09-12T00:47:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8d2a21640 1 -- start start 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8d2a21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8d2a21640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc10ab70 con 0x7fa8cc106540 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34880/0 (socket says 172.21.15.16:34880) 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8cbfff640 1 -- 172.21.15.16:0/4207110396 learned_addr learned my addr 172.21.15.16:0/4207110396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.405+0000 7fa8cbfff640 1 -- 172.21.15.16:0/4207110396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc197650 con 0x7fa8cc106540 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8cbfff640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0045e0 tx=0x7fa8bc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8bc030e70 con 0x7fa8cc106540 2024-09-12T00:47:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1978e0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8bc0318e0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8bc0049c0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.406+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc19b9e0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa8bc039040 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8c97fa640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 0x7fa8b403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8bc06d2b0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc101bb0 con 0x7fa8cc106540 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8cb7fe640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 0x7fa8b403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.407+0000 7fa8cb7fe640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 0x7fa8b403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098c0 tx=0x7fa8c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.412+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8bc040030 con 0x7fa8cc106540 2024-09-12T00:47:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.593+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa8cc198470 con 0x7fa8cc106540 2024-09-12T00:47:35.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.948+0000 7fa8c97fa640 1 -- 172.21.15.16:0/4207110396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fa8bc037d70 con 0x7fa8cc106540 2024-09-12T00:47:35.947 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 msgr2=0x7fa8b403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 0x7fa8b403b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00098c0 tx=0x7fa8c0006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 msgr2=0x7fa8cc197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0045e0 tx=0x7fa8bc004610 comp rx=0 tx=0).stop 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 shutdown_connections 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8b4039060 0x7fa8b403b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.951+0000 7fa8d2a21640 1 --2- 172.21.15.16:0/4207110396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc106540 0x7fa8cc197110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.952+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 >> 172.21.15.16:0/4207110396 conn(0x7fa8cc077310 msgr2=0x7fa8cc0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.952+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 shutdown_connections 2024-09-12T00:47:35.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:35.952+0000 7fa8d2a21640 1 -- 172.21.15.16:0/4207110396 wait complete. 2024-09-12T00:47:36.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:35 smithi016 ceph-mon[30972]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:36.256 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4207110396' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T00:47:36.558 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-12T00:47:36.558 INFO:tasks.cephadm:Adding mon.b on smithi099 2024-09-12T00:47:36.558 INFO:tasks.cephadm:Adding mon.c on smithi159 2024-09-12T00:47:36.559 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi099:172.21.15.99=b;smithi159:172.21.15.159=c' 2024-09-12T00:47:36.803 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T00:47:36.848 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4207110396' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:47:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:37.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:36 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.337+0000 7f45445ba640 1 -- 172.21.15.159:0/1133043956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 msgr2=0x7f453c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.337+0000 7f45445ba640 1 --2- 172.21.15.159:0/1133043956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c1008e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4538009c20 tx=0x7f453802aa90 comp rx=0 tx=0).stop 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 -- 172.21.15.159:0/1133043956 shutdown_connections 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 --2- 172.21.15.159:0/1133043956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c1008e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 -- 172.21.15.159:0/1133043956 >> 172.21.15.159:0/1133043956 conn(0x7f453c0fbc90 msgr2=0x7f453c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 -- 172.21.15.159:0/1133043956 shutdown_connections 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 -- 172.21.15.159:0/1133043956 wait complete. 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 Processor -- start 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.338+0000 7f45445ba640 1 -- start start 2024-09-12T00:47:37.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f45445ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f45445ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f453c1065f0 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f4542356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f4542356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.159:45512/0 (socket says 172.21.15.159:45512) 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f4542356640 1 -- 172.21.15.159:0/215683000 learned_addr learned my addr 172.21.15.159:0/215683000 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f4542356640 1 -- 172.21.15.159:0/215683000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c199a90 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.339+0000 7f4542356640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4538005cc0 tx=0x7f453802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.340+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4538047070 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.340+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c199c60 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.340+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4538003e50 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.340+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4538033050 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.340+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c19a000 con 0x7f453c1004e0 2024-09-12T00:47:37.409 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.341+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f453804a070 con 0x7f453c1004e0 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.341+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c106d70 con 0x7f453c1004e0 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.341+0000 7f45337fe640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 0x7f452803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.341+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f453803e030 con 0x7f453c1004e0 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.341+0000 7f4541b55640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 0x7f452803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.342+0000 7f4541b55640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 0x7f452803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098c0 tx=0x7f452c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:37.410 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.346+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4538038450 con 0x7f453c1004e0 2024-09-12T00:47:37.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.534+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi099:172.21.15.99=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}) -- 0x7f453c02db40 con 0x7f4528039060 2024-09-12T00:47:37.540 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.541+0000 7f45337fe640 1 -- 172.21.15.159:0/215683000 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f453c02db40 con 0x7f4528039060 2024-09-12T00:47:37.540 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mon update... 2024-09-12T00:47:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 msgr2=0x7f452803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 0x7f452803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f452c0098c0 tx=0x7f452c006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 msgr2=0x7f453c06a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:37.542 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4538005cc0 tx=0x7f453802fc10 comp rx=0 tx=0).stop 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 shutdown_connections 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4528039060 0x7f452803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 --2- 172.21.15.159:0/215683000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f453c1004e0 0x7f453c06a950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 >> 172.21.15.159:0/215683000 conn(0x7f453c0fbc90 msgr2=0x7f453c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 shutdown_connections 2024-09-12T00:47:37.543 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:37.543+0000 7f45445ba640 1 -- 172.21.15.159:0/215683000 wait complete. 2024-09-12T00:47:38.275 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T00:47:38.278 DEBUG:teuthology.orchestra.run.smithi159:mon.c> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T00:47:38.280 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T00:47:38.280 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:38.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:38.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:38.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:38.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:38 smithi016 ceph-mon[30972]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:38 smithi016 ceph-mon[30972]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:47:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:38 smithi016 ceph-mon[30972]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi099:172.21.15.99=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:47:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:38 smithi016 ceph-mon[30972]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi099:172.21.15.99=b;smithi159:172.21.15.159=c;count:3 2024-09-12T00:47:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:38 smithi016 ceph-mon[30972]: Deploying daemon mon.c on smithi159 2024-09-12T00:47:39.725 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:40.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:39 smithi016 ceph-mon[30972]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- 172.21.15.159:0/139231389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 msgr2=0x7fd01c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 --2- 172.21.15.159:0/139231389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009880 tx=0x7fd00c02a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- 172.21.15.159:0/139231389 shutdown_connections 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 --2- 172.21.15.159:0/139231389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- 172.21.15.159:0/139231389 >> 172.21.15.159:0/139231389 conn(0x7fd01c0fc1b0 msgr2=0x7fd01c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- 172.21.15.159:0/139231389 shutdown_connections 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- 172.21.15.159:0/139231389 wait complete. 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 Processor -- start 2024-09-12T00:47:40.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.514+0000 7fd0222a8640 1 -- start start 2024-09-12T00:47:40.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd0222a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:40.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd0222a8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c10ab90 con 0x7fd01c106560 2024-09-12T00:47:40.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:40.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd01bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.159:45526/0 (socket says 172.21.15.159:45526) 2024-09-12T00:47:40.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd01bfff640 1 -- 172.21.15.159:0/1550779162 learned_addr learned my addr 172.21.15.159:0/1550779162 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.515+0000 7fd01bfff640 1 -- 172.21.15.159:0/1550779162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c196520 con 0x7fd01c106560 2024-09-12T00:47:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd01bfff640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009230 tx=0x7fd00c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd00c03f070 con 0x7fd01c106560 2024-09-12T00:47:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c1967b0 con 0x7fd01c106560 2024-09-12T00:47:40.516 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c03bd40 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd00c035030 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.516+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c19a8b0 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.517+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd00c042070 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.517+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01c101bd0 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.517+0000 7fd0197fa640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 0x7fd00403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.517+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd00c06cd00 con 0x7fd01c106560 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.517+0000 7fd01b7fe640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 0x7fd00403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:40.517 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.518+0000 7fd01b7fe640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 0x7fd00403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0100098c0 tx=0x7fd010006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:40.521 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.522+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd00c036df0 con 0x7fd01c106560 2024-09-12T00:47:40.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.788+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd01c1069c0 con 0x7fd01c106560 2024-09-12T00:47:40.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.788+0000 7fd0197fa640 1 -- 172.21.15.159:0/1550779162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd00c03b860 con 0x7fd01c106560 2024-09-12T00:47:40.788 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:47:40.788 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","modified":"2024-09-12T00:46:08.235575Z","created":"2024-09-12T00:46:08.235575Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T00:47:40.788 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.790+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 msgr2=0x7fd00403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 0x7fd00403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0100098c0 tx=0x7fd010006db0 comp rx=0 tx=0).stop 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 msgr2=0x7fd01c195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009230 tx=0x7fd00c003d80 comp rx=0 tx=0).stop 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 shutdown_connections 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd004039060 0x7fd00403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 --2- 172.21.15.159:0/1550779162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd01c106560 0x7fd01c195fe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:40.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 >> 172.21.15.159:0/1550779162 conn(0x7fd01c0fc1b0 msgr2=0x7fd01c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:40.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 shutdown_connections 2024-09-12T00:47:40.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:40.791+0000 7fd0222a8640 1 -- 172.21.15.159:0/1550779162 wait complete. 2024-09-12T00:47:41.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/1550779162' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:42.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:42 smithi159 systemd[1]: Started Ceph mon.c for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:47:42.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:41 smithi016 ceph-mon[30972]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:42.642 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T00:47:42.642 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T00:47:42.901 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: mon.a calling monitor election 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: mon.c calling monitor election 2024-09-12T00:47:47.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: monmap epoch 2 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: last_changed 2024-09-12T00:47:42.111035+0000 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: min_mon_release 19 (squid) 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: election_strategy: 1 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:47.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: fsmap 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: mgrmap e13: x(active, since 38s) 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:47.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: mon.a calling monitor election 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: mon.c calling monitor election 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: monmap epoch 2 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: last_changed 2024-09-12T00:47:42.111035+0000 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: election_strategy: 1 2024-09-12T00:47:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: fsmap 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: mgrmap e13: x(active, since 38s) 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:47.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:47.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 -- 172.21.15.159:0/3799584747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368003530 msgr2=0x7f93680059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:47.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 --2- 172.21.15.159:0/3799584747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368003530 0x7f93680059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f937c06a9e0 tx=0x7f937802bfe0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 -- 172.21.15.159:0/3799584747 shutdown_connections 2024-09-12T00:47:47.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 --2- 172.21.15.159:0/3799584747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9368003530 0x7f93680059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.743 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 -- 172.21.15.159:0/3799584747 >> 172.21.15.159:0/3799584747 conn(0x7f937c0fc220 msgr2=0x7f937c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:47.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 -- 172.21.15.159:0/3799584747 shutdown_connections 2024-09-12T00:47:47.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.743+0000 7f93837f7640 1 -- 172.21.15.159:0/3799584747 wait complete. 2024-09-12T00:47:47.746 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.744+0000 7f93837f7640 1 Processor -- start 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f93837f7640 1 -- start start 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f93837f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f93837f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 0x7f937c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f93837f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1097a0 con 0x7f937c197810 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f93837f7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f937c109620 con 0x7f9368003530 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9380d92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 0x7f937c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42282/0 (socket says 172.21.15.159:42282) 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 -- 172.21.15.159:0/3548953534 learned_addr learned my addr 172.21.15.159:0/3548953534 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:47.747 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 -- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 msgr2=0x7f937c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 -- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 msgr2=0x7f937c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9381593640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9380d92640 1 -- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 msgr2=0x7f937c1972d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9380d92640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.745+0000 7f9380d92640 1 -- 172.21.15.159:0/3548953534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c197e30 con 0x7f937c197810 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f9380d92640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 0x7f937c19bba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b3d0 tx=0x7f936c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f936c011630 con 0x7f937c197810 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c19c0e0 con 0x7f937c197810 2024-09-12T00:47:47.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c011cb0 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f936c010df0 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.746+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c19c620 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.747+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f936c018020 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.747+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c06ac90 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.747+0000 7f93727fc640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 0x7f936003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.747+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f936c04bcb0 con 0x7f937c197810 2024-09-12T00:47:47.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.748+0000 7f9381593640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 0x7f936003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:47.750 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.748+0000 7f9381593640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 0x7f936003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b30 tx=0x7f9378005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:47.756 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.757+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f936c01f020 con 0x7f937c197810 2024-09-12T00:47:47.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.993+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f937c10a8b0 con 0x7f937c197810 2024-09-12T00:47:47.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.993+0000 7f93727fc640 1 -- 172.21.15.159:0/3548953534 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f936c018310 con 0x7f937c197810 2024-09-12T00:47:47.993 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:47:47.993 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","modified":"2024-09-12T00:47:42.111035Z","created":"2024-09-12T00:46:08.235575Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T00:47:47.993 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2024-09-12T00:47:47.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 msgr2=0x7f936003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:47.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 0x7f936003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b30 tx=0x7f9378005f60 comp rx=0 tx=0).stop 2024-09-12T00:47:47.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 msgr2=0x7f937c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 0x7f937c19bba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b3d0 tx=0x7f936c00b890 comp rx=0 tx=0).stop 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 shutdown_connections 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9360038e00 0x7f936003b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f937c197810 0x7f937c19bba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 --2- 172.21.15.159:0/3548953534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9368003530 0x7f937c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 >> 172.21.15.159:0/3548953534 conn(0x7f937c0fc220 msgr2=0x7f937c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 shutdown_connections 2024-09-12T00:47:47.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:47.995+0000 7f93837f7640 1 -- 172.21.15.159:0/3548953534 wait complete. 2024-09-12T00:47:48.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:48 smithi159 ceph-mon[35566]: Deploying daemon mon.b on smithi099 2024-09-12T00:47:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:48 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/3548953534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:48 smithi016 ceph-mon[30972]: Deploying daemon mon.b on smithi099 2024-09-12T00:47:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:48 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/3548953534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:49 smithi016 ceph-mon[30972]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:49.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:49 smithi159 ceph-mon[35566]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:49.622 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T00:47:49.623 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T00:47:49.878 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:47:50.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 -- 172.21.15.159:0/2741430389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 msgr2=0x7f0058106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:50.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 --2- 172.21.15.159:0/2741430389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f0058106a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0048009880 tx=0x7f004802a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 -- 172.21.15.159:0/2741430389 shutdown_connections 2024-09-12T00:47:50.504 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 --2- 172.21.15.159:0/2741430389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f0058106a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 -- 172.21.15.159:0/2741430389 >> 172.21.15.159:0/2741430389 conn(0x7f00580fc280 msgr2=0x7f00580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 -- 172.21.15.159:0/2741430389 shutdown_connections 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 -- 172.21.15.159:0/2741430389 wait complete. 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.503+0000 7f005e179640 1 Processor -- start 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f005e179640 1 -- start start 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f005e179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f005e179640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f005e179640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f005810ac80 con 0x7f0058106650 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f005e179640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f005810ab00 con 0x7f0058196810 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f00577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:50.505 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f00577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.159:42916/0 (socket says 172.21.15.159:42916) 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42284/0 (socket says 172.21.15.159:42284) 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f00577fe640 1 -- 172.21.15.159:0/3528660355 learned_addr learned my addr 172.21.15.159:0/3528660355 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 -- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 msgr2=0x7f005819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 -- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 msgr2=0x7f005819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f0056ffd640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f00577fe640 1 -- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 msgr2=0x7f005819aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.504+0000 7f00577fe640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f00577fe640 1 -- 172.21.15.159:0/3528660355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0058196e30 con 0x7f0058106650 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f00577fe640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0048004650 tx=0x7f0048032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0048035070 con 0x7f0058106650 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005819b0e0 con 0x7f0058106650 2024-09-12T00:47:50.506 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048003ce0 con 0x7f0058106650 2024-09-12T00:47:50.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f004803bbd0 con 0x7f0058106650 2024-09-12T00:47:50.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.505+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005819b640 con 0x7f0058106650 2024-09-12T00:47:50.507 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.506+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0048042030 con 0x7f0058106650 2024-09-12T00:47:50.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.506+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0058101cc0 con 0x7f0058106650 2024-09-12T00:47:50.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.506+0000 7f0054ff9640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 0x7f004403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:50.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.506+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0048040070 con 0x7f0058106650 2024-09-12T00:47:50.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.506+0000 7f0056ffd640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 0x7f004403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:50.508 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.507+0000 7f0056ffd640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 0x7f004403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0040006fd0 tx=0x7f0040008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:50.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.511+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f004803bd70 con 0x7f0058106650 2024-09-12T00:47:50.744 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.745+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f005819b8f0 con 0x7f0058106650 2024-09-12T00:47:50.745 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.746+0000 7f0054ff9640 1 -- 172.21.15.159:0/3528660355 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f0048043c60 con 0x7f0058106650 2024-09-12T00:47:50.745 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:47:50.745 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","modified":"2024-09-12T00:47:42.111035Z","created":"2024-09-12T00:46:08.235575Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T00:47:50.745 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 msgr2=0x7f004403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 0x7f004403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0040006fd0 tx=0x7f0040008040 comp rx=0 tx=0).stop 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 msgr2=0x7f00581962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0048004650 tx=0x7f0048032c50 comp rx=0 tx=0).stop 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 shutdown_connections 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0044038e00 0x7f004403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0058196810 0x7f005819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 --2- 172.21.15.159:0/3528660355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0058106650 0x7f00581962d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:50.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 >> 172.21.15.159:0/3528660355 conn(0x7f00580fc280 msgr2=0x7f00580fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:50.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 shutdown_connections 2024-09-12T00:47:50.749 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:50.748+0000 7f005e179640 1 -- 172.21.15.159:0/3528660355 wait complete. 2024-09-12T00:47:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:50 smithi099 systemd[1]: Started Ceph mon.b for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:47:52.359 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T00:47:52.360 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T00:47:52.621 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:47:56.065 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.065+0000 7f6f1e314640 1 -- 172.21.15.159:0/1944889893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 msgr2=0x7f6f000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.065+0000 7f6f1e314640 1 --2- 172.21.15.159:0/1944889893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f000059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6f181016c0 tx=0x7f6f0802bce0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.065+0000 7f6f1e314640 1 -- 172.21.15.159:0/1944889893 shutdown_connections 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.065+0000 7f6f1e314640 1 --2- 172.21.15.159:0/1944889893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f000059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.066+0000 7f6f1e314640 1 -- 172.21.15.159:0/1944889893 >> 172.21.15.159:0/1944889893 conn(0x7f6f180fc280 msgr2=0x7f6f180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.066+0000 7f6f1e314640 1 -- 172.21.15.159:0/1944889893 shutdown_connections 2024-09-12T00:47:56.066 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.066+0000 7f6f1e314640 1 -- 172.21.15.159:0/1944889893 wait complete. 2024-09-12T00:47:56.067 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.066+0000 7f6f1e314640 1 Processor -- start 2024-09-12T00:47:56.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.066+0000 7f6f1e314640 1 -- start start 2024-09-12T00:47:56.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f181973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:56.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 0x7f6f1819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:56.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:56.071 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18108680 con 0x7f6f00003530 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18108500 con 0x7f6f18197910 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1e314640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18108800 con 0x7f6f1819c1e0 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46920/0 (socket says 172.21.15.159:46920) 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 -- 172.21.15.159:0/3651789002 learned_addr learned my addr 172.21.15.159:0/3651789002 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f181973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 msgr2=0x7f6f1819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 msgr2=0x7f6f1819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f1c8b1640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:47:56.072 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f177fe640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 0x7f6f1819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 msgr2=0x7f6f1819c690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 msgr2=0x7f6f1819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 0x7f6f1819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f17fff640 1 -- 172.21.15.159:0/3651789002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f181a3f50 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.067+0000 7f6f177fe640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 0x7f6f1819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f17fff640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f181973d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011c00 tx=0x7f6f0802db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08005460 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f08003ad0 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f181a4120 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004dd0 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.068+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181a46b0 con 0x7f6f00003530 2024-09-12T00:47:56.073 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.069+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6f08037480 con 0x7f6f00003530 2024-09-12T00:47:56.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.069+0000 7f6f157fa640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 0x7f6f0403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:56.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.069+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f0806d870 con 0x7f6f00003530 2024-09-12T00:47:56.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.069+0000 7f6f177fe640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 0x7f6f0403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:56.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.069+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f181097f0 con 0x7f6f00003530 2024-09-12T00:47:56.074 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.070+0000 7f6f177fe640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 0x7f6f0403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007920 tx=0x7f6f0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:56.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.074+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f08040600 con 0x7f6f00003530 2024-09-12T00:47:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: mon.c calling monitor election 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: mon.a calling monitor election 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T00:47:56.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: monmap epoch 3 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: election_strategy: 1 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: fsmap 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:56.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: mgrmap e13: x(active, since 47s) 2024-09-12T00:47:56.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T00:47:56.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.146 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.324+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6f181a48e0 con 0x7f6f00003530 2024-09-12T00:47:56.324 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.325+0000 7f6f157fa640 1 -- 172.21.15.159:0/3651789002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f6f080403a0 con 0x7f6f00003530 2024-09-12T00:47:56.324 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.324 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: mon.c calling monitor election 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: mon.a calling monitor election 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: monmap epoch 3 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: min_mon_release 19 (squid) 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: election_strategy: 1 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: fsmap 2024-09-12T00:47:56.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: mgrmap e13: x(active, since 47s) 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:56.327 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:47:56.327 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","modified":"2024-09-12T00:47:50.814337Z","created":"2024-09-12T00:46:08.235575Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T00:47:56.328 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 3 2024-09-12T00:47:56.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 msgr2=0x7f6f0403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:56.328 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 0x7f6f0403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007920 tx=0x7f6f0c008040 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 msgr2=0x7f6f181973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f181973d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f08011c00 tx=0x7f6f0802db00 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 shutdown_connections 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6f040394f0 0x7f6f0403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f1819c1e0 0x7f6f1819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f18197910 0x7f6f1819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 --2- 172.21.15.159:0/3651789002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f00003530 0x7f6f181973d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 >> 172.21.15.159:0/3651789002 conn(0x7f6f180fc280 msgr2=0x7f6f181059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 shutdown_connections 2024-09-12T00:47:56.329 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:56.327+0000 7f6f1e314640 1 -- 172.21.15.159:0/3651789002 wait complete. 2024-09-12T00:47:56.867 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T00:47:56.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T00:47:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:56 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/3651789002' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:47:57.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:57.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:56 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/3651789002' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:57.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 -- 172.21.15.16:0/2827505584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 msgr2=0x7ff05c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 --2- 172.21.15.16:0/2827505584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c107a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff058009880 tx=0x7ff05802a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 -- 172.21.15.16:0/2827505584 shutdown_connections 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 --2- 172.21.15.16:0/2827505584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c107a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 -- 172.21.15.16:0/2827505584 >> 172.21.15.16:0/2827505584 conn(0x7ff05c0fd2b0 msgr2=0x7ff05c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 -- 172.21.15.16:0/2827505584 shutdown_connections 2024-09-12T00:47:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.613+0000 7ff063f30640 1 -- 172.21.15.16:0/2827505584 wait complete. 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 Processor -- start 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 -- start start 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 0x7ff05c074e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 0x7ff05c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10bc40 con 0x7ff05c107660 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10bac0 con 0x7ff05c1a3550 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff063f30640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c10bdc0 con 0x7ff05c074a30 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff061ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff061ccc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39514/0 (socket says 172.21.15.16:39514) 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.614+0000 7ff061ccc640 1 -- 172.21.15.16:0/3306608612 learned_addr learned my addr 172.21.15.16:0/3306608612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:47:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff0614cb640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 0x7ff05c074e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff0624cd640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 0x7ff05c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 -- 172.21.15.16:0/3306608612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 msgr2=0x7ff05c074e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 0x7ff05c074e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 -- 172.21.15.16:0/3306608612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 msgr2=0x7ff05c1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 0x7ff05c1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 -- 172.21.15.16:0/3306608612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c1a6010 con 0x7ff05c107660 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff0624cd640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 0x7ff05c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff0614cb640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 0x7ff05c074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff061ccc640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff058031550 tx=0x7ff0580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05803f030 con 0x7ff05c107660 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff058003ce0 con 0x7ff05c107660 2024-09-12T00:47:57.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1a62a0 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058035070 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.615+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a67d0 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.616+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff058042030 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.616+0000 7ff052ffd640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 0x7ff04803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.616+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff05806cd80 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.616+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c107ac0 con 0x7ff05c107660 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.617+0000 7ff0614cb640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 0x7ff04803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:57.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.617+0000 7ff0614cb640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 0x7ff04803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009890 tx=0x7ff04c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.622+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff05803bce0 con 0x7ff05c107660 2024-09-12T00:47:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.800+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff05c1a6ad0 con 0x7ff05c107660 2024-09-12T00:47:57.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.801+0000 7ff052ffd640 1 -- 172.21.15.16:0/3306608612 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ff058043460 con 0x7ff05c107660 2024-09-12T00:47:57.801 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:57.801 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-12T00:47:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:57.801 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-09-12T00:47:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.803+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 msgr2=0x7ff04803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.803+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 0x7ff04803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009890 tx=0x7ff04c008040 comp rx=0 tx=0).stop 2024-09-12T00:47:57.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.803+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 msgr2=0x7ff05c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.803+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff058031550 tx=0x7ff0580042f0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 shutdown_connections 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff0480394f0 0x7ff04803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff05c1a3550 0x7ff05c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c074a30 0x7ff05c074e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 --2- 172.21.15.16:0/3306608612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff05c107660 0x7ff05c0744f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 >> 172.21.15.16:0/3306608612 conn(0x7ff05c0fd2b0 msgr2=0x7ff05c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 shutdown_connections 2024-09-12T00:47:57.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:47:57.804+0000 7ff063f30640 1 -- 172.21.15.16:0/3306608612 wait complete. 2024-09-12T00:47:58.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:58.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:47:58.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: mon.c calling monitor election 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: mon.a calling monitor election 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T00:47:58.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: monmap epoch 3 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: min_mon_release 19 (squid) 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: election_strategy: 1 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: fsmap 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: mgrmap e13: x(active, since 47s) 2024-09-12T00:47:58.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: overall HEALTH_OK 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/3651789002' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T00:47:58.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:58.508 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T00:47:58.509 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:58.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T00:47:58.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:47:58.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:58.604 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:47:58.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T00:47:58.627 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mon.b calling monitor election 2024-09-12T00:47:58.627 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mon.b calling monitor election 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mon.c calling monitor election 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mon.a calling monitor election 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: monmap epoch 3 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:58.628 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: min_mon_release 19 (squid) 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: election_strategy: 1 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: fsmap 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: mgrmap e13: x(active, since 49s) 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: overall HEALTH_OK 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:47:58.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:58.631 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:47:58.632 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:58.696 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:47:58.696 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T00:47:58.725 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:47:58.725 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:47:58.788 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mon.b calling monitor election 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mon.b calling monitor election 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mon.c calling monitor election 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mon.a calling monitor election 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: monmap epoch 3 2024-09-12T00:47:58.789 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: min_mon_release 19 (squid) 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: election_strategy: 1 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: fsmap 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: mgrmap e13: x(active, since 49s) 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.790 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:47:58.791 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:58.792 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-12T00:47:58.793 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-09-12T00:47:58.793 INFO:tasks.cephadm:Adding mgr.z on smithi159 2024-09-12T00:47:58.793 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi099=y;smithi159=z' 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mon.b calling monitor election 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mon.b calling monitor election 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mon.c calling monitor election 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mon.a calling monitor election 2024-09-12T00:47:58.851 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: monmap epoch 3 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: last_changed 2024-09-12T00:47:50.814337+0000 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: created 2024-09-12T00:46:08.235575+0000 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: min_mon_release 19 (squid) 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: election_strategy: 1 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: fsmap 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: mgrmap e13: x(active, since 49s) 2024-09-12T00:47:58.852 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:47:58.853 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:47:59.069 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.485+0000 7f2b872df640 1 -- 172.21.15.159:0/2414186590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801021f0 msgr2=0x7f2b801045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.485+0000 7f2b872df640 1 --2- 172.21.15.159:0/2414186590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801021f0 0x7f2b801045e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009880 tx=0x7f2b7c02a5d0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- 172.21.15.159:0/2414186590 shutdown_connections 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 --2- 172.21.15.159:0/2414186590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801021f0 0x7f2b801045e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- 172.21.15.159:0/2414186590 >> 172.21.15.159:0/2414186590 conn(0x7f2b800fbdb0 msgr2=0x7f2b800fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- 172.21.15.159:0/2414186590 shutdown_connections 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- 172.21.15.159:0/2414186590 wait complete. 2024-09-12T00:47:59.488 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 Processor -- start 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- start start 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 0x7f2b8019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 0x7f2b8019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80107790 con 0x7f2b801974b0 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80107610 con 0x7f2b80197930 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.486+0000 7f2b872df640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80107910 con 0x7f2b801021f0 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46936/0 (socket says 172.21.15.159:46936) 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 -- 172.21.15.159:0/4051531407 learned_addr learned my addr 172.21.15.159:0/4051531407 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8487a640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 0x7f2b8019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:59.489 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8587c640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 0x7f2b8019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 -- 172.21.15.159:0/4051531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 msgr2=0x7f2b8019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 0x7f2b8019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 -- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 msgr2=0x7f2b8019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 0x7f2b8019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 -- 172.21.15.159:0/4051531407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8019f370 con 0x7f2b801021f0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8587c640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 0x7f2b8019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8487a640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 0x7f2b8019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b8507b640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c000c00 tx=0x7f2b7c004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c03f030 con 0x7f2b801021f0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8019f600 con 0x7f2b801021f0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8019fbb0 con 0x7f2b801021f0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.487+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c003ce0 con 0x7f2b801021f0 2024-09-12T00:47:59.490 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.488+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c035070 con 0x7f2b801021f0 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.488+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2b7c042030 con 0x7f2b801021f0 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.489+0000 7f2b767fc640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 0x7f2b6c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.489+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b640052c0 con 0x7f2b801021f0 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.489+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2b7c06c610 con 0x7f2b801021f0 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.489+0000 7f2b8487a640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 0x7f2b6c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:47:59.491 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.489+0000 7f2b8487a640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 0x7f2b6c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006fd0 tx=0x7f2b70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:47:59.493 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.494+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b7c02fdf0 con 0x7f2b801021f0 2024-09-12T00:47:59.669 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.670+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi099=y;smithi159=z", "target": ["mon-mgr", ""]}) -- 0x7f2b64000bf0 con 0x7f2b6c0394f0 2024-09-12T00:47:59.674 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.675+0000 7f2b767fc640 1 -- 172.21.15.159:0/4051531407 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2b64000bf0 con 0x7f2b6c0394f0 2024-09-12T00:47:59.674 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mgr update... 2024-09-12T00:47:59.676 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 msgr2=0x7f2b6c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 0x7f2b6c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2b70006fd0 tx=0x7f2b70008040 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 msgr2=0x7f2b80196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c000c00 tx=0x7f2b7c004430 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 shutdown_connections 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0394f0 0x7f2b6c03b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b80197930 0x7f2b8019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b801974b0 0x7f2b8019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 --2- 172.21.15.159:0/4051531407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b801021f0 0x7f2b80196f70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 >> 172.21.15.159:0/4051531407 conn(0x7f2b800fbdb0 msgr2=0x7f2b800fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 shutdown_connections 2024-09-12T00:47:59.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:47:59.677+0000 7f2b872df640 1 -- 172.21.15.159:0/4051531407 wait complete. 2024-09-12T00:47:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:59 smithi099 ceph-mon[35654]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:59 smithi099 ceph-mon[35654]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T00:47:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:59 smithi099 ceph-mon[35654]: Reconfiguring daemon mon.a on smithi016 2024-09-12T00:47:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:47:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:59 smithi159 ceph-mon[35566]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:59 smithi159 ceph-mon[35566]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T00:47:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:59 smithi159 ceph-mon[35566]: Reconfiguring daemon mon.a on smithi016 2024-09-12T00:47:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:47:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:47:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:59 smithi016 ceph-mon[30972]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:47:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:59 smithi016 ceph-mon[30972]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T00:47:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:59 smithi016 ceph-mon[30972]: Reconfiguring daemon mon.a on smithi016 2024-09-12T00:47:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:47:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:48:00.279 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T00:48:00.281 DEBUG:teuthology.orchestra.run.smithi159:mgr.z> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T00:48:00.283 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T00:48:00.283 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:48:00.283 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T00:48:00.304 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T00:48:00.304 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-12T00:48:00.363 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T00:48:00.363 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:00.363 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T00:48:00.363 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:00.363 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:00.364 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 00:47:19.208411644 +0000 2024-09-12T00:48:00.364 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 00:42:47.506083061 +0000 2024-09-12T00:48:00.364 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 00:42:47.506083061 +0000 2024-09-12T00:48:00.364 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 00:42:47.506083061 +0000 2024-09-12T00:48:00.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T00:48:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T00:48:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T00:48:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000127326 s, 4.0 MB/s 2024-09-12T00:48:00.435 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T00:48:00.495 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 00:47:19.245411053 +0000 2024-09-12T00:48:00.554 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 00:42:47.819078046 +0000 2024-09-12T00:48:00.555 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 00:42:47.819078046 +0000 2024-09-12T00:48:00.555 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 00:42:47.819078046 +0000 2024-09-12T00:48:00.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T00:48:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T00:48:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T00:48:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000125201 s, 4.1 MB/s 2024-09-12T00:48:00.623 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T00:48:00.682 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-12T00:48:00.739 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 00:47:19.280410494 +0000 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 00:42:48.179072277 +0000 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 00:42:48.179072277 +0000 2024-09-12T00:48:00.740 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 00:42:48.179072277 +0000 2024-09-12T00:48:00.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T00:48:00.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: Reconfiguring mon.b (monmap changed)... 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: Reconfiguring daemon mon.b on smithi099 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi099=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: Saving service mgr spec with placement smithi016=x;smithi099=y;smithi159=z;count:3 2024-09-12T00:48:00.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T00:48:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T00:48:00.807 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000125168 s, 4.1 MB/s 2024-09-12T00:48:00.808 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T00:48:00.866 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-12T00:48:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: Reconfiguring mon.b (monmap changed)... 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: Reconfiguring daemon mon.b on smithi099 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi099=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: Saving service mgr spec with placement smithi016=x;smithi099=y;smithi159=z;count:3 2024-09-12T00:48:00.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: Reconfiguring mon.b (monmap changed)... 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: Reconfiguring daemon mon.b on smithi099 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi099=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:00.908 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: Saving service mgr spec with placement smithi016=x;smithi099=y;smithi159=z;count:3 2024-09-12T00:48:00.909 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:00.922 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 00:47:19.315409935 +0000 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 00:42:48.519066830 +0000 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 00:42:48.519066830 +0000 2024-09-12T00:48:00.923 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 00:42:48.519066830 +0000 2024-09-12T00:48:00.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T00:48:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-12T00:48:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-12T00:48:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000115375 s, 4.4 MB/s 2024-09-12T00:48:00.990 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T00:48:01.047 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-12T00:48:01.047 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T00:48:01.064 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T00:48:01.064 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:01.120 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-12 00:47:50.263504035 +0000 2024-09-12T00:48:01.121 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-12 00:42:55.557336721 +0000 2024-09-12T00:48:01.121 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-12 00:42:55.557336721 +0000 2024-09-12T00:48:01.121 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-12 00:42:55.557336721 +0000 2024-09-12T00:48:01.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T00:48:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-12T00:48:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-12T00:48:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000253548 s, 2.0 MB/s 2024-09-12T00:48:01.185 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T00:48:01.242 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-12T00:48:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T00:48:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-12 00:47:50.263504035 +0000 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-12 00:42:55.886328718 +0000 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-12 00:42:55.886328718 +0000 2024-09-12T00:48:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-12 00:42:55.886328718 +0000 2024-09-12T00:48:01.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T00:48:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-12T00:48:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-12T00:48:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000215827 s, 2.4 MB/s 2024-09-12T00:48:01.362 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T00:48:01.419 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-12T00:48:01.474 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T00:48:01.474 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:01.474 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T00:48:01.474 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:01.474 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:01.475 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-12 00:47:50.264504012 +0000 2024-09-12T00:48:01.475 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-12 00:42:56.206320933 +0000 2024-09-12T00:48:01.475 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-12 00:42:56.206320933 +0000 2024-09-12T00:48:01.475 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-12 00:42:56.206320933 +0000 2024-09-12T00:48:01.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T00:48:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-12T00:48:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-12T00:48:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000234643 s, 2.2 MB/s 2024-09-12T00:48:01.539 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T00:48:01.595 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-12T00:48:01.650 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-12 00:47:50.264504012 +0000 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-12 00:42:56.531313027 +0000 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-12 00:42:56.531313027 +0000 2024-09-12T00:48:01.651 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-12 00:42:56.531313027 +0000 2024-09-12T00:48:01.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T00:48:01.714 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-12T00:48:01.714 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-12T00:48:01.714 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000198113 s, 2.6 MB/s 2024-09-12T00:48:01.715 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T00:48:01.771 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-12T00:48:01.771 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T00:48:01.788 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T00:48:01.788 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:01.843 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-12 00:47:41.352720418 +0000 2024-09-12T00:48:01.844 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-12 00:42:56.878771567 +0000 2024-09-12T00:48:01.844 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-12 00:42:56.878771567 +0000 2024-09-12T00:48:01.844 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-12 00:42:56.878771567 +0000 2024-09-12T00:48:01.844 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T00:48:01.909 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-12T00:48:01.909 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-12T00:48:01.909 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000276278 s, 1.9 MB/s 2024-09-12T00:48:01.910 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T00:48:01.967 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-09-12T00:48:02.021 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-12 00:47:41.353720399 +0000 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-12 00:42:57.205765247 +0000 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-12 00:42:57.205765247 +0000 2024-09-12T00:48:02.022 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-12 00:42:57.205765247 +0000 2024-09-12T00:48:02.022 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: Reconfiguring mon.c (monmap changed)... 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:02.084 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:02.085 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:02.085 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:01 smithi159 ceph-mon[35566]: Reconfiguring daemon mon.c on smithi159 2024-09-12T00:48:02.086 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-12T00:48:02.087 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-12T00:48:02.087 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000232096 s, 2.2 MB/s 2024-09-12T00:48:02.088 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: Reconfiguring mon.c (monmap changed)... 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:01 smithi016 ceph-mon[30972]: Reconfiguring daemon mon.c on smithi159 2024-09-12T00:48:02.146 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-12 00:47:41.353720399 +0000 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-12 00:42:57.554758502 +0000 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-12 00:42:57.554758502 +0000 2024-09-12T00:48:02.202 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-12 00:42:57.554758502 +0000 2024-09-12T00:48:02.203 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T00:48:02.268 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-12T00:48:02.268 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-12T00:48:02.268 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000177219 s, 2.9 MB/s 2024-09-12T00:48:02.269 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T00:48:02.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: Reconfiguring mon.c (monmap changed)... 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:02.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:01 smithi099 ceph-mon[35654]: Reconfiguring daemon mon.c on smithi159 2024-09-12T00:48:02.326 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-12 00:47:41.354720381 +0000 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-12 00:42:57.874752317 +0000 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-12 00:42:57.874752317 +0000 2024-09-12T00:48:02.382 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-12 00:42:57.874752317 +0000 2024-09-12T00:48:02.383 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T00:48:02.445 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-12T00:48:02.446 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-12T00:48:02.446 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000171725 s, 3.0 MB/s 2024-09-12T00:48:02.447 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T00:48:02.502 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-12T00:48:02.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T00:48:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:02 smithi016 ceph-mon[30972]: Deploying daemon mgr.z on smithi159 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:02 smithi099 ceph-mon[35654]: Deploying daemon mgr.z on smithi159 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:03.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:03.354 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:03.354 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:03.354 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:02 smithi159 ceph-mon[35566]: Deploying daemon mgr.z on smithi159 2024-09-12T00:48:03.967 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:03 smithi159 systemd[1]: Starting Ceph mgr.z for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:48:04.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:03 smithi016 ceph-mon[30972]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:04.235 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:03 smithi159 ceph-mon[35566]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:04.235 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:03 smithi159 podman[37359]: 2024-09-12 00:48:03.990016317 +0000 UTC m=+0.108444658 container create 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T00:48:04.236 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:03 smithi159 podman[37359]: 2024-09-12 00:48:03.897553259 +0000 UTC m=+0.015981569 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:48:04.236 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 podman[37359]: 2024-09-12 00:48:04.209266245 +0000 UTC m=+0.327694540 container init 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T00:48:04.236 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 podman[37359]: 2024-09-12 00:48:04.212486611 +0000 UTC m=+0.330914910 container start 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T00:48:04.236 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-mgr[37378]: -- 172.21.15.159:0/4288244487 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572adb950e0 con 0x5572adb73000 2024-09-12T00:48:04.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:03 smithi099 ceph-mon[35654]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:04.545 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 bash[37359]: 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 2024-09-12T00:48:04.545 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 systemd[1]: Started Ceph mgr.z for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:48:04.545 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.335+0000 7ff029414040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:48:04.545 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.396+0000 7ff029414040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:48:04.545 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.545+0000 7ff029414040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:48:04.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.612+0000 7ff029414040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:48:04.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.760+0000 7ff029414040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:48:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:48:05.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-12T00:48:05.220 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:04.901+0000 7ff029414040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:48:05.220 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:05.221+0000 7ff029414040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:48:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:05.564 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:05 smithi016 ceph-mon[30972]: Deploying daemon mgr.y on smithi099 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:05.282+0000 7ff029414040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:05.469+0000 7ff029414040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:05.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:05.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:05.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:05.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:05 smithi159 ceph-mon[35566]: Deploying daemon mgr.y on smithi099 2024-09-12T00:48:05.683 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:05.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:05 smithi099 ceph-mon[35654]: Deploying daemon mgr.y on smithi099 2024-09-12T00:48:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.889+0000 7fc9e5840640 1 -- 172.21.15.16:0/998176716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 msgr2=0x7fc9e0104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.889+0000 7fc9e5840640 1 --2- 172.21.15.16:0/998176716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e0104a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009900 tx=0x7fc9d002a410 comp rx=0 tx=0).stop 2024-09-12T00:48:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.889+0000 7fc9e5840640 1 -- 172.21.15.16:0/998176716 shutdown_connections 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 --2- 172.21.15.16:0/998176716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 0x7fc9e0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 --2- 172.21.15.16:0/998176716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 0x7fc9e01073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 --2- 172.21.15.16:0/998176716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e0104a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 -- 172.21.15.16:0/998176716 >> 172.21.15.16:0/998176716 conn(0x7fc9e00fc2a0 msgr2=0x7fc9e00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 -- 172.21.15.16:0/998176716 shutdown_connections 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 -- 172.21.15.16:0/998176716 wait complete. 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.890+0000 7fc9e5840640 1 Processor -- start 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 -- start start 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 0x7fc9e019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 0x7fc9e01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0100ee0 con 0x7fc9e01026a0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0100d60 con 0x7fc9e0107900 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9e5840640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0101060 con 0x7fc9e0104fd0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49816/0 (socket says 172.21.15.16:49816) 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 -- 172.21.15.16:0/3375912585 learned_addr learned my addr 172.21.15.16:0/3375912585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9de7fc640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 0x7fc9e019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9df7fe640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 0x7fc9e01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 -- 172.21.15.16:0/3375912585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 msgr2=0x7fc9e019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 0x7fc9e019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 -- 172.21.15.16:0/3375912585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 msgr2=0x7fc9e01a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 0x7fc9e01a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 -- 172.21.15.16:0/3375912585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e01a1d90 con 0x7fc9e01026a0 2024-09-12T00:48:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.891+0000 7fc9deffd640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009230 tx=0x7fc9d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d003f030 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d0003ce0 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e01a2020 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0035070 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a2500 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.892+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc9d0042030 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.893+0000 7fc9e483e640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 0x7fc9c803b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.893+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e01a2820 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.893+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc9d006c850 con 0x7fc9e01026a0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.893+0000 7fc9de7fc640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 0x7fc9c803b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.893+0000 7fc9de7fc640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 0x7fc9c803b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9e019b680 tx=0x7fc9d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:05.898+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9d0030950 con 0x7fc9e01026a0 2024-09-12T00:48:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:06.084+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc9e0109460 con 0x7fc9c80392c0 2024-09-12T00:48:06.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 systemd[1]: Starting Ceph mgr.y for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:48:06.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-mon[35654]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 podman[37000]: 2024-09-12 00:48:06.238907959 +0000 UTC m=+0.015528542 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 podman[37000]: 2024-09-12 00:48:06.336912258 +0000 UTC m=+0.113532846 container create a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 podman[37000]: 2024-09-12 00:48:06.564328398 +0000 UTC m=+0.340948984 container init a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 podman[37000]: 2024-09-12 00:48:06.56705292 +0000 UTC m=+0.343673502 container start a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T00:48:06.591 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-mgr[37019]: -- 172.21.15.99:0/499784159 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb7e6f90e0 con 0x55eb7e6d7000 2024-09-12T00:48:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-mon[35566]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:06.617 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:06.360+0000 7ff029414040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:48:06.617 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:06.429+0000 7ff029414040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:48:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:06 smithi016 ceph-mon[30972]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:06.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 bash[37000]: a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 2024-09-12T00:48:06.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 systemd[1]: Started Ceph mgr.y for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:48:06.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:06.689+0000 7f02282b5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:48:06.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:06.749+0000 7f02282b5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:48:06.905 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:06.906+0000 7f02282b5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:48:07.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:06.974+0000 7f02282b5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:48:07.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:07.135+0000 7f02282b5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:48:07.469 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.204+0000 7ff029414040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:48:07.469 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.323+0000 7ff029414040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:48:07.470 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.470+0000 7ff029414040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:48:07.603 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:07.283+0000 7f02282b5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:48:07.754 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:07.754 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.535+0000 7ff029414040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.685+0000 7ff029414040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:48:07.755 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.755+0000 7ff029414040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:48:07.866 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:07.604+0000 7f02282b5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:07.666+0000 7f02282b5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.867 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:07.868 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:08.055 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.815+0000 7ff029414040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:48:08.055 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:07.987+0000 7ff029414040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:48:08.055 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:08.055+0000 7ff029414040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:08.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:07.868+0000 7f02282b5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:48:08.325 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:08.122+0000 7ff029414040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:48:08.325 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:08.206+0000 7ff029414040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:48:08.325 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:08.266+0000 7ff029414040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:48:08.325 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:08.326+0000 7ff029414040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:48:08.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]: dispatch 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]': finished 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T00:48:08.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:08.751+0000 7f02282b5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:08.817+0000 7f02282b5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]: dispatch 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]': finished 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T00:48:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: Reconfiguring daemon mgr.x on smithi016 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]: dispatch 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2661491342' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "717d4d96-381a-4246-826b-f3ca949a2f46"}]': finished 2024-09-12T00:48:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T00:48:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:09.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:48:09.121+0000 7ff029414040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:48:09.789 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:09.533+0000 7f02282b5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:48:09.790 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:09.649+0000 7f02282b5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:48:09.790 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:09.791+0000 7f02282b5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:48:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:09.926+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fc9d003b460 con 0x7fc9e01026a0 2024-09-12T00:48:10.070 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:09.851+0000 7f02282b5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:48:10.070 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.003+0000 7f02282b5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:48:10.070 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.071+0000 7f02282b5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3036479762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: Standby manager daemon z started 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:48:10.071 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:09 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:10.349 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.131+0000 7f02282b5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:48:10.349 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.287+0000 7f02282b5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:48:10.350 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.351+0000 7f02282b5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3036479762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: Standby manager daemon z started 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:48:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:09 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3036479762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: Standby manager daemon z started 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:48:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:09 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/2541431181' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:10.616 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.416+0000 7f02282b5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:48:10.616 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.498+0000 7f02282b5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:48:10.616 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.558+0000 7f02282b5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:48:10.616 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:10.617+0000 7f02282b5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:48:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-mon[35654]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T00:48:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:10 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:48:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:10 smithi159 ceph-mon[35566]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T00:48:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:10 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:48:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:10 smithi016 ceph-mon[30972]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T00:48:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:10 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:48:11.773 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:48:11.399+0000 7f02282b5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:48:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:11.943+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc9d003a3c0 con 0x7fc9e01026a0 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: Standby manager daemon y started 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:48:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:11 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: Standby manager daemon y started 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:48:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:11 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: Standby manager daemon y started 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:48:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:11 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/1955700914' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:48:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:12 smithi099 ceph-mon[35654]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T00:48:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:48:13.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:12 smithi016 ceph-mon[30972]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T00:48:13.344 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:48:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:12 smithi159 ceph-mon[35566]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T00:48:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:48:14.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:13 smithi016 ceph-mon[30972]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:14.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:13 smithi099 ceph-mon[35654]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:13 smithi159 ceph-mon[35566]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:16.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:15 smithi016 ceph-mon[30972]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:15 smithi099 ceph-mon[35654]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:15 smithi159 ceph-mon[35566]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:17.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:18.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T00:48:18.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:18.342 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:18 smithi016 ceph-mon[30972]: Deploying daemon osd.0 on smithi016 2024-09-12T00:48:18.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:18 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T00:48:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:18 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:18 smithi099 ceph-mon[35654]: Deploying daemon osd.0 on smithi016 2024-09-12T00:48:18.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:18 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T00:48:18.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:18 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:18.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:18 smithi159 ceph-mon[35566]: Deploying daemon osd.0 on smithi016 2024-09-12T00:48:19.574 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:19 smithi016 ceph-mon[30972]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:19 smithi159 ceph-mon[35566]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:19 smithi099 ceph-mon[35654]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:21 smithi016 ceph-mon[30972]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:21 smithi159 ceph-mon[35566]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:21.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:21 smithi099 ceph-mon[35654]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:23.494 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:23 smithi016 ceph-mon[30972]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:23 smithi159 ceph-mon[35566]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:23.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:23 smithi099 ceph-mon[35654]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:24.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:24 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:24.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:24 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:24.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:24 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:24 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:24 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:24 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:24.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:24 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:24.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:24 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:24.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:24 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:26.151 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:26 smithi016 ceph-mon[30972]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:26 smithi159 ceph-mon[35566]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:26 smithi099 ceph-mon[35654]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:27.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.331+0000 7fc9e483e640 1 -- 172.21.15.16:0/3375912585 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc9e0109460 con 0x7fc9c80392c0 2024-09-12T00:48:28.329 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 msgr2=0x7fc9c803b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 0x7fc9c803b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9e019b680 tx=0x7fc9d4005f90 comp rx=0 tx=0).stop 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 msgr2=0x7fc9e019a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009230 tx=0x7fc9d00042f0 comp rx=0 tx=0).stop 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 shutdown_connections 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc9c80392c0 0x7fc9c803b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9e0107900 0x7fc9e01a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9e0104fd0 0x7fc9e019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 --2- 172.21.15.16:0/3375912585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9e01026a0 0x7fc9e019a160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.333+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 >> 172.21.15.16:0/3375912585 conn(0x7fc9e00fc2a0 msgr2=0x7fc9e0107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:48:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.334+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 shutdown_connections 2024-09-12T00:48:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:48:28.334+0000 7fc9e5840640 1 -- 172.21.15.16:0/3375912585 wait complete. 2024-09-12T00:48:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:28.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:28.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:28.970 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T00:48:28.972 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2024-09-12T00:48:28.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T00:48:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:48:29.634 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:29 smithi099 ceph-mon[35654]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:29.721 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:29 smithi016 ceph-mon[30972]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:29 smithi159 ceph-mon[35566]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:30.143 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:48:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T00:48:29.721+0000 7f6a096ca640 -1 osd.0 0 log_to_monitors true 2024-09-12T00:48:30.755 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:30 smithi016 ceph-mon[30972]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T00:48:30.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:30 smithi099 ceph-mon[35654]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T00:48:30.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:30 smithi159 ceph-mon[35566]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T00:48:31.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-mon[30972]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:31.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-mon[30972]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T00:48:31.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-mon[30972]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T00:48:31.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-mon[30972]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:48:31.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:31.556 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:48:31.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-09-12T00:48:31.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:31 smithi099 ceph-mon[35654]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:31.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:31 smithi099 ceph-mon[35654]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T00:48:31.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:31 smithi099 ceph-mon[35654]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T00:48:31.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:31 smithi099 ceph-mon[35654]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:48:31.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:31 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:31.776 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T00:48:31.501+0000 7f6a0564d640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T00:48:31.776 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:48:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T00:48:31.504+0000 7f6a01475640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:48:31.831 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:48:31.865 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:31 smithi159 ceph-mon[35566]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:31.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:31 smithi159 ceph-mon[35566]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T00:48:31.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:31 smithi159 ceph-mon[35566]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T00:48:31.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:31 smithi159 ceph-mon[35566]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:48:31.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:31 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 -- 172.21.15.99:0/4129651091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2518104f90 msgr2=0x7f2518107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 --2- 172.21.15.99:0/4129651091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2518104f90 0x7f2518107380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f250c009900 tx=0x7f250c02a410 comp rx=0 tx=0).stop 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 -- 172.21.15.99:0/4129651091 shutdown_connections 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 --2- 172.21.15.99:0/4129651091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25181078c0 0x7f2518109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 --2- 172.21.15.99:0/4129651091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2518104f90 0x7f2518107380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 --2- 172.21.15.99:0/4129651091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518102660 0x7f2518104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 -- 172.21.15.99:0/4129651091 >> 172.21.15.99:0/4129651091 conn(0x7f25180fc280 msgr2=0x7f25180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 -- 172.21.15.99:0/4129651091 shutdown_connections 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.275+0000 7f251d1e3640 1 -- 172.21.15.99:0/4129651091 wait complete. 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 Processor -- start 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 -- start start 2024-09-12T00:48:32.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2518102660 0x7f2518197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518104f90 0x7f2518198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f251810d8b0 con 0x7f2518104f90 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f251810d730 con 0x7f25181078c0 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251d1e3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f251810da30 con 0x7f2518102660 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f2516d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2518102660 0x7f2518197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35458/0 (socket says 172.21.15.99:35458) 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 -- 172.21.15.99:0/1793178460 learned_addr learned my addr 172.21.15.99:0/1793178460 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251659c640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518104f90 0x7f2518198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 -- 172.21.15.99:0/1793178460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2518102660 msgr2=0x7f2518197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:32.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2518102660 0x7f2518197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 -- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518104f90 msgr2=0x7f2518198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518104f90 0x7f2518198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.276+0000 7f251759e640 1 -- 172.21.15.99:0/1793178460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251819fbf0 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f251759e640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f251000b970 tx=0x7f251000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510019070 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251000cb40 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251819fee0 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510007650 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.277+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25181a0340 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.278+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25180fdd70 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.279+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f251001c070 con 0x7f25181078c0 2024-09-12T00:48:32.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.280+0000 7f24fffff640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 0x7f25000a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:32.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.280+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f25100ccea0 con 0x7f25181078c0 2024-09-12T00:48:32.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.280+0000 7f2516d9d640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 0x7f25000a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:32.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.280+0000 7f2516d9d640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 0x7f25000a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2508005d00 tx=0x7f250800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:32.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.287+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f251009b340 con 0x7f25181078c0 2024-09-12T00:48:32.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:32.471+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2518062630 con 0x7f25000a2b20 2024-09-12T00:48:32.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:48:32.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T00:48:32.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi016 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:32.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:32.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:32.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:32.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:48:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T00:48:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi016 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:32.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:48:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T00:48:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi016 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:32.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:32.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:32.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:32.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:32.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:33.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:48:33.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:48:33.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:33.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:33.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] boot 2024-09-12T00:48:33.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T00:48:33.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:33.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:48:33.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:48:33.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:33.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:33.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] boot 2024-09-12T00:48:33.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T00:48:33.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: osd.0 [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] boot 2024-09-12T00:48:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T00:48:33.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:48:34.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:34 smithi099 ceph-mon[35654]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T00:48:34.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:34 smithi159 ceph-mon[35566]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T00:48:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:34 smithi016 ceph-mon[30972]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/190663166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]': finished 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:35.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/3606522425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/190663166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]': finished 2024-09-12T00:48:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T00:48:35.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:35.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/3606522425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/190663166' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]: dispatch 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5541ef54-29b2-4099-bc7b-3b758b6f0a6d"}]': finished 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T00:48:35.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:35.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/3606522425' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:48:37.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:37 smithi099 ceph-mon[35654]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:37.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:37 smithi159 ceph-mon[35566]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:37 smithi016 ceph-mon[30972]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:39.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:39 smithi099 ceph-mon[35654]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:39.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:39 smithi159 ceph-mon[35566]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:39 smithi016 ceph-mon[30972]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:41.700 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:41 smithi099 ceph-mon[35654]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:41.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:41 smithi159 ceph-mon[35566]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:41.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:41 smithi016 ceph-mon[30972]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:42.760 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T00:48:42.760 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:42.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T00:48:42.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:42.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T00:48:42.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:43.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:43 smithi159 ceph-mon[35566]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:43.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:43 smithi159 ceph-mon[35566]: Deploying daemon osd.1 on smithi099 2024-09-12T00:48:43.875 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:43 smithi099 ceph-mon[35654]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:43.876 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:43 smithi099 ceph-mon[35654]: Deploying daemon osd.1 on smithi099 2024-09-12T00:48:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:43 smithi016 ceph-mon[30972]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:43.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:43 smithi016 ceph-mon[30972]: Deploying daemon osd.1 on smithi099 2024-09-12T00:48:45.618 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:45 smithi099 ceph-mon[35654]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:45.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:45 smithi159 ceph-mon[35566]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:45 smithi016 ceph-mon[30972]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:47.578 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:47 smithi099 ceph-mon[35654]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:47 smithi159 ceph-mon[35566]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:47.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:47 smithi016 ceph-mon[30972]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:48.747 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:48.748 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:48.748 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:48.896 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:48.896 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:48.896 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:49.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:49 smithi159 ceph-mon[35566]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:49.873 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:49 smithi099 ceph-mon[35654]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:49.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:49 smithi016 ceph-mon[30972]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:51.752 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:51.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.036+0000 7f24fffff640 1 -- 172.21.15.99:0/1793178460 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2518062630 con 0x7f25000a2b20 2024-09-12T00:48:52.035 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-09-12T00:48:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 msgr2=0x7f25000a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 0x7f25000a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2508005d00 tx=0x7f250800b410 comp rx=0 tx=0).stop 2024-09-12T00:48:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 msgr2=0x7f251819f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f251000b970 tx=0x7f251000be30 comp rx=0 tx=0).stop 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 shutdown_connections 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f25000a2b20 0x7f25000a4fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25181078c0 0x7f251819f580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2518104f90 0x7f2518198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 --2- 172.21.15.99:0/1793178460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2518102660 0x7f2518197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 >> 172.21.15.99:0/1793178460 conn(0x7f25180fc280 msgr2=0x7f2518102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 shutdown_connections 2024-09-12T00:48:52.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:48:52.039+0000 7f251d1e3640 1 -- 172.21.15.99:0/1793178460 wait complete. 2024-09-12T00:48:52.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:52 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:52.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:52 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.639 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:52 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.666 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T00:48:52.709 INFO:tasks.cephadm:Deploying osd.2 on smithi159 with /dev/vg_nvme/lv_4... 2024-09-12T00:48:52.709 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T00:48:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:52 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:52 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:52 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:52 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:48:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:52 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:52 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:52.963 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:48:54.081 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:54 smithi159 ceph-mon[35566]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:54.265 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:54 smithi099 ceph-mon[35654]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:54.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:54 smithi016 ceph-mon[30972]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:54.524 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:48:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T00:48:54.266+0000 7fc0c0c00640 -1 osd.1 0 log_to_monitors true 2024-09-12T00:48:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:55 smithi099 ceph-mon[35654]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:55 smithi099 ceph-mon[35654]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:55 smithi159 ceph-mon[35566]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:55 smithi159 ceph-mon[35566]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:55 smithi016 ceph-mon[30972]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:55 smithi016 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T00:48:55.525 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:48:55.549 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-09-12T00:48:55.790 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:48:56.175 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:56.175 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T00:48:56.175 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.175 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T00:48:56.175 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:56.176 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.258+0000 7f39fa8c0640 1 -- 172.21.15.159:0/2314082127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f41037c0 msgr2=0x7f39f410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:56.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.258+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/2314082127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f41037c0 0x7f39f410a050 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009ca0 tx=0x7f39ec02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 -- 172.21.15.159:0/2314082127 shutdown_connections 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/2314082127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f41037c0 0x7f39f410a050 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/2314082127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f4102e00 0x7f39f4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/2314082127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39f4101c00 0x7f39f4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 -- 172.21.15.159:0/2314082127 >> 172.21.15.159:0/2314082127 conn(0x7f39f40fd390 msgr2=0x7f39f40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 -- 172.21.15.159:0/2314082127 shutdown_connections 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 -- 172.21.15.159:0/2314082127 wait complete. 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 Processor -- start 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 -- start start 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.259+0000 7f39fa8c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39f4102e00 0x7f39f4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:56.267 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f41037c0 0x7f39f419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39f410d950 con 0x7f39f41037c0 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39f410d7d0 con 0x7f39f4102e00 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39f410dad0 con 0x7f39f4101c00 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:50988/0 (socket says 172.21.15.159:50988) 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 -- 172.21.15.159:0/3487450892 learned_addr learned my addr 172.21.15.159:0/3487450892 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f8e5d640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f41037c0 0x7f39f419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 -- 172.21.15.159:0/3487450892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39f4102e00 msgr2=0x7f39f4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39f4102e00 0x7f39f4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 -- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f41037c0 msgr2=0x7f39f419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f41037c0 0x7f39f419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 -- 172.21.15.159:0/3487450892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f419fb20 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f3fff640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c970 tx=0x7f39e400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4004220 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e40043c0 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4004a00 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f419fe10 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.260+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f41a0340 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.261+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f4108f30 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.262+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f39e40079b0 con 0x7f39f4101c00 2024-09-12T00:48:56.269 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.263+0000 7f39f17fa640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 0x7f39dc0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:48:56.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.263+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f39e40cd420 con 0x7f39f4101c00 2024-09-12T00:48:56.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.263+0000 7f39f37fe640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 0x7f39dc0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:48:56.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.264+0000 7f39f37fe640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 0x7f39dc0a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39f4199410 tx=0x7f39e8006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:48:56.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.267+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e4014030 con 0x7f39f4101c00 2024-09-12T00:48:56.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:56.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T00:48:56.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T00:48:56.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:56.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:48:56.442+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f39f4062630 con 0x7f39dc0a2ba0 2024-09-12T00:48:56.524 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T00:48:56.048+0000 7fc0bcb83640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T00:48:56.524 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T00:48:56.051+0000 7fc0b89ab640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:48:56.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:56.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T00:48:56.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:48:56.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:48:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi099 2024-09-12T00:48:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-12T00:48:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:48:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T00:48:57.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:57.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:57.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:57.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi099 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:57.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi099 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:48:57.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:48:57.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:48:58.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:48:58.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:48:58.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:58.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:58.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] boot 2024-09-12T00:48:58.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T00:48:58.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] boot 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T00:48:58.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: osd.1 [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] boot 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T00:48:58.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:48:59.256 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T00:48:59.256 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/988128146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.257 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.257 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]': finished 2024-09-12T00:48:59.257 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T00:48:59.257 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:48:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:48:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T00:48:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/988128146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]': finished 2024-09-12T00:48:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T00:48:59.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:48:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/988128146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]: dispatch 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f46058bc-91f3-4062-ab02-30e93a02b1d4"}]': finished 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T00:48:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:48:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:00 smithi159 ceph-mon[35566]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:00 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/3760488474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:00 smithi016 ceph-mon[30972]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:00 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/3760488474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:00 smithi099 ceph-mon[35654]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:00 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/3760488474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:02.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:02 smithi159 ceph-mon[35566]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:02.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:02 smithi016 ceph-mon[30972]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:02.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:02 smithi099 ceph-mon[35654]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:04.180 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:04 smithi159 ceph-mon[35566]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:04.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:04 smithi016 ceph-mon[30972]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:04.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:04 smithi099 ceph-mon[35654]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:06.354 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:06 smithi159 ceph-mon[35566]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:06.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:06 smithi016 ceph-mon[30972]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:06.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:06 smithi099 ceph-mon[35654]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:07.340 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T00:49:07.340 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T00:49:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T00:49:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:08.271 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:08 smithi159 ceph-mon[35566]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:08.556 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:08 smithi159 ceph-mon[35566]: Deploying daemon osd.2 on smithi159 2024-09-12T00:49:08.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:08 smithi099 ceph-mon[35654]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:08.557 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:08 smithi099 ceph-mon[35654]: Deploying daemon osd.2 on smithi159 2024-09-12T00:49:08.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:08 smithi016 ceph-mon[30972]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:08.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:08 smithi016 ceph-mon[30972]: Deploying daemon osd.2 on smithi159 2024-09-12T00:49:10.302 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:10 smithi159 ceph-mon[35566]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:10 smithi016 ceph-mon[30972]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:10.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:10 smithi099 ceph-mon[35654]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:12 smithi159 ceph-mon[35566]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:12 smithi016 ceph-mon[30972]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:12.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:12 smithi099 ceph-mon[35654]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:14 smithi159 ceph-mon[35566]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:14 smithi016 ceph-mon[30972]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:14.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:14 smithi099 ceph-mon[35654]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:14.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:14.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:17.268 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.268+0000 7f39f17fa640 1 -- 172.21.15.159:0/3487450892 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39f4062630 con 0x7f39dc0a2ba0 2024-09-12T00:49:17.268 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 2 on host 'smithi159' 2024-09-12T00:49:17.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 msgr2=0x7f39dc0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:17.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 0x7f39dc0a5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39f4199410 tx=0x7f39e8006c90 comp rx=0 tx=0).stop 2024-09-12T00:49:17.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 msgr2=0x7f39f4197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:17.270 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39e400c970 tx=0x7f39e400ce30 comp rx=0 tx=0).stop 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 shutdown_connections 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f39dc0a2ba0 0x7f39dc0a5060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f41037c0 0x7f39f419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39f4102e00 0x7f39f4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 --2- 172.21.15.159:0/3487450892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f4101c00 0x7f39f4197ef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 >> 172.21.15.159:0/3487450892 conn(0x7f39f40fd390 msgr2=0x7f39f4106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 shutdown_connections 2024-09-12T00:49:17.271 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:49:17.270+0000 7f39fa8c0640 1 -- 172.21.15.159:0/3487450892 wait complete. 2024-09-12T00:49:17.879 DEBUG:teuthology.orchestra.run.smithi159:osd.2> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T00:49:17.882 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-12T00:49:17.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T00:49:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:49:18.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:18 smithi016 ceph-mon[30972]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:18.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:18.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:18 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:18 smithi159 ceph-mon[35566]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:18 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:18 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:18 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:18 smithi099 ceph-mon[35654]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:18 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:18 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:18 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:18.995 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T00:49:18.898+0000 7f63e72e1640 -1 osd.2 0 log_to_monitors true 2024-09-12T00:49:19.296 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:19 smithi159 ceph-mon[35566]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:19.296 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:19 smithi159 ceph-mon[35566]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:19.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:19 smithi016 ceph-mon[30972]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:19.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:19 smithi016 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:19 smithi099 ceph-mon[35654]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:19 smithi099 ceph-mon[35654]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T00:49:20.366 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T00:49:20.118+0000 7f63e3264640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T00:49:20.366 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T00:49:20.122+0000 7f63df08c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:49:20.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:20.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T00:49:20.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T00:49:20.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:20.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:20 smithi159 ceph-mon[35566]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T00:49:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T00:49:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:20 smithi016 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T00:49:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T00:49:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:20.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: from='osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:20 smithi099 ceph-mon[35654]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:49:20.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:49:20.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-12T00:49:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:49:21.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:49:21.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T00:49:21.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:21.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:21.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:21.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:21.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 -- 172.21.15.16:0/2565265718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 msgr2=0x7ff82810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/2565265718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82810a100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff820009cd0 tx=0x7ff82002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 -- 172.21.15.16:0/2565265718 shutdown_connections 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/2565265718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82810a100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/2565265718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 0x7ff828103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/2565265718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff8281002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 -- 172.21.15.16:0/2565265718 >> 172.21.15.16:0/2565265718 conn(0x7ff8280fba40 msgr2=0x7ff8280fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 -- 172.21.15.16:0/2565265718 shutdown_connections 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.705+0000 7ff82cb6d640 1 -- 172.21.15.16:0/2565265718 wait complete. 2024-09-12T00:49:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 Processor -- start 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 -- start start 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff828197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 0x7ff828198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d890 con 0x7ff828103870 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d710 con 0x7ff8280ffed0 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82cb6d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810da10 con 0x7ff828102eb0 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59422/0 (socket says 172.21.15.16:59422) 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 -- 172.21.15.16:0/211413927 learned_addr learned my addr 172.21.15.16:0/211413927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:49:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff825d9b640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 0x7ff828198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 -- 172.21.15.16:0/211413927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 msgr2=0x7ff828198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 0x7ff828198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82659c640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff828197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 -- 172.21.15.16:0/211413927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 msgr2=0x7ff828197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff828197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff826d9d640 1 -- 172.21.15.16:0/211413927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82819fb50 con 0x7ff828103870 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.706+0000 7ff82659c640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff828197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:49:21.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff826d9d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff820009df0 tx=0x7ff820003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820004450 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82819fde0 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff820003e40 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200466e0 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.707+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a0290 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.708+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff828109100 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.709+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ff8200305a0 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.710+0000 7ff8177fe640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 0x7ff8100a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.710+0000 7ff82659c640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 0x7ff8100a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.710+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7ff82002e030 con 0x7ff828103870 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.710+0000 7ff82659c640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 0x7ff8100a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff82806a590 tx=0x7ff818005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:49:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.714+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff82004b060 con 0x7ff828103870 2024-09-12T00:49:21.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:21.901+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff828062630 con 0x7ff8100a2b20 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi159 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] boot 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:22.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:22.320 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi159 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-12T00:49:22.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] boot 2024-09-12T00:49:22.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T00:49:22.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:22.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:22.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:22.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:22.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:49:22.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:49:22.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T00:49:22.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi159 2024-09-12T00:49:22.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-12T00:49:22.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: osd.2 [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] boot 2024-09-12T00:49:22.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T00:49:22.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:49:22.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:22.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:22.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:23 smithi016 ceph-mon[30972]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:23 smithi016 ceph-mon[30972]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T00:49:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:23 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:23 smithi099 ceph-mon[35654]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:23 smithi099 ceph-mon[35654]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T00:49:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:23 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:23 smithi159 ceph-mon[35566]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:23 smithi159 ceph-mon[35566]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T00:49:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:23 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:24 smithi016 ceph-mon[30972]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:24 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:24 smithi016 ceph-mon[30972]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T00:49:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:24 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:24 smithi099 ceph-mon[35654]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:24 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:24 smithi099 ceph-mon[35654]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T00:49:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:24 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:24 smithi159 ceph-mon[35566]: pgmap v76: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:24 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:24 smithi159 ceph-mon[35566]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T00:49:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:24 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T00:49:25.103 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:24 smithi016 sudo[49549]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:25.103 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:24 smithi016 sudo[49549]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:25.104 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:24 smithi016 sudo[49549]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:25.104 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:24 smithi016 sudo[49549]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:25.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:25 smithi016 sudo[49583]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:25.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:25 smithi016 sudo[49583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:25.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:25 smithi016 sudo[49583]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:25.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 00:49:25 smithi016 sudo[49583]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]: dispatch 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]': finished 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:25.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1963341926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]: dispatch 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]': finished 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:25.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1963341926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]: dispatch 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2477716507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae799129-d588-4925-a79d-869205ffc0d8"}]': finished 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1963341926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:25.669 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:25.669 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:25.669 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43824]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:25.669 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43824]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:26.024 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:26.024 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:26.024 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43827]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:26.024 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 00:49:25 smithi099 sudo[43827]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:26.194 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:26 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:26.194 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:26 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:26.194 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:25 smithi159 sudo[44403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:26.194 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:25 smithi159 sudo[44403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:26.194 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:25 smithi159 sudo[44403]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:26.194 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:25 smithi159 sudo[44403]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:26 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:26 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:26.616 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:26 smithi159 sudo[44406]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:26.616 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:26 smithi159 sudo[44406]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:26.616 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:26 smithi159 sudo[44406]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:26.616 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 00:49:26 smithi159 sudo[44406]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:27.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 sudo[50001]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:27.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 sudo[50001]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:27.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 sudo[50001]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:27.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:26 smithi016 sudo[50001]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:27.144+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8200b9590 con 0x7ff828103870 2024-09-12T00:49:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 sudo[50008]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 sudo[50008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 sudo[50008]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 sudo[50008]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:27.633 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:27.634 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:27.634 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44409]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:27.634 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44409]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:27 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:27.894 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:27.895 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:28.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:28.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:28.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44412]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:28.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 sudo[44412]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:28.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:27 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:28.162 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 sudo[43830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T00:49:28.162 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 sudo[43830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:28.162 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 sudo[43830]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:28.162 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:27 smithi099 sudo[43830]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:28.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 sudo[43833]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T00:49:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 sudo[43833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T00:49:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 sudo[43833]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T00:49:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 sudo[43833]: pam_unix(sudo:session): session closed for user root 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:28 smithi099 ceph-mon[35654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:28 smithi159 ceph-mon[35566]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:49:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:28 smithi016 ceph-mon[30972]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T00:49:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:29 smithi099 ceph-mon[35654]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:30.039 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:29 smithi159 ceph-mon[35566]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:30.042 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:29 smithi016 ceph-mon[30972]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:31.741 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:31 smithi016 ceph-mon[30972]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:32.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:31 smithi099 ceph-mon[35654]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:32.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:31 smithi159 ceph-mon[35566]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:33.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:33 smithi016 ceph-mon[30972]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:33.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T00:49:33.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:33.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:33 smithi016 ceph-mon[30972]: Deploying daemon osd.3 on smithi016 2024-09-12T00:49:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:33 smithi099 ceph-mon[35654]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T00:49:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:33 smithi099 ceph-mon[35654]: Deploying daemon osd.3 on smithi016 2024-09-12T00:49:34.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:33 smithi159 ceph-mon[35566]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:34.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T00:49:34.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:34.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:33 smithi159 ceph-mon[35566]: Deploying daemon osd.3 on smithi016 2024-09-12T00:49:35.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:35 smithi016 ceph-mon[30972]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T00:49:36.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:35 smithi099 ceph-mon[35654]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T00:49:36.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:35 smithi159 ceph-mon[35566]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T00:49:38.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:37 smithi016 ceph-mon[30972]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T00:49:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:37 smithi099 ceph-mon[35654]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T00:49:38.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:37 smithi159 ceph-mon[35566]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T00:49:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:39 smithi016 ceph-mon[30972]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T00:49:40.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:39 smithi099 ceph-mon[35654]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T00:49:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:39 smithi159 ceph-mon[35566]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T00:49:40.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:40.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:40.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:41.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:41 smithi016 ceph-mon[30972]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:41 smithi099 ceph-mon[35654]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:41 smithi159 ceph-mon[35566]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:43.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:43.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.926+0000 7ff8177fe640 1 -- 172.21.15.16:0/211413927 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff828062630 con 0x7ff8100a2b20 2024-09-12T00:49:43.925 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-12T00:49:43.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.928+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 msgr2=0x7ff8100a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.928+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 0x7ff8100a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff82806a590 tx=0x7ff818005d30 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 msgr2=0x7ff82819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff820009df0 tx=0x7ff820003c20 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 shutdown_connections 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff8100a2b20 0x7ff8100a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff828103870 0x7ff82819f4e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff828102eb0 0x7ff828198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 --2- 172.21.15.16:0/211413927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8280ffed0 0x7ff828197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 >> 172.21.15.16:0/211413927 conn(0x7ff8280fba40 msgr2=0x7ff8281062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 shutdown_connections 2024-09-12T00:49:43.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:49:43.929+0000 7ff82cb6d640 1 -- 172.21.15.16:0/211413927 wait complete. 2024-09-12T00:49:44.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:44 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:44.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:44 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:44 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:44 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:44 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:44 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.572 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T00:49:44.574 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2024-09-12T00:49:44.574 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T00:49:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:44 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:49:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:44 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:44 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:44.828 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:49:45.393 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 00:49:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T00:49:45.269+0000 7fb79c79d640 -1 osd.3 0 log_to_monitors true 2024-09-12T00:49:46.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:45 smithi016 ceph-mon[30972]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:46.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:45 smithi016 ceph-mon[30972]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T00:49:46.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:45 smithi099 ceph-mon[35654]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:46.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:45 smithi099 ceph-mon[35654]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T00:49:46.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:45 smithi159 ceph-mon[35566]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T00:49:46.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:45 smithi159 ceph-mon[35566]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T00:49:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-mon[30972]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T00:49:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-mon[30972]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T00:49:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-mon[30972]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:49:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-mon[30972]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:49:47.143 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T00:49:46.933+0000 7fb798720640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T00:49:47.144 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 00:49:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T00:49:46.938+0000 7fb794548640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:49:47.267 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:46 smithi159 ceph-mon[35566]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T00:49:47.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:46 smithi159 ceph-mon[35566]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T00:49:47.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:46 smithi159 ceph-mon[35566]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:49:47.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:46 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:47.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:46 smithi159 ceph-mon[35566]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:49:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:46 smithi099 ceph-mon[35654]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T00:49:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:46 smithi099 ceph-mon[35654]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T00:49:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:46 smithi099 ceph-mon[35654]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:49:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:46 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:46 smithi099 ceph-mon[35654]: from='osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:49:47.364 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:49:47.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-09-12T00:49:47.637 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:49:48.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 -- 172.21.15.99:0/1809319664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d81026d0 msgr2=0x7f02d8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:48.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/1809319664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d81026d0 0x7f02d8108f60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009ca0 tx=0x7f02d002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 -- 172.21.15.99:0/1809319664 shutdown_connections 2024-09-12T00:49:48.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/1809319664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d81026d0 0x7f02d8108f60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/1809319664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8101d10 0x7f02d8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/1809319664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d8100b10 0x7f02d8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 -- 172.21.15.99:0/1809319664 >> 172.21.15.99:0/1809319664 conn(0x7f02d80fc2a0 msgr2=0x7f02d80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 -- 172.21.15.99:0/1809319664 shutdown_connections 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.100+0000 7f02dcfe9640 1 -- 172.21.15.99:0/1809319664 wait complete. 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 Processor -- start 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 -- start start 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 0x7f02d806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d81026d0 0x7f02d819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810c790 con 0x7f02d8100b10 2024-09-12T00:49:48.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810c610 con 0x7f02d8101d10 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.101+0000 7f02dcfe9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810c910 con 0x7f02d81026d0 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 0x7f02d806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37106/0 (socket says 172.21.15.99:37106) 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 -- 172.21.15.99:0/2918152658 learned_addr learned my addr 172.21.15.99:0/2918152658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d6d9d640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d81026d0 0x7f02d819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 -- 172.21.15.99:0/2918152658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d81026d0 msgr2=0x7f02d819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d81026d0 0x7f02d819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 -- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 msgr2=0x7f02d806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 0x7f02d806b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 -- 172.21.15.99:0/2918152658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d819fab0 con 0x7f02d8101d10 2024-09-12T00:49:48.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d659c640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 0x7f02d806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02d5d9b640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00d950 tx=0x7f02cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc017070 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d819fda0 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc009c30 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00f6b0 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.102+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81a02b0 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.104+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d8100f10 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.104+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f02cc0109d0 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.105+0000 7f02c77fe640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 0x7f02c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.105+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f02cc0cda80 con 0x7f02d8101d10 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.106+0000 7f02d659c640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 0x7f02c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.106+0000 7f02d659c640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 0x7f02c00a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02d806b870 tx=0x7f02c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:49:48.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.109+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02cc0cfe20 con 0x7f02d8101d10 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] boot 2024-09-12T00:49:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:48.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:49:48.294+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f02d8062630 con 0x7f02c00a2b80 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] boot 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-12T00:49:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T00:49:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: osd.3 [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] boot 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:49:48.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi016 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi016 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:49:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:49:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi016 2024-09-12T00:49:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-12T00:49:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T00:49:49.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:49:49.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:49:49.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:49:50.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:49 smithi099 ceph-mon[35654]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:50.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:49 smithi099 ceph-mon[35654]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:50.084 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:49 smithi099 ceph-mon[35654]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T00:49:50.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:49 smithi016 ceph-mon[30972]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:50.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:49 smithi016 ceph-mon[30972]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:50.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:49 smithi016 ceph-mon[30972]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T00:49:50.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:49 smithi159 ceph-mon[35566]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:50.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:49 smithi159 ceph-mon[35566]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:49:50.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:49 smithi159 ceph-mon[35566]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T00:49:51.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/2369028435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]': finished 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T00:49:51.391 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/2369028435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]': finished 2024-09-12T00:49:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T00:49:51.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/2369028435' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]: dispatch 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70a24b6e-0414-4620-a0ec-89f48dedbef7"}]': finished 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T00:49:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:49:52.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:52 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/297503745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:52.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:52 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/297503745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:52.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:52 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/297503745' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:49:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:53 smithi099 ceph-mon[35654]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:53 smithi159 ceph-mon[35566]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:53 smithi016 ceph-mon[30972]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:49:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:55 smithi099 ceph-mon[35654]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:49:55.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:55 smithi159 ceph-mon[35566]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:49:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:55 smithi016 ceph-mon[30972]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:49:57.384 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:57 smithi099 ceph-mon[35654]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T00:49:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:57 smithi159 ceph-mon[35566]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T00:49:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:57 smithi016 ceph-mon[30972]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-12T00:49:59.228 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:49:59 smithi099 ceph-mon[35654]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:49:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:49:59 smithi159 ceph-mon[35566]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:49:59.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:49:59 smithi016 ceph-mon[30972]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:50:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T00:50:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:00 smithi099 ceph-mon[35654]: Deploying daemon osd.4 on smithi099 2024-09-12T00:50:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:00 smithi099 ceph-mon[35654]: overall HEALTH_OK 2024-09-12T00:50:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T00:50:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:00 smithi159 ceph-mon[35566]: Deploying daemon osd.4 on smithi099 2024-09-12T00:50:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:00 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T00:50:00.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T00:50:00.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:00.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:00 smithi016 ceph-mon[30972]: Deploying daemon osd.4 on smithi099 2024-09-12T00:50:00.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:00 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T00:50:01.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:01 smithi099 ceph-mon[35654]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:01 smithi159 ceph-mon[35566]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:01.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:01 smithi016 ceph-mon[30972]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:03 smithi099 ceph-mon[35654]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T00:50:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:03 smithi159 ceph-mon[35566]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T00:50:03.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:03 smithi016 ceph-mon[30972]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T00:50:05.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:05 smithi099 ceph-mon[35654]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T00:50:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:05 smithi159 ceph-mon[35566]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T00:50:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:05 smithi016 ceph-mon[30972]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T00:50:06.349 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:06.349 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:06.349 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:06 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:06 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:06 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:07.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:07 smithi016 ceph-mon[30972]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:07.968 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:07 smithi099 ceph-mon[35654]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:07 smithi159 ceph-mon[35566]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:09.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:09.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.422 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:09.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:09.423 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.465+0000 7f02c77fe640 1 -- 172.21.15.99:0/2918152658 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02d8062630 con 0x7f02c00a2b80 2024-09-12T00:50:09.464 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2024-09-12T00:50:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.467+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 msgr2=0x7f02c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.467+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 0x7f02c00a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02d806b870 tx=0x7f02c8005f90 comp rx=0 tx=0).stop 2024-09-12T00:50:09.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.467+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 msgr2=0x7f02d819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.467+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00d950 tx=0x7f02cc00de10 comp rx=0 tx=0).stop 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 shutdown_connections 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02c00a2b80 0x7f02c00a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d81026d0 0x7f02d819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02d8101d10 0x7f02d819cba0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 --2- 172.21.15.99:0/2918152658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02d8100b10 0x7f02d806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 >> 172.21.15.99:0/2918152658 conn(0x7f02d80fc2a0 msgr2=0x7f02d8105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 shutdown_connections 2024-09-12T00:50:09.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:50:09.468+0000 7f02dcfe9640 1 -- 172.21.15.99:0/2918152658 wait complete. 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.097 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T00:50:10.099 INFO:tasks.cephadm:Deploying osd.5 on smithi159 with /dev/vg_nvme/lv_3... 2024-09-12T00:50:10.099 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T00:50:10.318 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:10 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:10.318 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:10 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.318 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:10 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.359 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:50:10.481 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:10 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:10.481 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:10 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.481 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:10 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:10 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:10.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:10 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:10.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:10 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:11 smithi099 ceph-mon[35654]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:11.818 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:11 smithi159 ceph-mon[35566]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:11.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:11 smithi016 ceph-mon[30972]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:12.271 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 00:50:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T00:50:12.020+0000 7fbb0c7fd640 -1 osd.4 0 log_to_monitors true 2024-09-12T00:50:12.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:12 smithi099 ceph-mon[35654]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:12 smithi099 ceph-mon[35654]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:12 smithi159 ceph-mon[35566]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:12 smithi159 ceph-mon[35566]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:12 smithi016 ceph-mon[30972]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:12 smithi016 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T00:50:12.942 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:50:12.969 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-09-12T00:50:13.228 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:50:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T00:50:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T00:50:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi099 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:13.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T00:50:13.478+0000 7fbb08780640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T00:50:13.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T00:50:13.484+0000 7fbb045a8640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:50:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T00:50:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T00:50:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi099 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:13.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 -- 172.21.15.159:0/2309252637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc1037c0 msgr2=0x7f76fc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 --2- 172.21.15.159:0/2309252637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc1037c0 0x7f76fc10a050 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009ca0 tx=0x7f76f402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 -- 172.21.15.159:0/2309252637 shutdown_connections 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 --2- 172.21.15.159:0/2309252637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc1037c0 0x7f76fc10a050 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 --2- 172.21.15.159:0/2309252637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc102e00 0x7f76fc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 --2- 172.21.15.159:0/2309252637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc101c00 0x7f76fc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 -- 172.21.15.159:0/2309252637 >> 172.21.15.159:0/2309252637 conn(0x7f76fc0fd390 msgr2=0x7f76fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 -- 172.21.15.159:0/2309252637 shutdown_connections 2024-09-12T00:50:13.776 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.759+0000 7f770432c640 1 -- 172.21.15.159:0/2309252637 wait complete. 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 Processor -- start 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 -- start start 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc102e00 0x7f76fc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 0x7f76fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d880 con 0x7f76fc1037c0 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10d700 con 0x7f76fc102e00 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f770432c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc10da00 con 0x7f76fc101c00 2024-09-12T00:50:13.777 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33054/0 (socket says 172.21.15.159:33054) 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 -- 172.21.15.159:0/1118576681 learned_addr learned my addr 172.21.15.159:0/1118576681 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77028c9640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 0x7f76fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77018c7640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc102e00 0x7f76fc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 -- 172.21.15.159:0/1118576681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc102e00 msgr2=0x7f76fc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc102e00 0x7f76fc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.760+0000 7f77020c8640 1 -- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 msgr2=0x7f76fc074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f77020c8640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 0x7f76fc074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f77020c8640 1 -- 172.21.15.159:0/1118576681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc0753e0 con 0x7f76fc101c00 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f77028c9640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 0x7f76fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f77020c8640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c970 tx=0x7f76ec00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004220 con 0x7f76fc101c00 2024-09-12T00:50:13.778 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec0043c0 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc1a1fe0 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004a00 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.761+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc1a23d0 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.762+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc108fb0 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.763+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f76ec0079b0 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.764+0000 7f76f37fe640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 0x7f76e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.764+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f76ec0cdd00 con 0x7f76fc101c00 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.764+0000 7f77018c7640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 0x7f76e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.764+0000 7f77018c7640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 0x7f76e80a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f8005fd0 tx=0x7f76f8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:50:13.779 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.768+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76ec014030 con 0x7f76fc101c00 2024-09-12T00:50:13.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:13.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T00:50:13.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T00:50:13.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:13.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi099 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:13.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-12T00:50:13.896 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.896 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:13.896 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:13.896 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:13.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:13.944+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f76fc062630 con 0x7f76e80a2b80 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:14 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:14 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:50:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T00:50:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:14.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:14.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:14.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:14 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] boot 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T00:50:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:15 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] boot 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T00:50:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:15 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: osd.4 [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] boot 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T00:50:15.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:15 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/1279087534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]': finished 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T00:50:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T00:50:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/1279087534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]': finished 2024-09-12T00:50:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T00:50:16.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/1279087534' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]: dispatch 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff6ef9d-4e03-4343-8e20-fa384843bc9b"}]': finished 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T00:50:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:17 smithi099 ceph-mon[35654]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:17 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/3779647663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:17.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:17 smithi159 ceph-mon[35566]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:17.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:17 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/3779647663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:17.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:17 smithi016 ceph-mon[30972]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:17.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:17 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/3779647663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:19 smithi099 ceph-mon[35654]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:19.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:19 smithi159 ceph-mon[35566]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:19.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:19 smithi016 ceph-mon[30972]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:21.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:21 smithi099 ceph-mon[35654]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:21.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:21 smithi159 ceph-mon[35566]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:21.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:21 smithi016 ceph-mon[30972]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:23.614 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:23 smithi159 ceph-mon[35566]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:23.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:23 smithi099 ceph-mon[35654]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:23.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:23 smithi016 ceph-mon[30972]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:25 smithi159 ceph-mon[35566]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T00:50:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:25 smithi159 ceph-mon[35566]: Deploying daemon osd.5 on smithi159 2024-09-12T00:50:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:25 smithi099 ceph-mon[35654]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T00:50:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:25 smithi099 ceph-mon[35654]: Deploying daemon osd.5 on smithi159 2024-09-12T00:50:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:25 smithi016 ceph-mon[30972]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T00:50:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:25 smithi016 ceph-mon[30972]: Deploying daemon osd.5 on smithi159 2024-09-12T00:50:27.583 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:27 smithi159 ceph-mon[35566]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:27 smithi099 ceph-mon[35654]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:27 smithi016 ceph-mon[30972]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:29.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:29 smithi099 ceph-mon[35654]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:29.816 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:29 smithi159 ceph-mon[35566]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:29.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:29 smithi016 ceph-mon[30972]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:31.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:31 smithi099 ceph-mon[35654]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:31.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:31 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:31.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:31 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:31.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:31 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:31.799 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:31 smithi159 ceph-mon[35566]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:31.799 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:31 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:31.799 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:31 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:31.799 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:31 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:31.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:31 smithi016 ceph-mon[30972]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:31.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:31 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:31.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:31 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:31.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:31 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:33.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:33 smithi099 ceph-mon[35654]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:33.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:33 smithi159 ceph-mon[35566]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:33 smithi016 ceph-mon[30972]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:34.405 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 5 on host 'smithi159' 2024-09-12T00:50:34.405 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.405+0000 7f76f37fe640 1 -- 172.21.15.159:0/1118576681 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76fc062630 con 0x7f76e80a2b80 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 msgr2=0x7f76e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 0x7f76e80a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f8005fd0 tx=0x7f76f8005850 comp rx=0 tx=0).stop 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 msgr2=0x7f76fc075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c970 tx=0x7f76ec00ce30 comp rx=0 tx=0).stop 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 shutdown_connections 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f76e80a2b80 0x7f76e80a5040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76fc1037c0 0x7f76fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76fc102e00 0x7f76fc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:34.407 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 --2- 172.21.15.159:0/1118576681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76fc101c00 0x7f76fc075f90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:34.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 >> 172.21.15.159:0/1118576681 conn(0x7f76fc0fd390 msgr2=0x7f76fc106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:50:34.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 shutdown_connections 2024-09-12T00:50:34.408 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:50:34.408+0000 7f770432c640 1 -- 172.21.15.159:0/1118576681 wait complete. 2024-09-12T00:50:35.000 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:34 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:34 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.028 DEBUG:teuthology.orchestra.run.smithi159:osd.5> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T00:50:35.030 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-12T00:50:35.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:35.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:34 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:50:35.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:35 smithi159 ceph-mon[35566]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:36.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:35 smithi099 ceph-mon[35654]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:36.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:35 smithi016 ceph-mon[30972]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:36.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:36 smithi159 ceph-mon[35566]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:36.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:36 smithi159 ceph-mon[35566]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:36.866 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 00:50:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T00:50:36.618+0000 7f71a9ca5640 -1 osd.5 0 log_to_monitors true 2024-09-12T00:50:36.963 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:36 smithi016 ceph-mon[30972]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:36.963 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:36 smithi016 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:36 smithi099 ceph-mon[35654]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:36 smithi099 ceph-mon[35654]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:37.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:37 smithi016 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:37.932 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:37.932 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T00:50:37.932 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T00:50:37.932 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:37.932 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:37.933 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:37 smithi159 ceph-mon[35566]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:38.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T00:50:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T00:50:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: from='osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:38.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:37 smithi099 ceph-mon[35654]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:50:38.171 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:50:38.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-12T00:50:38.366 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T00:50:38.176+0000 7f71a5c28640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T00:50:38.366 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T00:50:38.183+0000 7f719da48640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:50:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi159 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:38.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:38.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:38 smithi016 ceph-mon[30972]: from='osd.5 ' entity='osd.5' 2024-09-12T00:50:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.975+0000 7fd4af253640 1 -- 172.21.15.16:0/2092229142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a81026a0 msgr2=0x7fd4a8104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.975+0000 7fd4af253640 1 --2- 172.21.15.16:0/2092229142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a81026a0 0x7fd4a8104a90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009900 tx=0x7fd49c02a410 comp rx=0 tx=0).stop 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 -- 172.21.15.16:0/2092229142 shutdown_connections 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 --2- 172.21.15.16:0/2092229142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8107900 0x7fd4a8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 --2- 172.21.15.16:0/2092229142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a8104fd0 0x7fd4a81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 --2- 172.21.15.16:0/2092229142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a81026a0 0x7fd4a8104a90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 -- 172.21.15.16:0/2092229142 >> 172.21.15.16:0/2092229142 conn(0x7fd4a80fc2a0 msgr2=0x7fd4a80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 -- 172.21.15.16:0/2092229142 shutdown_connections 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 -- 172.21.15.16:0/2092229142 wait complete. 2024-09-12T00:50:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 Processor -- start 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 -- start start 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.976+0000 7fd4af253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 0x7fd4a8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8104fd0 0x7fd4a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810d8d0 con 0x7fd4a8107900 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810d750 con 0x7fd4a81026a0 2024-09-12T00:50:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810da50 con 0x7fd4a8104fd0 2024-09-12T00:50:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60948/0 (socket says 172.21.15.16:60948) 2024-09-12T00:50:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 -- 172.21.15.16:0/3559477071 learned_addr learned my addr 172.21.15.16:0/3559477071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:50:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4a7fff640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8104fd0 0x7fd4a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4acfef640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 0x7fd4a8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 -- 172.21.15.16:0/3559477071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8104fd0 msgr2=0x7fd4a81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8104fd0 0x7fd4a81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 -- 172.21.15.16:0/3559477071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 msgr2=0x7fd4a8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:50:38.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 0x7fd4a8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:50:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 -- 172.21.15.16:0/3559477071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a819fad0 con 0x7fd4a8107900 2024-09-12T00:50:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4acfef640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 0x7fd4a8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:50:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4ad7f0640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000eaa0 tx=0x7fd4a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:50:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0004d20 con 0x7fd4a8107900 2024-09-12T00:50:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a0005020 con 0x7fd4a8107900 2024-09-12T00:50:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a819fdc0 con 0x7fd4a8107900 2024-09-12T00:50:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.977+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00116d0 con 0x7fd4a8107900 2024-09-12T00:50:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.978+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81a02d0 con 0x7fd4a8107900 2024-09-12T00:50:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.979+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a80fdd90 con 0x7fd4a8107900 2024-09-12T00:50:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.979+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4a00051d0 con 0x7fd4a8107900 2024-09-12T00:50:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.980+0000 7fd4a5ffb640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 0x7fd4980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:50:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.980+0000 7fd4acfef640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 0x7fd4980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:50:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.980+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7fd4a00ce2c0 con 0x7fd4a8107900 2024-09-12T00:50:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.981+0000 7fd4acfef640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 0x7fd4980a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:50:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:38.985+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4a0016030 con 0x7fd4a8107900 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi159 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:38 smithi099 ceph-mon[35654]: from='osd.5 ' entity='osd.5' 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi159 2024-09-12T00:50:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:39.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:38 smithi159 ceph-mon[35566]: from='osd.5 ' entity='osd.5' 2024-09-12T00:50:39.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:50:39.171+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd4a8062630 con 0x7fd4980a2c30 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] boot 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T00:50:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] boot 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:40.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:40.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:40.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: osd.5 [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] boot 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T00:50:40.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:50:40.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:50:40.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:50:40.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:40.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:40 smithi016 ceph-mon[30972]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:40.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:40 smithi016 ceph-mon[30972]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T00:50:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:40 smithi099 ceph-mon[35654]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:40 smithi099 ceph-mon[35654]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T00:50:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:40 smithi159 ceph-mon[35566]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:50:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:40 smithi159 ceph-mon[35566]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]: dispatch 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]': finished 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T00:50:42.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:41 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]: dispatch 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]': finished 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T00:50:42.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:41 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]: dispatch 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1642141284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c365b94-3cbc-4aec-b6a6-d503fd51fd0e"}]': finished 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T00:50:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:41 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:42 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3024261181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:42 smithi159 ceph-mon[35566]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T00:50:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:42 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3024261181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:42 smithi016 ceph-mon[30972]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T00:50:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:42 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3024261181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:50:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:42 smithi099 ceph-mon[35654]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T00:50:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:50:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:43 smithi159 ceph-mon[35566]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:44.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:43 smithi016 ceph-mon[30972]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:43 smithi099 ceph-mon[35654]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:46.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:45 smithi159 ceph-mon[35566]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T00:50:46.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:45 smithi016 ceph-mon[30972]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T00:50:46.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:45 smithi099 ceph-mon[35654]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-12T00:50:48.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:47 smithi016 ceph-mon[30972]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:50:48.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:47 smithi159 ceph-mon[35566]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:50:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:47 smithi099 ceph-mon[35654]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T00:50:50.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:49 smithi099 ceph-mon[35654]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T00:50:50.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:49 smithi159 ceph-mon[35566]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T00:50:50.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:49 smithi016 ceph-mon[30972]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T00:50:51.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:50 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T00:50:51.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:50 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:50 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T00:50:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:50 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:50 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T00:50:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:50 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:50:51.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:51 smithi016 ceph-mon[30972]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T00:50:51.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:51 smithi016 ceph-mon[30972]: Deploying daemon osd.6 on smithi016 2024-09-12T00:50:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:51 smithi159 ceph-mon[35566]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T00:50:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:51 smithi159 ceph-mon[35566]: Deploying daemon osd.6 on smithi016 2024-09-12T00:50:52.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:51 smithi099 ceph-mon[35654]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T00:50:52.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:51 smithi099 ceph-mon[35654]: Deploying daemon osd.6 on smithi016 2024-09-12T00:50:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:53 smithi016 ceph-mon[30972]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:53 smithi159 ceph-mon[35566]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:53 smithi099 ceph-mon[35654]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:50:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:55 smithi159 ceph-mon[35566]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T00:50:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:55 smithi016 ceph-mon[30972]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T00:50:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:55 smithi099 ceph-mon[35654]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T00:50:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:50:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:50:58.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:57 smithi159 ceph-mon[35566]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:58.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:57 smithi016 ceph-mon[30972]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:50:58.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:57 smithi099 ceph-mon[35654]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:50:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:00.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:50:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:00.277 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:50:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.616+0000 7fd4a5ffb640 1 -- 172.21.15.16:0/3559477071 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4a8062630 con 0x7fd4980a2c30 2024-09-12T00:51:00.616 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 msgr2=0x7fd4980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 0x7fd4980a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000c00 tx=0x7fd49c031400 comp rx=0 tx=0).stop 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 msgr2=0x7fd4a819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000eaa0 tx=0x7fd4a000eea0 comp rx=0 tx=0).stop 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 shutdown_connections 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd4980a2c30 0x7fd4980a50f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:00.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4a8107900 0x7fd4a819f460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4a8104fd0 0x7fd4a81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 --2- 172.21.15.16:0/3559477071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4a81026a0 0x7fd4a8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.618+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 >> 172.21.15.16:0/3559477071 conn(0x7fd4a80fc2a0 msgr2=0x7fd4a8103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.619+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 shutdown_connections 2024-09-12T00:51:00.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:00.619+0000 7fd4af253640 1 -- 172.21.15.16:0/3559477071 wait complete. 2024-09-12T00:51:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:01.383 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T00:51:01.385 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2024-09-12T00:51:01.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T00:51:01.641 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:51:02.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:01 smithi099 ceph-mon[35654]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:02.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:01 smithi159 ceph-mon[35566]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:02.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:01 smithi016 ceph-mon[30972]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:02.621 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 00:51:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T00:51:02.368+0000 7fbc7a22b640 -1 osd.6 0 log_to_monitors true 2024-09-12T00:51:02.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:02 smithi016 ceph-mon[30972]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T00:51:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:02 smithi099 ceph-mon[35654]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T00:51:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:02 smithi159 ceph-mon[35566]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T00:51:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:03 smithi016 ceph-mon[30972]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:03 smithi016 ceph-mon[30972]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T00:51:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:03 smithi016 ceph-mon[30972]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T00:51:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:03 smithi016 ceph-mon[30972]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:51:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:03 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:03 smithi159 ceph-mon[35566]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:03 smithi159 ceph-mon[35566]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T00:51:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:03 smithi159 ceph-mon[35566]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T00:51:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:03 smithi159 ceph-mon[35566]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:51:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:03 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:04.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:03 smithi099 ceph-mon[35654]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:04.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:03 smithi099 ceph-mon[35654]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T00:51:04.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:03 smithi099 ceph-mon[35654]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T00:51:04.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:03 smithi099 ceph-mon[35654]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:51:04.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:03 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:04.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:51:04.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-09-12T00:51:04.601 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:51:04.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T00:51:04.265+0000 7fbc761ae640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T00:51:04.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T00:51:04.274+0000 7fbc6dfce640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:04.963 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:04.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:04.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:04.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:04.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:04.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 -- 172.21.15.99:0/1670483720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 msgr2=0x7fb0e0107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/1670483720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0107380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02a410 comp rx=0 tx=0).stop 2024-09-12T00:51:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 -- 172.21.15.99:0/1670483720 shutdown_connections 2024-09-12T00:51:05.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/1670483720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 0x7fb0e0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/1670483720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0107380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/1670483720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 0x7fb0e0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 -- 172.21.15.99:0/1670483720 >> 172.21.15.99:0/1670483720 conn(0x7fb0e00fc280 msgr2=0x7fb0e00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 -- 172.21.15.99:0/1670483720 shutdown_connections 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.049+0000 7fb0e79a0640 1 -- 172.21.15.99:0/1670483720 wait complete. 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 Processor -- start 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 -- start start 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 0x7fb0e0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 0x7fb0e019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e010d860 con 0x7fb0e0102660 2024-09-12T00:51:05.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e010d6e0 con 0x7fb0e0104f90 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e79a0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e010d9e0 con 0x7fb0e01078c0 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e4f3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e4f3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42816/0 (socket says 172.21.15.99:42816) 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e4f3b640 1 -- 172.21.15.99:0/3394890638 learned_addr learned my addr 172.21.15.99:0/3394890638 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e573c640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 0x7fb0e0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.050+0000 7fb0e5f3d640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 0x7fb0e019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 -- 172.21.15.99:0/3394890638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 msgr2=0x7fb0e019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 0x7fb0e019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 -- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 msgr2=0x7fb0e0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 0x7fb0e0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 -- 172.21.15.99:0/3394890638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e019faf0 con 0x7fb0e0104f90 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e4f3b640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc031610 tx=0x7fb0dc0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:05.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc03f030 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e019fd80 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc003ce0 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035070 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.051+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e01a0200 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.052+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e00fdd70 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.053+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0dc042030 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.054+0000 7fb0d67fc640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 0x7fb0cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.054+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7fb0dc0ee860 con 0x7fb0e0104f90 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.055+0000 7fb0e573c640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 0x7fb0cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.055+0000 7fb0e573c640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 0x7fb0cc0a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000abe0 tx=0x7fb0d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:05.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.059+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0dc0bc5f0 con 0x7fb0e0104f90 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:05.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:05.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256]' entity='osd.6' 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:05.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:05.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:05.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:05.240+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb0e0062630 con 0x7fb0cc0a2bb0 2024-09-12T00:51:05.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:51:05.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:51:05.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:05.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi016 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] boot 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T00:51:05.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:05 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:51:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:51:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi016 2024-09-12T00:51:06.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T00:51:06.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:06.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:06.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:06.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:06.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] boot 2024-09-12T00:51:06.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T00:51:06.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:05 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:51:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:51:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-12T00:51:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi016 2024-09-12T00:51:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-12T00:51:06.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:06.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:06.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:06.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:06.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: osd.6 [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] boot 2024-09-12T00:51:06.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T00:51:06.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:05 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:51:06.992 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:06 smithi099 ceph-mon[35654]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:06.992 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:06 smithi099 ceph-mon[35654]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T00:51:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:06 smithi159 ceph-mon[35566]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:06 smithi159 ceph-mon[35566]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T00:51:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:06 smithi016 ceph-mon[30972]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:06 smithi016 ceph-mon[30972]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/2313209237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]': finished 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T00:51:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/2313209237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]': finished 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T00:51:08.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:08.236 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:08.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/2313209237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]: dispatch 2024-09-12T00:51:08.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e1b90b2-de0e-4bd2-9cab-01595a0ec06e"}]': finished 2024-09-12T00:51:08.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T00:51:08.237 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/2720429376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:09.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/2720429376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/2720429376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:10.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:09 smithi159 ceph-mon[35566]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:10.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:09 smithi016 ceph-mon[30972]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:09 smithi099 ceph-mon[35654]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:11 smithi159 ceph-mon[35566]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:12.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:11 smithi016 ceph-mon[30972]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:12.242 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:11 smithi099 ceph-mon[35654]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:13.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:13 smithi099 ceph-mon[35654]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:13 smithi159 ceph-mon[35566]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:13 smithi016 ceph-mon[30972]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:15.969 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:15 smithi099 ceph-mon[35654]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:15 smithi159 ceph-mon[35566]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:16.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:15 smithi016 ceph-mon[30972]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T00:51:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:16 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:16 smithi159 ceph-mon[35566]: Deploying daemon osd.7 on smithi099 2024-09-12T00:51:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T00:51:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:16 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:16 smithi016 ceph-mon[30972]: Deploying daemon osd.7 on smithi099 2024-09-12T00:51:17.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T00:51:17.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:16 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:17.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:16 smithi099 ceph-mon[35654]: Deploying daemon osd.7 on smithi099 2024-09-12T00:51:18.061 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:17 smithi099 ceph-mon[35654]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:17 smithi159 ceph-mon[35566]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:18.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:17 smithi016 ceph-mon[30972]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:19 smithi099 ceph-mon[35654]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:19 smithi159 ceph-mon[35566]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:19 smithi016 ceph-mon[30972]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:22.075 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:21 smithi099 ceph-mon[35654]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:22.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:21 smithi159 ceph-mon[35566]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:21 smithi016 ceph-mon[30972]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:22.901 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:22.901 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:22.901 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:23.975 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:23 smithi099 ceph-mon[35654]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:24.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:23 smithi159 ceph-mon[35566]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:24.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:23 smithi016 ceph-mon[30972]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.618+0000 7fb0d67fc640 1 -- 172.21.15.99:0/3394890638 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb0e0062630 con 0x7fb0cc0a2bb0 2024-09-12T00:51:25.619 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.621+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 msgr2=0x7fb0cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.621+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 0x7fb0cc0a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000abe0 tx=0x7fb0d000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.621+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 msgr2=0x7fb0e0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.621+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc031610 tx=0x7fb0dc0043b0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 shutdown_connections 2024-09-12T00:51:25.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0cc0a2bb0 0x7fb0cc0a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e01078c0 0x7fb0e019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e0104f90 0x7fb0e0198400 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 --2- 172.21.15.99:0/3394890638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0102660 0x7fb0e0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 >> 172.21.15.99:0/3394890638 conn(0x7fb0e00fc280 msgr2=0x7fb0e0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 shutdown_connections 2024-09-12T00:51:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:51:25.622+0000 7fb0e79a0640 1 -- 172.21.15.99:0/3394890638 wait complete. 2024-09-12T00:51:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:25 smithi016 ceph-mon[30972]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:25 smithi099 ceph-mon[35654]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:26.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.324 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T00:51:26.326 INFO:tasks.cephadm:Deploying osd.8 on smithi159 with /dev/vg_nvme/lv_2... 2024-09-12T00:51:26.327 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T00:51:26.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:25 smithi159 ceph-mon[35566]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:26.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:26.583 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:51:27.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:26 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.024 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:26 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:26 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:26 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:27.996 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:27 smithi159 ceph-mon[35566]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:27.996 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:27 smithi159 ceph-mon[35566]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:27.996 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:27 smithi159 ceph-mon[35566]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:28.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:27 smithi099 ceph-mon[35654]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:28.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:27 smithi099 ceph-mon[35654]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:28.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:27 smithi099 ceph-mon[35654]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:28.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 00:51:27 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T00:51:27.799+0000 7f8e82dc8640 -1 osd.7 0 log_to_monitors true 2024-09-12T00:51:28.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:27 smithi016 ceph-mon[30972]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:28.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:27 smithi016 ceph-mon[30972]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:28.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:27 smithi016 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T00:51:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T00:51:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T00:51:29.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:29.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:28 smithi099 ceph-mon[35654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:51:29.254 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T00:51:29.254 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.254 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T00:51:29.255 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:29.255 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.255 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:28 smithi159 ceph-mon[35566]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:51:29.257 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:51:29.284 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: from='osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:51:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:28 smithi016 ceph-mon[30972]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:51:29.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T00:51:29.343+0000 7f8e7ed4b640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T00:51:29.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T00:51:29.353+0000 7f8e76b6b640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:51:29.578 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:51:30.053 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 -- 172.21.15.159:0/273971653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd874100b10 msgr2=0x7fd874100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:30.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/273971653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd874100b10 0x7fd874100f10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd870009900 tx=0x7fd87002a410 comp rx=0 tx=0).stop 2024-09-12T00:51:30.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 -- 172.21.15.159:0/273971653 shutdown_connections 2024-09-12T00:51:30.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/273971653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8741026d0 0x7fd874108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:30.054 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/273971653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/273971653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd874100b10 0x7fd874100f10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 -- 172.21.15.159:0/273971653 >> 172.21.15.159:0/273971653 conn(0x7fd8740fc2a0 msgr2=0x7fd8740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 -- 172.21.15.159:0/273971653 shutdown_connections 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.050+0000 7fd87bcf0640 1 -- 172.21.15.159:0/273971653 wait complete. 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 Processor -- start 2024-09-12T00:51:30.055 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 -- start start 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd874100b10 0x7fd874104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 0x7fd874103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410c770 con 0x7fd874100b10 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410c5f0 con 0x7fd874101d10 2024-09-12T00:51:30.056 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87bcf0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd87410c8f0 con 0x7fd8741026d0 2024-09-12T00:51:30.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87a28d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 0x7fd874103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:30.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:30.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87a28d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 0x7fd874103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48216/0 (socket says 172.21.15.159:48216) 2024-09-12T00:51:30.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87a28d640 1 -- 172.21.15.159:0/3492152433 learned_addr learned my addr 172.21.15.159:0/3492152433 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:51:30.057 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd879a8c640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd874100b10 0x7fd874104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 -- 172.21.15.159:0/3492152433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 msgr2=0x7fd874103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 0x7fd874103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 -- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd874100b10 msgr2=0x7fd874104be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd874100b10 0x7fd874104be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.051+0000 7fd87928b640 1 -- 172.21.15.159:0/3492152433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd874104030 con 0x7fd874101d10 2024-09-12T00:51:30.058 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd87928b640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874103230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e9f0 tx=0x7fd86400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:30.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86400ccc0 con 0x7fd874101d10 2024-09-12T00:51:30.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864010910 con 0x7fd874101d10 2024-09-12T00:51:30.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741a20c0 con 0x7fd874101d10 2024-09-12T00:51:30.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864018960 con 0x7fd874101d10 2024-09-12T00:51:30.059 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.052+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741a25d0 con 0x7fd874101d10 2024-09-12T00:51:30.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.053+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8741a2800 con 0x7fd874101d10 2024-09-12T00:51:30.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.054+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd864010ac0 con 0x7fd874101d10 2024-09-12T00:51:30.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.054+0000 7fd86affd640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 0x7fd8600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:30.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.055+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7fd8640ce8f0 con 0x7fd874101d10 2024-09-12T00:51:30.060 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.055+0000 7fd879a8c640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 0x7fd8600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:30.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.055+0000 7fd879a8c640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 0x7fd8600a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd870005ab0 tx=0x7fd870000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:30.061 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.059+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd86409bca0 con 0x7fd874101d10 2024-09-12T00:51:30.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:30.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T00:51:30.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='osd.7 ' entity='osd.7' 2024-09-12T00:51:30.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] boot 2024-09-12T00:51:30.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T00:51:30.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.232 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:30.233+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd87406aea0 con 0x7fd8600a2b80 2024-09-12T00:51:30.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:30.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T00:51:30.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='osd.7 ' entity='osd.7' 2024-09-12T00:51:30.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] boot 2024-09-12T00:51:30.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T00:51:30.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T00:51:30.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T00:51:30.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='osd.7 ' entity='osd.7' 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: osd.7 [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] boot 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T00:51:30.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi099 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:30 smithi016 ceph-mon[30972]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi099 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:30 smithi099 ceph-mon[35654]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T00:51:31.364 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi099 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:30 smithi159 ceph-mon[35566]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T00:51:32.202 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:31 smithi159 ceph-mon[35566]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:32.202 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:31 smithi159 ceph-mon[35566]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:32.203 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:31 smithi159 ceph-mon[35566]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T00:51:32.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:31 smithi099 ceph-mon[35654]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:32.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:31 smithi099 ceph-mon[35654]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:32.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:31 smithi099 ceph-mon[35654]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:31 smithi016 ceph-mon[30972]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:31 smithi016 ceph-mon[30972]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:32.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:31 smithi016 ceph-mon[30972]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/3411790848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]': finished 2024-09-12T00:51:33.240 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T00:51:33.241 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T00:51:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/3411790848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]': finished 2024-09-12T00:51:33.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T00:51:33.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T00:51:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/3411790848' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]: dispatch 2024-09-12T00:51:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c93a157-0688-48d0-a978-1cf9ef2cd1ba"}]': finished 2024-09-12T00:51:33.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T00:51:33.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:34.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:34 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/2631320731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:34.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:34 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/2631320731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:34.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:34 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/2631320731' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:35 smithi099 ceph-mon[35654]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:35 smithi159 ceph-mon[35566]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:35 smithi016 ceph-mon[30972]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:37 smithi099 ceph-mon[35654]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:37.578 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:37 smithi159 ceph-mon[35566]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:37 smithi016 ceph-mon[30972]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:39 smithi159 ceph-mon[35566]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:39 smithi099 ceph-mon[35654]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:39 smithi016 ceph-mon[30972]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:41 smithi159 ceph-mon[35566]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:41 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T00:51:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:41 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:41 smithi099 ceph-mon[35654]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:41 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T00:51:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:41 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:41 smithi016 ceph-mon[30972]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:41 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T00:51:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:41 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:42 smithi099 ceph-mon[35654]: Deploying daemon osd.8 on smithi159 2024-09-12T00:51:42.589 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:42 smithi159 ceph-mon[35566]: Deploying daemon osd.8 on smithi159 2024-09-12T00:51:42.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:42 smithi016 ceph-mon[30972]: Deploying daemon osd.8 on smithi159 2024-09-12T00:51:43.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:43 smithi159 ceph-mon[35566]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:43 smithi099 ceph-mon[35654]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:43 smithi016 ceph-mon[30972]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:45 smithi099 ceph-mon[35654]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:45 smithi159 ceph-mon[35566]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:45 smithi016 ceph-mon[30972]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:47 smithi099 ceph-mon[35654]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:47.547 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:47 smithi159 ceph-mon[35566]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:47.547 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:47.547 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:47.547 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:47 smithi016 ceph-mon[30972]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:49 smithi159 ceph-mon[35566]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:49 smithi099 ceph-mon[35654]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:49.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:49 smithi016 ceph-mon[30972]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:50.533 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.534+0000 7fd86affd640 1 -- 172.21.15.159:0/3492152433 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd87406aea0 con 0x7fd8600a2b80 2024-09-12T00:51:50.533 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 8 on host 'smithi159' 2024-09-12T00:51:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 msgr2=0x7fd8600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:50.535 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 0x7fd8600a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd870005ab0 tx=0x7fd870000bc0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 msgr2=0x7fd874103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874103230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd86400e9f0 tx=0x7fd86400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 shutdown_connections 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd8600a2b80 0x7fd8600a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8741026d0 0x7fd874103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd874101d10 0x7fd874103230 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 --2- 172.21.15.159:0/3492152433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd874100b10 0x7fd874104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 >> 172.21.15.159:0/3492152433 conn(0x7fd8740fc2a0 msgr2=0x7fd87406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 shutdown_connections 2024-09-12T00:51:50.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:51:50.536+0000 7fd87bcf0640 1 -- 172.21.15.159:0/3492152433 wait complete. 2024-09-12T00:51:51.186 DEBUG:teuthology.orchestra.run.smithi159:osd.8> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T00:51:51.188 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-12T00:51:51.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T00:51:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:51:51.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:51 smithi159 ceph-mon[35566]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:51.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:51 smithi016 ceph-mon[30972]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:51.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:52.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:51 smithi099 ceph-mon[35654]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:52.132 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:52.132 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:52.132 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:52.132 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:53.336 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 00:51:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T00:51:53.035+0000 7f5ec8c0b640 -1 osd.8 0 log_to_monitors true 2024-09-12T00:51:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:53 smithi159 ceph-mon[35566]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:53 smithi159 ceph-mon[35566]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:53 smithi159 ceph-mon[35566]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:53 smithi016 ceph-mon[30972]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:53 smithi016 ceph-mon[30972]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:53 smithi016 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:54.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:53 smithi099 ceph-mon[35654]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:54.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:53 smithi099 ceph-mon[35654]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:54.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:53 smithi099 ceph-mon[35654]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T00:51:54.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:51:54.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-12T00:51:54.626 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:51:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:54 smithi016 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T00:51:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:54 smithi016 ceph-mon[30972]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T00:51:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:54 smithi016 ceph-mon[30972]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:54 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:54 smithi016 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:54.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T00:51:54.549+0000 7f5ec4b8e640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T00:51:54.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T00:51:54.561+0000 7f5ec09b6640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:51:54.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-mon[35566]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T00:51:54.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-mon[35566]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T00:51:54.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-mon[35566]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:54.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:54.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:54 smithi159 ceph-mon[35566]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:54 smithi099 ceph-mon[35654]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T00:51:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:54 smithi099 ceph-mon[35654]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T00:51:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:54 smithi099 ceph-mon[35654]: from='osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:54 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:54 smithi099 ceph-mon[35654]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1741149575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 msgr2=0x7f68e010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1741149575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e010a050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009be0 tx=0x7f68d802a6c0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1741149575 shutdown_connections 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1741149575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e010a050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1741149575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1741149575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 0x7f68e0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1741149575 >> 172.21.15.16:0/1741149575 conn(0x7f68e00fd390 msgr2=0x7f68e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:51:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.117+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1741149575 shutdown_connections 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.118+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1741149575 wait complete. 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.118+0000 7f68e7c3a640 1 Processor -- start 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.118+0000 7f68e7c3a640 1 -- start start 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 0x7f68e0197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010d810 con 0x7f68e01037c0 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010d690 con 0x7f68e0101c00 2024-09-12T00:51:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e7c3a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68e010d990 con 0x7f68e0102e00 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e61d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:55142/0 (socket says 172.21.15.16:55142) 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e61d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41280/0 (socket says 172.21.15.16:41280) 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e59d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 0x7f68e0197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 -- 172.21.15.16:0/1733389266 learned_addr learned my addr 172.21.15.16:0/1733389266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 -- 172.21.15.16:0/1733389266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 msgr2=0x7f68e0197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 0x7f68e0197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 -- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 msgr2=0x7f68e019f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 -- 172.21.15.16:0/1733389266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e00744a0 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e61d7640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.119+0000 7f68e51d5640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9c0 tx=0x7f68d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.120+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d000ccc0 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.120+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d0005500 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.120+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e0074790 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.120+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0010430 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.120+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e0074cd0 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.121+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e0076110 con 0x7f68e0102e00 2024-09-12T00:51:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.122+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68d00046c0 con 0x7f68e0102e00 2024-09-12T00:51:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.122+0000 7f68d6ffd640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 0x7f68cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:51:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.123+0000 7f68e59d6640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 0x7f68cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:51:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.123+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7f68d00ce8a0 con 0x7f68e0102e00 2024-09-12T00:51:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.123+0000 7f68e59d6640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 0x7f68cc0a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f68e0102c60 tx=0x7f68dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:51:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.125+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68d009c230 con 0x7f68e0102e00 2024-09-12T00:51:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:51:55.311+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f68e0108d60 con 0x7f68cc0a2bb0 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi159 2024-09-12T00:51:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:55.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:55.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:55.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:55.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:55.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi159 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:55.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:55.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:55.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:55.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:56.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-12T00:51:56.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:51:56.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T00:51:56.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi159 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:56.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:51:56.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] boot 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T00:51:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:56 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] boot 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T00:51:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:56 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: osd.8 [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] boot 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T00:51:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:56 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:51:57.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:57 smithi016 ceph-mon[30972]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:51:57.814 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:57 smithi016 ceph-mon[30972]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T00:51:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:57 smithi159 ceph-mon[35566]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:51:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:57 smithi159 ceph-mon[35566]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T00:51:58.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:57 smithi099 ceph-mon[35654]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:51:58.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:57 smithi099 ceph-mon[35654]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T00:51:58.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:58 smithi016 ceph-mon[30972]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T00:51:58.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:58 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]: dispatch 2024-09-12T00:51:58.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:58 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]': finished 2024-09-12T00:51:58.759 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:58 smithi016 ceph-mon[30972]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T00:51:58.760 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:51:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:58 smithi159 ceph-mon[35566]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T00:51:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:58 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]: dispatch 2024-09-12T00:51:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:58 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]': finished 2024-09-12T00:51:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:58 smithi159 ceph-mon[35566]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T00:51:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:51:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:58 smithi099 ceph-mon[35654]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T00:51:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:58 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]: dispatch 2024-09-12T00:51:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:58 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2560591782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd2b63bf-d329-4318-87c5-945c8d0b12fb"}]': finished 2024-09-12T00:51:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:58 smithi099 ceph-mon[35654]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T00:51:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:51:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:59 smithi159 ceph-mon[35566]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:51:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:59 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3948186405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:59 smithi159 ceph-mon[35566]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T00:51:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:51:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:51:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:59 smithi016 ceph-mon[30972]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:51:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:59 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3948186405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:51:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:59 smithi016 ceph-mon[30972]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T00:51:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:51:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:59 smithi099 ceph-mon[35654]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:59 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3948186405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:59 smithi099 ceph-mon[35654]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T00:52:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:51:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:01.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:01 smithi159 ceph-mon[35566]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:01.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:01 smithi016 ceph-mon[30972]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:02.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:01 smithi099 ceph-mon[35654]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:03.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:03 smithi016 ceph-mon[30972]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T00:52:03.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:03 smithi099 ceph-mon[35654]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T00:52:03.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:03 smithi159 ceph-mon[35566]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T00:52:05.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:05 smithi159 ceph-mon[35566]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T00:52:05.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:05 smithi016 ceph-mon[30972]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T00:52:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:05 smithi099 ceph-mon[35654]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T00:52:07.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:07 smithi016 ceph-mon[30972]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:52:07.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T00:52:07.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:07.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:07 smithi016 ceph-mon[30972]: Deploying daemon osd.9 on smithi016 2024-09-12T00:52:07.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:07 smithi159 ceph-mon[35566]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:52:07.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T00:52:07.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:07.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:07 smithi159 ceph-mon[35566]: Deploying daemon osd.9 on smithi016 2024-09-12T00:52:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:07 smithi099 ceph-mon[35654]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T00:52:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T00:52:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:08.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:07 smithi099 ceph-mon[35654]: Deploying daemon osd.9 on smithi016 2024-09-12T00:52:09.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:09 smithi159 ceph-mon[35566]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:52:09.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:09 smithi016 ceph-mon[30972]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:52:10.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:09 smithi099 ceph-mon[35654]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T00:52:11.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:11 smithi159 ceph-mon[35566]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T00:52:11.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:11 smithi016 ceph-mon[30972]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T00:52:12.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:11 smithi099 ceph-mon[35654]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T00:52:13.842 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:13 smithi016 ceph-mon[30972]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:13.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:13.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:13.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:13 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:13.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:13 smithi159 ceph-mon[35566]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:13.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:13.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:13.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:13 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:14.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:13 smithi099 ceph-mon[35654]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:13 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:15.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:15 smithi016 ceph-mon[30972]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:15.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:15 smithi159 ceph-mon[35566]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:16.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:15 smithi099 ceph-mon[35654]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.149+0000 7f68d6ffd640 1 -- 172.21.15.16:0/1733389266 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f68e0108d60 con 0x7f68cc0a2bb0 2024-09-12T00:52:17.149 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-12T00:52:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 msgr2=0x7f68cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 0x7f68cc0a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f68e0102c60 tx=0x7f68dc00b390 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 msgr2=0x7f68e01983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e9c0 tx=0x7f68d000ee80 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 shutdown_connections 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f68cc0a2bb0 0x7f68cc0a5070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68e01037c0 0x7f68e019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68e0102e00 0x7f68e01983a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 --2- 172.21.15.16:0/1733389266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68e0101c00 0x7f68e0197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 >> 172.21.15.16:0/1733389266 conn(0x7f68e00fd390 msgr2=0x7f68e0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 shutdown_connections 2024-09-12T00:52:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:52:17.151+0000 7f68e7c3a640 1 -- 172.21.15.16:0/1733389266 wait complete. 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:17.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:17 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:17 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:17 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:17.939 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T00:52:17.941 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2024-09-12T00:52:17.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T00:52:18.198 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:52:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:19 smithi016 ceph-mon[30972]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:19.560 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:19 smithi099 ceph-mon[35654]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:19 smithi159 ceph-mon[35566]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:20.120 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 00:52:19 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T00:52:19.713+0000 7f432674c640 -1 osd.9 0 log_to_monitors true 2024-09-12T00:52:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:20 smithi016 ceph-mon[30972]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T00:52:20.477 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:20 smithi099 ceph-mon[35654]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T00:52:20.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:20 smithi159 ceph-mon[35566]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T00:52:20.888 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-12T00:52:20.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-09-12T00:52:21.176 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi016 2024-09-12T00:52:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:21 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:21.630 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi016 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.631 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:21.632 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:21.632 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:21 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi016 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:21.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:21.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 -- 172.21.15.99:0/118755718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 msgr2=0x7f0910102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 --2- 172.21.15.99:0/118755718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f0910102000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c02a410 comp rx=0 tx=0).stop 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 -- 172.21.15.99:0/118755718 shutdown_connections 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 --2- 172.21.15.99:0/118755718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09101037c0 0x7f091010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 --2- 172.21.15.99:0/118755718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0910102e00 0x7f0910103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 --2- 172.21.15.99:0/118755718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f0910102000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 -- 172.21.15.99:0/118755718 >> 172.21.15.99:0/118755718 conn(0x7f09100fd390 msgr2=0x7f09100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 -- 172.21.15.99:0/118755718 shutdown_connections 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.715+0000 7f0917af2640 1 -- 172.21.15.99:0/118755718 wait complete. 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 Processor -- start 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 -- start start 2024-09-12T00:52:21.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0910102e00 0x7f091019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09101037c0 0x7f09101a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f091010d890 con 0x7f09101037c0 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f091010d710 con 0x7f0910101c00 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f0917af2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f091010da10 con 0x7f0910102e00 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41454/0 (socket says 172.21.15.99:41454) 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 -- 172.21.15.99:0/1635189844 learned_addr learned my addr 172.21.15.99:0/1635189844 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091608f640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09101037c0 0x7f09101a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091508d640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0910102e00 0x7f091019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 -- 172.21.15.99:0/1635189844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0910102e00 msgr2=0x7f091019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:21.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0910102e00 0x7f091019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 -- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09101037c0 msgr2=0x7f09101a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09101037c0 0x7f09101a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 -- 172.21.15.99:0/1635189844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09101a1d60 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.716+0000 7f091588e640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f090c031610 tx=0x7f090c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.717+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c03f030 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.717+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090c003ce0 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.717+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101a1ff0 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.717+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c035070 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.717+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101a23f0 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.718+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09101a2620 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.719+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f090c042030 con 0x7f0910101c00 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.719+0000 7f0906ffd640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 0x7f08fc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:21.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.720+0000 7f091508d640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 0x7f08fc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:21.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.720+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f090c0eee00 con 0x7f0910101c00 2024-09-12T00:52:21.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.720+0000 7f091508d640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 0x7f08fc0a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f091019b650 tx=0x7f090000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:52:21.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.723+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f090c0bc030 con 0x7f0910101c00 2024-09-12T00:52:21.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:21.900+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0910075b40 con 0x7f08fc0a2bb0 2024-09-12T00:52:22.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T00:52:21.749+0000 7f43226cf640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T00:52:22.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 00:52:21 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T00:52:21.761+0000 7f431a4ef640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:52:22.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:22.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:22.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397]' entity='osd.9' 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] boot 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T00:52:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:23 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] boot 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T00:52:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:23 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:23.655 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:52:23.656 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:52:23.656 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T00:52:23.656 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: osd.9 [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] boot 2024-09-12T00:52:23.656 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T00:52:23.656 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:23 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:52:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/541250263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]': finished 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.99:0/843004106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/541250263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]': finished 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T00:52:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:25.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.99:0/843004106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T00:52:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/541250263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]: dispatch 2024-09-12T00:52:25.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df53f8db-e889-404c-a107-c3fde59b80b5"}]': finished 2024-09-12T00:52:25.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T00:52:25.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:25.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.99:0/843004106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:27 smithi159 ceph-mon[35566]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:27.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:27 smithi016 ceph-mon[30972]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:27 smithi099 ceph-mon[35654]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:29 smithi159 ceph-mon[35566]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:29.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:29 smithi016 ceph-mon[30972]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:29 smithi099 ceph-mon[35654]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:31 smithi159 ceph-mon[35566]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:31.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:31 smithi016 ceph-mon[30972]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:31.711 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:31 smithi099 ceph-mon[35654]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:33.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:33 smithi099 ceph-mon[35654]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:33.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T00:52:33.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:33.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:33 smithi099 ceph-mon[35654]: Deploying daemon osd.10 on smithi099 2024-09-12T00:52:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:33 smithi159 ceph-mon[35566]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T00:52:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:33 smithi159 ceph-mon[35566]: Deploying daemon osd.10 on smithi099 2024-09-12T00:52:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:33 smithi016 ceph-mon[30972]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T00:52:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:33 smithi016 ceph-mon[30972]: Deploying daemon osd.10 on smithi099 2024-09-12T00:52:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:35 smithi159 ceph-mon[35566]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:35.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:35 smithi016 ceph-mon[30972]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:35.742 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:35 smithi099 ceph-mon[35654]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:37 smithi159 ceph-mon[35566]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:37.632 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:37 smithi099 ceph-mon[35654]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:37 smithi016 ceph-mon[30972]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:39 smithi159 ceph-mon[35566]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:39 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:39.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:39 smithi099 ceph-mon[35654]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:39.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:39.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:39.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:39 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:39 smithi016 ceph-mon[30972]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:39 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:41 smithi159 ceph-mon[35566]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:41 smithi016 ceph-mon[30972]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:41.681 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:41 smithi099 ceph-mon[35654]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:42.366 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2024-09-12T00:52:42.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.366+0000 7f0906ffd640 1 -- 172.21.15.99:0/1635189844 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0910075b40 con 0x7f08fc0a2bb0 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 msgr2=0x7f08fc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 0x7f08fc0a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f091019b650 tx=0x7f090000b390 comp rx=0 tx=0).stop 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 msgr2=0x7f091019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f090c031610 tx=0x7f090c0043b0 comp rx=0 tx=0).stop 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 shutdown_connections 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08fc0a2bb0 0x7f08fc0a5070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:42.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09101037c0 0x7f09101a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:42.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0910102e00 0x7f091019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:42.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 --2- 172.21.15.99:0/1635189844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0910101c00 0x7f091019a130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:42.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 >> 172.21.15.99:0/1635189844 conn(0x7f09100fd390 msgr2=0x7f0910074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:52:42.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 shutdown_connections 2024-09-12T00:52:42.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-12T00:52:42.368+0000 7f0917af2640 1 -- 172.21.15.99:0/1635189844 wait complete. 2024-09-12T00:52:43.051 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T00:52:43.053 INFO:tasks.cephadm:Deploying osd.11 on smithi159 with /dev/vg_nvme/lv_1... 2024-09-12T00:52:43.054 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T00:52:43.314 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:52:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:43 smithi159 ceph-mon[35566]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:43 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.641 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:43 smithi099 ceph-mon[35654]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:43.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:43 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:43 smithi016 ceph-mon[30972]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:43 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:45.275 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 00:52:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T00:52:44.975+0000 7f3286bf1640 -1 osd.10 0 log_to_monitors true 2024-09-12T00:52:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:45 smithi159 ceph-mon[35566]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:45 smithi159 ceph-mon[35566]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:45 smithi159 ceph-mon[35566]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:45 smithi016 ceph-mon[30972]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:45 smithi016 ceph-mon[30972]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:45 smithi016 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:45.709 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:45 smithi099 ceph-mon[35654]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:45.709 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:45 smithi099 ceph-mon[35654]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:45.709 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:45 smithi099 ceph-mon[35654]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T00:52:46.000 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-12T00:52:46.027 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-09-12T00:52:46.284 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:52:46.535 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:46 smithi159 ceph-mon[35566]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T00:52:46.535 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:46 smithi159 ceph-mon[35566]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.536 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:46 smithi159 ceph-mon[35566]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T00:52:46.536 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:46 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:46.536 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:46 smithi159 ceph-mon[35566]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:46 smithi016 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T00:52:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:46 smithi016 ceph-mon[30972]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:46 smithi016 ceph-mon[30972]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T00:52:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:46 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:46 smithi016 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 -- 172.21.15.159:0/462423279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c101d10 msgr2=0x7f2e7c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:46.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 --2- 172.21.15.159:0/462423279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c101d10 0x7f2e7c102190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009900 tx=0x7f2e7802a410 comp rx=0 tx=0).stop 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 -- 172.21.15.159:0/462423279 shutdown_connections 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 --2- 172.21.15.159:0/462423279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 --2- 172.21.15.159:0/462423279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c101d10 0x7f2e7c102190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 --2- 172.21.15.159:0/462423279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c100b10 0x7f2e7c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 -- 172.21.15.159:0/462423279 >> 172.21.15.159:0/462423279 conn(0x7f2e7c0fc2a0 msgr2=0x7f2e7c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 -- 172.21.15.159:0/462423279 shutdown_connections 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.746+0000 7f2e847bc640 1 -- 172.21.15.159:0/462423279 wait complete. 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 Processor -- start 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 -- start start 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:46.760 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c101d10 0x7f2e7c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c10d8c0 con 0x7f2e7c1026d0 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c10d740 con 0x7f2e7c101d10 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e847bc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c10da40 con 0x7f2e7c100b10 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:47874/0 (socket says 172.21.15.159:47874) 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 -- 172.21.15.159:0/3870565023 learned_addr learned my addr 172.21.15.159:0/3870565023 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e81d57640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c101d10 0x7f2e7c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82d59640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 -- 172.21.15.159:0/3870565023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c101d10 msgr2=0x7f2e7c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c101d10 0x7f2e7c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:46.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 -- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 msgr2=0x7f2e7c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 -- 172.21.15.159:0/3870565023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7c19fad0 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82d59640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.747+0000 7f2e82558640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c970 tx=0x7f2e6c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.748+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c004220 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.748+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e6c0043c0 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.748+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7c19fdc0 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.748+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00f450 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.748+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7c1a0210 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.749+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7c107ec0 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.750+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2e6c0079b0 con 0x7f2e7c100b10 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.750+0000 7f2e737fe640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 0x7f2e680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:52:46.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.751+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7f2e6c0ce9f0 con 0x7f2e7c100b10 2024-09-12T00:52:46.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.751+0000 7f2e81d57640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 0x7f2e680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:52:46.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.751+0000 7f2e81d57640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 0x7f2e680a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e780314e0 tx=0x7f2e78000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:52:46.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.754+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e6c014030 con 0x7f2e7c100b10 2024-09-12T00:52:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-mon[35654]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T00:52:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-mon[35654]: from='osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-mon[35654]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T00:52:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-mon[35654]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-12T00:52:46.932 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:52:46.933+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2e7c062630 con 0x7f2e680a2bb0 2024-09-12T00:52:47.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T00:52:46.783+0000 7f3282b74640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T00:52:47.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 00:52:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T00:52:46.793+0000 7f327a994640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi099 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.597 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.598 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='osd.10 ' entity='osd.10' 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] boot 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T00:52:47.599 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:52:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi099 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='osd.10 ' entity='osd.10' 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] boot 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T00:52:47.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:47.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi099 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='osd.10 ' entity='osd.10' 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: osd.10 [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] boot 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T00:52:47.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:52:48.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:48 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:52:48.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:48 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:52:48.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:48 smithi016 ceph-mon[30972]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T00:52:48.690 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:48 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:52:48.690 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:48 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:52:48.690 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:48 smithi159 ceph-mon[35566]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T00:52:48.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:48 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:52:48.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:48 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:52:48.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:48 smithi099 ceph-mon[35654]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T00:52:49.639 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:49 smithi159 ceph-mon[35566]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:52:49.639 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:49 smithi159 ceph-mon[35566]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T00:52:49.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:49 smithi016 ceph-mon[30972]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:52:49.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:49 smithi016 ceph-mon[30972]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T00:52:49.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:49 smithi099 ceph-mon[35654]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:52:49.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:49 smithi099 ceph-mon[35654]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/559701259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]': finished 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:50.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:50 smithi016 ceph-mon[30972]: from='client.? 172.21.15.159:0/1517245918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/559701259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]': finished 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:50 smithi099 ceph-mon[35654]: from='client.? 172.21.15.159:0/1517245918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/559701259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]: dispatch 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6545934-6ceb-4f96-aecf-d9ded7ceefde"}]': finished 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:50 smithi159 ceph-mon[35566]: from='client.? 172.21.15.159:0/1517245918' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T00:52:51.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:51 smithi099 ceph-mon[35654]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T00:52:51.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:51 smithi099 ceph-mon[35654]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T00:52:51.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:51.824 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:51 smithi016 ceph-mon[30972]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T00:52:51.825 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:51 smithi016 ceph-mon[30972]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T00:52:51.825 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:51 smithi159 ceph-mon[35566]: pgmap v232: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T00:52:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:51 smithi159 ceph-mon[35566]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T00:52:51.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:52.675 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:52 smithi099 ceph-mon[35654]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T00:52:52.675 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:52 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:52 smithi159 ceph-mon[35566]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T00:52:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:52 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:52 smithi016 ceph-mon[30972]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T00:52:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:52 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:52:53.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:53 smithi099 ceph-mon[35654]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:53.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:53 smithi159 ceph-mon[35566]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:53 smithi016 ceph-mon[30972]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T00:52:55.698 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:55 smithi159 ceph-mon[35566]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T00:52:55.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:55 smithi099 ceph-mon[35654]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T00:52:55.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:55 smithi016 ceph-mon[30972]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T00:52:57.537 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:57 smithi159 ceph-mon[35566]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T00:52:57.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:57 smithi099 ceph-mon[35654]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T00:52:57.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:57 smithi016 ceph-mon[30972]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T00:52:58.723 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T00:52:58.724 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:58 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:58.724 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:58 smithi159 ceph-mon[35566]: Deploying daemon osd.11 on smithi159 2024-09-12T00:52:58.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T00:52:58.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:58 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:58.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:58 smithi099 ceph-mon[35654]: Deploying daemon osd.11 on smithi159 2024-09-12T00:52:58.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T00:52:58.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:58 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:52:58.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:58 smithi016 ceph-mon[30972]: Deploying daemon osd.11 on smithi159 2024-09-12T00:52:59.738 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:52:59 smithi159 ceph-mon[35566]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:52:59.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:52:59 smithi099 ceph-mon[35654]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:52:59.892 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:52:59 smithi016 ceph-mon[30972]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:01 smithi159 ceph-mon[35566]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:01.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:01 smithi016 ceph-mon[30972]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:01 smithi099 ceph-mon[35654]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:03.716 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:03 smithi159 ceph-mon[35566]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:03.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:03 smithi099 ceph-mon[35654]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:03 smithi016 ceph-mon[30972]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:04.562 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:04.562 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:04.562 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:04 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:04.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:04.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:04.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:04 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:04 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:05 smithi159 ceph-mon[35566]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:05 smithi016 ceph-mon[30972]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:05 smithi099 ceph-mon[35654]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:06.748 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.749+0000 7f2e737fe640 1 -- 172.21.15.159:0/3870565023 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2e7c062630 con 0x7f2e680a2bb0 2024-09-12T00:53:06.748 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 11 on host 'smithi159' 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 msgr2=0x7f2e680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 0x7f2e680a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e780314e0 tx=0x7f2e78000bc0 comp rx=0 tx=0).stop 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 msgr2=0x7f2e7c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c970 tx=0x7f2e6c00ce30 comp rx=0 tx=0).stop 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 shutdown_connections 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2e680a2bb0 0x7f2e680a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e7c1026d0 0x7f2e7c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e7c101d10 0x7f2e7c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:06.753 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 --2- 172.21.15.159:0/3870565023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e7c100b10 0x7f2e7c197ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:06.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 >> 172.21.15.159:0/3870565023 conn(0x7f2e7c0fc2a0 msgr2=0x7f2e7c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:06.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 shutdown_connections 2024-09-12T00:53:06.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:06.751+0000 7f2e847bc640 1 -- 172.21.15.159:0/3870565023 wait complete. 2024-09-12T00:53:07.359 DEBUG:teuthology.orchestra.run.smithi159:osd.11> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T00:53:07.362 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T00:53:07.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T00:53:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:07 smithi099 ceph-mon[35654]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:07 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:07 smithi016 ceph-mon[30972]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:08.052 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:07 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:07 smithi159 ceph-mon[35566]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:07 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/2367400540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec102690 msgr2=0x7fb9ec108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/2367400540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec102690 0x7fb9ec108f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009ca0 tx=0x7fb9e402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/2367400540 shutdown_connections 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/2367400540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec102690 0x7fb9ec108f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/2367400540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec101cd0 0x7fb9ec102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/2367400540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/2367400540 >> 172.21.15.16:0/2367400540 conn(0x7fb9ec0fc280 msgr2=0x7fb9ec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/2367400540 shutdown_connections 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.138+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/2367400540 wait complete. 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 Processor -- start 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 -- start start 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec101cd0 0x7fb9ec1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec102690 0x7fb9ec1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec10c750 con 0x7fb9ec100ad0 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec10c5d0 con 0x7fb9ec102690 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9f0ef8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec10c8d0 con 0x7fb9ec101cd0 2024-09-12T00:53:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9e9d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec101cd0 0x7fb9ec1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44984/0 (socket says 172.21.15.16:44984) 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.139+0000 7fb9ea59c640 1 -- 172.21.15.16:0/1030227593 learned_addr learned my addr 172.21.15.16:0/1030227593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ead9d640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec102690 0x7fb9ec1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 -- 172.21.15.16:0/1030227593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec101cd0 msgr2=0x7fb9ec1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec101cd0 0x7fb9ec1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 -- 172.21.15.16:0/1030227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec102690 msgr2=0x7fb9ec1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec102690 0x7fb9ec1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 -- 172.21.15.16:0/1030227593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ec103eb0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9ea59c640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400e9c0 tx=0x7fb9d400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400ccc0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d40035f0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ec1a1ff0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400ce60 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.140+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ec1a2450 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.141+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ec1a26e0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.142+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9d40037a0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.143+0000 7fb9e37fe640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 0x7fb9d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.144+0000 7fb9e9d9b640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 0x7fb9d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.144+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7fb9d40ceeb0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.144+0000 7fb9e9d9b640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 0x7fb9d80a50c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec1047e0 tx=0x7fb9dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.146+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9d409c3f0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.326+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb9ec1a29f0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.326+0000 7fb9e37fe640 1 -- 172.21.15.16:0/1030227593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fb9d409bcf0 con 0x7fb9ec100ad0 2024-09-12T00:53:08.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 msgr2=0x7fb9d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 0x7fb9d80a50c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec1047e0 tx=0x7fb9dc00a2d0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 msgr2=0x7fb9ec104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400e9c0 tx=0x7fb9d400edc0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 shutdown_connections 2024-09-12T00:53:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb9d80a2c00 0x7fb9d80a50c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9ec102690 0x7fb9ec1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9ec101cd0 0x7fb9ec1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 --2- 172.21.15.16:0/1030227593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9ec100ad0 0x7fb9ec104a60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.328+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 >> 172.21.15.16:0/1030227593 conn(0x7fb9ec0fc280 msgr2=0x7fb9ec0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.329+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 shutdown_connections 2024-09-12T00:53:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:08.329+0000 7fb9f0ef8640 1 -- 172.21.15.16:0/1030227593 wait complete. 2024-09-12T00:53:09.091 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1030227593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:09.093 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726102367,"num_in_osds":12,"osd_in_since":1726102369,"num_remapped_pgs":0} 2024-09-12T00:53:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1030227593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1030227593' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:09.916 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 00:53:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T00:53:09.857+0000 7f45db5f1640 -1 osd.11 0 log_to_monitors true 2024-09-12T00:53:10.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T00:53:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:09 smithi099 ceph-mon[35654]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:09 smithi099 ceph-mon[35654]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:09 smithi099 ceph-mon[35654]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.365 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:10.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:09 smithi159 ceph-mon[35566]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:09 smithi159 ceph-mon[35566]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:09 smithi159 ceph-mon[35566]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:09 smithi016 ceph-mon[30972]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:09 smithi016 ceph-mon[30972]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:09 smithi016 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 -- 172.21.15.16:0/4136399815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0100b10 msgr2=0x7fb5b0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/4136399815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0100b10 0x7fb5b0100f10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009900 tx=0x7fb5a402a410 comp rx=0 tx=0).stop 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 -- 172.21.15.16:0/4136399815 shutdown_connections 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/4136399815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b01026d0 0x7fb5b0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/4136399815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b0101d10 0x7fb5b0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/4136399815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0100b10 0x7fb5b0100f10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 -- 172.21.15.16:0/4136399815 >> 172.21.15.16:0/4136399815 conn(0x7fb5b00fc2a0 msgr2=0x7fb5b00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:10.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 -- 172.21.15.16:0/4136399815 shutdown_connections 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.878+0000 7fb5b6b98640 1 -- 172.21.15.16:0/4136399815 wait complete. 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 Processor -- start 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 -- start start 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 0x7fb5b006b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 0x7fb5b019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c7a0 con 0x7fb5b0101d10 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c620 con 0x7fb5b01026d0 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b6b98640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b010c920 con 0x7fb5b0100b10 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41258/0 (socket says 172.21.15.16:41258) 2024-09-12T00:53:10.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 -- 172.21.15.16:0/3594318274 learned_addr learned my addr 172.21.15.16:0/3594318274 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b4934640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 0x7fb5b006b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b5135640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 0x7fb5b019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 -- 172.21.15.16:0/3594318274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 msgr2=0x7fb5b006b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 0x7fb5b006b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 -- 172.21.15.16:0/3594318274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 msgr2=0x7fb5b019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 0x7fb5b019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5affff640 1 -- 172.21.15.16:0/3594318274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b019fb00 con 0x7fb5b0101d10 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b4934640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 0x7fb5b006b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.879+0000 7fb5b5135640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 0x7fb5b019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5affff640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00d950 tx=0x7fb59c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c017070 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c009c30 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b019fdf0 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00f6b0 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.880+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b01a02f0 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.881+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b0107f80 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.882+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb59c0109d0 con 0x7fb5b0101d10 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.882+0000 7fb5adffb640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 0x7fb5a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.883+0000 7fb5b4934640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 0x7fb5a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.883+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6831+0+0 (secure 0 0 0) 0x7fb59c0ce750 con 0x7fb5b0101d10 2024-09-12T00:53:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.883+0000 7fb5b4934640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 0x7fb5a00a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40314e0 tx=0x7fb5a4032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:10.887+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb59c009de0 con 0x7fb5b0101d10 2024-09-12T00:53:11.066 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T00:53:11.066 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.067 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T00:53:11.067 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:11.067 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.067 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:53:11.067 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:10 smithi159 ceph-mon[35566]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T00:53:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.068+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb5b0062df0 con 0x7fb5b0101d10 2024-09-12T00:53:11.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:53:11.069 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:10 smithi016 ceph-mon[30972]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T00:53:11.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.069+0000 7fb5adffb640 1 -- 172.21.15.16:0/3594318274 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fb59c09bbe0 con 0x7fb5b0101d10 2024-09-12T00:53:11.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 msgr2=0x7fb5a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 0x7fb5a00a50c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40314e0 tx=0x7fb5a4032cf0 comp rx=0 tx=0).stop 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 msgr2=0x7fb5b019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00d950 tx=0x7fb59c00de10 comp rx=0 tx=0).stop 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 shutdown_connections 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb5a00a2c00 0x7fb5a00a50c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5b01026d0 0x7fb5b019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b0101d10 0x7fb5b019cbf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 --2- 172.21.15.16:0/3594318274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5b0100b10 0x7fb5b006b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 >> 172.21.15.16:0/3594318274 conn(0x7fb5b00fc2a0 msgr2=0x7fb5b0105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.071+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 shutdown_connections 2024-09-12T00:53:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:11.072+0000 7fb5b6b98640 1 -- 172.21.15.16:0/3594318274 wait complete. 2024-09-12T00:53:11.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: from='osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-12T00:53:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:10 smithi099 ceph-mon[35654]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T00:53:11.786 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726102367,"num_in_osds":12,"osd_in_since":1726102369,"num_remapped_pgs":0} 2024-09-12T00:53:11.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T00:53:11.389+0000 7f45d7574640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T00:53:11.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T00:53:11.400+0000 7f45cf394640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3594318274' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: Detected new or changed devices on smithi159 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:11 smithi099 ceph-mon[35654]: from='osd.11 ' entity='osd.11' 2024-09-12T00:53:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3594318274' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:12.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: Detected new or changed devices on smithi159 2024-09-12T00:53:12.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-12T00:53:12.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.370 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.370 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:11 smithi159 ceph-mon[35566]: from='osd.11 ' entity='osd.11' 2024-09-12T00:53:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3594318274' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:12.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: Detected new or changed devices on smithi159 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:12.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:11 smithi016 ceph-mon[30972]: from='osd.11 ' entity='osd.11' 2024-09-12T00:53:12.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T00:53:13.057 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: purged_snaps scrub starts 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: purged_snaps scrub ok 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] boot 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T00:53:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: purged_snaps scrub starts 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: purged_snaps scrub ok 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] boot 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T00:53:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: purged_snaps scrub starts 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: purged_snaps scrub ok 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: osd.11 [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] boot 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T00:53:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:53:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 -- 172.21.15.16:0/730108840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 msgr2=0x7f1c10100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 --2- 172.21.15.16:0/730108840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c10100ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a410 comp rx=0 tx=0).stop 2024-09-12T00:53:13.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 -- 172.21.15.16:0/730108840 shutdown_connections 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 --2- 172.21.15.16:0/730108840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10102690 0x7f1c10108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 --2- 172.21.15.16:0/730108840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10101cd0 0x7f1c10102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 --2- 172.21.15.16:0/730108840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c10100ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 -- 172.21.15.16:0/730108840 >> 172.21.15.16:0/730108840 conn(0x7f1c100fc280 msgr2=0x7f1c100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 -- 172.21.15.16:0/730108840 shutdown_connections 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 -- 172.21.15.16:0/730108840 wait complete. 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.551+0000 7f1c16097640 1 Processor -- start 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 -- start start 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10101cd0 0x7f1c1019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10102690 0x7f1c101a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:13.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c750 con 0x7f1c10100ad0 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c5d0 con 0x7f1c10101cd0 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c16097640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c8d0 con 0x7f1c10102690 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41270/0 (socket says 172.21.15.16:41270) 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 -- 172.21.15.16:0/1518630542 learned_addr learned my addr 172.21.15.16:0/1518630542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0ffff640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10102690 0x7f1c101a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0effd640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10101cd0 0x7f1c1019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 -- 172.21.15.16:0/1518630542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10102690 msgr2=0x7f1c101a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10102690 0x7f1c101a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 -- 172.21.15.16:0/1518630542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10101cd0 msgr2=0x7f1c1019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:13.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10101cd0 0x7f1c1019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 -- 172.21.15.16:0/1518630542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c101a1db0 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.552+0000 7f1c0f7fe640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031610 tx=0x7f1c000043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.553+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0003f030 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.553+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c00003ce0 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.553+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a1fe0 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.553+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00035070 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.553+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a2430 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.554+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c10107f70 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.555+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c00042030 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.555+0000 7f1c0cff9640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 0x7f1bf80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.556+0000 7f1c0effd640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 0x7f1bf80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.556+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f1c000ef280 con 0x7f1c10100ad0 2024-09-12T00:53:13.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.556+0000 7f1c0effd640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 0x7f1bf80a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006b050 tx=0x7f1c0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:13.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.559+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c000bc030 con 0x7f1c10100ad0 2024-09-12T00:53:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.734+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1c1002da10 con 0x7f1c10100ad0 2024-09-12T00:53:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.734+0000 7f1c0cff9640 1 -- 172.21.15.16:0/1518630542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f1c000c0330 con 0x7f1c10100ad0 2024-09-12T00:53:13.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 msgr2=0x7f1bf80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 0x7f1bf80a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006b050 tx=0x7f1c0400b390 comp rx=0 tx=0).stop 2024-09-12T00:53:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 msgr2=0x7f1c1006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c00031610 tx=0x7f1c000043b0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 shutdown_connections 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1bf80a2bb0 0x7f1bf80a5070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c10102690 0x7f1c101a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c10101cd0 0x7f1c1019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 --2- 172.21.15.16:0/1518630542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10100ad0 0x7f1c1006aad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 >> 172.21.15.16:0/1518630542 conn(0x7f1c100fc280 msgr2=0x7f1c10105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 shutdown_connections 2024-09-12T00:53:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:13.736+0000 7f1c16097640 1 -- 172.21.15.16:0/1518630542 wait complete. 2024-09-12T00:53:14.055 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:13 smithi016 ceph-mon[30972]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:14.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:13 smithi016 ceph-mon[30972]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T00:53:14.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1518630542' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:13 smithi099 ceph-mon[35654]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:13 smithi099 ceph-mon[35654]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T00:53:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:13 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1518630542' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:13 smithi159 ceph-mon[35566]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-12T00:53:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:13 smithi159 ceph-mon[35566]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T00:53:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:13 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1518630542' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T00:53:14.492 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1726102392,"num_in_osds":12,"osd_in_since":1726102369,"num_remapped_pgs":0} 2024-09-12T00:53:14.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:53:14.752 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 -- 172.21.15.16:0/1463163504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540112b90 msgr2=0x7ff540114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 --2- 172.21.15.16:0/1463163504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540112b90 0x7ff540114f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff538009ca0 tx=0x7ff53802a950 comp rx=0 tx=0).stop 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 -- 172.21.15.16:0/1463163504 shutdown_connections 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 --2- 172.21.15.16:0/1463163504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540112b90 0x7ff540114f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 --2- 172.21.15.16:0/1463163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540074eb0 0x7ff540112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 --2- 172.21.15.16:0/1463163504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074590 0x7ff540074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 -- 172.21.15.16:0/1463163504 >> 172.21.15.16:0/1463163504 conn(0x7ff5400fd8c0 msgr2=0x7ff5400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 -- 172.21.15.16:0/1463163504 shutdown_connections 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.209+0000 7ff548131640 1 -- 172.21.15.16:0/1463163504 wait complete. 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 Processor -- start 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 -- start start 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074eb0 0x7ff5401a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540112b90 0x7ff54019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff540118550 con 0x7ff540074590 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5401183d0 con 0x7ff540074eb0 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff548131640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5401186d0 con 0x7ff540112b90 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41298/0 (socket says 172.21.15.16:41298) 2024-09-12T00:53:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 -- 172.21.15.16:0/3285336363 learned_addr learned my addr 172.21.15.16:0/3285336363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff5466ce640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540112b90 0x7ff54019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 -- 172.21.15.16:0/3285336363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540112b90 msgr2=0x7ff54019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540112b90 0x7ff54019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff5456cc640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074eb0 0x7ff5401a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 -- 172.21.15.16:0/3285336363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074eb0 msgr2=0x7ff5401a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074eb0 0x7ff5401a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.210+0000 7ff545ecd640 1 -- 172.21.15.16:0/3285336363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54019efe0 con 0x7ff540074590 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff545ecd640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9f0 tx=0x7ff53c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cd80 con 0x7ff540074590 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53c010910 con 0x7ff540074590 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54019f2d0 con 0x7ff540074590 2024-09-12T00:53:15.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c018990 con 0x7ff540074590 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.211+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5401ab520 con 0x7ff540074590 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.212+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff540075cd0 con 0x7ff540074590 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.213+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff53c010ac0 con 0x7ff540074590 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.214+0000 7ff536ffd640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.214+0000 7ff5456cc640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.214+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7ff53c0cf380 con 0x7ff540074590 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.215+0000 7ff5456cc640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5400fff50 tx=0x7ff530005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:15.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.218+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff53c09c760 con 0x7ff540074590 2024-09-12T00:53:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.395+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5401ab850 con 0x7ff540074590 2024-09-12T00:53:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.396+0000 7ff536ffd640 1 -- 172.21.15.16:0/3285336363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18380 (secure 0 0 0) 0x7ff53c09c060 con 0x7ff540074590 2024-09-12T00:53:15.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:15.396 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:53:13.394126+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:53:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 msgr2=0x7ff52c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5400fff50 tx=0x7ff530005be0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 msgr2=0x7ff5401a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9f0 tx=0x7ff53c00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 shutdown_connections 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff540112b90 0x7ff54019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540074eb0 0x7ff5401a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 --2- 172.21.15.16:0/3285336363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff540074590 0x7ff5401a46c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 >> 172.21.15.16:0/3285336363 conn(0x7ff5400fd8c0 msgr2=0x7ff5400fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 shutdown_connections 2024-09-12T00:53:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:15.398+0000 7ff548131640 1 -- 172.21.15.16:0/3285336363 wait complete. 2024-09-12T00:53:16.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:15 smithi016 ceph-mon[30972]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:16.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:15 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3285336363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:53:16.171 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T00:49:22.631453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T00:53:16.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T00:53:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:15 smithi099 ceph-mon[35654]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:15 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3285336363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:53:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:15 smithi159 ceph-mon[35566]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:15 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3285336363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:53:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:16.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 -- 172.21.15.16:0/1674207606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f81036c0 msgr2=0x7fa2f810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:16.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 --2- 172.21.15.16:0/1674207606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f81036c0 0x7fa2f810bbb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009900 tx=0x7fa2e802a480 comp rx=0 tx=0).stop 2024-09-12T00:53:16.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 -- 172.21.15.16:0/1674207606 shutdown_connections 2024-09-12T00:53:16.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 --2- 172.21.15.16:0/1674207606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f810c180 0x7fa2f810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 --2- 172.21.15.16:0/1674207606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f81036c0 0x7fa2f810bbb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 --2- 172.21.15.16:0/1674207606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f8102da0 0x7fa2f8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 -- 172.21.15.16:0/1674207606 >> 172.21.15.16:0/1674207606 conn(0x7fa2f80fc7a0 msgr2=0x7fa2f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 -- 172.21.15.16:0/1674207606 shutdown_connections 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.903+0000 7fa3001a8640 1 -- 172.21.15.16:0/1674207606 wait complete. 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 Processor -- start 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 -- start start 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 0x7fa2f81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f81036c0 0x7fa2f8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8112020 con 0x7fa2f810c180 2024-09-12T00:53:16.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f8111ea0 con 0x7fa2f8102da0 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa3001a8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f81121a0 con 0x7fa2f81036c0 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa2fe745640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa2fe745640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41326/0 (socket says 172.21.15.16:41326) 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa2fe745640 1 -- 172.21.15.16:0/41603871 learned_addr learned my addr 172.21.15.16:0/41603871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.904+0000 7fa2fd743640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f81036c0 0x7fa2f8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fdf44640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 0x7fa2f81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 -- 172.21.15.16:0/41603871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f81036c0 msgr2=0x7fa2f8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f81036c0 0x7fa2f8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 -- 172.21.15.16:0/41603871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 msgr2=0x7fa2f81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 0x7fa2f81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:16.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 -- 172.21.15.16:0/41603871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f819d550 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fdf44640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 0x7fa2f81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2fe745640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000b970 tx=0x7fa2f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0019070 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2f000cb40 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f819d840 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0007650 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.905+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81a5120 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.906+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f8103200 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.907+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa2f001c070 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.908+0000 7fa2eeffd640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 0x7fa2e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.908+0000 7fa2fdf44640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 0x7fa2e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.908+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa2f00ce190 con 0x7fa2f810c180 2024-09-12T00:53:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.909+0000 7fa2fdf44640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 0x7fa2e40a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009930 tx=0x7fa2f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:16.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:16.913+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2f009af40 con 0x7fa2f810c180 2024-09-12T00:53:17.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.088+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2f8107220 con 0x7fa2f810c180 2024-09-12T00:53:17.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.089+0000 7fa2eeffd640 1 -- 172.21.15.16:0/41603871 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fa2f009f240 con 0x7fa2f810c180 2024-09-12T00:53:17.088 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T00:53:17.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 msgr2=0x7fa2e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 0x7fa2e40a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009930 tx=0x7fa2f4008040 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 msgr2=0x7fa2f819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000b970 tx=0x7fa2f000be30 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 shutdown_connections 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa2e40a2b80 0x7fa2e40a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f810c180 0x7fa2f819ce70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2f81036c0 0x7fa2f8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 --2- 172.21.15.16:0/41603871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2f8102da0 0x7fa2f81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 >> 172.21.15.16:0/41603871 conn(0x7fa2f80fc7a0 msgr2=0x7fa2f80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 shutdown_connections 2024-09-12T00:53:17.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:17.091+0000 7fa3001a8640 1 -- 172.21.15.16:0/41603871 wait complete. 2024-09-12T00:53:17.900 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.b on smithi099 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.e on smithi099 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.h on smithi099 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.k on smithi099 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.c on smithi159 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.f on smithi159 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.i on smithi159 2024-09-12T00:53:17.901 INFO:tasks.cephadm:Adding mds.l on smithi159 2024-09-12T00:53:17.901 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l' 2024-09-12T00:53:18.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:17 smithi016 ceph-mon[30972]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:18.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:17 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/41603871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:53:18.164 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T00:53:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:17 smithi099 ceph-mon[35654]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:17 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/41603871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:53:18.285 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:17 smithi159 ceph-mon[35566]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:18.285 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:17 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/41603871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.708+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/1427767848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94100ad0 msgr2=0x7f1d94100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.708+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/1427767848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94100ad0 0x7f1d94100ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009900 tx=0x7f1d8402a3f0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/1427767848 shutdown_connections 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/1427767848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94102690 0x7f1d94108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/1427767848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94101cd0 0x7f1d94102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/1427767848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94100ad0 0x7f1d94100ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/1427767848 >> 172.21.15.159:0/1427767848 conn(0x7f1d940fc280 msgr2=0x7f1d940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:18.729 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/1427767848 shutdown_connections 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/1427767848 wait complete. 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 Processor -- start 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.709+0000 7f1d9c2cb640 1 -- start start 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94100ad0 0x7f1d9406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94101cd0 0x7f1d9419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410c7b0 con 0x7f1d94101cd0 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410c630 con 0x7f1d94100ad0 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9c2cb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9410c930 con 0x7f1d94102690 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55906/0 (socket says 172.21.15.159:55906) 2024-09-12T00:53:18.730 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 -- 172.21.15.159:0/3309128984 learned_addr learned my addr 172.21.15.159:0/3309128984 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d99866640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94101cd0 0x7f1d9419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a067640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94100ad0 0x7f1d9406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 -- 172.21.15.159:0/3309128984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94100ad0 msgr2=0x7f1d9406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94100ad0 0x7f1d9406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 -- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94101cd0 msgr2=0x7f1d9419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94101cd0 0x7f1d9419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 -- 172.21.15.159:0/3309128984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9419fbc0 con 0x7f1d94102690 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d9a868640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cce0 tx=0x7f1d8c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.710+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00f450 con 0x7f1d94102690 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.711+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9419fe50 con 0x7f1d94102690 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.711+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d8c00fad0 con 0x7f1d94102690 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.711+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c018b60 con 0x7f1d94102690 2024-09-12T00:53:18.731 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.711+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941a03e0 con 0x7f1d94102690 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.712+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94107ef0 con 0x7f1d94102690 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.714+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d8c00fc80 con 0x7f1d94102690 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.714+0000 7f1d8b7fe640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 0x7f1d800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.714+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f1d8c0cf0f0 con 0x7f1d94102690 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.714+0000 7f1d9a067640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 0x7f1d800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.715+0000 7f1d9a067640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 0x7f1d800a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006010 tx=0x7f1d84000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:18.732 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.718+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8c014030 con 0x7f1d94102690 2024-09-12T00:53:18.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.897+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}) -- 0x7f1d940627f0 con 0x7f1d800a2b80 2024-09-12T00:53:18.903 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.905+0000 7f1d8b7fe640 1 -- 172.21.15.159:0/3309128984 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1d940627f0 con 0x7f1d800a2b80 2024-09-12T00:53:18.904 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mds.all update... 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 msgr2=0x7f1d800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 0x7f1d800a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006010 tx=0x7f1d84000bc0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 msgr2=0x7f1d9419f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00cce0 tx=0x7f1d8c00a620 comp rx=0 tx=0).stop 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 shutdown_connections 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f1d800a2b80 0x7f1d800a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.906 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d94102690 0x7f1d9419f2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d94101cd0 0x7f1d9419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 --2- 172.21.15.159:0/3309128984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d94100ad0 0x7f1d9406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:18.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 >> 172.21.15.159:0/3309128984 conn(0x7f1d940fc280 msgr2=0x7f1d941050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:18.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 shutdown_connections 2024-09-12T00:53:18.907 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-12T00:53:18.907+0000 7f1d9c2cb640 1 -- 172.21.15.159:0/3309128984 wait complete. 2024-09-12T00:53:19.594 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T00:53:19.597 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T00:53:19.640 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T00:53:19.683 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T00:53:19.726 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T00:53:19.729 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T00:53:19.771 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T00:53:19.814 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T00:53:19.858 DEBUG:teuthology.orchestra.run.smithi159:mds.c> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T00:53:19.862 DEBUG:teuthology.orchestra.run.smithi159:mds.f> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T00:53:19.905 DEBUG:teuthology.orchestra.run.smithi159:mds.i> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T00:53:19.948 DEBUG:teuthology.orchestra.run.smithi159:mds.l> sudo journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T00:53:19.992 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T00:53:19.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:53:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:19 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:53:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-12T00:53:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:20.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:19 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:20.205 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:20.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:20.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:20.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:20.206 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:19 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:20.246 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 -- 172.21.15.16:0/3703267831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04106f10 msgr2=0x7f9d0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 --2- 172.21.15.16:0/3703267831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04106f10 0x7f9d0410ea90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009ca0 tx=0x7f9cfc02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 -- 172.21.15.16:0/3703267831 shutdown_connections 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 --2- 172.21.15.16:0/3703267831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04106f10 0x7f9d0410ea90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 --2- 172.21.15.16:0/3703267831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04102790 0x7f9d041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 --2- 172.21.15.16:0/3703267831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04101de0 0x7f9d041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 -- 172.21.15.16:0/3703267831 >> 172.21.15.16:0/3703267831 conn(0x7f9d040fd8c0 msgr2=0x7f9d040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 -- 172.21.15.16:0/3703267831 shutdown_connections 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.727+0000 7f9d0b799640 1 -- 172.21.15.16:0/3703267831 wait complete. 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 Processor -- start 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 -- start start 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04102790 0x7f9d04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04106f10 0x7f9d04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04111f70 con 0x7f9d04101de0 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04111df0 con 0x7f9d04102790 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.728+0000 7f9d0b799640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d041120f0 con 0x7f9d04106f10 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44670/0 (socket says 172.21.15.16:44670) 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 -- 172.21.15.16:0/2851900391 learned_addr learned my addr 172.21.15.16:0/2851900391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d08d34640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04102790 0x7f9d04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09d36640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04106f10 0x7f9d04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 -- 172.21.15.16:0/2851900391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04106f10 msgr2=0x7f9d04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04106f10 0x7f9d04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 -- 172.21.15.16:0/2851900391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04102790 msgr2=0x7f9d04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04102790 0x7f9d04074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 -- 172.21.15.16:0/2851900391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041a68b0 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d09535640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d950 tx=0x7f9d0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00017070 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d00009c30 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041a6ba0 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000f6b0 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.729+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d041a7130 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.730+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d041021c0 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.731+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9d000109d0 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.732+0000 7f9cfa7fc640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 0x7f9cf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.732+0000 7f9d08d34640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 0x7f9cf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.732+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f9d000ce680 con 0x7f9d04101de0 2024-09-12T00:53:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.732+0000 7f9d08d34640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 0x7f9cf00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d04075f80 tx=0x7f9cf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.736+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d00009de0 con 0x7f9d04101de0 2024-09-12T00:53:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.953+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d0402d7c0 con 0x7f9d04101de0 2024-09-12T00:53:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.953+0000 7f9cfa7fc640 1 -- 172.21.15.16:0/2851900391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f9d0009b430 con 0x7f9d04101de0 2024-09-12T00:53:20.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:20.953 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-12T00:46:20:927044+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T00:53:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-12T00:53:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 msgr2=0x7f9cf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 0x7f9cf00a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d04075f80 tx=0x7f9cf4005f90 comp rx=0 tx=0).stop 2024-09-12T00:53:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 msgr2=0x7f9d040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000d950 tx=0x7f9d0000de10 comp rx=0 tx=0).stop 2024-09-12T00:53:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 shutdown_connections 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9cf00a2c00 0x7f9cf00a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d04106f10 0x7f9d04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d04102790 0x7f9d04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 --2- 172.21.15.16:0/2851900391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d04101de0 0x7f9d040744f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 >> 172.21.15.16:0/2851900391 conn(0x7f9d040fd8c0 msgr2=0x7f9d040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 shutdown_connections 2024-09-12T00:53:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:20.956+0000 7f9d0b799640 1 -- 172.21.15.16:0/2851900391 wait complete. 2024-09-12T00:53:21.227 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:20 smithi159 ceph-mon[35566]: Deploying daemon mds.l on smithi159 2024-09-12T00:53:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:20 smithi099 ceph-mon[35654]: Deploying daemon mds.l on smithi159 2024-09-12T00:53:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:20 smithi016 ceph-mon[30972]: Deploying daemon mds.l on smithi159 2024-09-12T00:53:21.616 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 systemd[1]: Starting Ceph mds.l for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:21.881 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 podman[65105]: 2024-09-12 00:53:21.530658347 +0000 UTC m=+0.016125378 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:21.881 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 podman[65105]: 2024-09-12 00:53:21.646323292 +0000 UTC m=+0.131790326 container create 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T00:53:21.881 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 podman[65105]: 2024-09-12 00:53:21.882479564 +0000 UTC m=+0.367946595 container init 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T00:53:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:21 smithi016 ceph-mon[30972]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:22.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:21 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2851900391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:22.175 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:21 smithi099 ceph-mon[35654]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:22.175 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:21 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2851900391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:22.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:21 smithi159 ceph-mon[35566]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:22.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:21 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2851900391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:22.366 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 podman[65105]: 2024-09-12 00:53:21.885310115 +0000 UTC m=+0.370777149 container start 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T00:53:22.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 ceph-mds[65124]: -- 172.21.15.159:0/3349317088 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562501027c20 con 0x562501005000 2024-09-12T00:53:22.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: starting mds.l at 2024-09-12T00:53:22.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 bash[65105]: 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 2024-09-12T00:53:22.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 00:53:21 smithi159 systemd[1]: Started Ceph mds.l for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6832/310857906,v1:172.21.15.159:6833/310857906] up:boot 2024-09-12T00:53:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: fsmap 1 up:standby 2024-09-12T00:53:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:53:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:22 smithi099 ceph-mon[35654]: Deploying daemon mds.k on smithi099 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6832/310857906,v1:172.21.15.159:6833/310857906] up:boot 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: fsmap 1 up:standby 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:23.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:23.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:23.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:22 smithi159 ceph-mon[35566]: Deploying daemon mds.k on smithi099 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6832/310857906,v1:172.21.15.159:6833/310857906] up:boot 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: fsmap 1 up:standby 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:23.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:23.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:23.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:23.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:22 smithi016 ceph-mon[30972]: Deploying daemon mds.k on smithi099 2024-09-12T00:53:23.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:24.108 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:24 smithi099 ceph-mon[35654]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:24.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:24 smithi016 ceph-mon[30972]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:24.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:24 smithi159 ceph-mon[35566]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 -- 172.21.15.16:0/3008792288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 msgr2=0x7f6b4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 --2- 172.21.15.16:0/3008792288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4010ea90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009ca0 tx=0x7f6b3802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 -- 172.21.15.16:0/3008792288 shutdown_connections 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 --2- 172.21.15.16:0/3008792288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4010ea90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 --2- 172.21.15.16:0/3008792288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 0x7f6b401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 --2- 172.21.15.16:0/3008792288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 0x7f6b401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 -- 172.21.15.16:0/3008792288 >> 172.21.15.16:0/3008792288 conn(0x7f6b400fd8c0 msgr2=0x7f6b400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 -- 172.21.15.16:0/3008792288 shutdown_connections 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 -- 172.21.15.16:0/3008792288 wait complete. 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.443+0000 7f6b45c31640 1 Processor -- start 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 -- start start 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 0x7f6b40198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 0x7f6b40198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40111f70 con 0x7f6b40106f10 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b40111df0 con 0x7f6b40101de0 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b45c31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b401120f0 con 0x7f6b40102790 2024-09-12T00:53:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44700/0 (socket says 172.21.15.16:44700) 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 -- 172.21.15.16:0/1063926048 learned_addr learned my addr 172.21.15.16:0/1063926048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3f7fe640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 0x7f6b40198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3effd640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 0x7f6b40198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 -- 172.21.15.16:0/1063926048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 msgr2=0x7f6b40198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 0x7f6b40198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 -- 172.21.15.16:0/1063926048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 msgr2=0x7f6b40198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 0x7f6b40198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.444+0000 7f6b3ffff640 1 -- 172.21.15.16:0/1063926048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4019d2e0 con 0x7f6b40106f10 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b3ffff640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b38031a60 tx=0x7f6b38031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:24.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38003b70 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4019d570 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b38003d10 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b38004770 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.445+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b401a4eb0 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.446+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b401021c0 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.447+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b3802f790 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.448+0000 7f6b3cff9640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 0x7f6b280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.448+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6b380efed0 con 0x7f6b40106f10 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.448+0000 7f6b3f7fe640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 0x7f6b280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.448+0000 7f6b3f7fe640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 0x7f6b280a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000abe0 tx=0x7f6b3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.452+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b380bd2d0 con 0x7f6b40106f10 2024-09-12T00:53:24.524 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 podman[65087]: 2024-09-12 00:53:24.334438138 +0000 UTC m=+0.130263914 container create 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T00:53:24.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.664+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b4010a150 con 0x7f6b40106f10 2024-09-12T00:53:24.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.665+0000 7f6b3cff9640 1 -- 172.21.15.16:0/1063926048 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f6b380bcbd0 con 0x7f6b40106f10 2024-09-12T00:53:24.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:24.665 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-12T00:53:21:914580+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T00:53:24.665 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-12T00:53:24.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 msgr2=0x7f6b280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:24.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 0x7f6b280a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000abe0 tx=0x7f6b3000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 msgr2=0x7f6b4019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6b38031a60 tx=0x7f6b38031a90 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 shutdown_connections 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6b280a2b80 0x7f6b280a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b40106f10 0x7f6b4019cc00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b40102790 0x7f6b40198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 --2- 172.21.15.16:0/1063926048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b40101de0 0x7f6b40198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 >> 172.21.15.16:0/1063926048 conn(0x7f6b400fd8c0 msgr2=0x7f6b400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 shutdown_connections 2024-09-12T00:53:24.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:24.667+0000 7f6b45c31640 1 -- 172.21.15.16:0/1063926048 wait complete. 2024-09-12T00:53:25.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 podman[65087]: 2024-09-12 00:53:24.695758968 +0000 UTC m=+0.491584743 container init 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T00:53:25.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 podman[65087]: 2024-09-12 00:53:24.698579658 +0000 UTC m=+0.494405434 container start 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T00:53:25.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 ceph-mds[65106]: -- 172.21.15.99:0/131088348 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddc9961860 con 0x55ddc993f400 2024-09-12T00:53:25.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: starting mds.k at 2024-09-12T00:53:25.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 bash[65087]: 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 2024-09-12T00:53:25.025 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 00:53:24 smithi099 systemd[1]: Started Ceph mds.k for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1063926048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:25.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:25.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:25 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1063926048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:25 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1063926048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:25.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:25.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:25 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:26 smithi159 ceph-mon[35566]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:26 smithi159 ceph-mon[35566]: Deploying daemon mds.j on smithi016 2024-09-12T00:53:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:26 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6832/4113169911,v1:172.21.15.99:6833/4113169911] up:boot 2024-09-12T00:53:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:26 smithi159 ceph-mon[35566]: fsmap 2 up:standby 2024-09-12T00:53:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:26 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:53:26.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:26 smithi016 ceph-mon[30972]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:26.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:26 smithi016 ceph-mon[30972]: Deploying daemon mds.j on smithi016 2024-09-12T00:53:26.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:26 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6832/4113169911,v1:172.21.15.99:6833/4113169911] up:boot 2024-09-12T00:53:26.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:26 smithi016 ceph-mon[30972]: fsmap 2 up:standby 2024-09-12T00:53:26.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:26 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:53:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:26 smithi099 ceph-mon[35654]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:26 smithi099 ceph-mon[35654]: Deploying daemon mds.j on smithi016 2024-09-12T00:53:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:26 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6832/4113169911,v1:172.21.15.99:6833/4113169911] up:boot 2024-09-12T00:53:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:26 smithi099 ceph-mon[35654]: fsmap 2 up:standby 2024-09-12T00:53:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:26 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:53:27.393 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 systemd[1]: Starting Ceph mds.j for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:27.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:27.727 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 podman[69876]: 2024-09-12 00:53:27.361387985 +0000 UTC m=+0.016469032 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:27.728 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 podman[69876]: 2024-09-12 00:53:27.476672922 +0000 UTC m=+0.131753963 container create bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T00:53:27.728 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 podman[69876]: 2024-09-12 00:53:27.729343298 +0000 UTC m=+0.384424332 container init bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T00:53:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:28.023 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 podman[69876]: 2024-09-12 00:53:27.732603791 +0000 UTC m=+0.387684843 container start bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:53:28.023 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 ceph-mds[69925]: -- 172.21.15.16:0/509898961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cad9ebf860 con 0x55cad9e9d400 2024-09-12T00:53:28.023 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j[69909]: starting mds.j at 2024-09-12T00:53:28.023 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 bash[69876]: bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 2024-09-12T00:53:28.023 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 00:53:27 smithi016 systemd[1]: Started Ceph mds.j for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:28.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:28.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:28 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:28 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.391+0000 7f3a499c4640 1 -- 172.21.15.16:0/1515759193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 msgr2=0x7f3a4410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.391+0000 7f3a499c4640 1 --2- 172.21.15.16:0/1515759193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a4410d950 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009cd0 tx=0x7f3a3c02a8e0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.391+0000 7f3a499c4640 1 -- 172.21.15.16:0/1515759193 shutdown_connections 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.391+0000 7f3a499c4640 1 --2- 172.21.15.16:0/1515759193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a4410d950 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 --2- 172.21.15.16:0/1515759193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 0x7f3a44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 --2- 172.21.15.16:0/1515759193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 -- 172.21.15.16:0/1515759193 >> 172.21.15.16:0/1515759193 conn(0x7f3a440fc7a0 msgr2=0x7f3a440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 -- 172.21.15.16:0/1515759193 shutdown_connections 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 -- 172.21.15.16:0/1515759193 wait complete. 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 Processor -- start 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.392+0000 7f3a499c4640 1 -- start start 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 0x7f3a441992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a441a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44110e80 con 0x7f3a44101420 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44110d00 con 0x7f3a44106a80 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44111000 con 0x7f3a44100a80 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44736/0 (socket says 172.21.15.16:44736) 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 -- 172.21.15.16:0/2282278449 learned_addr learned my addr 172.21.15.16:0/2282278449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a42ffd640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 0x7f3a441992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a437fe640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a441a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 -- 172.21.15.16:0/2282278449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 msgr2=0x7f3a441992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 0x7f3a441992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 -- 172.21.15.16:0/2282278449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 msgr2=0x7f3a441a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a441a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 -- 172.21.15.16:0/2282278449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a441a4870 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a437fe640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a441a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a427fc640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d770 tx=0x7f3a3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38017070 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a38005500 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a441a4a40 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38004e00 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.393+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a441a4f50 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.394+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a441021c0 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.396+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a38004fa0 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.397+0000 7f3a489c2640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 0x7f3a2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.397+0000 7f3a42ffd640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 0x7f3a2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.397+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f3a380ceb30 con 0x7f3a44101420 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.397+0000 7f3a42ffd640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 0x7f3a2c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a34005e00 tx=0x7f3a34005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.400+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a3809bf10 con 0x7f3a44101420 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:28 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:28.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.627+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a44100ee0 con 0x7f3a44101420 2024-09-12T00:53:28.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.628+0000 7f3a489c2640 1 -- 172.21.15.16:0/2282278449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f3a3809b810 con 0x7f3a44101420 2024-09-12T00:53:28.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:28.626 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-09-12T00:53:28:036474+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T00:53:28.626 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 msgr2=0x7f3a2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 0x7f3a2c0a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a34005e00 tx=0x7f3a34005c50 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 msgr2=0x7f3a441997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d770 tx=0x7f3a3800dc30 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 shutdown_connections 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3a2c0a2c00 0x7f3a2c0a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a44106a80 0x7f3a441a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a44101420 0x7f3a441997f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 --2- 172.21.15.16:0/2282278449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a44100a80 0x7f3a441992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 >> 172.21.15.16:0/2282278449 conn(0x7f3a440fc7a0 msgr2=0x7f3a440fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 shutdown_connections 2024-09-12T00:53:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:28.630+0000 7f3a499c4640 1 -- 172.21.15.16:0/2282278449 wait complete. 2024-09-12T00:53:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:29 smithi016 ceph-mon[30972]: Deploying daemon mds.d on smithi016 2024-09-12T00:53:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:29 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6834/2008522291,v1:172.21.15.16:6835/2008522291] up:boot 2024-09-12T00:53:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:29 smithi016 ceph-mon[30972]: fsmap 3 up:standby 2024-09-12T00:53:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:29 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:53:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:29 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2282278449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:29 smithi159 ceph-mon[35566]: Deploying daemon mds.d on smithi016 2024-09-12T00:53:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:29 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6834/2008522291,v1:172.21.15.16:6835/2008522291] up:boot 2024-09-12T00:53:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:29 smithi159 ceph-mon[35566]: fsmap 3 up:standby 2024-09-12T00:53:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:29 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:53:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:29 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2282278449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:29 smithi099 ceph-mon[35654]: Deploying daemon mds.d on smithi016 2024-09-12T00:53:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:29 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6834/2008522291,v1:172.21.15.16:6835/2008522291] up:boot 2024-09-12T00:53:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:29 smithi099 ceph-mon[35654]: fsmap 3 up:standby 2024-09-12T00:53:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:29 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:53:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:29 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2282278449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:30.343 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:30 smithi016 ceph-mon[30972]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:30.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:30 smithi159 ceph-mon[35566]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:30.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:30 smithi099 ceph-mon[35654]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:31.055 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:30 smithi016 systemd[1]: Starting Ceph mds.d for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:31.339 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 podman[70291]: 2024-09-12 00:53:30.959368013 +0000 UTC m=+0.016325949 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:31.340 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 podman[70291]: 2024-09-12 00:53:31.089166426 +0000 UTC m=+0.146124350 container create dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:53:31.340 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 podman[70291]: 2024-09-12 00:53:31.341659378 +0000 UTC m=+0.398617317 container init dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:53:31.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:31.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 podman[70291]: 2024-09-12 00:53:31.34477097 +0000 UTC m=+0.401728904 container start dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T00:53:31.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 ceph-mds[70311]: -- 172.21.15.16:0/2493812713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597b754b860 con 0x5597b7529000 2024-09-12T00:53:31.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d[70307]: starting mds.d at 2024-09-12T00:53:31.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 bash[70291]: dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 2024-09-12T00:53:31.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 00:53:31 smithi016 systemd[1]: Started Ceph mds.d for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:31.740 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:32.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:32.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:32 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:32.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:32 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:32.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 -- 172.21.15.16:0/343894899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102570 msgr2=0x7f0f4c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:32.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 --2- 172.21.15.16:0/343894899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102570 0x7f0f4c1029b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02a480 comp rx=0 tx=0).stop 2024-09-12T00:53:32.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 -- 172.21.15.16:0/343894899 shutdown_connections 2024-09-12T00:53:32.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 --2- 172.21.15.16:0/343894899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 --2- 172.21.15.16:0/343894899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102570 0x7f0f4c1029b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 --2- 172.21.15.16:0/343894899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 -- 172.21.15.16:0/343894899 >> 172.21.15.16:0/343894899 conn(0x7f0f4c0fc7d0 msgr2=0x7f0f4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 -- 172.21.15.16:0/343894899 shutdown_connections 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.287+0000 7f0f54520640 1 -- 172.21.15.16:0/343894899 wait complete. 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 Processor -- start 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 -- start start 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102570 0x7f0f4c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c1120a0 con 0x7f0f4c102ef0 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c111f20 con 0x7f0f4c100e20 2024-09-12T00:53:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f54520640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c112220 con 0x7f0f4c102570 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42462/0 (socket says 172.21.15.16:42462) 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 -- 172.21.15.16:0/3111516473 learned_addr learned my addr 172.21.15.16:0/3111516473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f522bc640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f51abb640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102570 0x7f0f4c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 -- 172.21.15.16:0/3111516473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102570 msgr2=0x7f0f4c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102570 0x7f0f4c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 -- 172.21.15.16:0/3111516473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 msgr2=0x7f0f4c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f52abd640 1 -- 172.21.15.16:0/3111516473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1a6a50 con 0x7f0f4c102ef0 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.288+0000 7f0f522bc640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f52abd640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400eaa0 tx=0x7f0f4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400cdb0 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f44005020 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1a6d40 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440116b0 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.289+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1a7250 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.290+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4c101200 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.291+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f44005560 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.291+0000 7f0f437fe640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 0x7f0f380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.292+0000 7f0f522bc640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 0x7f0f380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.292+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f0f440ceca0 con 0x7f0f4c102ef0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.292+0000 7f0f522bc640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 0x7f0f380a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bfd0 tx=0x7f0f4800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:32.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.296+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f44016030 con 0x7f0f4c102ef0 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:32 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:32.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.506+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f4c109090 con 0x7f0f4c102ef0 2024-09-12T00:53:32.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.507+0000 7f0f437fe640 1 -- 172.21.15.16:0/3111516473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f0f4409ba50 con 0x7f0f4c102ef0 2024-09-12T00:53:32.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-12T00:53:32:048924+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T00:53:32.506 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-12T00:53:32.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 msgr2=0x7f0f380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 0x7f0f380a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800bfd0 tx=0x7f0f4800b390 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 msgr2=0x7f0f4c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400eaa0 tx=0x7f0f4400ef60 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 shutdown_connections 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0f380a2b80 0x7f0f380a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c102ef0 0x7f0f4c10ef50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f4c102570 0x7f0f4c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 --2- 172.21.15.16:0/3111516473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f4c100e20 0x7f0f4c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 >> 172.21.15.16:0/3111516473 conn(0x7f0f4c0fc7d0 msgr2=0x7f0f4c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:32.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 shutdown_connections 2024-09-12T00:53:32.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:32.509+0000 7f0f54520640 1 -- 172.21.15.16:0/3111516473 wait complete. 2024-09-12T00:53:33.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:33 smithi016 ceph-mon[30972]: Deploying daemon mds.e on smithi099 2024-09-12T00:53:33.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:33 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6836/2601027197,v1:172.21.15.16:6837/2601027197] up:boot 2024-09-12T00:53:33.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:33 smithi016 ceph-mon[30972]: fsmap 4 up:standby 2024-09-12T00:53:33.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:33 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:53:33.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:33 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3111516473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:33 smithi099 ceph-mon[35654]: Deploying daemon mds.e on smithi099 2024-09-12T00:53:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:33 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6836/2601027197,v1:172.21.15.16:6837/2601027197] up:boot 2024-09-12T00:53:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:33 smithi099 ceph-mon[35654]: fsmap 4 up:standby 2024-09-12T00:53:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:33 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:53:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:33 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3111516473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:33.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:33 smithi159 ceph-mon[35566]: Deploying daemon mds.e on smithi099 2024-09-12T00:53:33.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:33 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6836/2601027197,v1:172.21.15.16:6837/2601027197] up:boot 2024-09-12T00:53:33.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:33 smithi159 ceph-mon[35566]: fsmap 4 up:standby 2024-09-12T00:53:33.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:33 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:53:33.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:33 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3111516473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:33.774 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:33 smithi099 systemd[1]: Starting Ceph mds.e for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:34.060 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:33 smithi099 podman[65390]: 2024-09-12 00:53:33.684313506 +0000 UTC m=+0.015313311 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:34.060 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:33 smithi099 podman[65390]: 2024-09-12 00:53:33.815002726 +0000 UTC m=+0.146002520 container create bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T00:53:34.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:34 smithi159 ceph-mon[35566]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:34.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:34 smithi016 ceph-mon[30972]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:34 smithi099 ceph-mon[35654]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 podman[65390]: 2024-09-12 00:53:34.076077351 +0000 UTC m=+0.407077145 container init bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 podman[65390]: 2024-09-12 00:53:34.079032066 +0000 UTC m=+0.410031866 container start bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 ceph-mds[65409]: -- 172.21.15.99:0/883570431 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56364f92f860 con 0x56364f90d000 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: starting mds.e at 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 bash[65390]: bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f 2024-09-12T00:53:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 00:53:34 smithi099 systemd[1]: Started Ceph mds.e for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:35.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:35.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:35.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: Deploying daemon mds.f on smithi159 2024-09-12T00:53:35.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6834/3000703552,v1:172.21.15.99:6835/3000703552] up:boot 2024-09-12T00:53:35.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: fsmap 5 up:standby 2024-09-12T00:53:35.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:35 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: Deploying daemon mds.f on smithi159 2024-09-12T00:53:35.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6834/3000703552,v1:172.21.15.99:6835/3000703552] up:boot 2024-09-12T00:53:35.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: fsmap 5 up:standby 2024-09-12T00:53:35.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:35 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: Deploying daemon mds.f on smithi159 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6834/3000703552,v1:172.21.15.99:6835/3000703552] up:boot 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: fsmap 5 up:standby 2024-09-12T00:53:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:35 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:53:35.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:36.318 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 systemd[1]: Starting Ceph mds.f for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- 172.21.15.16:0/2063391066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8102ef0 msgr2=0x7f12f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/2063391066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8102ef0 0x7f12f810da40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009900 tx=0x7f12f002a480 comp rx=0 tx=0).stop 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- 172.21.15.16:0/2063391066 shutdown_connections 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/2063391066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8102ef0 0x7f12f810da40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/2063391066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102570 0x7f12f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/2063391066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8100e20 0x7f12f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- 172.21.15.16:0/2063391066 >> 172.21.15.16:0/2063391066 conn(0x7f12f80fc7d0 msgr2=0x7f12f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- 172.21.15.16:0/2063391066 shutdown_connections 2024-09-12T00:53:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- 172.21.15.16:0/2063391066 wait complete. 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 Processor -- start 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.305+0000 7f12ff3cd640 1 -- start start 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8102570 0x7f12f80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102ef0 0x7f12f80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12f8110f80 con 0x7f12f8100e20 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12f8110e00 con 0x7f12f8102ef0 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f8111100 con 0x7f12f8102570 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42484/0 (socket says 172.21.15.16:42484) 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 -- 172.21.15.16:0/4187691502 learned_addr learned my addr 172.21.15.16:0/4187691502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd96a640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102ef0 0x7f12f80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fc968640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8102570 0x7f12f80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 -- 172.21.15.16:0/4187691502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8102570 msgr2=0x7f12f80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8102570 0x7f12f80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 -- 172.21.15.16:0/4187691502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102ef0 msgr2=0x7f12f80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102ef0 0x7f12f80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 -- 172.21.15.16:0/4187691502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f81a6890 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12fd169640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12e800cce0 tx=0x7f12e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800f650 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f81a6b80 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e800fcd0 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.306+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8018c70 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.307+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f81a7110 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.308+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f8101200 con 0x7f12f8100e20 2024-09-12T00:53:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.309+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12e800f7f0 con 0x7f12f8100e20 2024-09-12T00:53:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.309+0000 7f12f67fc640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 0x7f12e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.310+0000 7f12fc968640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 0x7f12e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.310+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f12e80cf1b0 con 0x7f12f8100e20 2024-09-12T00:53:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.310+0000 7f12fc968640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 0x7f12e00a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12f80fe710 tx=0x7f12ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.314+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12e8014030 con 0x7f12f8100e20 2024-09-12T00:53:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.523+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12f810a0a0 con 0x7f12f8100e20 2024-09-12T00:53:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.523+0000 7f12f67fc640 1 -- 172.21.15.16:0/4187691502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f12e809c510 con 0x7f12f8100e20 2024-09-12T00:53:36.522 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:36.522 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-12T00:53:35:061916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T00:53:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 msgr2=0x7f12e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 0x7f12e00a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12f80fe710 tx=0x7f12ec005f90 comp rx=0 tx=0).stop 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 msgr2=0x7f12f80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12e800cce0 tx=0x7f12e800a620 comp rx=0 tx=0).stop 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 shutdown_connections 2024-09-12T00:53:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f12e00a2b80 0x7f12e00a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f8102ef0 0x7f12f80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f8102570 0x7f12f80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 --2- 172.21.15.16:0/4187691502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f8100e20 0x7f12f80fcc80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 >> 172.21.15.16:0/4187691502 conn(0x7f12f80fc7d0 msgr2=0x7f12f806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 shutdown_connections 2024-09-12T00:53:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:36.525+0000 7f12ff3cd640 1 -- 172.21.15.16:0/4187691502 wait complete. 2024-09-12T00:53:36.585 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 podman[65406]: 2024-09-12 00:53:36.222464424 +0000 UTC m=+0.016244822 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:36.585 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 podman[65406]: 2024-09-12 00:53:36.342209787 +0000 UTC m=+0.135990215 container create 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T00:53:36.586 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 podman[65406]: 2024-09-12 00:53:36.586813187 +0000 UTC m=+0.380593577 container init 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T00:53:36.869 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 podman[65406]: 2024-09-12 00:53:36.589757012 +0000 UTC m=+0.383537408 container start 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T00:53:36.869 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 ceph-mds[65426]: -- 172.21.15.159:0/1651417556 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2a55d3c20 con 0x55d2a55b1000 2024-09-12T00:53:36.869 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: starting mds.f at 2024-09-12T00:53:36.870 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 bash[65406]: 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 2024-09-12T00:53:36.870 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 00:53:36 smithi159 systemd[1]: Started Ceph mds.f for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4187691502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:37.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:37.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:37.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:37 smithi159 ceph-mon[35566]: Deploying daemon mds.c on smithi159 2024-09-12T00:53:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4187691502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:37.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:37.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:37.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:37 smithi099 ceph-mon[35654]: Deploying daemon mds.c on smithi159 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4187691502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:37.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:37 smithi016 ceph-mon[30972]: Deploying daemon mds.c on smithi159 2024-09-12T00:53:38.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:38 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:boot 2024-09-12T00:53:38.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:38 smithi159 ceph-mon[35566]: fsmap 6 up:standby 2024-09-12T00:53:38.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:38 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:53:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:38 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:boot 2024-09-12T00:53:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:38 smithi099 ceph-mon[35654]: fsmap 6 up:standby 2024-09-12T00:53:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:38 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:53:38.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:38 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:boot 2024-09-12T00:53:38.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:38 smithi016 ceph-mon[30972]: fsmap 6 up:standby 2024-09-12T00:53:38.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:38 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:53:38.866 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:38 smithi159 systemd[1]: Starting Ceph mds.c for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:39.120 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:38 smithi159 podman[65707]: 2024-09-12 00:53:38.770819314 +0000 UTC m=+0.016343606 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:39.120 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:38 smithi159 podman[65707]: 2024-09-12 00:53:38.881429132 +0000 UTC m=+0.126953467 container create 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:53:39.120 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 podman[65707]: 2024-09-12 00:53:39.11796261 +0000 UTC m=+0.363486910 container init 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T00:53:39.120 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 podman[65707]: 2024-09-12 00:53:39.121662986 +0000 UTC m=+0.367187278 container start 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T00:53:39.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:39.509 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:39.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:39 smithi099 ceph-mon[35654]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:39 smithi159 ceph-mon[35566]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:39.616 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 ceph-mds[65726]: -- 172.21.15.159:0/102491152 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fdb6f2f860 con 0x55fdb6f0d400 2024-09-12T00:53:39.616 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: starting mds.c at 2024-09-12T00:53:39.616 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 bash[65707]: 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 2024-09-12T00:53:39.617 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 00:53:39 smithi159 systemd[1]: Started Ceph mds.c for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:39.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:39 smithi016 ceph-mon[30972]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 -- 172.21.15.16:0/1140603963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102570 msgr2=0x7f14701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 --2- 172.21.15.16:0/1140603963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102570 0x7f14701029b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f145c009900 tx=0x7f145c02a480 comp rx=0 tx=0).stop 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 -- 172.21.15.16:0/1140603963 shutdown_connections 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 --2- 172.21.15.16:0/1140603963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102ef0 0x7f147010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 --2- 172.21.15.16:0/1140603963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102570 0x7f14701029b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 --2- 172.21.15.16:0/1140603963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 0x7f1470101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.013+0000 7f1476f66640 1 -- 172.21.15.16:0/1140603963 >> 172.21.15.16:0/1140603963 conn(0x7f14700fc7d0 msgr2=0x7f14700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.014+0000 7f1476f66640 1 -- 172.21.15.16:0/1140603963 shutdown_connections 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.014+0000 7f1476f66640 1 -- 172.21.15.16:0/1140603963 wait complete. 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.014+0000 7f1476f66640 1 Processor -- start 2024-09-12T00:53:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.014+0000 7f1476f66640 1 -- start start 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 0x7f1470198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 0x7f1470198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1470110fa0 con 0x7f1470102ef0 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1470110e20 con 0x7f1470102570 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1470111120 con 0x7f1470100e20 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50354/0 (socket says 172.21.15.16:50354) 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 -- 172.21.15.16:0/1391329358 learned_addr learned my addr 172.21.15.16:0/1391329358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f146ffff640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 0x7f1470198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1474d02640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 0x7f1470198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 -- 172.21.15.16:0/1391329358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 msgr2=0x7f1470198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 0x7f1470198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 -- 172.21.15.16:0/1391329358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 msgr2=0x7f1470198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 0x7f1470198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 -- 172.21.15.16:0/1391329358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147019d500 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f146ffff640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 0x7f1470198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1475503640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f146800eaa0 tx=0x7f146800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146800cdb0 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147019d7f0 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1468005020 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.015+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680116b0 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.016+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701a50d0 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.017+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1470101200 con 0x7f1470102ef0 2024-09-12T00:53:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.018+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1468005560 con 0x7f1470102ef0 2024-09-12T00:53:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.018+0000 7f146dffb640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 0x7f14600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.019+0000 7f1474d02640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 0x7f14600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.019+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f14680cee20 con 0x7f1470102ef0 2024-09-12T00:53:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.019+0000 7f1474d02640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 0x7f14600a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1464009890 tx=0x7f1464008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.023+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1468016030 con 0x7f1470102ef0 2024-09-12T00:53:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.234+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14701a52a0 con 0x7f1470102ef0 2024-09-12T00:53:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.235+0000 7f146dffb640 1 -- 172.21.15.16:0/1391329358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f146809c220 con 0x7f1470102ef0 2024-09-12T00:53:40.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:40.234 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-12T00:53:39:240883+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T00:53:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.237+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 msgr2=0x7f14600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.237+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 0x7f14600a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1464009890 tx=0x7f1464008040 comp rx=0 tx=0).stop 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.237+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 msgr2=0x7f147019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.237+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f146800eaa0 tx=0x7f146800ef60 comp rx=0 tx=0).stop 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 shutdown_connections 2024-09-12T00:53:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f14600a2b80 0x7f14600a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1470102ef0 0x7f147019ce20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1470102570 0x7f1470198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 --2- 172.21.15.16:0/1391329358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1470100e20 0x7f1470198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 >> 172.21.15.16:0/1391329358 conn(0x7f14700fc7d0 msgr2=0x7f1470069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 shutdown_connections 2024-09-12T00:53:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:40.238+0000 7f1476f66640 1 -- 172.21.15.16:0/1391329358 wait complete. 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:boot 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: fsmap 7 up:standby 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: Deploying daemon mds.h on smithi099 2024-09-12T00:53:40.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:40 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1391329358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:boot 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: fsmap 7 up:standby 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:40.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:40.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:40.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: Deploying daemon mds.h on smithi099 2024-09-12T00:53:40.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:40 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1391329358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:boot 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: fsmap 7 up:standby 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: Deploying daemon mds.h on smithi099 2024-09-12T00:53:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1391329358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:41 smithi099 ceph-mon[35654]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:41.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 systemd[1]: Starting Ceph mds.h for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:41 smithi159 ceph-mon[35566]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:41 smithi016 ceph-mon[30972]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:41.814 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 podman[65688]: 2024-09-12 00:53:41.427821723 +0000 UTC m=+0.015489336 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:41.814 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 podman[65688]: 2024-09-12 00:53:41.554539316 +0000 UTC m=+0.142206973 container create 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-12T00:53:41.814 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 podman[65688]: 2024-09-12 00:53:41.815891031 +0000 UTC m=+0.403558633 container init 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T00:53:42.274 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 podman[65688]: 2024-09-12 00:53:41.818722255 +0000 UTC m=+0.406389856 container start 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T00:53:42.274 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 ceph-mds[65707]: -- 172.21.15.99:0/2500848839 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2c3c13860 con 0x55e2c3bf0800 2024-09-12T00:53:42.274 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: starting mds.h at 2024-09-12T00:53:42.274 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 bash[65688]: 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 2024-09-12T00:53:42.274 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 00:53:41 smithi099 systemd[1]: Started Ceph mds.h for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:42.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:43.169 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: Deploying daemon mds.i on smithi159 2024-09-12T00:53:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:boot 2024-09-12T00:53:43.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: fsmap 8 up:standby 2024-09-12T00:53:43.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:42 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:53:43.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: Deploying daemon mds.i on smithi159 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:boot 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: fsmap 8 up:standby 2024-09-12T00:53:43.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:42 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:53:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: Deploying daemon mds.i on smithi159 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:boot 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: fsmap 8 up:standby 2024-09-12T00:53:43.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:42 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:53:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- 172.21.15.16:0/519449901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4100a80 msgr2=0x7fd3e4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 --2- 172.21.15.16:0/519449901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4100a80 0x7fd3e4100ee0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009930 tx=0x7fd3d802a420 comp rx=0 tx=0).stop 2024-09-12T00:53:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- 172.21.15.16:0/519449901 shutdown_connections 2024-09-12T00:53:43.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 --2- 172.21.15.16:0/519449901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4101420 0x7fd3e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 --2- 172.21.15.16:0/519449901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4100a80 0x7fd3e4100ee0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 --2- 172.21.15.16:0/519449901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4106a80 0x7fd3e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- 172.21.15.16:0/519449901 >> 172.21.15.16:0/519449901 conn(0x7fd3e40fc7a0 msgr2=0x7fd3e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- 172.21.15.16:0/519449901 shutdown_connections 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- 172.21.15.16:0/519449901 wait complete. 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 Processor -- start 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.696+0000 7fd3ea735640 1 -- start start 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 0x7fd3e4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4106a80 0x7fd3e419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4110ed0 con 0x7fd3e4101420 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4110d50 con 0x7fd3e4106a80 2024-09-12T00:53:43.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3ea735640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111050 con 0x7fd3e4100a80 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 0x7fd3e4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50374/0 (socket says 172.21.15.16:50374) 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 -- 172.21.15.16:0/3312422986 learned_addr learned my addr 172.21.15.16:0/3312422986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e8cd2640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4106a80 0x7fd3e419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 -- 172.21.15.16:0/3312422986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 msgr2=0x7fd3e4197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 0x7fd3e4197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 -- 172.21.15.16:0/3312422986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4106a80 msgr2=0x7fd3e419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4106a80 0x7fd3e419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 -- 172.21.15.16:0/3312422986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e419c2e0 con 0x7fd3e4101420 2024-09-12T00:53:43.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 0x7fd3e4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e37fe640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3d802a940 tx=0x7fd3d8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.697+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d803f070 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.698+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d8031c90 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.698+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e419c570 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.698+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8035030 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.698+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a3eb0 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.698+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3e41021c0 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.700+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3d8042070 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.701+0000 7fd3e17fa640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 0x7fd3cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.701+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 0x7fd3cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.701+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd3d80efa90 con 0x7fd3e4101420 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.701+0000 7fd3e3fff640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 0x7fd3cc0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009f40 tx=0x7fd3d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:43.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.704+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3d80bcdf0 con 0x7fd3e4101420 2024-09-12T00:53:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.916+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3e41a4080 con 0x7fd3e4101420 2024-09-12T00:53:43.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.917+0000 7fd3e17fa640 1 -- 172.21.15.16:0/3312422986 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7fd3d80bc6f0 con 0x7fd3e4101420 2024-09-12T00:53:43.916 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:43.916 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-12T00:53:42:288135+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T00:53:43.916 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 msgr2=0x7fd3cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 0x7fd3cc0a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009f40 tx=0x7fd3d4005f90 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 msgr2=0x7fd3e4197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3d802a940 tx=0x7fd3d8003b90 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 shutdown_connections 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3cc0a2b80 0x7fd3cc0a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3e4106a80 0x7fd3e419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e4101420 0x7fd3e4197870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 --2- 172.21.15.16:0/3312422986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e4100a80 0x7fd3e4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 >> 172.21.15.16:0/3312422986 conn(0x7fd3e40fc7a0 msgr2=0x7fd3e40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:43.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.919+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 shutdown_connections 2024-09-12T00:53:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:43.920+0000 7fd3ea735640 1 -- 172.21.15.16:0/3312422986 wait complete. 2024-09-12T00:53:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:43 smithi159 ceph-mon[35566]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:43 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3312422986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:43 smithi099 ceph-mon[35654]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:43 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3312422986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:44.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:43 smithi016 ceph-mon[30972]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:44.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:43 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3312422986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:44.866 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:53:44 smithi159 systemd[1]: Started Ceph mds.i for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: Deploying daemon mds.g on smithi016 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:boot 2024-09-12T00:53:45.869 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: fsmap 9 up:standby 2024-09-12T00:53:45.870 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:45 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:45.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:45.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: Deploying daemon mds.g on smithi016 2024-09-12T00:53:45.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:boot 2024-09-12T00:53:45.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: fsmap 9 up:standby 2024-09-12T00:53:45.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:45 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:53:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: Deploying daemon mds.g on smithi016 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:boot 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: fsmap 9 up:standby 2024-09-12T00:53:46.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:45 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:53:46.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:46.643 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:46 smithi016 systemd[1]: Starting Ceph mds.g for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:46.920 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:46 smithi016 podman[71067]: 2024-09-12 00:53:46.652158304 +0000 UTC m=+0.016508727 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:46.921 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:46 smithi016 podman[71067]: 2024-09-12 00:53:46.786271614 +0000 UTC m=+0.150622014 container create a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T00:53:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:47.320 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 podman[71067]: 2024-09-12 00:53:47.055661863 +0000 UTC m=+0.420012281 container init a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T00:53:47.320 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 podman[71067]: 2024-09-12 00:53:47.05872507 +0000 UTC m=+0.423075482 container start a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T00:53:47.320 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mds[71131]: -- 172.21.15.16:0/2899915401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56365a30b860 con 0x56365a2e8800 2024-09-12T00:53:47.320 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: starting mds.g at 2024-09-12T00:53:47.320 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 bash[71067]: a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb 2024-09-12T00:53:47.321 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 00:53:47 smithi016 systemd[1]: Started Ceph mds.g for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:47.632 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:47.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:47 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 -- 172.21.15.16:0/705425979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0101de0 msgr2=0x7f88d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 --2- 172.21.15.16:0/705425979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0101de0 0x7f88d01021c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009900 tx=0x7f88cc02a3f0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 -- 172.21.15.16:0/705425979 shutdown_connections 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 --2- 172.21.15.16:0/705425979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0106f10 0x7f88d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 --2- 172.21.15.16:0/705425979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0102790 0x7f88d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 --2- 172.21.15.16:0/705425979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0101de0 0x7f88d01021c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 -- 172.21.15.16:0/705425979 >> 172.21.15.16:0/705425979 conn(0x7f88d00fd8c0 msgr2=0x7f88d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 -- 172.21.15.16:0/705425979 shutdown_connections 2024-09-12T00:53:47.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.717+0000 7f88d7987640 1 -- 172.21.15.16:0/705425979 wait complete. 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 Processor -- start 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 -- start start 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101de0 0x7f88d0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 0x7f88d0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0111f30 con 0x7f88d0106f10 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0111db0 con 0x7f88d0101de0 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d7987640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01120b0 con 0x7f88d0102790 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d5f24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d5f24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50402/0 (socket says 172.21.15.16:50402) 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d5f24640 1 -- 172.21.15.16:0/2939937620 learned_addr learned my addr 172.21.15.16:0/2939937620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.718+0000 7f88d4f22640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 0x7f88d0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:47.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 -- 172.21.15.16:0/2939937620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 msgr2=0x7f88d0198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5723640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101de0 0x7f88d0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 0x7f88d0198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 -- 172.21.15.16:0/2939937620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101de0 msgr2=0x7f88d0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101de0 0x7f88d0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 -- 172.21.15.16:0/2939937620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d019d2c0 con 0x7f88d0106f10 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d4f22640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 0x7f88d0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d5f24640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88c800cce0 tx=0x7f88c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c800f450 con 0x7f88d0106f10 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c800fad0 con 0x7f88d0106f10 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d019d5b0 con 0x7f88d0106f10 2024-09-12T00:53:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8018c10 con 0x7f88d0106f10 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.719+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01a4e90 con 0x7f88d0106f10 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.720+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d01021c0 con 0x7f88d0106f10 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.721+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f88c800f5f0 con 0x7f88d0106f10 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.722+0000 7f88c67fc640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 0x7f88bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.722+0000 7f88d5723640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 0x7f88bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.722+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f88c80cf1d0 con 0x7f88d0106f10 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.722+0000 7f88d5723640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 0x7f88bc0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0096c0 tx=0x7f88cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.726+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88c8014030 con 0x7f88d0106f10 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:47 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.947+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88d01a5130 con 0x7f88d0106f10 2024-09-12T00:53:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.948+0000 7f88c67fc640 1 -- 172.21.15.16:0/2939937620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f88c809c5d0 con 0x7f88d0106f10 2024-09-12T00:53:47.947 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:47.947 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"btime":"2024-09-12T00:53:47:557919+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T00:53:47.947 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 11 2024-09-12T00:53:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 msgr2=0x7f88bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 0x7f88bc0a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0096c0 tx=0x7f88cc000bc0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 msgr2=0x7f88d019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:47.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f88c800cce0 tx=0x7f88c800a620 comp rx=0 tx=0).stop 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 shutdown_connections 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f88bc0a2c00 0x7f88bc0a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d0106f10 0x7f88d019cbe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d0102790 0x7f88d0198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 --2- 172.21.15.16:0/2939937620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101de0 0x7f88d0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 >> 172.21.15.16:0/2939937620 conn(0x7f88d00fd8c0 msgr2=0x7f88d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 shutdown_connections 2024-09-12T00:53:47.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:47.950+0000 7f88d7987640 1 -- 172.21.15.16:0/2939937620 wait complete. 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:47 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:48.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:48 smithi016 ceph-mon[30972]: Deploying daemon mds.a on smithi016 2024-09-12T00:53:48.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:48 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6838/3482588678,v1:172.21.15.16:6839/3482588678] up:boot 2024-09-12T00:53:48.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:48 smithi016 ceph-mon[30972]: fsmap 10 up:standby 2024-09-12T00:53:48.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:48 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:53:48.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:48 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2939937620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:48 smithi159 ceph-mon[35566]: Deploying daemon mds.a on smithi016 2024-09-12T00:53:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:48 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6838/3482588678,v1:172.21.15.16:6839/3482588678] up:boot 2024-09-12T00:53:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:48 smithi159 ceph-mon[35566]: fsmap 10 up:standby 2024-09-12T00:53:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:48 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:53:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:48 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2939937620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:48 smithi099 ceph-mon[35654]: Deploying daemon mds.a on smithi016 2024-09-12T00:53:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:48 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6838/3482588678,v1:172.21.15.16:6839/3482588678] up:boot 2024-09-12T00:53:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:48 smithi099 ceph-mon[35654]: fsmap 10 up:standby 2024-09-12T00:53:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:48 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:53:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:48 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2939937620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:49.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:49 smithi016 ceph-mon[30972]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:49.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:49 smithi159 ceph-mon[35566]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:50.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:49 smithi099 ceph-mon[35654]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:50.372 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 systemd[1]: Starting Ceph mds.a for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:50.633 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 podman[71484]: 2024-09-12 00:53:50.276166222 +0000 UTC m=+0.015900203 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:50.633 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 podman[71484]: 2024-09-12 00:53:50.390979751 +0000 UTC m=+0.130713717 container create 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T00:53:50.633 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 podman[71484]: 2024-09-12 00:53:50.635206355 +0000 UTC m=+0.374940322 container init 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T00:53:50.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:50.893 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 podman[71484]: 2024-09-12 00:53:50.638294822 +0000 UTC m=+0.378028798 container start 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T00:53:50.893 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 ceph-mds[71503]: -- 172.21.15.16:0/575453432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6dd18d860 con 0x55b6dd16b000 2024-09-12T00:53:50.893 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: starting mds.a at 2024-09-12T00:53:50.893 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 bash[71484]: 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 2024-09-12T00:53:50.893 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 00:53:50 smithi016 systemd[1]: Started Ceph mds.a for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 -- 172.21.15.16:0/1756417938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef0101280 msgr2=0x7f2ef010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 --2- 172.21.15.16:0/1756417938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef0101280 0x7f2ef010df40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009900 tx=0x7f2ee002a410 comp rx=0 tx=0).stop 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 -- 172.21.15.16:0/1756417938 shutdown_connections 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 --2- 172.21.15.16:0/1756417938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 0x7f2ef0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 --2- 172.21.15.16:0/1756417938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef0101280 0x7f2ef010df40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 --2- 172.21.15.16:0/1756417938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef01008d0 0x7f2ef0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.522+0000 7f2ef7959640 1 -- 172.21.15.16:0/1756417938 >> 172.21.15.16:0/1756417938 conn(0x7f2ef00fc7a0 msgr2=0x7f2ef00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 -- 172.21.15.16:0/1756417938 shutdown_connections 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 -- 172.21.15.16:0/1756417938 wait complete. 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 Processor -- start 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 -- start start 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef0101280 0x7f2ef019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.523+0000 7f2ef7959640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 0x7f2ef019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef7959640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01141b0 con 0x7f2ef01008d0 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef7959640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0114030 con 0x7f2ef010e480 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef7959640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0114330 con 0x7f2ef0101280 2024-09-12T00:53:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 -- 172.21.15.16:0/2385204507 learned_addr learned my addr 172.21.15.16:0/2385204507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef5ef6640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 0x7f2ef019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef4ef4640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef0101280 0x7f2ef019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 -- 172.21.15.16:0/2385204507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef0101280 msgr2=0x7f2ef019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef0101280 0x7f2ef019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 -- 172.21.15.16:0/2385204507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 msgr2=0x7f2ef019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 0x7f2ef019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 -- 172.21.15.16:0/2385204507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef019f700 con 0x7f2ef01008d0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef56f5640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c970 tx=0x7f2eec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007bb0 con 0x7f2ef01008d0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eec007d50 con 0x7f2ef01008d0 2024-09-12T00:53:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef019f9f0 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00f450 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.524+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01a72d0 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.525+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0100cb0 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.526+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2eec005020 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.527+0000 7f2ee67fc640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 0x7f2edc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.527+0000 7f2ef4ef4640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 0x7f2edc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.527+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2eec0cea20 con 0x7f2ef01008d0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.528+0000 7f2ef4ef4640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 0x7f2edc0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0102370 tx=0x7f2ee0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.531+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2eec014030 con 0x7f2ef01008d0 2024-09-12T00:53:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.744+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ef0109640 con 0x7f2ef01008d0 2024-09-12T00:53:51.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.745+0000 7f2ee67fc640 1 -- 172.21.15.16:0/2385204507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f2eec09bd80 con 0x7f2ef01008d0 2024-09-12T00:53:51.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:51.743 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-12T00:53:51:573148+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T00:53:51.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-12T00:53:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 msgr2=0x7f2edc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:51.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 0x7f2edc0a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0102370 tx=0x7f2ee0005a20 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 msgr2=0x7f2ef019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00c970 tx=0x7f2eec00ce30 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 shutdown_connections 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2edc0a2b80 0x7f2edc0a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ef010e480 0x7f2ef019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ef0101280 0x7f2ef019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 --2- 172.21.15.16:0/2385204507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ef01008d0 0x7f2ef019a750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 >> 172.21.15.16:0/2385204507 conn(0x7f2ef00fc7a0 msgr2=0x7f2ef00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 shutdown_connections 2024-09-12T00:53:51.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:51.747+0000 7f2ef7959640 1 -- 172.21.15.16:0/2385204507 wait complete. 2024-09-12T00:53:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: Deploying daemon mds.b on smithi099 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6840/2826352912,v1:172.21.15.16:6841/2826352912] up:boot 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: fsmap 11 up:standby 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:53:52.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:51 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2385204507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:52.124 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: Deploying daemon mds.b on smithi099 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6840/2826352912,v1:172.21.15.16:6841/2826352912] up:boot 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: fsmap 11 up:standby 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:53:52.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:51 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2385204507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:52.267 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:52.267 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.267 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.267 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: Deploying daemon mds.b on smithi099 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6840/2826352912,v1:172.21.15.16:6841/2826352912] up:boot 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: fsmap 11 up:standby 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:53:52.268 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:51 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2385204507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:53.236 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 systemd[1]: Starting Ceph mds.b for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T00:53:53.488 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 podman[65990]: 2024-09-12 00:53:53.140572247 +0000 UTC m=+0.019263541 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T00:53:53.488 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 podman[65990]: 2024-09-12 00:53:53.244028048 +0000 UTC m=+0.122719322 container create 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T00:53:53.489 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 podman[65990]: 2024-09-12 00:53:53.471977792 +0000 UTC m=+0.350669067 container init 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T00:53:53.489 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 podman[65990]: 2024-09-12 00:53:53.47507176 +0000 UTC m=+0.353763037 container start 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T00:53:53.774 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mds[66016]: -- 172.21.15.99:0/3130030807 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ec56b5860 con 0x556ec5693400 2024-09-12T00:53:53.774 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: starting mds.b at 2024-09-12T00:53:53.774 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 bash[65990]: 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 2024-09-12T00:53:53.774 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 00:53:53 smithi099 systemd[1]: Started Ceph mds.b for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:53 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:54.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:53 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:54.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:53 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:54.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:53:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:54 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:boot 2024-09-12T00:53:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:54 smithi159 ceph-mon[35566]: fsmap 12 up:standby 2024-09-12T00:53:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:54 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:53:55.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:54 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:boot 2024-09-12T00:53:55.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:54 smithi016 ceph-mon[30972]: fsmap 12 up:standby 2024-09-12T00:53:55.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:54 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:53:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:54 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:boot 2024-09-12T00:53:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:54 smithi099 ceph-mon[35654]: fsmap 12 up:standby 2024-09-12T00:53:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:54 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:53:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.435+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3224535624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b8102d30 msgr2=0x7fc1b8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.435+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3224535624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b8102d30 0x7fc1b8103110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009930 tx=0x7fc1a802a420 comp rx=0 tx=0).stop 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3224535624 shutdown_connections 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3224535624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 0x7fc1b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3224535624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b81036e0 0x7fc1b8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3224535624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b8102d30 0x7fc1b8103110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3224535624 >> 172.21.15.16:0/3224535624 conn(0x7fc1b80fc7d0 msgr2=0x7fc1b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3224535624 shutdown_connections 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3224535624 wait complete. 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.436+0000 7fc1bc8b6640 1 Processor -- start 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 -- start start 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8102d30 0x7fc1b81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 0x7fc1b819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b81120a0 con 0x7fc1b81036e0 2024-09-12T00:53:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8111f20 con 0x7fc1b8102d30 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1bc8b6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b8112220 con 0x7fc1b81082a0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59014/0 (socket says 172.21.15.16:59014) 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 -- 172.21.15.16:0/3271493796 learned_addr learned my addr 172.21.15.16:0/3271493796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b659c640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8102d30 0x7fc1b81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b6d9d640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 0x7fc1b819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 -- 172.21.15.16:0/3271493796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 msgr2=0x7fc1b819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 0x7fc1b819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 -- 172.21.15.16:0/3271493796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8102d30 msgr2=0x7fc1b81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8102d30 0x7fc1b81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 -- 172.21.15.16:0/3271493796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b819d550 con 0x7fc1b81036e0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.437+0000 7fc1b5d9b640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00e9f0 tx=0x7fc1ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:55.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.438+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00ccc0 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.438+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1ac010910 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.438+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b819d840 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.438+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac018920 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.438+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b81a5120 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.439+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b806a850 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.440+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1ac010ac0 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.441+0000 7fc1a77fe640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 0x7fc1a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.441+0000 7fc1b659c640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 0x7fc1a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.441+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc1ac0cee30 con 0x7fc1b81036e0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.441+0000 7fc1b659c640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 0x7fc1a00a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009900 tx=0x7fc1a80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:55.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.445+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1ac09bbe0 con 0x7fc1b81036e0 2024-09-12T00:53:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.661+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1b810a2b0 con 0x7fc1b81036e0 2024-09-12T00:53:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.662+0000 7fc1a77fe640 1 -- 172.21.15.16:0/3271493796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7fc1ac09fee0 con 0x7fc1b81036e0 2024-09-12T00:53:55.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:55.662 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-12T00:53:53:799730+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T00:53:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 msgr2=0x7fc1a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 0x7fc1a00a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009900 tx=0x7fc1a80058d0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 msgr2=0x7fc1b8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00e9f0 tx=0x7fc1ac00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 shutdown_connections 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc1a00a2b80 0x7fc1a00a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1b81082a0 0x7fc1b819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b81036e0 0x7fc1b8198ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 --2- 172.21.15.16:0/3271493796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1b8102d30 0x7fc1b81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 >> 172.21.15.16:0/3271493796 conn(0x7fc1b80fc7d0 msgr2=0x7fc1b8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 shutdown_connections 2024-09-12T00:53:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:55.664+0000 7fc1bc8b6640 1 -- 172.21.15.16:0/3271493796 wait complete. 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:55 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3271493796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:55 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3271493796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:56.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:55 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3271493796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:53:56.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T00:53:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 -- 172.21.15.16:0/1728919956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c0745c0 msgr2=0x7fa91c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 --2- 172.21.15.16:0/1728919956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c0745c0 0x7fa91c0749a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa910009900 tx=0x7fa91002a480 comp rx=0 tx=0).stop 2024-09-12T00:53:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 -- 172.21.15.16:0/1728919956 shutdown_connections 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 --2- 172.21.15.16:0/1728919956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e420 0x7fa91c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 --2- 172.21.15.16:0/1728919956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 --2- 172.21.15.16:0/1728919956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c0745c0 0x7fa91c0749a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 -- 172.21.15.16:0/1728919956 >> 172.21.15.16:0/1728919956 conn(0x7fa91c0fd8c0 msgr2=0x7fa91c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 -- 172.21.15.16:0/1728919956 shutdown_connections 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 -- 172.21.15.16:0/1728919956 wait complete. 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.167+0000 7fa923054640 1 Processor -- start 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 -- start start 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c0745c0 0x7fa91c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c114170 con 0x7fa91c10e420 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c113ff0 con 0x7fa91c0745c0 2024-09-12T00:53:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa923054640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1142f0 con 0x7fa91c074ee0 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59036/0 (socket says 172.21.15.16:59036) 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 -- 172.21.15.16:0/3441030671 learned_addr learned my addr 172.21.15.16:0/3441030671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa920df0640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c0745c0 0x7fa91c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa91bfff640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 -- 172.21.15.16:0/3441030671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 msgr2=0x7fa91c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 -- 172.21.15.16:0/3441030671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c0745c0 msgr2=0x7fa91c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c0745c0 0x7fa91c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 -- 172.21.15.16:0/3441030671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c19f620 con 0x7fa91c10e420 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa91bfff640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.168+0000 7fa9215f1640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9d0 tx=0x7fa91400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.169+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91400cd80 con 0x7fa91c10e420 2024-09-12T00:53:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.169+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c19f910 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.169+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa914005020 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.169+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9140116d0 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.169+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a71f0 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.170+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c0749a0 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.171+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa9140051d0 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.172+0000 7fa919ffb640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.172+0000 7fa920df0640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.172+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fa9140cf1d0 con 0x7fa91c10e420 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.172+0000 7fa920df0640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 0x7fa90c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa910009230 tx=0x7fa9100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.176+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa914016030 con 0x7fa91c10e420 2024-09-12T00:53:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.384+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa91c109010 con 0x7fa91c10e420 2024-09-12T00:53:57.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.385+0000 7fa919ffb640 1 -- 172.21.15.16:0/3441030671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7fa91409c530 con 0x7fa91c10e420 2024-09-12T00:53:57.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:53:57.385 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 msgr2=0x7fa90c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 0x7fa90c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa910009230 tx=0x7fa9100058d0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 msgr2=0x7fa91c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9d0 tx=0x7fa91400ee90 comp rx=0 tx=0).stop 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 shutdown_connections 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa90c0a2b80 0x7fa90c0a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c10e420 0x7fa91c19ef40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa91c074ee0 0x7fa91c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 --2- 172.21.15.16:0/3441030671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c0745c0 0x7fa91c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 >> 172.21.15.16:0/3441030671 conn(0x7fa91c0fd8c0 msgr2=0x7fa91c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 shutdown_connections 2024-09-12T00:53:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:57.387+0000 7fa923054640 1 -- 172.21.15.16:0/3441030671 wait complete. 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.463 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:57.464 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:57 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3441030671' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:57 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3441030671' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:53:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:57 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3441030671' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T00:53:58.723 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T00:53:58.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T00:53:58.992 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:53:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:53:59 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.281 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:53:59 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.369 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:53:59 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:53:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 -- 172.21.15.16:0/3356440398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed810c170 msgr2=0x7f6ed810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/3356440398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed810c170 0x7f6ed810e610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009ca0 tx=0x7f6ed002a950 comp rx=0 tx=0).stop 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 -- 172.21.15.16:0/3356440398 shutdown_connections 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/3356440398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed810c170 0x7f6ed810e610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/3356440398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8103660 0x7f6ed810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/3356440398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8102d40 0x7f6ed8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 -- 172.21.15.16:0/3356440398 >> 172.21.15.16:0/3356440398 conn(0x7f6ed80fc7a0 msgr2=0x7f6ed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 -- 172.21.15.16:0/3356440398 shutdown_connections 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.509+0000 7f6ee02a3640 1 -- 172.21.15.16:0/3356440398 wait complete. 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 Processor -- start 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 -- start start 2024-09-12T00:53:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111f30 con 0x7f6ed8102d40 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111db0 con 0x7f6ed810c170 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ee02a3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81120b0 con 0x7f6ed8103660 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35492/0 (socket says 172.21.15.16:35492) 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 -- 172.21.15.16:0/2822594497 learned_addr learned my addr 172.21.15.16:0/2822594497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede840640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6edd83e640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 -- 172.21.15.16:0/2822594497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 msgr2=0x7f6ed8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 -- 172.21.15.16:0/2822594497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 msgr2=0x7f6ed8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede03f640 1 -- 172.21.15.16:0/2822594497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81a6890 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6ede840640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.510+0000 7f6edd83e640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ede03f640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d950 tx=0x7f6ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4017070 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed4009c30 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81a6b80 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400f6b0 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.511+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a70c0 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.512+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8103120 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.513+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6ed40109d0 con 0x7f6ed8102d40 2024-09-12T00:53:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.514+0000 7f6ecf7fe640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 0x7f6ec40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:53:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.514+0000 7f6edd83e640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 0x7f6ec40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:53:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.514+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f6ed40ce940 con 0x7f6ed8102d40 2024-09-12T00:53:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.514+0000 7f6edd83e640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 0x7f6ec40a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005fd0 tx=0x7f6ec8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:53:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.518+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ed4009de0 con 0x7f6ed8102d40 2024-09-12T00:53:59.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:53:59.702+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6ed802d7c0 con 0x7f6ed8102d40 2024-09-12T00:54:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:54:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:54:00.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:00 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:54:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:00 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:54:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:00 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T00:54:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.653+0000 7f6ecf7fe640 1 -- 172.21.15.16:0/2822594497 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f6ed409bca0 con 0x7f6ed8102d40 2024-09-12T00:54:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 msgr2=0x7f6ec40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 0x7f6ec40a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8005fd0 tx=0x7f6ec8005850 comp rx=0 tx=0).stop 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 msgr2=0x7f6ed81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400d950 tx=0x7f6ed400de10 comp rx=0 tx=0).stop 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 shutdown_connections 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f6ec40a2b80 0x7f6ec40a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 --2- 172.21.15.16:0/2822594497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.655+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 >> 172.21.15.16:0/2822594497 conn(0x7f6ed80fc7a0 msgr2=0x7f6ed80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.656+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 shutdown_connections 2024-09-12T00:54:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:00.656+0000 7f6ee02a3640 1 -- 172.21.15.16:0/2822594497 wait complete. 2024-09-12T00:54:01.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T00:54:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:01.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:01 smithi016 ceph-mon[30972]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:01.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:01 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T00:54:01.922 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:01 smithi016 ceph-mon[30972]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T00:54:02.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:01 smithi099 ceph-mon[35654]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:02.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:01 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T00:54:02.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:01 smithi099 ceph-mon[35654]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T00:54:02.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:01 smithi159 ceph-mon[35566]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:02.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:01 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2822594497' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T00:54:02.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:01 smithi159 ceph-mon[35566]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T00:54:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 -- 172.21.15.16:0/3261647060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4108300 msgr2=0x7efde410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 --2- 172.21.15.16:0/3261647060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4108300 0x7efde410ec70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efddc009930 tx=0x7efddc02a420 comp rx=0 tx=0).stop 2024-09-12T00:54:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 -- 172.21.15.16:0/3261647060 shutdown_connections 2024-09-12T00:54:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 --2- 172.21.15.16:0/3261647060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4108300 0x7efde410ec70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 --2- 172.21.15.16:0/3261647060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4103780 0x7efde4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 --2- 172.21.15.16:0/3261647060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4102dd0 0x7efde41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 -- 172.21.15.16:0/3261647060 >> 172.21.15.16:0/3261647060 conn(0x7efde40fc7d0 msgr2=0x7efde40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 -- 172.21.15.16:0/3261647060 shutdown_connections 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 -- 172.21.15.16:0/3261647060 wait complete. 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.112+0000 7efde9cb4640 1 Processor -- start 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 -- start start 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 0x7efde4100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4108300 0x7efde41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efde4112120 con 0x7efde4102dd0 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efde4111fa0 con 0x7efde4103780 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde9cb4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efde41122a0 con 0x7efde4108300 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35508/0 (socket says 172.21.15.16:35508) 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 -- 172.21.15.16:0/2414095765 learned_addr learned my addr 172.21.15.16:0/2414095765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde3fff640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4108300 0x7efde41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde2ffd640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 0x7efde4100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 -- 172.21.15.16:0/2414095765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4108300 msgr2=0x7efde41014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4108300 0x7efde41014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 -- 172.21.15.16:0/2414095765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 msgr2=0x7efde4100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 0x7efde4100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde37fe640 1 -- 172.21.15.16:0/2414095765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde41a69c0 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.113+0000 7efde2ffd640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 0x7efde4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde37fe640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdd400eab0 tx=0x7efdd400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400ce40 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd4011950 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde41a6cb0 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4010ae0 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.114+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde41a71a0 con 0x7efde4102dd0 2024-09-12T00:54:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.115+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde41031b0 con 0x7efde4102dd0 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.116+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efdd4011b00 con 0x7efde4102dd0 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.117+0000 7efde0ff9640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.117+0000 7efde2ffd640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.117+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7efdd40cf500 con 0x7efde4102dd0 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.118+0000 7efde2ffd640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 0x7efdcc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efde41024c0 tx=0x7efdd8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.121+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdd4016030 con 0x7efde4102dd0 2024-09-12T00:54:02.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.298+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efde4101f60 con 0x7efde4102dd0 2024-09-12T00:54:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.732+0000 7efde0ff9640 1 -- 172.21.15.16:0/2414095765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7efdd409c6f0 con 0x7efde4102dd0 2024-09-12T00:54:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-12T00:54:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 msgr2=0x7efdcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 0x7efdcc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efde41024c0 tx=0x7efdd8005d20 comp rx=0 tx=0).stop 2024-09-12T00:54:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 msgr2=0x7efde4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdd400eab0 tx=0x7efdd400ef70 comp rx=0 tx=0).stop 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 shutdown_connections 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4108300 0x7efde41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efde4103780 0x7efde4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 --2- 172.21.15.16:0/2414095765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde4102dd0 0x7efde4100a30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 >> 172.21.15.16:0/2414095765 conn(0x7efde40fc7d0 msgr2=0x7efde40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 shutdown_connections 2024-09-12T00:54:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:02.735+0000 7efde9cb4640 1 -- 172.21.15.16:0/2414095765 wait complete. 2024-09-12T00:54:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:02 smithi099 ceph-mon[35654]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T00:54:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:02 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T00:54:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:02 smithi159 ceph-mon[35566]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T00:54:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:02 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T00:54:03.117 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:02 smithi016 ceph-mon[30972]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T00:54:03.117 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:02 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T00:54:03.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T00:54:03.725 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:03 smithi016 ceph-mon[30972]: pgmap v276: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:03.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:03 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T00:54:04.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:03 smithi016 ceph-mon[30972]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T00:54:04.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:03 smithi016 ceph-mon[30972]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T00:54:04.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:03 smithi016 ceph-mon[30972]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T00:54:04.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:03 smithi099 ceph-mon[35654]: pgmap v276: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:04.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:03 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T00:54:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:03 smithi099 ceph-mon[35654]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T00:54:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:03 smithi099 ceph-mon[35654]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T00:54:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:03 smithi099 ceph-mon[35654]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T00:54:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:03 smithi159 ceph-mon[35566]: pgmap v276: 33 pgs: 3 creating+peering, 26 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:03 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2414095765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T00:54:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:03 smithi159 ceph-mon[35566]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T00:54:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:03 smithi159 ceph-mon[35566]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T00:54:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:03 smithi159 ceph-mon[35566]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T00:54:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 -- 172.21.15.16:0/796814778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 msgr2=0x7f4fec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/796814778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec10bb50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe002a480 comp rx=0 tx=0).stop 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 -- 172.21.15.16:0/796814778 shutdown_connections 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/796814778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec10c170 0x7f4fec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/796814778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec10bb50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/796814778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec102d40 0x7f4fec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.240+0000 7f4ff0d08640 1 -- 172.21.15.16:0/796814778 >> 172.21.15.16:0/796814778 conn(0x7f4fec0fc7a0 msgr2=0x7f4fec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- 172.21.15.16:0/796814778 shutdown_connections 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- 172.21.15.16:0/796814778 wait complete. 2024-09-12T00:54:04.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 Processor -- start 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- start start 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec102d40 0x7f4fec198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec10c170 0x7f4fec19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec112030 con 0x7f4fec103660 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec111eb0 con 0x7f4fec102d40 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.241+0000 7f4ff0d08640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1121b0 con 0x7f4fec10c170 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35524/0 (socket says 172.21.15.16:35524) 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 -- 172.21.15.16:0/1083192153 learned_addr learned my addr 172.21.15.16:0/1083192153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fea59c640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec102d40 0x7f4fec198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fead9d640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec10c170 0x7f4fec19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 -- 172.21.15.16:0/1083192153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec10c170 msgr2=0x7f4fec19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec10c170 0x7f4fec19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 -- 172.21.15.16:0/1083192153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec102d40 msgr2=0x7f4fec198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec102d40 0x7f4fec198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 -- 172.21.15.16:0/1083192153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19d640 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fe9d9b640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005ab0 tx=0x7f4fe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe003f030 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe0003ce0 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19d8d0 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0035070 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.242+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a5210 con 0x7f4fec103660 2024-09-12T00:54:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.243+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec06b070 con 0x7f4fec103660 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.245+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fe0042030 con 0x7f4fec103660 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.246+0000 7f4fdb7fe640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 0x7f4fd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.246+0000 7f4fea59c640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 0x7f4fd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.246+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f4fe00ef940 con 0x7f4fec103660 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.246+0000 7f4fea59c640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 0x7f4fd40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009990 tx=0x7f4fdc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.249+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fe00bca30 con 0x7f4fec103660 2024-09-12T00:54:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.463+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4fec1071e0 con 0x7f4fec103660 2024-09-12T00:54:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.757+0000 7f4fdb7fe640 1 -- 172.21.15.16:0/1083192153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4fe00bc330 con 0x7f4fec103660 2024-09-12T00:54:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 msgr2=0x7f4fd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 0x7f4fd40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009990 tx=0x7f4fdc009110 comp rx=0 tx=0).stop 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 msgr2=0x7f4fec198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0005ab0 tx=0x7f4fe00042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 shutdown_connections 2024-09-12T00:54:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4fd40a2b80 0x7f4fd40a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4fec10c170 0x7f4fec19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fec103660 0x7f4fec198bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 --2- 172.21.15.16:0/1083192153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fec102d40 0x7f4fec198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.759+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 >> 172.21.15.16:0/1083192153 conn(0x7f4fec0fc7a0 msgr2=0x7f4fec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.760+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 shutdown_connections 2024-09-12T00:54:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:04.760+0000 7f4ff0d08640 1 -- 172.21.15.16:0/1083192153 wait complete. 2024-09-12T00:54:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T00:54:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:04 smithi099 ceph-mon[35654]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T00:54:05.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T00:54:05.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:04 smithi016 ceph-mon[30972]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T00:54:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T00:54:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:04 smithi159 ceph-mon[35566]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T00:54:05.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:54:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:06.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:05 smithi016 ceph-mon[30972]: pgmap v279: 97 pgs: 34 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:06.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:05 smithi016 ceph-mon[30972]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T00:54:06.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T00:54:06.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:05 smithi016 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T00:54:06.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:05 smithi016 ceph-mon[30972]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T00:54:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:05 smithi099 ceph-mon[35654]: pgmap v279: 97 pgs: 34 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:05 smithi099 ceph-mon[35654]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T00:54:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T00:54:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:05 smithi099 ceph-mon[35654]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T00:54:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:05 smithi099 ceph-mon[35654]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T00:54:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:05 smithi159 ceph-mon[35566]: pgmap v279: 97 pgs: 34 creating+peering, 50 unknown, 13 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:54:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:05 smithi159 ceph-mon[35566]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T00:54:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1083192153' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T00:54:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:05 smithi159 ceph-mon[35566]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T00:54:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:05 smithi159 ceph-mon[35566]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T00:54:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 -- 172.21.15.16:0/3751722969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420106f10 msgr2=0x7fc42010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3751722969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420106f10 0x7fc42010ea90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc418009ca0 tx=0x7fc41802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 -- 172.21.15.16:0/3751722969 shutdown_connections 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3751722969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420106f10 0x7fc42010ea90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3751722969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420102790 0x7fc4201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3751722969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420101de0 0x7fc4201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 -- 172.21.15.16:0/3751722969 >> 172.21.15.16:0/3751722969 conn(0x7fc4200fd8c0 msgr2=0x7fc4200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 -- 172.21.15.16:0/3751722969 shutdown_connections 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.230+0000 7fc4283e2640 1 -- 172.21.15.16:0/3751722969 wait complete. 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 Processor -- start 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 -- start start 2024-09-12T00:54:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 0x7fc420198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 0x7fc42019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc420111f70 con 0x7fc420101de0 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc420111df0 con 0x7fc420106f10 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc4283e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4201120f0 con 0x7fc420102790 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc42617e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc42617e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35538/0 (socket says 172.21.15.16:35538) 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.231+0000 7fc42617e640 1 -- 172.21.15.16:0/3196560349 learned_addr learned my addr 172.21.15.16:0/3196560349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42697f640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 0x7fc42019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42597d640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 0x7fc420198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 -- 172.21.15.16:0/3196560349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 msgr2=0x7fc420198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 0x7fc420198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 -- 172.21.15.16:0/3196560349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 msgr2=0x7fc42019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 0x7fc42019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 -- 172.21.15.16:0/3196560349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42019d2e0 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42697f640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 0x7fc42019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42597d640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 0x7fc420198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc42617e640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c920 tx=0x7fc41c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c007c70 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c007e10 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42019d5d0 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c018890 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.232+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4201a4eb0 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.233+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4201021c0 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.235+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc41c018a50 con 0x7fc420101de0 2024-09-12T00:54:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.236+0000 7fc4177fe640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 0x7fc40c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.236+0000 7fc42597d640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 0x7fc40c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.236+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fc41c0cf430 con 0x7fc420101de0 2024-09-12T00:54:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.236+0000 7fc42597d640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 0x7fc40c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc4100094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.239+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc41c014030 con 0x7fc420101de0 2024-09-12T00:54:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.414+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc420199690 con 0x7fc420101de0 2024-09-12T00:54:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.415+0000 7fc4177fe640 1 -- 172.21.15.16:0/3196560349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22191 (secure 0 0 0) 0x7fc41c09c4d0 con 0x7fc420101de0 2024-09-12T00:54:06.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:06.415 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:54:04.465131+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:54:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 msgr2=0x7fc40c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 0x7fc40c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc410005fd0 tx=0x7fc4100094d0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 msgr2=0x7fc420198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c920 tx=0x7fc41c00cde0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 shutdown_connections 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc40c0a2b80 0x7fc40c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc420106f10 0x7fc42019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc420102790 0x7fc420198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 --2- 172.21.15.16:0/3196560349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc420101de0 0x7fc420198330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 >> 172.21.15.16:0/3196560349 conn(0x7fc4200fd8c0 msgr2=0x7fc4200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 shutdown_connections 2024-09-12T00:54:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:06.418+0000 7fc4283e2640 1 -- 172.21.15.16:0/3196560349 wait complete. 2024-09-12T00:54:07.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:06 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:active 2024-09-12T00:54:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:06 smithi099 ceph-mon[35654]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:06 smithi099 ceph-mon[35654]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:06 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3196560349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:07.102 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:06 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:active 2024-09-12T00:54:07.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:06 smithi016 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:06 smithi016 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:06 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3196560349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:06 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] up:active 2024-09-12T00:54:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:06 smithi159 ceph-mon[35566]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:06 smithi159 ceph-mon[35566]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:06 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3196560349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:07.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:54:07.399 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:07.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:07 smithi016 ceph-mon[30972]: pgmap v281: 97 pgs: 31 creating+peering, 8 unknown, 58 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s wr, 6 op/s 2024-09-12T00:54:07.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:07 smithi016 ceph-mon[30972]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T00:54:07.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:07 smithi016 ceph-mon[30972]: Cluster is now healthy 2024-09-12T00:54:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 -- 172.21.15.16:0/3106111333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 msgr2=0x7f659c1003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 --2- 172.21.15.16:0/3106111333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c1003d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6598009900 tx=0x7f659802a410 comp rx=0 tx=0).stop 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 -- 172.21.15.16:0/3106111333 shutdown_connections 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 --2- 172.21.15.16:0/3106111333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f659c105120 0x7f659c10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 --2- 172.21.15.16:0/3106111333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 0x7f659c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 --2- 172.21.15.16:0/3106111333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c1003d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 -- 172.21.15.16:0/3106111333 >> 172.21.15.16:0/3106111333 conn(0x7f659c0fba90 msgr2=0x7f659c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 -- 172.21.15.16:0/3106111333 shutdown_connections 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.943+0000 7f65a358e640 1 -- 172.21.15.16:0/3106111333 wait complete. 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 Processor -- start 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 -- start start 2024-09-12T00:54:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 0x7f659c101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f659c105120 0x7f659c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f659c110e60 con 0x7f659c0ffff0 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f659c110ce0 con 0x7f659c1009a0 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a358e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f659c110fe0 con 0x7f659c105120 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a132a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a132a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35550/0 (socket says 172.21.15.16:35550) 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a132a640 1 -- 172.21.15.16:0/4223315725 learned_addr learned my addr 172.21.15.16:0/4223315725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.944+0000 7f65a0b29640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 0x7f659c101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 -- 172.21.15.16:0/4223315725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f659c105120 msgr2=0x7f659c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f659c105120 0x7f659c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 -- 172.21.15.16:0/4223315725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 msgr2=0x7f659c101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 0x7f659c101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 -- 172.21.15.16:0/4223315725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1a6a10 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a132a640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6598006010 tx=0x7f65980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659803f030 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6598003ce0 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a6ca0 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598035070 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.945+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a7180 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.946+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c1003d0 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.947+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6598042030 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.948+0000 7f658e7fc640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.948+0000 7f65a0b29640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.948+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f65980ef3b0 con 0x7f659c0ffff0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.948+0000 7f65a0b29640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 0x7f65880a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f659c103520 tx=0x7f659000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:07.952+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65980bc520 con 0x7f659c0ffff0 2024-09-12T00:54:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:07 smithi099 ceph-mon[35654]: pgmap v281: 97 pgs: 31 creating+peering, 8 unknown, 58 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s wr, 6 op/s 2024-09-12T00:54:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:07 smithi099 ceph-mon[35654]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T00:54:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:07 smithi099 ceph-mon[35654]: Cluster is now healthy 2024-09-12T00:54:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:07 smithi159 ceph-mon[35566]: pgmap v281: 97 pgs: 31 creating+peering, 8 unknown, 58 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 675 B/s wr, 6 op/s 2024-09-12T00:54:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:07 smithi159 ceph-mon[35566]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T00:54:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:07 smithi159 ceph-mon[35566]: Cluster is now healthy 2024-09-12T00:54:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.129+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f659c109310 con 0x7f659c0ffff0 2024-09-12T00:54:08.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.131+0000 7f658e7fc640 1 -- 172.21.15.16:0/4223315725 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22191 (secure 0 0 0) 0x7f65980bbe20 con 0x7f659c0ffff0 2024-09-12T00:54:08.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:08.131 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:54:04.465131+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 msgr2=0x7f65880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 0x7f65880a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f659c103520 tx=0x7f659000b390 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 msgr2=0x7f659c101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6598006010 tx=0x7f65980042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 shutdown_connections 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f65880a2c00 0x7f65880a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f659c105120 0x7f659c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f659c1009a0 0x7f659c101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 --2- 172.21.15.16:0/4223315725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659c0ffff0 0x7f659c101a90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 >> 172.21.15.16:0/4223315725 conn(0x7f659c0fba90 msgr2=0x7f659c10dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:08.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 shutdown_connections 2024-09-12T00:54:08.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:08.133+0000 7f65a358e640 1 -- 172.21.15.16:0/4223315725 wait complete. 2024-09-12T00:54:08.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T00:54:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4223315725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4223315725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:09.123 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:09.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4223315725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.624+0000 7f06427a0640 1 -- 172.21.15.16:0/680247501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c101de0 msgr2=0x7f063c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:09.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.624+0000 7f06427a0640 1 --2- 172.21.15.16:0/680247501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c101de0 0x7f063c1021c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f062c009900 tx=0x7f062c02a410 comp rx=0 tx=0).stop 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 -- 172.21.15.16:0/680247501 shutdown_connections 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 --2- 172.21.15.16:0/680247501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c106f10 0x7f063c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 --2- 172.21.15.16:0/680247501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c102790 0x7f063c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 --2- 172.21.15.16:0/680247501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c101de0 0x7f063c1021c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 -- 172.21.15.16:0/680247501 >> 172.21.15.16:0/680247501 conn(0x7f063c0fd8c0 msgr2=0x7f063c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 -- 172.21.15.16:0/680247501 shutdown_connections 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 -- 172.21.15.16:0/680247501 wait complete. 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 Processor -- start 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.625+0000 7f06427a0640 1 -- start start 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c101de0 0x7f063c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:09.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c106f10 0x7f063c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f063c112010 con 0x7f063c102790 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f063c111e90 con 0x7f063c106f10 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f063c112190 con 0x7f063c101de0 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35474/0 (socket says 172.21.15.16:35474) 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 -- 172.21.15.16:0/2883215852 learned_addr learned my addr 172.21.15.16:0/2883215852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063bfff640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c101de0 0x7f063c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f0640d3d640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c106f10 0x7f063c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 -- 172.21.15.16:0/2883215852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c101de0 msgr2=0x7f063c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c101de0 0x7f063c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 -- 172.21.15.16:0/2883215852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c106f10 msgr2=0x7f063c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c106f10 0x7f063c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 -- 172.21.15.16:0/2883215852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c19d560 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f063b7fe640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f063000b4a0 tx=0x7f063000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630014070 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c19d850 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.626+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0630010af0 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.627+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630017d10 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.627+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c1a5130 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.628+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c102240 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.629+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0630017070 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.630+0000 7f06397fa640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 0x7f06240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.630+0000 7f063bfff640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 0x7f06240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.630+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f06300ce640 con 0x7f063c102790 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.630+0000 7f063bfff640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 0x7f06240a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f062c000c00 tx=0x7f062c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.633+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f063009b000 con 0x7f063c102790 2024-09-12T00:54:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:09.850+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f063c199910 con 0x7f063c102790 2024-09-12T00:54:09.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:09 smithi016 ceph-mon[30972]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T00:54:10.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:09 smithi099 ceph-mon[35654]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T00:54:10.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:09 smithi159 ceph-mon[35566]: pgmap v282: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T00:54:10.116 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:54:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T00:54:09.991+0000 7f9267075640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T00:54:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.774+0000 7f06397fa640 1 -- 172.21.15.16:0/2883215852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f063c199910 con 0x7f063c102790 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 msgr2=0x7f06240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 0x7f06240a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f062c000c00 tx=0x7f062c0058d0 comp rx=0 tx=0).stop 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 msgr2=0x7f063c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f063000b4a0 tx=0x7f063000b960 comp rx=0 tx=0).stop 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 shutdown_connections 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f06240a2b80 0x7f06240a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f063c106f10 0x7f063c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063c102790 0x7f063c198af0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 --2- 172.21.15.16:0/2883215852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f063c101de0 0x7f063c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.777+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 >> 172.21.15.16:0/2883215852 conn(0x7f063c0fd8c0 msgr2=0x7f063c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.778+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 shutdown_connections 2024-09-12T00:54:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:10.778+0000 7f06427a0640 1 -- 172.21.15.16:0/2883215852 wait complete. 2024-09-12T00:54:10.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:10 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T00:54:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:10 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T00:54:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:10 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T00:54:11.490 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T00:54:11.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-12T00:54:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:12.032 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:11 smithi016 ceph-mon[30972]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T00:54:12.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T00:54:12.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:11 smithi016 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:11 smithi159 ceph-mon[35566]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T00:54:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T00:54:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:11 smithi159 ceph-mon[35566]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:11 smithi099 ceph-mon[35654]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-12T00:54:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2883215852' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T00:54:12.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:11 smithi099 ceph-mon[35654]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 -- 172.21.15.16:0/2639811801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac102ef0 msgr2=0x7f47ac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/2639811801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac102ef0 0x7f47ac10da40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009ca0 tx=0x7f47a402a950 comp rx=0 tx=0).stop 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 -- 172.21.15.16:0/2639811801 shutdown_connections 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/2639811801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac102ef0 0x7f47ac10da40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/2639811801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/2639811801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac100e20 0x7f47ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 -- 172.21.15.16:0/2639811801 >> 172.21.15.16:0/2639811801 conn(0x7f47ac0fc7d0 msgr2=0x7f47ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 -- 172.21.15.16:0/2639811801 shutdown_connections 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.268+0000 7f47b0bbf640 1 -- 172.21.15.16:0/2639811801 wait complete. 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 Processor -- start 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 -- start start 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102ef0 0x7f47ac19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac110f80 con 0x7f47ac100e20 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac110e00 con 0x7f47ac102570 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47b0bbf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac111100 con 0x7f47ac102ef0 2024-09-12T00:54:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35494/0 (socket says 172.21.15.16:35494) 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.269+0000 7f47aa59c640 1 -- 172.21.15.16:0/1106607889 learned_addr learned my addr 172.21.15.16:0/1106607889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aad9d640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102ef0 0x7f47ac19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 -- 172.21.15.16:0/1106607889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102ef0 msgr2=0x7f47ac19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47a3fff640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102ef0 0x7f47ac19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 -- 172.21.15.16:0/1106607889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 msgr2=0x7f47ac198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 -- 172.21.15.16:0/1106607889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac19d4b0 con 0x7f47ac100e20 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47a3fff640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47aa59c640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c970 tx=0x7f479c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c007bb0 con 0x7f47ac100e20 2024-09-12T00:54:12.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ac19d7a0 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f479c007d50 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c00f450 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.270+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ac1a5080 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.271+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ac101200 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.272+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f479c005020 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.273+0000 7f47a37fe640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 0x7f47980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.273+0000 7f47a3fff640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 0x7f47980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.273+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f479c0cee20 con 0x7f47ac100e20 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.273+0000 7f47a3fff640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 0x7f47980a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0ff0c0 tx=0x7f4794005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:12.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.277+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f479c014030 con 0x7f47ac100e20 2024-09-12T00:54:12.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.488+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f47ac10a0a0 con 0x7f47ac100e20 2024-09-12T00:54:12.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.784+0000 7f47a37fe640 1 -- 172.21.15.16:0/1106607889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f479c09be30 con 0x7f47ac100e20 2024-09-12T00:54:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 msgr2=0x7f47980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 0x7f47980a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0ff0c0 tx=0x7f4794005cb0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 msgr2=0x7f47ac198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f479c00c970 tx=0x7f479c00ce30 comp rx=0 tx=0).stop 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 shutdown_connections 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f47980a2b80 0x7f47980a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47ac102ef0 0x7f47ac19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47ac102570 0x7f47ac198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 --2- 172.21.15.16:0/1106607889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47ac100e20 0x7f47ac198500 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 >> 172.21.15.16:0/1106607889 conn(0x7f47ac0fc7d0 msgr2=0x7f47ac06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 shutdown_connections 2024-09-12T00:54:12.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:12.786+0000 7f47b0bbf640 1 -- 172.21.15.16:0/1106607889 wait complete. 2024-09-12T00:54:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:12 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T00:54:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:12 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T00:54:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:12 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T00:54:13.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T00:54:13.810 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:14.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T00:54:14.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:13 smithi016 ceph-mon[30972]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T00:54:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:13 smithi159 ceph-mon[35566]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1106607889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T00:54:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:13 smithi099 ceph-mon[35654]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T00:54:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 -- 172.21.15.16:0/3459544943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8106a80 msgr2=0x7fdec8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 --2- 172.21.15.16:0/3459544943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8106a80 0x7fdec8106e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009930 tx=0x7fdeb802a420 comp rx=0 tx=0).stop 2024-09-12T00:54:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 -- 172.21.15.16:0/3459544943 shutdown_connections 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 --2- 172.21.15.16:0/3459544943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8101420 0x7fdec810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 --2- 172.21.15.16:0/3459544943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8100a80 0x7fdec8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 --2- 172.21.15.16:0/3459544943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8106a80 0x7fdec8106e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 -- 172.21.15.16:0/3459544943 >> 172.21.15.16:0/3459544943 conn(0x7fdec80fc7a0 msgr2=0x7fdec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 -- 172.21.15.16:0/3459544943 shutdown_connections 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.300+0000 7fdecf22c640 1 -- 172.21.15.16:0/3459544943 wait complete. 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 Processor -- start 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 -- start start 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8100a80 0x7fdec81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8106a80 0x7fdec81a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8110e80 con 0x7fdec8101420 2024-09-12T00:54:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8110d00 con 0x7fdec8106a80 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdecf22c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8111000 con 0x7fdec8100a80 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdec7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdec7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35522/0 (socket says 172.21.15.16:35522) 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdec7fff640 1 -- 172.21.15.16:0/2226719130 learned_addr learned my addr 172.21.15.16:0/2226719130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.301+0000 7fdeccfc8640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8100a80 0x7fdec81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdecd7c9640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8106a80 0x7fdec81a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 -- 172.21.15.16:0/2226719130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8100a80 msgr2=0x7fdec81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8100a80 0x7fdec81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 -- 172.21.15.16:0/2226719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8106a80 msgr2=0x7fdec81a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8106a80 0x7fdec81a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 -- 172.21.15.16:0/2226719130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec81a4890 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec7fff640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d950 tx=0x7fdebc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc017070 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc009c30 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81a4a60 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00f6b0 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.302+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a4f70 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.303+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec8100ee0 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.304+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdebc0109d0 con 0x7fdec8101420 2024-09-12T00:54:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.305+0000 7fdec5ffb640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 0x7fdeb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.305+0000 7fdeccfc8640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 0x7fdeb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.305+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fdebc0cead0 con 0x7fdec8101420 2024-09-12T00:54:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.306+0000 7fdeccfc8640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 0x7fdeb00a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.309+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdebc009de0 con 0x7fdec8101420 2024-09-12T00:54:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.520+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdec8108ff0 con 0x7fdec8101420 2024-09-12T00:54:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.800+0000 7fdec5ffb640 1 -- 172.21.15.16:0/2226719130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fdebc09b530 con 0x7fdec8101420 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 msgr2=0x7fdeb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 0x7fdeb00a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb80058d0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 msgr2=0x7fdec8199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d950 tx=0x7fdebc00de10 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 shutdown_connections 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fdeb00a2b80 0x7fdeb00a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdec8106a80 0x7fdec81a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8101420 0x7fdec8199810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 --2- 172.21.15.16:0/2226719130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdec8100a80 0x7fdec81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 >> 172.21.15.16:0/2226719130 conn(0x7fdec80fc7a0 msgr2=0x7fdec80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 shutdown_connections 2024-09-12T00:54:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:14.803+0000 7fdecf22c640 1 -- 172.21.15.16:0/2226719130 wait complete. 2024-09-12T00:54:15.116 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T00:54:14.992+0000 7f9267075640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T00:54:15.116 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T00:54:14.992+0000 7f9267075640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T00:54:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:active 2024-09-12T00:54:15.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T00:54:15.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T00:54:15.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-12T00:54:15.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-12T00:54:15.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:14 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T00:54:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:active 2024-09-12T00:54:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T00:54:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T00:54:15.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-12T00:54:15.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-12T00:54:15.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:14 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T00:54:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] up:active 2024-09-12T00:54:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T00:54:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T00:54:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-09-12T00:54:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-09-12T00:54:15.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:14 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T00:54:15.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T00:54:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:active 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-09-12T00:54:16.071 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:15 smithi016 ceph-mon[30972]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:active 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-09-12T00:54:16.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:15 smithi159 ceph-mon[35566]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2226719130' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] up:active 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-09-12T00:54:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:15 smithi099 ceph-mon[35654]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-12T00:54:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 -- 172.21.15.16:0/53617046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 msgr2=0x7f60a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 --2- 172.21.15.16:0/53617046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a41067e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c02a410 comp rx=0 tx=0).stop 2024-09-12T00:54:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 -- 172.21.15.16:0/53617046 shutdown_connections 2024-09-12T00:54:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 --2- 172.21.15.16:0/53617046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 --2- 172.21.15.16:0/53617046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a41067e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 --2- 172.21.15.16:0/53617046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 0x7f60a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 -- 172.21.15.16:0/53617046 >> 172.21.15.16:0/53617046 conn(0x7f60a40fd8c0 msgr2=0x7f60a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 -- 172.21.15.16:0/53617046 shutdown_connections 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.293+0000 7f60a8d35640 1 -- 172.21.15.16:0/53617046 wait complete. 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 Processor -- start 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 -- start start 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 0x7f60a4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:16.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4111f90 con 0x7f60a4102790 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4111e10 con 0x7f60a4106f10 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a8d35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4112110 con 0x7f60a4101de0 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35546/0 (socket says 172.21.15.16:35546) 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 -- 172.21.15.16:0/1111382965 learned_addr learned my addr 172.21.15.16:0/1111382965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a259c640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 0x7f60a4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a2d9d640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 -- 172.21.15.16:0/1111382965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 msgr2=0x7f60a4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 0x7f60a4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 -- 172.21.15.16:0/1111382965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 msgr2=0x7f60a419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f609bfff640 1 -- 172.21.15.16:0/1111382965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a419d3c0 con 0x7f60a4102790 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.294+0000 7f60a2d9d640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f609bfff640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f608c005ab0 tx=0x7f608c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:16.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c03f030 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c003ce0 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a419d650 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c035070 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.295+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41a4f90 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.296+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a41021c0 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.297+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f608c042030 con 0x7f60a4102790 2024-09-12T00:54:16.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.298+0000 7f609b7fe640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 0x7f60900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.298+0000 7f60a259c640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 0x7f60900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.298+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f608c0ef8d0 con 0x7f60a4102790 2024-09-12T00:54:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.298+0000 7f60a259c640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 0x7f60900a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6094009890 tx=0x7f6094008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:16.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.302+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f608c0bc8e0 con 0x7f60a4102790 2024-09-12T00:54:16.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.514+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f60a410a0f0 con 0x7f60a4102790 2024-09-12T00:54:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.812+0000 7f609b7fe640 1 -- 172.21.15.16:0/1111382965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f608c0dae20 con 0x7f60a4102790 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 msgr2=0x7f60900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 0x7f60900a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6094009890 tx=0x7f6094008040 comp rx=0 tx=0).stop 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 msgr2=0x7f60a4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f608c005ab0 tx=0x7f608c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 shutdown_connections 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f60900a2c00 0x7f60900a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a4106f10 0x7f60a419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4102790 0x7f60a4198950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 --2- 172.21.15.16:0/1111382965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60a4101de0 0x7f60a4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 >> 172.21.15.16:0/1111382965 conn(0x7f60a40fd8c0 msgr2=0x7f60a40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 shutdown_connections 2024-09-12T00:54:16.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:16.814+0000 7f60a8d35640 1 -- 172.21.15.16:0/1111382965 wait complete. 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:active 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: Cluster is now healthy 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-09-12T00:54:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T00:54:17.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T00:54:17.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:16 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:active 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: Cluster is now healthy 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T00:54:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T00:54:17.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:16 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] up:active 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: Cluster is now healthy 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T00:54:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:16 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T00:54:17.539 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T00:54:17.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T00:54:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:18.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:17 smithi016 ceph-mon[30972]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 9 op/s 2024-09-12T00:54:18.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:17 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T00:54:18.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:17 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:active 2024-09-12T00:54:18.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:17 smithi016 ceph-mon[30972]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T00:54:18.082 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:17 smithi016 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:54:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:17 smithi159 ceph-mon[35566]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 9 op/s 2024-09-12T00:54:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:17 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T00:54:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:17 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:active 2024-09-12T00:54:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:17 smithi159 ceph-mon[35566]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T00:54:18.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:17 smithi159 ceph-mon[35566]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:54:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:17 smithi099 ceph-mon[35654]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 9 op/s 2024-09-12T00:54:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:17 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1111382965' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T00:54:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:17 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] up:active 2024-09-12T00:54:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:17 smithi099 ceph-mon[35654]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-12T00:54:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:17 smithi099 ceph-mon[35654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 -- 172.21.15.16:0/922938872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0106a80 msgr2=0x7f46b0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/922938872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0106a80 0x7f46b0106e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009930 tx=0x7f46ac02a420 comp rx=0 tx=0).stop 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 -- 172.21.15.16:0/922938872 shutdown_connections 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/922938872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0101420 0x7f46b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/922938872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 0x7f46b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/922938872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0106a80 0x7f46b0106e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 -- 172.21.15.16:0/922938872 >> 172.21.15.16:0/922938872 conn(0x7f46b00fc7a0 msgr2=0x7f46b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 -- 172.21.15.16:0/922938872 shutdown_connections 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.308+0000 7f46b7f0f640 1 -- 172.21.15.16:0/922938872 wait complete. 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 Processor -- start 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 -- start start 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 0x7f46b00fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0106a80 0x7f46b00fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110ed0 con 0x7f46b0101420 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110d50 con 0x7f46b0106a80 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b7f0f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0111050 con 0x7f46b0100a80 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b54aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:18.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b54aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35574/0 (socket says 172.21.15.16:35574) 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.309+0000 7f46b54aa640 1 -- 172.21.15.16:0/1453851459 learned_addr learned my addr 172.21.15.16:0/1453851459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b5cab640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 0x7f46b00fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b64ac640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0106a80 0x7f46b00fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 -- 172.21.15.16:0/1453851459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 msgr2=0x7f46b00fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 0x7f46b00fcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 -- 172.21.15.16:0/1453851459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0106a80 msgr2=0x7f46b00fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0106a80 0x7f46b00fd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 -- 172.21.15.16:0/1453851459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b01a6960 con 0x7f46b0101420 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b54aa640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46a000d770 tx=0x7f46a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0017070 con 0x7f46b0101420 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b01a6c50 con 0x7f46b0101420 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a0005500 con 0x7f46b0101420 2024-09-12T00:54:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0004e00 con 0x7f46b0101420 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.310+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a7140 con 0x7f46b0101420 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.311+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b0100ee0 con 0x7f46b0101420 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.312+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f46a0004fa0 con 0x7f46b0101420 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.313+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 0x7f469c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.313+0000 7f46b5cab640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 0x7f469c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.313+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f46a00ce960 con 0x7f46b0101420 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.313+0000 7f46b5cab640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 0x7f469c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006010 tx=0x7f46ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:18.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.317+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a009b320 con 0x7f46b0101420 2024-09-12T00:54:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.517+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f46b010a070 con 0x7f469c0a2b80 2024-09-12T00:54:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.544+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1453851459 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f46ac004270 con 0x7f469c0a2b80 2024-09-12T00:54:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 msgr2=0x7f469c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 0x7f469c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006010 tx=0x7f46ac000bc0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 msgr2=0x7f46b00fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f46a000d770 tx=0x7f46a000dc30 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 shutdown_connections 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f469c0a2b80 0x7f469c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46b0106a80 0x7f46b00fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46b0101420 0x7f46b00fd290 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 --2- 172.21.15.16:0/1453851459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46b0100a80 0x7f46b00fcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 >> 172.21.15.16:0/1453851459 conn(0x7f46b00fc7a0 msgr2=0x7f46b006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 shutdown_connections 2024-09-12T00:54:18.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:18.546+0000 7f46b7f0f640 1 -- 172.21.15.16:0/1453851459 wait complete. 2024-09-12T00:54:18.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:54:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:54:18.522+0000 7fa627a08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:54:18.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:54:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:54:18.522+0000 7fa627a08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:54:18.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:54:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:54:18.522+0000 7fa627a08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:54:18.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:54:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:54:18.522+0000 7fa627a08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:54:18.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:54:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:54:18.522+0000 7fa627a08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:54:19.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T00:54:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:20.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:19 smithi016 ceph-mon[30972]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-12T00:54:20.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:19 smithi016 ceph-mon[30972]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:19 smithi159 ceph-mon[35566]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-12T00:54:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:19 smithi159 ceph-mon[35566]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:20.116 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:54:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T00:54:19.992+0000 7f9267075640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-12T00:54:20.117 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 00:54:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T00:54:19.992+0000 7f9267075640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-12T00:54:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 -- 172.21.15.16:0/1258326412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c101de0 msgr2=0x7f4a0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 --2- 172.21.15.16:0/1258326412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c101de0 0x7f4a0c1021c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009ca0 tx=0x7f49fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 -- 172.21.15.16:0/1258326412 shutdown_connections 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 --2- 172.21.15.16:0/1258326412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c106f10 0x7f4a0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 --2- 172.21.15.16:0/1258326412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c102790 0x7f4a0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 --2- 172.21.15.16:0/1258326412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c101de0 0x7f4a0c1021c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 -- 172.21.15.16:0/1258326412 >> 172.21.15.16:0/1258326412 conn(0x7f4a0c0fd8c0 msgr2=0x7f4a0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 -- 172.21.15.16:0/1258326412 shutdown_connections 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.103+0000 7f4a1334e640 1 -- 172.21.15.16:0/1258326412 wait complete. 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 Processor -- start 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 -- start start 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c101de0 0x7f4a0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c102790 0x7f4a0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c111f70 con 0x7f4a0c106f10 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c111df0 con 0x7f4a0c102790 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.104+0000 7f4a1334e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1120f0 con 0x7f4a0c101de0 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57282/0 (socket says 172.21.15.16:57282) 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 -- 172.21.15.16:0/3552199049 learned_addr learned my addr 172.21.15.16:0/3552199049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a108e9640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c102790 0x7f4a0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a110ea640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c101de0 0x7f4a0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 -- 172.21.15.16:0/3552199049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c101de0 msgr2=0x7f4a0c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c101de0 0x7f4a0c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 -- 172.21.15.16:0/3552199049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c102790 msgr2=0x7f4a0c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c102790 0x7f4a0c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 -- 172.21.15.16:0/3552199049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c1a6930 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a118eb640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c970 tx=0x7f4a0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04007bb0 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a6c20 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04007d50 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400f450 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.105+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a7160 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.106+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c1021c0 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.108+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a04004f00 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.109+0000 7f4a0a7fc640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 0x7f49f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.109+0000 7f4a110ea640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 0x7f49f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.109+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f4a040cef70 con 0x7f4a0c106f10 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.109+0000 7f4a110ea640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 0x7f49f40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000f80 tx=0x7f49fc0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.112+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a04014030 con 0x7f4a0c106f10 2024-09-12T00:54:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:19 smithi099 ceph-mon[35654]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 11 op/s 2024-09-12T00:54:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:19 smithi099 ceph-mon[35654]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:20.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.316+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4a0c062630 con 0x7f49f40a2b80 2024-09-12T00:54:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.341+0000 7f4a0a7fc640 1 -- 172.21.15.16:0/3552199049 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a0c062630 con 0x7f49f40a2b80 2024-09-12T00:54:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 msgr2=0x7f49f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 0x7f49f40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49fc000f80 tx=0x7f49fc0093f0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 msgr2=0x7f4a0c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c970 tx=0x7f4a0400ce30 comp rx=0 tx=0).stop 2024-09-12T00:54:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 shutdown_connections 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f49f40a2b80 0x7f49f40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a0c106f10 0x7f4a0c074f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a0c102790 0x7f4a0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 --2- 172.21.15.16:0/3552199049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a0c101de0 0x7f4a0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 >> 172.21.15.16:0/3552199049 conn(0x7f4a0c0fd8c0 msgr2=0x7f4a0c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 shutdown_connections 2024-09-12T00:54:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:20.344+0000 7f4a1334e640 1 -- 172.21.15.16:0/3552199049 wait complete. 2024-09-12T00:54:21.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T00:54:21.480 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:21.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:21 smithi016 ceph-mon[30972]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-09-12T00:54:21.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:21 smithi016 ceph-mon[30972]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:21.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:21 smithi016 ceph-mon[30972]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T00:54:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- 172.21.15.16:0/2704017653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4101420 msgr2=0x7f73b410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 --2- 172.21.15.16:0/2704017653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4101420 0x7f73b410d950 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009cd0 tx=0x7f73ac02a8e0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- 172.21.15.16:0/2704017653 shutdown_connections 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 --2- 172.21.15.16:0/2704017653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4101420 0x7f73b410d950 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 --2- 172.21.15.16:0/2704017653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4100a80 0x7f73b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 --2- 172.21.15.16:0/2704017653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4106a80 0x7f73b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- 172.21.15.16:0/2704017653 >> 172.21.15.16:0/2704017653 conn(0x7f73b40fc7a0 msgr2=0x7f73b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- 172.21.15.16:0/2704017653 shutdown_connections 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- 172.21.15.16:0/2704017653 wait complete. 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 Processor -- start 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.971+0000 7f73b9532640 1 -- start start 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4101420 0x7f73b41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4106a80 0x7f73b41a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4110e80 con 0x7f73b4100a80 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4110d00 con 0x7f73b4101420 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b9532640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73b4111000 con 0x7f73b4106a80 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57298/0 (socket says 172.21.15.16:57298) 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 -- 172.21.15.16:0/3493709585 learned_addr learned my addr 172.21.15.16:0/3493709585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73abfff640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4101420 0x7f73b41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b37fe640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4106a80 0x7f73b41a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 -- 172.21.15.16:0/3493709585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4106a80 msgr2=0x7f73b41a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4106a80 0x7f73b41a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 -- 172.21.15.16:0/3493709585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4101420 msgr2=0x7f73b41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4101420 0x7f73b41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 -- 172.21.15.16:0/3493709585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b41a4870 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.972+0000 7f73b2ffd640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73a400d950 tx=0x7f73a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.973+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4017030 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.973+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a4004020 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.973+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b41a4a40 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.973+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a40047a0 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.973+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b41a4f50 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.974+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b4100ee0 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.975+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f73a40041d0 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.975+0000 7f73b0ff9640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 0x7f73a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.976+0000 7f73abfff640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 0x7f73a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.976+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f73a40ce9a0 con 0x7f73b4100a80 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.976+0000 7f73abfff640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 0x7f73a00a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73b406b2e0 tx=0x7f739c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:21.980+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73a409b950 con 0x7f73b4100a80 2024-09-12T00:54:22.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:21 smithi099 ceph-mon[35654]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-09-12T00:54:22.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:21 smithi099 ceph-mon[35654]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:22.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:21 smithi099 ceph-mon[35654]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T00:54:22.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:21 smithi159 ceph-mon[35566]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-09-12T00:54:22.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:21 smithi159 ceph-mon[35566]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:22.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:21 smithi159 ceph-mon[35566]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T00:54:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.184+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f73b4109080 con 0x7f73a00a2c10 2024-09-12T00:54:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.190+0000 7f73b0ff9640 1 -- 172.21.15.16:0/3493709585 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f739c003890 con 0x7f73a00a2c10 2024-09-12T00:54:22.189 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/d7d57e90-e97d-47f1-b26f-9a8497f213a4 2024-09-12T00:54:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 msgr2=0x7f73a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 0x7f73a00a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73b406b2e0 tx=0x7f739c00b390 comp rx=0 tx=0).stop 2024-09-12T00:54:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 msgr2=0x7f73b41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73a400d950 tx=0x7f73a400de10 comp rx=0 tx=0).stop 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 shutdown_connections 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f73a00a2c10 0x7f73a00a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b4106a80 0x7f73b41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73b4101420 0x7f73b41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 --2- 172.21.15.16:0/3493709585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73b4100a80 0x7f73b41992b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 >> 172.21.15.16:0/3493709585 conn(0x7f73b40fc7a0 msgr2=0x7f73b40fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 shutdown_connections 2024-09-12T00:54:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:22.192+0000 7f73b9532640 1 -- 172.21.15.16:0/3493709585 wait complete. 2024-09-12T00:54:22.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T00:54:23.246 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:23.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:23 smithi016 ceph-mon[30972]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:23.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:23 smithi016 ceph-mon[30972]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-12T00:54:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:23 smithi099 ceph-mon[35654]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:23 smithi099 ceph-mon[35654]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-12T00:54:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:23 smithi159 ceph-mon[35566]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:23 smithi159 ceph-mon[35566]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- 172.21.15.16:0/1179730762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750100a80 msgr2=0x7fe750100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 --2- 172.21.15.16:0/1179730762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750100a80 0x7fe750100ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe738009930 tx=0x7fe73802a440 comp rx=0 tx=0).stop 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- 172.21.15.16:0/1179730762 shutdown_connections 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 --2- 172.21.15.16:0/1179730762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe750101420 0x7fe75010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 --2- 172.21.15.16:0/1179730762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750100a80 0x7fe750100ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 --2- 172.21.15.16:0/1179730762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750106a80 0x7fe750106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- 172.21.15.16:0/1179730762 >> 172.21.15.16:0/1179730762 conn(0x7fe7500fc7a0 msgr2=0x7fe7500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- 172.21.15.16:0/1179730762 shutdown_connections 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- 172.21.15.16:0/1179730762 wait complete. 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 Processor -- start 2024-09-12T00:54:23.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.749+0000 7fe75696d640 1 -- start start 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750100a80 0x7fe750199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe750106a80 0x7fe7501a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110f10 con 0x7fe750101420 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe750110d90 con 0x7fe750100a80 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111090 con 0x7fe750106a80 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57312/0 (socket says 172.21.15.16:57312) 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 -- 172.21.15.16:0/2127662664 learned_addr learned my addr 172.21.15.16:0/2127662664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe74ffff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750100a80 0x7fe750199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 -- 172.21.15.16:0/2127662664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe750106a80 msgr2=0x7fe7501a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:23.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe750106a80 0x7fe7501a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 -- 172.21.15.16:0/2127662664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750100a80 msgr2=0x7fe750199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750100a80 0x7fe750199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 -- 172.21.15.16:0/2127662664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7501a4920 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe747fff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe738009900 tx=0x7fe738003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73803f070 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe738003e10 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7501a4af0 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.750+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738035030 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.751+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a4f40 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.752+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe750100ee0 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.752+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe738042070 con 0x7fe750101420 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.753+0000 7fe74dffb640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:23.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.753+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fe7380ef880 con 0x7fe750101420 2024-09-12T00:54:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.753+0000 7fe74ffff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.754+0000 7fe74ffff640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe74000bbe0 tx=0x7fe74000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:23.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.757+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7380bc830 con 0x7fe750101420 2024-09-12T00:54:23.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.958+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe7500fd5c0 con 0x7fe73c0a2c10 2024-09-12T00:54:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.985+0000 7fe74dffb640 1 -- 172.21.15.16:0/2127662664 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7500fd5c0 con 0x7fe73c0a2c10 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 msgr2=0x7fe73c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe74000bbe0 tx=0x7fe74000b390 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 msgr2=0x7fe7501998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe738009900 tx=0x7fe738003b60 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 shutdown_connections 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe73c0a2c10 0x7fe73c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe750106a80 0x7fe7501a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750101420 0x7fe7501998a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 --2- 172.21.15.16:0/2127662664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe750100a80 0x7fe750199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 >> 172.21.15.16:0/2127662664 conn(0x7fe7500fc7a0 msgr2=0x7fe7500fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 shutdown_connections 2024-09-12T00:54:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:23.987+0000 7fe75696d640 1 -- 172.21.15.16:0/2127662664 wait complete. 2024-09-12T00:54:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:24 smithi016 ceph-mon[30972]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:24 smithi099 ceph-mon[35654]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:24 smithi159 ceph-mon[35566]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:24.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T00:54:24.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:25 smithi016 ceph-mon[30972]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-12T00:54:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- 172.21.15.16:0/1582425927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c102ef0 msgr2=0x7f447c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- 172.21.15.16:0/1582425927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c102ef0 0x7f447c10da40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4474009ca0 tx=0x7f447402a950 comp rx=0 tx=0).stop 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- 172.21.15.16:0/1582425927 shutdown_connections 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- 172.21.15.16:0/1582425927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c102ef0 0x7f447c10da40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- 172.21.15.16:0/1582425927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102570 0x7f447c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- 172.21.15.16:0/1582425927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f447c100e20 0x7f447c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- 172.21.15.16:0/1582425927 >> 172.21.15.16:0/1582425927 conn(0x7f447c0fc7d0 msgr2=0x7f447c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- 172.21.15.16:0/1582425927 shutdown_connections 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- 172.21.15.16:0/1582425927 wait complete. 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 Processor -- start 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 -- start start 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.502+0000 7f4483af3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f447c102570 0x7f447c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 0x7f447c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f447c110f80 con 0x7f447c100e20 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f447c110e00 con 0x7f447c102ef0 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f447c111100 con 0x7f447c102570 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57340/0 (socket says 172.21.15.16:57340) 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 -- 172.21.15.16:0/3517558464 learned_addr learned my addr 172.21.15.16:0/3517558464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4482090640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 0x7f447c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 -- 172.21.15.16:0/3517558464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f447c102570 msgr2=0x7f447c0fd1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f447c102570 0x7f447c0fd1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 -- 172.21.15.16:0/3517558464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 msgr2=0x7f447c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 0x7f447c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 -- 172.21.15.16:0/3517558464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1a6910 con 0x7f447c100e20 2024-09-12T00:54:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4482090640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 0x7f447c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f448188f640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f447800d770 tx=0x7f447800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478017070 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1a6c00 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4478005500 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478004e00 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.503+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a70f0 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.504+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c101200 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.505+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4478004fa0 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.506+0000 7f4472ffd640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 0x7f44680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.507+0000 7f448108e640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 0x7f44680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.507+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f44780ceb50 con 0x7f447c100e20 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.507+0000 7f448108e640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 0x7f44680a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f447c0fe710 tx=0x7f446c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.511+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f447809bb00 con 0x7f447c100e20 2024-09-12T00:54:25.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:25 smithi099 ceph-mon[35654]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-12T00:54:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:25 smithi159 ceph-mon[35566]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.3 KiB/s wr, 21 op/s 2024-09-12T00:54:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.713+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f447c106f10 con 0x7f44680a2c10 2024-09-12T00:54:25.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.718+0000 7f4472ffd640 1 -- 172.21.15.16:0/3517558464 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f447c106f10 con 0x7f44680a2c10 2024-09-12T00:54:25.717 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/eddbff54-14e7-4eaf-b025-9f9443c2e321 2024-09-12T00:54:25.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 msgr2=0x7f44680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 0x7f44680a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f447c0fe710 tx=0x7f446c005ed0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 msgr2=0x7f447c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f447800d770 tx=0x7f447800dc30 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 shutdown_connections 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f44680a2c10 0x7f44680a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f447c102ef0 0x7f447c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f447c102570 0x7f447c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 --2- 172.21.15.16:0/3517558464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c100e20 0x7f447c0fcc80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 >> 172.21.15.16:0/3517558464 conn(0x7f447c0fc7d0 msgr2=0x7f447c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 shutdown_connections 2024-09-12T00:54:25.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:25.720+0000 7f4483af3640 1 -- 172.21.15.16:0/3517558464 wait complete. 2024-09-12T00:54:26.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:26 smithi016 ceph-mon[30972]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:26.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:54:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:26 smithi099 ceph-mon[35654]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:26.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:26 smithi159 ceph-mon[35566]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:54:26.657 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:27.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- 172.21.15.16:0/972051470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 msgr2=0x7f71341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 --2- 172.21.15.16:0/972051470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f71341067e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7128009900 tx=0x7f712802a410 comp rx=0 tx=0).stop 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- 172.21.15.16:0/972051470 shutdown_connections 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 --2- 172.21.15.16:0/972051470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 0x7f713410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 --2- 172.21.15.16:0/972051470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f71341067e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 --2- 172.21.15.16:0/972051470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 0x7f71341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- 172.21.15.16:0/972051470 >> 172.21.15.16:0/972051470 conn(0x7f71340fd8c0 msgr2=0x7f71340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- 172.21.15.16:0/972051470 shutdown_connections 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- 172.21.15.16:0/972051470 wait complete. 2024-09-12T00:54:27.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 Processor -- start 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.154+0000 7f7139ca7640 1 -- start start 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 0x7f71340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 0x7f7134074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111f90 con 0x7f7134102790 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7134111e10 con 0x7f7134101de0 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7139ca7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7134112110 con 0x7f7134106f10 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57348/0 (socket says 172.21.15.16:57348) 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 -- 172.21.15.16:0/3534486962 learned_addr learned my addr 172.21.15.16:0/3534486962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f71337fe640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 0x7f71340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7133fff640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 0x7f7134074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 -- 172.21.15.16:0/3534486962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 msgr2=0x7f7134074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 0x7f7134074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 -- 172.21.15.16:0/3534486962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 msgr2=0x7f71340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 0x7f71340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 -- 172.21.15.16:0/3534486962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71341a69e0 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.155+0000 7f7132ffd640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f71280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.156+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712803f030 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.156+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128003ce0 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.156+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71341a6c70 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.156+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128035070 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.156+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a7150 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.157+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71341021c0 con 0x7f7134102790 2024-09-12T00:54:27.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.158+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7128042030 con 0x7f7134102790 2024-09-12T00:54:27.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.159+0000 7f7130ff9640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 0x7f711c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:27.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.159+0000 7f71337fe640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 0x7f711c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:27.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.159+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f71280ef580 con 0x7f7134102790 2024-09-12T00:54:27.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.159+0000 7f71337fe640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 0x7f711c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7124009bd0 tx=0x7f7124005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:27.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.163+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71280bc5f0 con 0x7f7134102790 2024-09-12T00:54:27.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.377+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f713410a150 con 0x7f7134102790 2024-09-12T00:54:27.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:27 smithi016 ceph-mon[30972]: pgmap v291: 97 pgs: 97 active+clean; 606 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-12T00:54:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.378+0000 7f7130ff9640 1 -- 172.21.15.16:0/3534486962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7f71280bbef0 con 0x7f7134102790 2024-09-12T00:54:27.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:27.378 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:54:27.378 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:54:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 msgr2=0x7f711c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 0x7f711c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7124009bd0 tx=0x7f7124005f90 comp rx=0 tx=0).stop 2024-09-12T00:54:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 msgr2=0x7f7134074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7128006010 tx=0x7f71280042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 shutdown_connections 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f711c0a2c10 0x7f711c0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7134106f10 0x7f7134074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7134102790 0x7f7134074a30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 --2- 172.21.15.16:0/3534486962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7134101de0 0x7f71340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 >> 172.21.15.16:0/3534486962 conn(0x7f71340fd8c0 msgr2=0x7f71340fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 shutdown_connections 2024-09-12T00:54:27.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:27.380+0000 7f7139ca7640 1 -- 172.21.15.16:0/3534486962 wait complete. 2024-09-12T00:54:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:27 smithi099 ceph-mon[35654]: pgmap v291: 97 pgs: 97 active+clean; 606 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-12T00:54:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:27 smithi159 ceph-mon[35566]: pgmap v291: 97 pgs: 97 active+clean; 606 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-12T00:54:28.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:54:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:28.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:28 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3534486962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:54:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:28 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3534486962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:54:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:28 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3534486962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:54:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 -- 172.21.15.16:0/3035561959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 msgr2=0x7f4b3c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 --2- 172.21.15.16:0/3035561959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1029b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009900 tx=0x7f4b2402a480 comp rx=0 tx=0).stop 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 -- 172.21.15.16:0/3035561959 shutdown_connections 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 --2- 172.21.15.16:0/3035561959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 --2- 172.21.15.16:0/3035561959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1029b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 --2- 172.21.15.16:0/3035561959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c100e20 0x7f4b3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 -- 172.21.15.16:0/3035561959 >> 172.21.15.16:0/3035561959 conn(0x7f4b3c0fc7d0 msgr2=0x7f4b3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 -- 172.21.15.16:0/3035561959 shutdown_connections 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.846+0000 7f4b4200b640 1 -- 172.21.15.16:0/3035561959 wait complete. 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 Processor -- start 2024-09-12T00:54:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 -- start start 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c100e20 0x7f4b3c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c110f80 con 0x7f4b3c102570 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c110e00 con 0x7f4b3c100e20 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b4200b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c111100 con 0x7f4b3c102ef0 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45088/0 (socket says 172.21.15.16:45088) 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b3affd640 1 -- 172.21.15.16:0/1629582625 learned_addr learned my addr 172.21.15.16:0/1629582625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b3bfff640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.847+0000 7f4b3b7fe640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c100e20 0x7f4b3c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 -- 172.21.15.16:0/1629582625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c102ef0 msgr2=0x7f4b3c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 -- 172.21.15.16:0/1629582625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c100e20 msgr2=0x7f4b3c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c100e20 0x7f4b3c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 -- 172.21.15.16:0/1629582625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19d460 con 0x7f4b3c102570 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b3affd640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006010 tx=0x7f4b24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2403f070 con 0x7f4b3c102570 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19d6f0 con 0x7f4b3c102570 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b24031c30 con 0x7f4b3c102570 2024-09-12T00:54:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24035030 con 0x7f4b3c102570 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.848+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a5030 con 0x7f4b3c102570 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.849+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c1021b0 con 0x7f4b3c102570 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.850+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4b24042070 con 0x7f4b3c102570 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.851+0000 7f4b38ff9640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 0x7f4b280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.851+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f4b240ef880 con 0x7f4b3c102570 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.851+0000 7f4b3b7fe640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 0x7f4b280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.851+0000 7f4b3b7fe640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 0x7f4b280a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005fd0 tx=0x7f4b2c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:28.855+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b24031de0 con 0x7f4b3c102570 2024-09-12T00:54:29.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.030+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b3c10a0a0 con 0x7f4b3c102570 2024-09-12T00:54:29.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.032+0000 7f4b38ff9640 1 -- 172.21.15.16:0/1629582625 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22191 (secure 0 0 0) 0x7f4b240bc830 con 0x7f4b3c102570 2024-09-12T00:54:29.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:29.031 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:54:04.465131+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:54:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 msgr2=0x7f4b280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 0x7f4b280a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c005fd0 tx=0x7f4b2c009870 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 msgr2=0x7f4b3c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b24006010 tx=0x7f4b24003c10 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 shutdown_connections 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4b280a2c10 0x7f4b280a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b3c102ef0 0x7f4b3c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c102570 0x7f4b3c1989f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 --2- 172.21.15.16:0/1629582625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b3c100e20 0x7f4b3c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 >> 172.21.15.16:0/1629582625 conn(0x7f4b3c0fc7d0 msgr2=0x7f4b3c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 shutdown_connections 2024-09-12T00:54:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:29.034+0000 7f4b4200b640 1 -- 172.21.15.16:0/1629582625 wait complete. 2024-09-12T00:54:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:29 smithi016 ceph-mon[30972]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-12T00:54:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:29 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1629582625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:29 smithi099 ceph-mon[35654]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-12T00:54:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:29 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1629582625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:29 smithi159 ceph-mon[35566]: pgmap v292: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-12T00:54:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:29 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1629582625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:29.879 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T00:54:29.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:54:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 -- 172.21.15.16:0/4263513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c106a80 msgr2=0x7f3d1c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 --2- 172.21.15.16:0/4263513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c106a80 0x7f3d1c106e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009930 tx=0x7f3d0c02a420 comp rx=0 tx=0).stop 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 -- 172.21.15.16:0/4263513623 shutdown_connections 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 --2- 172.21.15.16:0/4263513623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c101420 0x7f3d1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 --2- 172.21.15.16:0/4263513623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 --2- 172.21.15.16:0/4263513623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c106a80 0x7f3d1c106e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 -- 172.21.15.16:0/4263513623 >> 172.21.15.16:0/4263513623 conn(0x7f3d1c0fc7a0 msgr2=0x7f3d1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 -- 172.21.15.16:0/4263513623 shutdown_connections 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.640+0000 7f3d20964640 1 -- 172.21.15.16:0/4263513623 wait complete. 2024-09-12T00:54:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 Processor -- start 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 -- start start 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 0x7f3d1c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c112020 con 0x7f3d1c101420 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c111ea0 con 0x7f3d1c100a80 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d20964640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1121a0 con 0x7f3d1c106a80 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45102/0 (socket says 172.21.15.16:45102) 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 -- 172.21.15.16:0/3795477938 learned_addr learned my addr 172.21.15.16:0/3795477938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d1a59c640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d1ad9d640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 0x7f3d1c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 -- 172.21.15.16:0/3795477938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 msgr2=0x7f3d1c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 0x7f3d1c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 -- 172.21.15.16:0/3795477938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 msgr2=0x7f3d1c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d19d9b640 1 -- 172.21.15.16:0/3795477938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c19d570 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d1a59c640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.641+0000 7f3d1ad9d640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 0x7f3d1c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d19d9b640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000b4a0 tx=0x7f3d1000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10014070 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c19d860 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d10010af0 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10017d10 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.642+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c1a5140 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.643+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1c100f60 con 0x7f3d1c101420 2024-09-12T00:54:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.644+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3d10017070 con 0x7f3d1c101420 2024-09-12T00:54:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.645+0000 7f3d0b7fe640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 0x7f3d040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.645+0000 7f3d1a59c640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 0x7f3d040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.645+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f3d100ce490 con 0x7f3d1c101420 2024-09-12T00:54:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.645+0000 7f3d1a59c640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 0x7f3d040a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:30.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.649+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d1009b450 con 0x7f3d1c101420 2024-09-12T00:54:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.821+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d1c062630 con 0x7f3d040a2c10 2024-09-12T00:54:30.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.830+0000 7f3d0b7fe640 1 -- 172.21.15.16:0/3795477938 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258870 (secure 0 0 0) 0x7f3d1c062630 con 0x7f3d040a2c10 2024-09-12T00:54:30.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 msgr2=0x7f3d040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 0x7f3d040a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c032d50 comp rx=0 tx=0).stop 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 msgr2=0x7f3d1c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000b4a0 tx=0x7f3d1000b960 comp rx=0 tx=0).stop 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 shutdown_connections 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3d040a2c10 0x7f3d040a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d1c106a80 0x7f3d1c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d1c101420 0x7f3d1c198b00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 --2- 172.21.15.16:0/3795477938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d1c100a80 0x7f3d1c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 >> 172.21.15.16:0/3795477938 conn(0x7f3d1c0fc7a0 msgr2=0x7f3d1c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 shutdown_connections 2024-09-12T00:54:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:30.833+0000 7f3d20964640 1 -- 172.21.15.16:0/3795477938 wait complete. 2024-09-12T00:54:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:31 smithi099 ceph-mon[35654]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-12T00:54:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:31 smithi099 ceph-mon[35654]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:31 smithi159 ceph-mon[35566]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-12T00:54:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:31 smithi159 ceph-mon[35566]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:31 smithi016 ceph-mon[30972]: pgmap v293: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-12T00:54:31.628 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:31 smithi016 ceph-mon[30972]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:31.663 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-12T00:54:30.213339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":286,"num_read_kb":253,"num_write":414,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372732,"kb_used_data":11004,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569092,"statfs":{"total":1151940427776,"available":1151558750208,"internally_reserved":0,"allocated":11268096,"data_stored":7045176,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48266,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":139,"num_read_kb":125,"num_write":76,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002884"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755743+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:54:04.755743+0000","last_peered":"2024-09-12T00:54:04.755743+0000","last_clean":"2024-09-12T00:54:04.755743+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:54:04.755743+0000","last_undegraded":"2024-09-12T00:54:04.755743+0000","last_fullsized":"2024-09-12T00:54:04.755743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771673+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:54:04.771673+0000","last_peered":"2024-09-12T00:54:04.771673+0000","last_clean":"2024-09-12T00:54:04.771673+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:54:04.771673+0000","last_undegraded":"2024-09-12T00:54:04.771673+0000","last_fullsized":"2024-09-12T00:54:04.771673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771688+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:54:04.771688+0000","last_peered":"2024-09-12T00:54:04.771688+0000","last_clean":"2024-09-12T00:54:04.771688+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:54:04.771688+0000","last_undegraded":"2024-09-12T00:54:04.771688+0000","last_fullsized":"2024-09-12T00:54:04.771688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.754332+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:54:04.754332+0000","last_peered":"2024-09-12T00:54:04.754332+0000","last_clean":"2024-09-12T00:54:04.754332+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:54:04.754332+0000","last_undegraded":"2024-09-12T00:54:04.754332+0000","last_fullsized":"2024-09-12T00:54:04.754332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768621+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:54:04.768621+0000","last_peered":"2024-09-12T00:54:04.768621+0000","last_clean":"2024-09-12T00:54:04.768621+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:54:04.768621+0000","last_undegraded":"2024-09-12T00:54:04.768621+0000","last_fullsized":"2024-09-12T00:54:04.768621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.767888+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:54:04.767888+0000","last_peered":"2024-09-12T00:54:04.767888+0000","last_clean":"2024-09-12T00:54:04.767888+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:54:04.767888+0000","last_undegraded":"2024-09-12T00:54:04.767888+0000","last_fullsized":"2024-09-12T00:54:04.767888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771153+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:54:04.771153+0000","last_peered":"2024-09-12T00:54:04.771153+0000","last_clean":"2024-09-12T00:54:04.771153+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:54:04.771153+0000","last_undegraded":"2024-09-12T00:54:04.771153+0000","last_fullsized":"2024-09-12T00:54:04.771153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770212+0000","last_change":"2024-09-12T00:54:03.755510+0000","last_active":"2024-09-12T00:54:04.770212+0000","last_peered":"2024-09-12T00:54:04.770212+0000","last_clean":"2024-09-12T00:54:04.770212+0000","last_became_active":"2024-09-12T00:54:03.754586+0000","last_became_peered":"2024-09-12T00:54:03.754586+0000","last_unstale":"2024-09-12T00:54:04.770212+0000","last_undegraded":"2024-09-12T00:54:04.770212+0000","last_fullsized":"2024-09-12T00:54:04.770212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:25:15.505506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770626+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:54:04.770626+0000","last_peered":"2024-09-12T00:54:04.770626+0000","last_clean":"2024-09-12T00:54:04.770626+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:54:04.770626+0000","last_undegraded":"2024-09-12T00:54:04.770626+0000","last_fullsized":"2024-09-12T00:54:04.770626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768153+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:54:04.768153+0000","last_peered":"2024-09-12T00:54:04.768153+0000","last_clean":"2024-09-12T00:54:04.768153+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:54:04.768153+0000","last_undegraded":"2024-09-12T00:54:04.768153+0000","last_fullsized":"2024-09-12T00:54:04.768153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772424+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:54:04.772424+0000","last_peered":"2024-09-12T00:54:04.772424+0000","last_clean":"2024-09-12T00:54:04.772424+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:54:04.772424+0000","last_undegraded":"2024-09-12T00:54:04.772424+0000","last_fullsized":"2024-09-12T00:54:04.772424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.754328+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:54:04.754328+0000","last_peered":"2024-09-12T00:54:04.754328+0000","last_clean":"2024-09-12T00:54:04.754328+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:54:04.754328+0000","last_undegraded":"2024-09-12T00:54:04.754328+0000","last_fullsized":"2024-09-12T00:54:04.754328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752648+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:54:04.752648+0000","last_peered":"2024-09-12T00:54:04.752648+0000","last_clean":"2024-09-12T00:54:04.752648+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:54:04.752648+0000","last_undegraded":"2024-09-12T00:54:04.752648+0000","last_fullsized":"2024-09-12T00:54:04.752648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771902+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:54:04.771902+0000","last_peered":"2024-09-12T00:54:04.771902+0000","last_clean":"2024-09-12T00:54:04.771902+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:54:04.771902+0000","last_undegraded":"2024-09-12T00:54:04.771902+0000","last_fullsized":"2024-09-12T00:54:04.771902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771797+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:54:04.771797+0000","last_peered":"2024-09-12T00:54:04.771797+0000","last_clean":"2024-09-12T00:54:04.771797+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:54:04.771797+0000","last_undegraded":"2024-09-12T00:54:04.771797+0000","last_fullsized":"2024-09-12T00:54:04.771797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770578+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:54:04.770578+0000","last_peered":"2024-09-12T00:54:04.770578+0000","last_clean":"2024-09-12T00:54:04.770578+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:54:04.770578+0000","last_undegraded":"2024-09-12T00:54:04.770578+0000","last_fullsized":"2024-09-12T00:54:04.770578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755788+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:54:04.755788+0000","last_peered":"2024-09-12T00:54:04.755788+0000","last_clean":"2024-09-12T00:54:04.755788+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:54:04.755788+0000","last_undegraded":"2024-09-12T00:54:04.755788+0000","last_fullsized":"2024-09-12T00:54:04.755788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769817+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:54:04.769817+0000","last_peered":"2024-09-12T00:54:04.769817+0000","last_clean":"2024-09-12T00:54:04.769817+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:54:04.769817+0000","last_undegraded":"2024-09-12T00:54:04.769817+0000","last_fullsized":"2024-09-12T00:54:04.769817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771241+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:54:04.771241+0000","last_peered":"2024-09-12T00:54:04.771241+0000","last_clean":"2024-09-12T00:54:04.771241+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:54:04.771241+0000","last_undegraded":"2024-09-12T00:54:04.771241+0000","last_fullsized":"2024-09-12T00:54:04.771241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755469+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:54:04.755469+0000","last_peered":"2024-09-12T00:54:04.755469+0000","last_clean":"2024-09-12T00:54:04.755469+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:54:04.755469+0000","last_undegraded":"2024-09-12T00:54:04.755469+0000","last_fullsized":"2024-09-12T00:54:04.755469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752826+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:54:04.752826+0000","last_peered":"2024-09-12T00:54:04.752826+0000","last_clean":"2024-09-12T00:54:04.752826+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:54:04.752826+0000","last_undegraded":"2024-09-12T00:54:04.752826+0000","last_fullsized":"2024-09-12T00:54:04.752826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770710+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:54:04.770710+0000","last_peered":"2024-09-12T00:54:04.770710+0000","last_clean":"2024-09-12T00:54:04.770710+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:54:04.770710+0000","last_undegraded":"2024-09-12T00:54:04.770710+0000","last_fullsized":"2024-09-12T00:54:04.770710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768471+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:54:04.768471+0000","last_peered":"2024-09-12T00:54:04.768471+0000","last_clean":"2024-09-12T00:54:04.768471+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:54:04.768471+0000","last_undegraded":"2024-09-12T00:54:04.768471+0000","last_fullsized":"2024-09-12T00:54:04.768471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768736+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:54:04.768736+0000","last_peered":"2024-09-12T00:54:04.768736+0000","last_clean":"2024-09-12T00:54:04.768736+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:54:04.768736+0000","last_undegraded":"2024-09-12T00:54:04.768736+0000","last_fullsized":"2024-09-12T00:54:04.768736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769177+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:54:04.769177+0000","last_peered":"2024-09-12T00:54:04.769177+0000","last_clean":"2024-09-12T00:54:04.769177+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:54:04.769177+0000","last_undegraded":"2024-09-12T00:54:04.769177+0000","last_fullsized":"2024-09-12T00:54:04.769177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772372+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:54:04.772372+0000","last_peered":"2024-09-12T00:54:04.772372+0000","last_clean":"2024-09-12T00:54:04.772372+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:54:04.772372+0000","last_undegraded":"2024-09-12T00:54:04.772372+0000","last_fullsized":"2024-09-12T00:54:04.772372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755640+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:54:04.755640+0000","last_peered":"2024-09-12T00:54:04.755640+0000","last_clean":"2024-09-12T00:54:04.755640+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:54:04.755640+0000","last_undegraded":"2024-09-12T00:54:04.755640+0000","last_fullsized":"2024-09-12T00:54:04.755640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752591+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:54:04.752591+0000","last_peered":"2024-09-12T00:54:04.752591+0000","last_clean":"2024-09-12T00:54:04.752591+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:54:04.752591+0000","last_undegraded":"2024-09-12T00:54:04.752591+0000","last_fullsized":"2024-09-12T00:54:04.752591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771746+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:54:04.771746+0000","last_peered":"2024-09-12T00:54:04.771746+0000","last_clean":"2024-09-12T00:54:04.771746+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:54:04.771746+0000","last_undegraded":"2024-09-12T00:54:04.771746+0000","last_fullsized":"2024-09-12T00:54:04.771746+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768486+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:54:04.768486+0000","last_peered":"2024-09-12T00:54:04.768486+0000","last_clean":"2024-09-12T00:54:04.768486+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:54:04.768486+0000","last_undegraded":"2024-09-12T00:54:04.768486+0000","last_fullsized":"2024-09-12T00:54:04.768486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768681+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:54:04.768681+0000","last_peered":"2024-09-12T00:54:04.768681+0000","last_clean":"2024-09-12T00:54:04.768681+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:54:04.768681+0000","last_undegraded":"2024-09-12T00:54:04.768681+0000","last_fullsized":"2024-09-12T00:54:04.768681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755330+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:54:04.755330+0000","last_peered":"2024-09-12T00:54:04.755330+0000","last_clean":"2024-09-12T00:54:04.755330+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:54:04.755330+0000","last_undegraded":"2024-09-12T00:54:04.755330+0000","last_fullsized":"2024-09-12T00:54:04.755330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770789+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:54:04.770789+0000","last_peered":"2024-09-12T00:54:04.770789+0000","last_clean":"2024-09-12T00:54:04.770789+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:54:04.770789+0000","last_undegraded":"2024-09-12T00:54:04.770789+0000","last_fullsized":"2024-09-12T00:54:04.770789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:27.842049+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:54:27.842049+0000","last_peered":"2024-09-12T00:54:27.842049+0000","last_clean":"2024-09-12T00:54:27.842049+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:54:27.842049+0000","last_undegraded":"2024-09-12T00:54:27.842049+0000","last_fullsized":"2024-09-12T00:54:27.842049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769887+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:54:04.769887+0000","last_peered":"2024-09-12T00:54:04.769887+0000","last_clean":"2024-09-12T00:54:04.769887+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:54:04.769887+0000","last_undegraded":"2024-09-12T00:54:04.769887+0000","last_fullsized":"2024-09-12T00:54:04.769887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.817789+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:54:14.817789+0000","last_peered":"2024-09-12T00:54:14.817789+0000","last_clean":"2024-09-12T00:54:14.817789+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:54:14.817789+0000","last_undegraded":"2024-09-12T00:54:14.817789+0000","last_fullsized":"2024-09-12T00:54:14.817789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:13.807787+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:54:13.807787+0000","last_peered":"2024-09-12T00:54:13.807787+0000","last_clean":"2024-09-12T00:54:13.807787+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:54:13.807787+0000","last_undegraded":"2024-09-12T00:54:13.807787+0000","last_fullsized":"2024-09-12T00:54:13.807787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770308+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:54:04.770308+0000","last_peered":"2024-09-12T00:54:04.770308+0000","last_clean":"2024-09-12T00:54:04.770308+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:54:04.770308+0000","last_undegraded":"2024-09-12T00:54:04.770308+0000","last_fullsized":"2024-09-12T00:54:04.770308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.820882+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:54:15.820882+0000","last_peered":"2024-09-12T00:54:15.820882+0000","last_clean":"2024-09-12T00:54:15.820882+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:54:15.820882+0000","last_undegraded":"2024-09-12T00:54:15.820882+0000","last_fullsized":"2024-09-12T00:54:15.820882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768125+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:54:04.768125+0000","last_peered":"2024-09-12T00:54:04.768125+0000","last_clean":"2024-09-12T00:54:04.768125+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:54:04.768125+0000","last_undegraded":"2024-09-12T00:54:04.768125+0000","last_fullsized":"2024-09-12T00:54:04.768125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.840061+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:54:26.840061+0000","last_peered":"2024-09-12T00:54:26.840061+0000","last_clean":"2024-09-12T00:54:26.840061+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:54:26.840061+0000","last_undegraded":"2024-09-12T00:54:26.840061+0000","last_fullsized":"2024-09-12T00:54:26.840061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770497+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:54:04.770497+0000","last_peered":"2024-09-12T00:54:04.770497+0000","last_clean":"2024-09-12T00:54:04.770497+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:54:04.770497+0000","last_undegraded":"2024-09-12T00:54:04.770497+0000","last_fullsized":"2024-09-12T00:54:04.770497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822432+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:54:15.822432+0000","last_peered":"2024-09-12T00:54:15.822432+0000","last_clean":"2024-09-12T00:54:15.822432+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:54:15.822432+0000","last_undegraded":"2024-09-12T00:54:15.822432+0000","last_fullsized":"2024-09-12T00:54:15.822432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772148+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:54:04.772148+0000","last_peered":"2024-09-12T00:54:04.772148+0000","last_clean":"2024-09-12T00:54:04.772148+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:54:04.772148+0000","last_undegraded":"2024-09-12T00:54:04.772148+0000","last_fullsized":"2024-09-12T00:54:04.772148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.816658+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:54:14.816658+0000","last_peered":"2024-09-12T00:54:14.816658+0000","last_clean":"2024-09-12T00:54:14.816658+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:54:14.816658+0000","last_undegraded":"2024-09-12T00:54:14.816658+0000","last_fullsized":"2024-09-12T00:54:14.816658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770432+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:54:04.770432+0000","last_peered":"2024-09-12T00:54:04.770432+0000","last_clean":"2024-09-12T00:54:04.770432+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:54:04.770432+0000","last_undegraded":"2024-09-12T00:54:04.770432+0000","last_fullsized":"2024-09-12T00:54:04.770432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.843213+0000","last_change":"2024-09-12T00:54:01.669518+0000","last_active":"2024-09-12T00:54:26.843213+0000","last_peered":"2024-09-12T00:54:26.843213+0000","last_clean":"2024-09-12T00:54:26.843213+0000","last_became_active":"2024-09-12T00:54:01.669145+0000","last_became_peered":"2024-09-12T00:54:01.669145+0000","last_unstale":"2024-09-12T00:54:26.843213+0000","last_undegraded":"2024-09-12T00:54:26.843213+0000","last_fullsized":"2024-09-12T00:54:26.843213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:48.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.773321+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:54:04.773321+0000","last_peered":"2024-09-12T00:54:04.773321+0000","last_clean":"2024-09-12T00:54:04.773321+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:54:04.773321+0000","last_undegraded":"2024-09-12T00:54:04.773321+0000","last_fullsized":"2024-09-12T00:54:04.773321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.823087+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:54:16.823087+0000","last_peered":"2024-09-12T00:54:16.823087+0000","last_clean":"2024-09-12T00:54:16.823087+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:54:16.823087+0000","last_undegraded":"2024-09-12T00:54:16.823087+0000","last_fullsized":"2024-09-12T00:54:16.823087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752793+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:54:04.752793+0000","last_peered":"2024-09-12T00:54:04.752793+0000","last_clean":"2024-09-12T00:54:04.752793+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:54:04.752793+0000","last_undegraded":"2024-09-12T00:54:04.752793+0000","last_fullsized":"2024-09-12T00:54:04.752793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821246+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:54:15.821246+0000","last_peered":"2024-09-12T00:54:15.821246+0000","last_clean":"2024-09-12T00:54:15.821246+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:54:15.821246+0000","last_undegraded":"2024-09-12T00:54:15.821246+0000","last_fullsized":"2024-09-12T00:54:15.821246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771725+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:54:04.771725+0000","last_peered":"2024-09-12T00:54:04.771725+0000","last_clean":"2024-09-12T00:54:04.771725+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:54:04.771725+0000","last_undegraded":"2024-09-12T00:54:04.771725+0000","last_fullsized":"2024-09-12T00:54:04.771725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:23.836213+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:54:23.836213+0000","last_peered":"2024-09-12T00:54:23.836213+0000","last_clean":"2024-09-12T00:54:23.836213+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:54:23.836213+0000","last_undegraded":"2024-09-12T00:54:23.836213+0000","last_fullsized":"2024-09-12T00:54:23.836213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771172+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:54:04.771172+0000","last_peered":"2024-09-12T00:54:04.771172+0000","last_clean":"2024-09-12T00:54:04.771172+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:54:04.771172+0000","last_undegraded":"2024-09-12T00:54:04.771172+0000","last_fullsized":"2024-09-12T00:54:04.771172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821838+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:54:15.821838+0000","last_peered":"2024-09-12T00:54:15.821838+0000","last_clean":"2024-09-12T00:54:15.821838+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:54:15.821838+0000","last_undegraded":"2024-09-12T00:54:15.821838+0000","last_fullsized":"2024-09-12T00:54:15.821838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768802+0000","last_change":"2024-09-12T00:54:03.763987+0000","last_active":"2024-09-12T00:54:04.768802+0000","last_peered":"2024-09-12T00:54:04.768802+0000","last_clean":"2024-09-12T00:54:04.768802+0000","last_became_active":"2024-09-12T00:54:03.763634+0000","last_became_peered":"2024-09-12T00:54:03.763634+0000","last_unstale":"2024-09-12T00:54:04.768802+0000","last_undegraded":"2024-09-12T00:54:04.768802+0000","last_fullsized":"2024-09-12T00:54:04.768802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:04.348736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.803434+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:54:15.803434+0000","last_peered":"2024-09-12T00:54:15.803434+0000","last_clean":"2024-09-12T00:54:15.803434+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:54:15.803434+0000","last_undegraded":"2024-09-12T00:54:15.803434+0000","last_fullsized":"2024-09-12T00:54:15.803434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755554+0000","last_change":"2024-09-12T00:54:03.762778+0000","last_active":"2024-09-12T00:54:04.755554+0000","last_peered":"2024-09-12T00:54:04.755554+0000","last_clean":"2024-09-12T00:54:04.755554+0000","last_became_active":"2024-09-12T00:54:03.762469+0000","last_became_peered":"2024-09-12T00:54:03.762469+0000","last_unstale":"2024-09-12T00:54:04.755554+0000","last_undegraded":"2024-09-12T00:54:04.755554+0000","last_fullsized":"2024-09-12T00:54:04.755554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:01.749510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821379+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:54:15.821379+0000","last_peered":"2024-09-12T00:54:15.821379+0000","last_clean":"2024-09-12T00:54:15.821379+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:54:15.821379+0000","last_undegraded":"2024-09-12T00:54:15.821379+0000","last_fullsized":"2024-09-12T00:54:15.821379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772428+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:54:04.772428+0000","last_peered":"2024-09-12T00:54:04.772428+0000","last_clean":"2024-09-12T00:54:04.772428+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:54:04.772428+0000","last_undegraded":"2024-09-12T00:54:04.772428+0000","last_fullsized":"2024-09-12T00:54:04.772428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.823361+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:54:15.823361+0000","last_peered":"2024-09-12T00:54:15.823361+0000","last_clean":"2024-09-12T00:54:15.823361+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:54:15.823361+0000","last_undegraded":"2024-09-12T00:54:15.823361+0000","last_fullsized":"2024-09-12T00:54:15.823361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.953262+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:54:04.953262+0000","last_peered":"2024-09-12T00:54:04.953262+0000","last_clean":"2024-09-12T00:54:04.953262+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:54:04.953262+0000","last_undegraded":"2024-09-12T00:54:04.953262+0000","last_fullsized":"2024-09-12T00:54:04.953262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:25.839313+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:54:25.839313+0000","last_peered":"2024-09-12T00:54:25.839313+0000","last_clean":"2024-09-12T00:54:25.839313+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:54:25.839313+0000","last_undegraded":"2024-09-12T00:54:25.839313+0000","last_fullsized":"2024-09-12T00:54:25.839313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772594+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:54:04.772594+0000","last_peered":"2024-09-12T00:54:04.772594+0000","last_clean":"2024-09-12T00:54:04.772594+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:54:04.772594+0000","last_undegraded":"2024-09-12T00:54:04.772594+0000","last_fullsized":"2024-09-12T00:54:04.772594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771446+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:54:04.771446+0000","last_peered":"2024-09-12T00:54:04.771446+0000","last_clean":"2024-09-12T00:54:04.771446+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:54:04.771446+0000","last_undegraded":"2024-09-12T00:54:04.771446+0000","last_fullsized":"2024-09-12T00:54:04.771446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.823163+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:54:16.823163+0000","last_peered":"2024-09-12T00:54:16.823163+0000","last_clean":"2024-09-12T00:54:16.823163+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:54:16.823163+0000","last_undegraded":"2024-09-12T00:54:16.823163+0000","last_fullsized":"2024-09-12T00:54:16.823163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768811+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:54:04.768811+0000","last_peered":"2024-09-12T00:54:04.768811+0000","last_clean":"2024-09-12T00:54:04.768811+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:54:04.768811+0000","last_undegraded":"2024-09-12T00:54:04.768811+0000","last_fullsized":"2024-09-12T00:54:04.768811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:13.806698+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:54:13.806698+0000","last_peered":"2024-09-12T00:54:13.806698+0000","last_clean":"2024-09-12T00:54:13.806698+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:54:13.806698+0000","last_undegraded":"2024-09-12T00:54:13.806698+0000","last_fullsized":"2024-09-12T00:54:13.806698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752347+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:54:04.752347+0000","last_peered":"2024-09-12T00:54:04.752347+0000","last_clean":"2024-09-12T00:54:04.752347+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:54:04.752347+0000","last_undegraded":"2024-09-12T00:54:04.752347+0000","last_fullsized":"2024-09-12T00:54:04.752347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:24.837079+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:54:24.837079+0000","last_peered":"2024-09-12T00:54:24.837079+0000","last_clean":"2024-09-12T00:54:24.837079+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:54:24.837079+0000","last_undegraded":"2024-09-12T00:54:24.837079+0000","last_fullsized":"2024-09-12T00:54:24.837079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768894+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:54:04.768894+0000","last_peered":"2024-09-12T00:54:04.768894+0000","last_clean":"2024-09-12T00:54:04.768894+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:54:04.768894+0000","last_undegraded":"2024-09-12T00:54:04.768894+0000","last_fullsized":"2024-09-12T00:54:04.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:28.842057+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:54:28.842057+0000","last_peered":"2024-09-12T00:54:28.842057+0000","last_clean":"2024-09-12T00:54:28.842057+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:54:28.842057+0000","last_undegraded":"2024-09-12T00:54:28.842057+0000","last_fullsized":"2024-09-12T00:54:28.842057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768097+0000","last_change":"2024-09-12T00:54:03.761993+0000","last_active":"2024-09-12T00:54:04.768097+0000","last_peered":"2024-09-12T00:54:04.768097+0000","last_clean":"2024-09-12T00:54:04.768097+0000","last_became_active":"2024-09-12T00:54:03.761620+0000","last_became_peered":"2024-09-12T00:54:03.761620+0000","last_unstale":"2024-09-12T00:54:04.768097+0000","last_undegraded":"2024-09-12T00:54:04.768097+0000","last_fullsized":"2024-09-12T00:54:04.768097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:35.657480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768582+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:54:04.768582+0000","last_peered":"2024-09-12T00:54:04.768582+0000","last_clean":"2024-09-12T00:54:04.768582+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:54:04.768582+0000","last_undegraded":"2024-09-12T00:54:04.768582+0000","last_fullsized":"2024-09-12T00:54:04.768582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:07.386806+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:54:07.386806+0000","last_peered":"2024-09-12T00:54:07.386806+0000","last_clean":"2024-09-12T00:54:07.386806+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:54:07.386806+0000","last_undegraded":"2024-09-12T00:54:07.386806+0000","last_fullsized":"2024-09-12T00:54:07.386806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822137+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:54:15.822137+0000","last_peered":"2024-09-12T00:54:15.822137+0000","last_clean":"2024-09-12T00:54:15.822137+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:54:15.822137+0000","last_undegraded":"2024-09-12T00:54:15.822137+0000","last_fullsized":"2024-09-12T00:54:15.822137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752645+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:54:04.752645+0000","last_peered":"2024-09-12T00:54:04.752645+0000","last_clean":"2024-09-12T00:54:04.752645+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:54:04.752645+0000","last_undegraded":"2024-09-12T00:54:04.752645+0000","last_fullsized":"2024-09-12T00:54:04.752645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:25.771111+0000","last_change":"2024-09-12T00:54:01.667655+0000","last_active":"2024-09-12T00:54:25.771111+0000","last_peered":"2024-09-12T00:54:25.771111+0000","last_clean":"2024-09-12T00:54:25.771111+0000","last_became_active":"2024-09-12T00:54:01.666917+0000","last_became_peered":"2024-09-12T00:54:01.666917+0000","last_unstale":"2024-09-12T00:54:25.771111+0000","last_undegraded":"2024-09-12T00:54:25.771111+0000","last_fullsized":"2024-09-12T00:54:25.771111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:07:44.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769200+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:54:04.769200+0000","last_peered":"2024-09-12T00:54:04.769200+0000","last_clean":"2024-09-12T00:54:04.769200+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:54:04.769200+0000","last_undegraded":"2024-09-12T00:54:04.769200+0000","last_fullsized":"2024-09-12T00:54:04.769200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822229+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:54:15.822229+0000","last_peered":"2024-09-12T00:54:15.822229+0000","last_clean":"2024-09-12T00:54:15.822229+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:54:15.822229+0000","last_undegraded":"2024-09-12T00:54:15.822229+0000","last_fullsized":"2024-09-12T00:54:15.822229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771950+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:54:04.771950+0000","last_peered":"2024-09-12T00:54:04.771950+0000","last_clean":"2024-09-12T00:54:04.771950+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:54:04.771950+0000","last_undegraded":"2024-09-12T00:54:04.771950+0000","last_fullsized":"2024-09-12T00:54:04.771950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.799867+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:54:12.799867+0000","last_peered":"2024-09-12T00:54:12.799867+0000","last_clean":"2024-09-12T00:54:12.799867+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:54:12.799867+0000","last_undegraded":"2024-09-12T00:54:12.799867+0000","last_fullsized":"2024-09-12T00:54:12.799867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768440+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:54:04.768440+0000","last_peered":"2024-09-12T00:54:04.768440+0000","last_clean":"2024-09-12T00:54:04.768440+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:54:04.768440+0000","last_undegraded":"2024-09-12T00:54:04.768440+0000","last_fullsized":"2024-09-12T00:54:04.768440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.823344+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:54:15.823344+0000","last_peered":"2024-09-12T00:54:15.823344+0000","last_clean":"2024-09-12T00:54:15.823344+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:54:15.823344+0000","last_undegraded":"2024-09-12T00:54:15.823344+0000","last_fullsized":"2024-09-12T00:54:15.823344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755557+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:54:04.755557+0000","last_peered":"2024-09-12T00:54:04.755557+0000","last_clean":"2024-09-12T00:54:04.755557+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:54:04.755557+0000","last_undegraded":"2024-09-12T00:54:04.755557+0000","last_fullsized":"2024-09-12T00:54:04.755557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.817700+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:54:14.817700+0000","last_peered":"2024-09-12T00:54:14.817700+0000","last_clean":"2024-09-12T00:54:14.817700+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:54:14.817700+0000","last_undegraded":"2024-09-12T00:54:14.817700+0000","last_fullsized":"2024-09-12T00:54:14.817700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768828+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:54:04.768828+0000","last_peered":"2024-09-12T00:54:04.768828+0000","last_clean":"2024-09-12T00:54:04.768828+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:54:04.768828+0000","last_undegraded":"2024-09-12T00:54:04.768828+0000","last_fullsized":"2024-09-12T00:54:04.768828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.800746+0000","last_change":"2024-09-12T00:54:01.670253+0000","last_active":"2024-09-12T00:54:12.800746+0000","last_peered":"2024-09-12T00:54:12.800746+0000","last_clean":"2024-09-12T00:54:12.800746+0000","last_became_active":"2024-09-12T00:54:01.669438+0000","last_became_peered":"2024-09-12T00:54:01.669438+0000","last_unstale":"2024-09-12T00:54:12.800746+0000","last_undegraded":"2024-09-12T00:54:12.800746+0000","last_fullsized":"2024-09-12T00:54:12.800746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:46.323570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771941+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:54:04.771941+0000","last_peered":"2024-09-12T00:54:04.771941+0000","last_clean":"2024-09-12T00:54:04.771941+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:54:04.771941+0000","last_undegraded":"2024-09-12T00:54:04.771941+0000","last_fullsized":"2024-09-12T00:54:04.771941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769751+0000","last_change":"2024-09-12T00:54:03.769913+0000","last_active":"2024-09-12T00:54:04.769751+0000","last_peered":"2024-09-12T00:54:04.769751+0000","last_clean":"2024-09-12T00:54:04.769751+0000","last_became_active":"2024-09-12T00:54:03.769550+0000","last_became_peered":"2024-09-12T00:54:03.769550+0000","last_unstale":"2024-09-12T00:54:04.769751+0000","last_undegraded":"2024-09-12T00:54:04.769751+0000","last_fullsized":"2024-09-12T00:54:04.769751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:19.126729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822142+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:54:15.822142+0000","last_peered":"2024-09-12T00:54:15.822142+0000","last_clean":"2024-09-12T00:54:15.822142+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:54:15.822142+0000","last_undegraded":"2024-09-12T00:54:15.822142+0000","last_fullsized":"2024-09-12T00:54:15.822142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770106+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:54:04.770106+0000","last_peered":"2024-09-12T00:54:04.770106+0000","last_clean":"2024-09-12T00:54:04.770106+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:54:04.770106+0000","last_undegraded":"2024-09-12T00:54:04.770106+0000","last_fullsized":"2024-09-12T00:54:04.770106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.817499+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:54:16.817499+0000","last_peered":"2024-09-12T00:54:16.817499+0000","last_clean":"2024-09-12T00:54:16.817499+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:54:16.817499+0000","last_undegraded":"2024-09-12T00:54:16.817499+0000","last_fullsized":"2024-09-12T00:54:16.817499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772140+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:54:04.772140+0000","last_peered":"2024-09-12T00:54:04.772140+0000","last_clean":"2024-09-12T00:54:04.772140+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:54:04.772140+0000","last_undegraded":"2024-09-12T00:54:04.772140+0000","last_fullsized":"2024-09-12T00:54:04.772140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770949+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:54:04.770949+0000","last_peered":"2024-09-12T00:54:04.770949+0000","last_clean":"2024-09-12T00:54:04.770949+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:54:04.770949+0000","last_undegraded":"2024-09-12T00:54:04.770949+0000","last_fullsized":"2024-09-12T00:54:04.770949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769067+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:54:04.769067+0000","last_peered":"2024-09-12T00:54:04.769067+0000","last_clean":"2024-09-12T00:54:04.769067+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:54:04.769067+0000","last_undegraded":"2024-09-12T00:54:04.769067+0000","last_fullsized":"2024-09-12T00:54:04.769067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.839390+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:54:26.839390+0000","last_peered":"2024-09-12T00:54:26.839390+0000","last_clean":"2024-09-12T00:54:26.839390+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:54:26.839390+0000","last_undegraded":"2024-09-12T00:54:26.839390+0000","last_fullsized":"2024-09-12T00:54:26.839390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":231,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":162030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214598,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1065361,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":427067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168607,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":427445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1019965,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":426784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":1019250,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":426609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":475112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":474499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":428516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:54:31.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:54:31.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 -- 172.21.15.16:0/2541833651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102570 msgr2=0x7fdea81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 --2- 172.21.15.16:0/2541833651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102570 0x7fdea81029b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde98009ca0 tx=0x7fde9802a950 comp rx=0 tx=0).stop 2024-09-12T00:54:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 -- 172.21.15.16:0/2541833651 shutdown_connections 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 --2- 172.21.15.16:0/2541833651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102ef0 0x7fdea810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 --2- 172.21.15.16:0/2541833651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102570 0x7fdea81029b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 --2- 172.21.15.16:0/2541833651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 0x7fdea8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 -- 172.21.15.16:0/2541833651 >> 172.21.15.16:0/2541833651 conn(0x7fdea80fc7d0 msgr2=0x7fdea80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 -- 172.21.15.16:0/2541833651 shutdown_connections 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.433+0000 7fdeaea68640 1 -- 172.21.15.16:0/2541833651 wait complete. 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 Processor -- start 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 -- start start 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 0x7fdea8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 0x7fdea8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8110f80 con 0x7fdea8102ef0 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8110e00 con 0x7fdea8100e20 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeaea68640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8111100 con 0x7fdea8102570 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45110/0 (socket says 172.21.15.16:45110) 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 -- 172.21.15.16:0/1209498613 learned_addr learned my addr 172.21.15.16:0/1209498613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdea7fff640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 0x7fdea8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdeac804640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 0x7fdea8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 -- 172.21.15.16:0/1209498613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 msgr2=0x7fdea8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 0x7fdea8199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 -- 172.21.15.16:0/1209498613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 msgr2=0x7fdea8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 0x7fdea8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdead005640 1 -- 172.21.15.16:0/1209498613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea81a4910 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.434+0000 7fdea7fff640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 0x7fdea8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdead005640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e970 tx=0x7fde9000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000ccc0 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea81a4ae0 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9000ce60 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde90010490 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.435+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a4ff0 con 0x7fdea8102ef0 2024-09-12T00:54:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.436+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8101200 con 0x7fdea8102ef0 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.437+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fde90005020 con 0x7fdea8102ef0 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.438+0000 7fdea5ffb640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 0x7fde940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.438+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fde900cf510 con 0x7fdea8102ef0 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.438+0000 7fdeac804640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 0x7fde940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.438+0000 7fdeac804640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 0x7fde940a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdea0005fd0 tx=0x7fdea0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.442+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde9009c4c0 con 0x7fdea8102ef0 2024-09-12T00:54:32.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.615+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdea8109100 con 0x7fde940a2c10 2024-09-12T00:54:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.624+0000 7fdea5ffb640 1 -- 172.21.15.16:0/1209498613 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258869 (secure 0 0 0) 0x7fdea000ea00 con 0x7fde940a2c10 2024-09-12T00:54:32.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 msgr2=0x7fde940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 0x7fde940a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdea0005fd0 tx=0x7fdea0005850 comp rx=0 tx=0).stop 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 msgr2=0x7fdea81a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fde9000e970 tx=0x7fde9000ed70 comp rx=0 tx=0).stop 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 shutdown_connections 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fde940a2c10 0x7fde940a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdea8102ef0 0x7fdea81a43d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdea8102570 0x7fdea8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 --2- 172.21.15.16:0/1209498613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea8100e20 0x7fdea8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 >> 172.21.15.16:0/1209498613 conn(0x7fdea80fc7d0 msgr2=0x7fdea81068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 shutdown_connections 2024-09-12T00:54:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:32.627+0000 7fdeaea68640 1 -- 172.21.15.16:0/1209498613 wait complete. 2024-09-12T00:54:33.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:33 smithi016 ceph-mon[30972]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-09-12T00:54:33.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:33 smithi016 ceph-mon[30972]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:33 smithi099 ceph-mon[35654]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-09-12T00:54:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:33 smithi099 ceph-mon[35654]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:33.529 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-12T00:54:32.213983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644612,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":300,"num_read_kb":257,"num_write":414,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372740,"kb_used_data":11012,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569084,"statfs":{"total":1151940427776,"available":1151558742016,"internally_reserved":0,"allocated":11276288,"data_stored":7053328,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46916,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":135,"num_read_kb":118,"num_write":45,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003059"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755743+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:54:04.755743+0000","last_peered":"2024-09-12T00:54:04.755743+0000","last_clean":"2024-09-12T00:54:04.755743+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:54:04.755743+0000","last_undegraded":"2024-09-12T00:54:04.755743+0000","last_fullsized":"2024-09-12T00:54:04.755743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771673+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:54:04.771673+0000","last_peered":"2024-09-12T00:54:04.771673+0000","last_clean":"2024-09-12T00:54:04.771673+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:54:04.771673+0000","last_undegraded":"2024-09-12T00:54:04.771673+0000","last_fullsized":"2024-09-12T00:54:04.771673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771688+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:54:04.771688+0000","last_peered":"2024-09-12T00:54:04.771688+0000","last_clean":"2024-09-12T00:54:04.771688+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:54:04.771688+0000","last_undegraded":"2024-09-12T00:54:04.771688+0000","last_fullsized":"2024-09-12T00:54:04.771688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.754332+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:54:04.754332+0000","last_peered":"2024-09-12T00:54:04.754332+0000","last_clean":"2024-09-12T00:54:04.754332+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:54:04.754332+0000","last_undegraded":"2024-09-12T00:54:04.754332+0000","last_fullsized":"2024-09-12T00:54:04.754332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768621+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:54:04.768621+0000","last_peered":"2024-09-12T00:54:04.768621+0000","last_clean":"2024-09-12T00:54:04.768621+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:54:04.768621+0000","last_undegraded":"2024-09-12T00:54:04.768621+0000","last_fullsized":"2024-09-12T00:54:04.768621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.767888+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:54:04.767888+0000","last_peered":"2024-09-12T00:54:04.767888+0000","last_clean":"2024-09-12T00:54:04.767888+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:54:04.767888+0000","last_undegraded":"2024-09-12T00:54:04.767888+0000","last_fullsized":"2024-09-12T00:54:04.767888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771153+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:54:04.771153+0000","last_peered":"2024-09-12T00:54:04.771153+0000","last_clean":"2024-09-12T00:54:04.771153+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:54:04.771153+0000","last_undegraded":"2024-09-12T00:54:04.771153+0000","last_fullsized":"2024-09-12T00:54:04.771153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770212+0000","last_change":"2024-09-12T00:54:03.755510+0000","last_active":"2024-09-12T00:54:04.770212+0000","last_peered":"2024-09-12T00:54:04.770212+0000","last_clean":"2024-09-12T00:54:04.770212+0000","last_became_active":"2024-09-12T00:54:03.754586+0000","last_became_peered":"2024-09-12T00:54:03.754586+0000","last_unstale":"2024-09-12T00:54:04.770212+0000","last_undegraded":"2024-09-12T00:54:04.770212+0000","last_fullsized":"2024-09-12T00:54:04.770212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:25:15.505506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770626+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:54:04.770626+0000","last_peered":"2024-09-12T00:54:04.770626+0000","last_clean":"2024-09-12T00:54:04.770626+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:54:04.770626+0000","last_undegraded":"2024-09-12T00:54:04.770626+0000","last_fullsized":"2024-09-12T00:54:04.770626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768153+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:54:04.768153+0000","last_peered":"2024-09-12T00:54:04.768153+0000","last_clean":"2024-09-12T00:54:04.768153+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:54:04.768153+0000","last_undegraded":"2024-09-12T00:54:04.768153+0000","last_fullsized":"2024-09-12T00:54:04.768153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772424+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:54:04.772424+0000","last_peered":"2024-09-12T00:54:04.772424+0000","last_clean":"2024-09-12T00:54:04.772424+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:54:04.772424+0000","last_undegraded":"2024-09-12T00:54:04.772424+0000","last_fullsized":"2024-09-12T00:54:04.772424+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.754328+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:54:04.754328+0000","last_peered":"2024-09-12T00:54:04.754328+0000","last_clean":"2024-09-12T00:54:04.754328+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:54:04.754328+0000","last_undegraded":"2024-09-12T00:54:04.754328+0000","last_fullsized":"2024-09-12T00:54:04.754328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752648+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:54:04.752648+0000","last_peered":"2024-09-12T00:54:04.752648+0000","last_clean":"2024-09-12T00:54:04.752648+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:54:04.752648+0000","last_undegraded":"2024-09-12T00:54:04.752648+0000","last_fullsized":"2024-09-12T00:54:04.752648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771902+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:54:04.771902+0000","last_peered":"2024-09-12T00:54:04.771902+0000","last_clean":"2024-09-12T00:54:04.771902+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:54:04.771902+0000","last_undegraded":"2024-09-12T00:54:04.771902+0000","last_fullsized":"2024-09-12T00:54:04.771902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771797+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:54:04.771797+0000","last_peered":"2024-09-12T00:54:04.771797+0000","last_clean":"2024-09-12T00:54:04.771797+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:54:04.771797+0000","last_undegraded":"2024-09-12T00:54:04.771797+0000","last_fullsized":"2024-09-12T00:54:04.771797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770578+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:54:04.770578+0000","last_peered":"2024-09-12T00:54:04.770578+0000","last_clean":"2024-09-12T00:54:04.770578+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:54:04.770578+0000","last_undegraded":"2024-09-12T00:54:04.770578+0000","last_fullsized":"2024-09-12T00:54:04.770578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755788+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:54:04.755788+0000","last_peered":"2024-09-12T00:54:04.755788+0000","last_clean":"2024-09-12T00:54:04.755788+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:54:04.755788+0000","last_undegraded":"2024-09-12T00:54:04.755788+0000","last_fullsized":"2024-09-12T00:54:04.755788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769817+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:54:04.769817+0000","last_peered":"2024-09-12T00:54:04.769817+0000","last_clean":"2024-09-12T00:54:04.769817+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:54:04.769817+0000","last_undegraded":"2024-09-12T00:54:04.769817+0000","last_fullsized":"2024-09-12T00:54:04.769817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771241+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:54:04.771241+0000","last_peered":"2024-09-12T00:54:04.771241+0000","last_clean":"2024-09-12T00:54:04.771241+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:54:04.771241+0000","last_undegraded":"2024-09-12T00:54:04.771241+0000","last_fullsized":"2024-09-12T00:54:04.771241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755469+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:54:04.755469+0000","last_peered":"2024-09-12T00:54:04.755469+0000","last_clean":"2024-09-12T00:54:04.755469+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:54:04.755469+0000","last_undegraded":"2024-09-12T00:54:04.755469+0000","last_fullsized":"2024-09-12T00:54:04.755469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752826+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:54:04.752826+0000","last_peered":"2024-09-12T00:54:04.752826+0000","last_clean":"2024-09-12T00:54:04.752826+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:54:04.752826+0000","last_undegraded":"2024-09-12T00:54:04.752826+0000","last_fullsized":"2024-09-12T00:54:04.752826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770710+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:54:04.770710+0000","last_peered":"2024-09-12T00:54:04.770710+0000","last_clean":"2024-09-12T00:54:04.770710+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:54:04.770710+0000","last_undegraded":"2024-09-12T00:54:04.770710+0000","last_fullsized":"2024-09-12T00:54:04.770710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768471+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:54:04.768471+0000","last_peered":"2024-09-12T00:54:04.768471+0000","last_clean":"2024-09-12T00:54:04.768471+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:54:04.768471+0000","last_undegraded":"2024-09-12T00:54:04.768471+0000","last_fullsized":"2024-09-12T00:54:04.768471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768736+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:54:04.768736+0000","last_peered":"2024-09-12T00:54:04.768736+0000","last_clean":"2024-09-12T00:54:04.768736+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:54:04.768736+0000","last_undegraded":"2024-09-12T00:54:04.768736+0000","last_fullsized":"2024-09-12T00:54:04.768736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769177+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:54:04.769177+0000","last_peered":"2024-09-12T00:54:04.769177+0000","last_clean":"2024-09-12T00:54:04.769177+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:54:04.769177+0000","last_undegraded":"2024-09-12T00:54:04.769177+0000","last_fullsized":"2024-09-12T00:54:04.769177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772372+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:54:04.772372+0000","last_peered":"2024-09-12T00:54:04.772372+0000","last_clean":"2024-09-12T00:54:04.772372+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:54:04.772372+0000","last_undegraded":"2024-09-12T00:54:04.772372+0000","last_fullsized":"2024-09-12T00:54:04.772372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755640+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:54:04.755640+0000","last_peered":"2024-09-12T00:54:04.755640+0000","last_clean":"2024-09-12T00:54:04.755640+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:54:04.755640+0000","last_undegraded":"2024-09-12T00:54:04.755640+0000","last_fullsized":"2024-09-12T00:54:04.755640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752591+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:54:04.752591+0000","last_peered":"2024-09-12T00:54:04.752591+0000","last_clean":"2024-09-12T00:54:04.752591+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:54:04.752591+0000","last_undegraded":"2024-09-12T00:54:04.752591+0000","last_fullsized":"2024-09-12T00:54:04.752591+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771746+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:54:04.771746+0000","last_peered":"2024-09-12T00:54:04.771746+0000","last_clean":"2024-09-12T00:54:04.771746+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:54:04.771746+0000","last_undegraded":"2024-09-12T00:54:04.771746+0000","last_fullsized":"2024-09-12T00:54:04.771746+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768486+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:54:04.768486+0000","last_peered":"2024-09-12T00:54:04.768486+0000","last_clean":"2024-09-12T00:54:04.768486+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:54:04.768486+0000","last_undegraded":"2024-09-12T00:54:04.768486+0000","last_fullsized":"2024-09-12T00:54:04.768486+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768681+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:54:04.768681+0000","last_peered":"2024-09-12T00:54:04.768681+0000","last_clean":"2024-09-12T00:54:04.768681+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:54:04.768681+0000","last_undegraded":"2024-09-12T00:54:04.768681+0000","last_fullsized":"2024-09-12T00:54:04.768681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755330+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:54:04.755330+0000","last_peered":"2024-09-12T00:54:04.755330+0000","last_clean":"2024-09-12T00:54:04.755330+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:54:04.755330+0000","last_undegraded":"2024-09-12T00:54:04.755330+0000","last_fullsized":"2024-09-12T00:54:04.755330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770789+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:54:04.770789+0000","last_peered":"2024-09-12T00:54:04.770789+0000","last_clean":"2024-09-12T00:54:04.770789+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:54:04.770789+0000","last_undegraded":"2024-09-12T00:54:04.770789+0000","last_fullsized":"2024-09-12T00:54:04.770789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:27.842049+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:54:27.842049+0000","last_peered":"2024-09-12T00:54:27.842049+0000","last_clean":"2024-09-12T00:54:27.842049+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:54:27.842049+0000","last_undegraded":"2024-09-12T00:54:27.842049+0000","last_fullsized":"2024-09-12T00:54:27.842049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769887+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:54:04.769887+0000","last_peered":"2024-09-12T00:54:04.769887+0000","last_clean":"2024-09-12T00:54:04.769887+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:54:04.769887+0000","last_undegraded":"2024-09-12T00:54:04.769887+0000","last_fullsized":"2024-09-12T00:54:04.769887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.817789+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:54:14.817789+0000","last_peered":"2024-09-12T00:54:14.817789+0000","last_clean":"2024-09-12T00:54:14.817789+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:54:14.817789+0000","last_undegraded":"2024-09-12T00:54:14.817789+0000","last_fullsized":"2024-09-12T00:54:14.817789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:13.807787+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:54:13.807787+0000","last_peered":"2024-09-12T00:54:13.807787+0000","last_clean":"2024-09-12T00:54:13.807787+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:54:13.807787+0000","last_undegraded":"2024-09-12T00:54:13.807787+0000","last_fullsized":"2024-09-12T00:54:13.807787+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770308+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:54:04.770308+0000","last_peered":"2024-09-12T00:54:04.770308+0000","last_clean":"2024-09-12T00:54:04.770308+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:54:04.770308+0000","last_undegraded":"2024-09-12T00:54:04.770308+0000","last_fullsized":"2024-09-12T00:54:04.770308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.820882+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:54:15.820882+0000","last_peered":"2024-09-12T00:54:15.820882+0000","last_clean":"2024-09-12T00:54:15.820882+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:54:15.820882+0000","last_undegraded":"2024-09-12T00:54:15.820882+0000","last_fullsized":"2024-09-12T00:54:15.820882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768125+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:54:04.768125+0000","last_peered":"2024-09-12T00:54:04.768125+0000","last_clean":"2024-09-12T00:54:04.768125+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:54:04.768125+0000","last_undegraded":"2024-09-12T00:54:04.768125+0000","last_fullsized":"2024-09-12T00:54:04.768125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.840061+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:54:26.840061+0000","last_peered":"2024-09-12T00:54:26.840061+0000","last_clean":"2024-09-12T00:54:26.840061+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:54:26.840061+0000","last_undegraded":"2024-09-12T00:54:26.840061+0000","last_fullsized":"2024-09-12T00:54:26.840061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770497+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:54:04.770497+0000","last_peered":"2024-09-12T00:54:04.770497+0000","last_clean":"2024-09-12T00:54:04.770497+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:54:04.770497+0000","last_undegraded":"2024-09-12T00:54:04.770497+0000","last_fullsized":"2024-09-12T00:54:04.770497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822432+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:54:15.822432+0000","last_peered":"2024-09-12T00:54:15.822432+0000","last_clean":"2024-09-12T00:54:15.822432+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:54:15.822432+0000","last_undegraded":"2024-09-12T00:54:15.822432+0000","last_fullsized":"2024-09-12T00:54:15.822432+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772148+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:54:04.772148+0000","last_peered":"2024-09-12T00:54:04.772148+0000","last_clean":"2024-09-12T00:54:04.772148+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:54:04.772148+0000","last_undegraded":"2024-09-12T00:54:04.772148+0000","last_fullsized":"2024-09-12T00:54:04.772148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.816658+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:54:14.816658+0000","last_peered":"2024-09-12T00:54:14.816658+0000","last_clean":"2024-09-12T00:54:14.816658+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:54:14.816658+0000","last_undegraded":"2024-09-12T00:54:14.816658+0000","last_fullsized":"2024-09-12T00:54:14.816658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770432+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:54:04.770432+0000","last_peered":"2024-09-12T00:54:04.770432+0000","last_clean":"2024-09-12T00:54:04.770432+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:54:04.770432+0000","last_undegraded":"2024-09-12T00:54:04.770432+0000","last_fullsized":"2024-09-12T00:54:04.770432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'24","reported_seq":69,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.843213+0000","last_change":"2024-09-12T00:54:01.669518+0000","last_active":"2024-09-12T00:54:26.843213+0000","last_peered":"2024-09-12T00:54:26.843213+0000","last_clean":"2024-09-12T00:54:26.843213+0000","last_became_active":"2024-09-12T00:54:01.669145+0000","last_became_peered":"2024-09-12T00:54:01.669145+0000","last_unstale":"2024-09-12T00:54:26.843213+0000","last_undegraded":"2024-09-12T00:54:26.843213+0000","last_fullsized":"2024-09-12T00:54:26.843213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:48.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.773321+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:54:04.773321+0000","last_peered":"2024-09-12T00:54:04.773321+0000","last_clean":"2024-09-12T00:54:04.773321+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:54:04.773321+0000","last_undegraded":"2024-09-12T00:54:04.773321+0000","last_fullsized":"2024-09-12T00:54:04.773321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.823087+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:54:16.823087+0000","last_peered":"2024-09-12T00:54:16.823087+0000","last_clean":"2024-09-12T00:54:16.823087+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:54:16.823087+0000","last_undegraded":"2024-09-12T00:54:16.823087+0000","last_fullsized":"2024-09-12T00:54:16.823087+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752793+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:54:04.752793+0000","last_peered":"2024-09-12T00:54:04.752793+0000","last_clean":"2024-09-12T00:54:04.752793+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:54:04.752793+0000","last_undegraded":"2024-09-12T00:54:04.752793+0000","last_fullsized":"2024-09-12T00:54:04.752793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821246+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:54:15.821246+0000","last_peered":"2024-09-12T00:54:15.821246+0000","last_clean":"2024-09-12T00:54:15.821246+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:54:15.821246+0000","last_undegraded":"2024-09-12T00:54:15.821246+0000","last_fullsized":"2024-09-12T00:54:15.821246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771725+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:54:04.771725+0000","last_peered":"2024-09-12T00:54:04.771725+0000","last_clean":"2024-09-12T00:54:04.771725+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:54:04.771725+0000","last_undegraded":"2024-09-12T00:54:04.771725+0000","last_fullsized":"2024-09-12T00:54:04.771725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:28.842559+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:54:28.842559+0000","last_peered":"2024-09-12T00:54:28.842559+0000","last_clean":"2024-09-12T00:54:28.842559+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:54:28.842559+0000","last_undegraded":"2024-09-12T00:54:28.842559+0000","last_fullsized":"2024-09-12T00:54:28.842559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771172+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:54:04.771172+0000","last_peered":"2024-09-12T00:54:04.771172+0000","last_clean":"2024-09-12T00:54:04.771172+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:54:04.771172+0000","last_undegraded":"2024-09-12T00:54:04.771172+0000","last_fullsized":"2024-09-12T00:54:04.771172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821838+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:54:15.821838+0000","last_peered":"2024-09-12T00:54:15.821838+0000","last_clean":"2024-09-12T00:54:15.821838+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:54:15.821838+0000","last_undegraded":"2024-09-12T00:54:15.821838+0000","last_fullsized":"2024-09-12T00:54:15.821838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768802+0000","last_change":"2024-09-12T00:54:03.763987+0000","last_active":"2024-09-12T00:54:04.768802+0000","last_peered":"2024-09-12T00:54:04.768802+0000","last_clean":"2024-09-12T00:54:04.768802+0000","last_became_active":"2024-09-12T00:54:03.763634+0000","last_became_peered":"2024-09-12T00:54:03.763634+0000","last_unstale":"2024-09-12T00:54:04.768802+0000","last_undegraded":"2024-09-12T00:54:04.768802+0000","last_fullsized":"2024-09-12T00:54:04.768802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:04.348736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.803434+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:54:15.803434+0000","last_peered":"2024-09-12T00:54:15.803434+0000","last_clean":"2024-09-12T00:54:15.803434+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:54:15.803434+0000","last_undegraded":"2024-09-12T00:54:15.803434+0000","last_fullsized":"2024-09-12T00:54:15.803434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755554+0000","last_change":"2024-09-12T00:54:03.762778+0000","last_active":"2024-09-12T00:54:04.755554+0000","last_peered":"2024-09-12T00:54:04.755554+0000","last_clean":"2024-09-12T00:54:04.755554+0000","last_became_active":"2024-09-12T00:54:03.762469+0000","last_became_peered":"2024-09-12T00:54:03.762469+0000","last_unstale":"2024-09-12T00:54:04.755554+0000","last_undegraded":"2024-09-12T00:54:04.755554+0000","last_fullsized":"2024-09-12T00:54:04.755554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:01.749510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.821379+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:54:15.821379+0000","last_peered":"2024-09-12T00:54:15.821379+0000","last_clean":"2024-09-12T00:54:15.821379+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:54:15.821379+0000","last_undegraded":"2024-09-12T00:54:15.821379+0000","last_fullsized":"2024-09-12T00:54:15.821379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772428+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:54:04.772428+0000","last_peered":"2024-09-12T00:54:04.772428+0000","last_clean":"2024-09-12T00:54:04.772428+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:54:04.772428+0000","last_undegraded":"2024-09-12T00:54:04.772428+0000","last_fullsized":"2024-09-12T00:54:04.772428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.823361+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:54:15.823361+0000","last_peered":"2024-09-12T00:54:15.823361+0000","last_clean":"2024-09-12T00:54:15.823361+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:54:15.823361+0000","last_undegraded":"2024-09-12T00:54:15.823361+0000","last_fullsized":"2024-09-12T00:54:15.823361+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.953262+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:54:04.953262+0000","last_peered":"2024-09-12T00:54:04.953262+0000","last_clean":"2024-09-12T00:54:04.953262+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:54:04.953262+0000","last_undegraded":"2024-09-12T00:54:04.953262+0000","last_fullsized":"2024-09-12T00:54:04.953262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:30.845998+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:54:30.845998+0000","last_peered":"2024-09-12T00:54:30.845998+0000","last_clean":"2024-09-12T00:54:30.845998+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:54:30.845998+0000","last_undegraded":"2024-09-12T00:54:30.845998+0000","last_fullsized":"2024-09-12T00:54:30.845998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772594+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:54:04.772594+0000","last_peered":"2024-09-12T00:54:04.772594+0000","last_clean":"2024-09-12T00:54:04.772594+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:54:04.772594+0000","last_undegraded":"2024-09-12T00:54:04.772594+0000","last_fullsized":"2024-09-12T00:54:04.772594+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771446+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:54:04.771446+0000","last_peered":"2024-09-12T00:54:04.771446+0000","last_clean":"2024-09-12T00:54:04.771446+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:54:04.771446+0000","last_undegraded":"2024-09-12T00:54:04.771446+0000","last_fullsized":"2024-09-12T00:54:04.771446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.823163+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:54:16.823163+0000","last_peered":"2024-09-12T00:54:16.823163+0000","last_clean":"2024-09-12T00:54:16.823163+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:54:16.823163+0000","last_undegraded":"2024-09-12T00:54:16.823163+0000","last_fullsized":"2024-09-12T00:54:16.823163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768811+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:54:04.768811+0000","last_peered":"2024-09-12T00:54:04.768811+0000","last_clean":"2024-09-12T00:54:04.768811+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:54:04.768811+0000","last_undegraded":"2024-09-12T00:54:04.768811+0000","last_fullsized":"2024-09-12T00:54:04.768811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:13.806698+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:54:13.806698+0000","last_peered":"2024-09-12T00:54:13.806698+0000","last_clean":"2024-09-12T00:54:13.806698+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:54:13.806698+0000","last_undegraded":"2024-09-12T00:54:13.806698+0000","last_fullsized":"2024-09-12T00:54:13.806698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752347+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:54:04.752347+0000","last_peered":"2024-09-12T00:54:04.752347+0000","last_clean":"2024-09-12T00:54:04.752347+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:54:04.752347+0000","last_undegraded":"2024-09-12T00:54:04.752347+0000","last_fullsized":"2024-09-12T00:54:04.752347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:28.842170+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:54:28.842170+0000","last_peered":"2024-09-12T00:54:28.842170+0000","last_clean":"2024-09-12T00:54:28.842170+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:54:28.842170+0000","last_undegraded":"2024-09-12T00:54:28.842170+0000","last_fullsized":"2024-09-12T00:54:28.842170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768894+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:54:04.768894+0000","last_peered":"2024-09-12T00:54:04.768894+0000","last_clean":"2024-09-12T00:54:04.768894+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:54:04.768894+0000","last_undegraded":"2024-09-12T00:54:04.768894+0000","last_fullsized":"2024-09-12T00:54:04.768894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:28.842057+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:54:28.842057+0000","last_peered":"2024-09-12T00:54:28.842057+0000","last_clean":"2024-09-12T00:54:28.842057+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:54:28.842057+0000","last_undegraded":"2024-09-12T00:54:28.842057+0000","last_fullsized":"2024-09-12T00:54:28.842057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768097+0000","last_change":"2024-09-12T00:54:03.761993+0000","last_active":"2024-09-12T00:54:04.768097+0000","last_peered":"2024-09-12T00:54:04.768097+0000","last_clean":"2024-09-12T00:54:04.768097+0000","last_became_active":"2024-09-12T00:54:03.761620+0000","last_became_peered":"2024-09-12T00:54:03.761620+0000","last_unstale":"2024-09-12T00:54:04.768097+0000","last_undegraded":"2024-09-12T00:54:04.768097+0000","last_fullsized":"2024-09-12T00:54:04.768097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:35.657480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768582+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:54:04.768582+0000","last_peered":"2024-09-12T00:54:04.768582+0000","last_clean":"2024-09-12T00:54:04.768582+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:54:04.768582+0000","last_undegraded":"2024-09-12T00:54:04.768582+0000","last_fullsized":"2024-09-12T00:54:04.768582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:07.386806+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:54:07.386806+0000","last_peered":"2024-09-12T00:54:07.386806+0000","last_clean":"2024-09-12T00:54:07.386806+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:54:07.386806+0000","last_undegraded":"2024-09-12T00:54:07.386806+0000","last_fullsized":"2024-09-12T00:54:07.386806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822137+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:54:15.822137+0000","last_peered":"2024-09-12T00:54:15.822137+0000","last_clean":"2024-09-12T00:54:15.822137+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:54:15.822137+0000","last_undegraded":"2024-09-12T00:54:15.822137+0000","last_fullsized":"2024-09-12T00:54:15.822137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.752645+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:54:04.752645+0000","last_peered":"2024-09-12T00:54:04.752645+0000","last_clean":"2024-09-12T00:54:04.752645+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:54:04.752645+0000","last_undegraded":"2024-09-12T00:54:04.752645+0000","last_fullsized":"2024-09-12T00:54:04.752645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:25.771111+0000","last_change":"2024-09-12T00:54:01.667655+0000","last_active":"2024-09-12T00:54:25.771111+0000","last_peered":"2024-09-12T00:54:25.771111+0000","last_clean":"2024-09-12T00:54:25.771111+0000","last_became_active":"2024-09-12T00:54:01.666917+0000","last_became_peered":"2024-09-12T00:54:01.666917+0000","last_unstale":"2024-09-12T00:54:25.771111+0000","last_undegraded":"2024-09-12T00:54:25.771111+0000","last_fullsized":"2024-09-12T00:54:25.771111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:07:44.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769200+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:54:04.769200+0000","last_peered":"2024-09-12T00:54:04.769200+0000","last_clean":"2024-09-12T00:54:04.769200+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:54:04.769200+0000","last_undegraded":"2024-09-12T00:54:04.769200+0000","last_fullsized":"2024-09-12T00:54:04.769200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822229+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:54:15.822229+0000","last_peered":"2024-09-12T00:54:15.822229+0000","last_clean":"2024-09-12T00:54:15.822229+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:54:15.822229+0000","last_undegraded":"2024-09-12T00:54:15.822229+0000","last_fullsized":"2024-09-12T00:54:15.822229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771950+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:54:04.771950+0000","last_peered":"2024-09-12T00:54:04.771950+0000","last_clean":"2024-09-12T00:54:04.771950+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:54:04.771950+0000","last_undegraded":"2024-09-12T00:54:04.771950+0000","last_fullsized":"2024-09-12T00:54:04.771950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.799867+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:54:12.799867+0000","last_peered":"2024-09-12T00:54:12.799867+0000","last_clean":"2024-09-12T00:54:12.799867+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:54:12.799867+0000","last_undegraded":"2024-09-12T00:54:12.799867+0000","last_fullsized":"2024-09-12T00:54:12.799867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768440+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:54:04.768440+0000","last_peered":"2024-09-12T00:54:04.768440+0000","last_clean":"2024-09-12T00:54:04.768440+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:54:04.768440+0000","last_undegraded":"2024-09-12T00:54:04.768440+0000","last_fullsized":"2024-09-12T00:54:04.768440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.823344+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:54:15.823344+0000","last_peered":"2024-09-12T00:54:15.823344+0000","last_clean":"2024-09-12T00:54:15.823344+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:54:15.823344+0000","last_undegraded":"2024-09-12T00:54:15.823344+0000","last_fullsized":"2024-09-12T00:54:15.823344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755557+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:54:04.755557+0000","last_peered":"2024-09-12T00:54:04.755557+0000","last_clean":"2024-09-12T00:54:04.755557+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:54:04.755557+0000","last_undegraded":"2024-09-12T00:54:04.755557+0000","last_fullsized":"2024-09-12T00:54:04.755557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.817700+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:54:14.817700+0000","last_peered":"2024-09-12T00:54:14.817700+0000","last_clean":"2024-09-12T00:54:14.817700+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:54:14.817700+0000","last_undegraded":"2024-09-12T00:54:14.817700+0000","last_fullsized":"2024-09-12T00:54:14.817700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768828+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:54:04.768828+0000","last_peered":"2024-09-12T00:54:04.768828+0000","last_clean":"2024-09-12T00:54:04.768828+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:54:04.768828+0000","last_undegraded":"2024-09-12T00:54:04.768828+0000","last_fullsized":"2024-09-12T00:54:04.768828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.800746+0000","last_change":"2024-09-12T00:54:01.670253+0000","last_active":"2024-09-12T00:54:12.800746+0000","last_peered":"2024-09-12T00:54:12.800746+0000","last_clean":"2024-09-12T00:54:12.800746+0000","last_became_active":"2024-09-12T00:54:01.669438+0000","last_became_peered":"2024-09-12T00:54:01.669438+0000","last_unstale":"2024-09-12T00:54:12.800746+0000","last_undegraded":"2024-09-12T00:54:12.800746+0000","last_fullsized":"2024-09-12T00:54:12.800746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:46.323570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.771941+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:54:04.771941+0000","last_peered":"2024-09-12T00:54:04.771941+0000","last_clean":"2024-09-12T00:54:04.771941+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:54:04.771941+0000","last_undegraded":"2024-09-12T00:54:04.771941+0000","last_fullsized":"2024-09-12T00:54:04.771941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769751+0000","last_change":"2024-09-12T00:54:03.769913+0000","last_active":"2024-09-12T00:54:04.769751+0000","last_peered":"2024-09-12T00:54:04.769751+0000","last_clean":"2024-09-12T00:54:04.769751+0000","last_became_active":"2024-09-12T00:54:03.769550+0000","last_became_peered":"2024-09-12T00:54:03.769550+0000","last_unstale":"2024-09-12T00:54:04.769751+0000","last_undegraded":"2024-09-12T00:54:04.769751+0000","last_fullsized":"2024-09-12T00:54:04.769751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:19.126729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.822142+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:54:15.822142+0000","last_peered":"2024-09-12T00:54:15.822142+0000","last_clean":"2024-09-12T00:54:15.822142+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:54:15.822142+0000","last_undegraded":"2024-09-12T00:54:15.822142+0000","last_fullsized":"2024-09-12T00:54:15.822142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770106+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:54:04.770106+0000","last_peered":"2024-09-12T00:54:04.770106+0000","last_clean":"2024-09-12T00:54:04.770106+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:54:04.770106+0000","last_undegraded":"2024-09-12T00:54:04.770106+0000","last_fullsized":"2024-09-12T00:54:04.770106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:16.817499+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:54:16.817499+0000","last_peered":"2024-09-12T00:54:16.817499+0000","last_clean":"2024-09-12T00:54:16.817499+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:54:16.817499+0000","last_undegraded":"2024-09-12T00:54:16.817499+0000","last_fullsized":"2024-09-12T00:54:16.817499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.772140+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:54:04.772140+0000","last_peered":"2024-09-12T00:54:04.772140+0000","last_clean":"2024-09-12T00:54:04.772140+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:54:04.772140+0000","last_undegraded":"2024-09-12T00:54:04.772140+0000","last_fullsized":"2024-09-12T00:54:04.772140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770949+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:54:04.770949+0000","last_peered":"2024-09-12T00:54:04.770949+0000","last_clean":"2024-09-12T00:54:04.770949+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:54:04.770949+0000","last_undegraded":"2024-09-12T00:54:04.770949+0000","last_fullsized":"2024-09-12T00:54:04.770949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769067+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:54:04.769067+0000","last_peered":"2024-09-12T00:54:04.769067+0000","last_clean":"2024-09-12T00:54:04.769067+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:54:04.769067+0000","last_undegraded":"2024-09-12T00:54:04.769067+0000","last_fullsized":"2024-09-12T00:54:04.769067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":44,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:26.839390+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:54:26.839390+0000","last_peered":"2024-09-12T00:54:26.839390+0000","last_clean":"2024-09-12T00:54:26.839390+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:54:26.839390+0000","last_undegraded":"2024-09-12T00:54:26.839390+0000","last_fullsized":"2024-09-12T00:54:26.839390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54010,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":164,"num_read_kb":138,"num_write":231,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":170182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214598,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1065361,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":427067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":427445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1019965,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":426784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":1019250,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":426609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":483264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":474499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":428516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:54:33.533 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T00:54:33.534 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T00:54:33.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T00:54:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:33 smithi159 ceph-mon[35566]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-09-12T00:54:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:33 smithi159 ceph-mon[35566]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:54:33.798 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.318+0000 7f860cf50640 1 -- 172.21.15.16:0/2934121718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 msgr2=0x7f8608102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.318+0000 7f860cf50640 1 --2- 172.21.15.16:0/2934121718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608102000 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009ca0 tx=0x7f85f802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 -- 172.21.15.16:0/2934121718 shutdown_connections 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 --2- 172.21.15.16:0/2934121718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 0x7f860810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 --2- 172.21.15.16:0/2934121718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 --2- 172.21.15.16:0/2934121718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608102000 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 -- 172.21.15.16:0/2934121718 >> 172.21.15.16:0/2934121718 conn(0x7f86080fd390 msgr2=0x7f86080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 -- 172.21.15.16:0/2934121718 shutdown_connections 2024-09-12T00:54:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 -- 172.21.15.16:0/2934121718 wait complete. 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 Processor -- start 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.319+0000 7f860cf50640 1 -- start start 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 0x7f860819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f860810d8b0 con 0x7f8608101c00 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f860810d730 con 0x7f86081037c0 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f860810da30 con 0x7f8608102e00 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45126/0 (socket says 172.21.15.16:45126) 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 -- 172.21.15.16:0/1620823147 learned_addr learned my addr 172.21.15.16:0/1620823147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f8605d9b640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f8606d9d640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 0x7f860819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 -- 172.21.15.16:0/1620823147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 msgr2=0x7f8608198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 -- 172.21.15.16:0/1620823147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 msgr2=0x7f860819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 0x7f860819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 -- 172.21.15.16:0/1620823147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860819fa20 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f8605d9b640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860659c640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f80 tx=0x7f85f8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8004040 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f80046c0 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860819fcb0 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80305c0 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.320+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a01a0 con 0x7f8608101c00 2024-09-12T00:54:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.321+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608108ed0 con 0x7f8608101c00 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.322+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85f802f610 con 0x7f8608101c00 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.323+0000 7f85ff7fe640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 0x7f85f40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.323+0000 7f8605d9b640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 0x7f85f40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.323+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f85f80f07c0 con 0x7f8608101c00 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.324+0000 7f8605d9b640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 0x7f85f40a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8608199310 tx=0x7f85f000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.327+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85f80bd780 con 0x7f8608101c00 2024-09-12T00:54:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.568+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8608062e90 con 0x7f8608101c00 2024-09-12T00:54:34.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.570+0000 7f85ff7fe640 1 -- 172.21.15.16:0/1620823147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f85f80098b0 con 0x7f8608101c00 2024-09-12T00:54:34.569 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-12T00:54:34.569 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDKO+JmDyfuIRAAMFmTsNAHSz6D72/L8iMVmQ== 2024-09-12T00:54:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 msgr2=0x7f85f40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 0x7f85f40a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8608199310 tx=0x7f85f000a5c0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 msgr2=0x7f8608197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000f80 tx=0x7f85f8003ba0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 shutdown_connections 2024-09-12T00:54:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f85f40a2c40 0x7f85f40a5100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86081037c0 0x7f860819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8608102e00 0x7f8608198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 --2- 172.21.15.16:0/1620823147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608101c00 0x7f8608197df0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.572+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 >> 172.21.15.16:0/1620823147 conn(0x7f86080fd390 msgr2=0x7f86081060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.573+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 shutdown_connections 2024-09-12T00:54:34.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:34.573+0000 7f860cf50640 1 -- 172.21.15.16:0/1620823147 wait complete. 2024-09-12T00:54:35.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:35 smithi016 ceph-mon[30972]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T00:54:35.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T00:54:35.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T00:54:35.456 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:54:35.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T00:54:35.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T00:54:35.497 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T00:54:35.498 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T00:54:35.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T00:54:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:35 smithi099 ceph-mon[35654]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T00:54:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T00:54:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T00:54:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:35 smithi159 ceph-mon[35566]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-09-12T00:54:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T00:54:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1620823147' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T00:54:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 -- 172.21.15.16:0/3910461238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24106a80 msgr2=0x7fab24106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 --2- 172.21.15.16:0/3910461238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24106a80 0x7fab24106e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab14009930 tx=0x7fab1402a420 comp rx=0 tx=0).stop 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 -- 172.21.15.16:0/3910461238 shutdown_connections 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 --2- 172.21.15.16:0/3910461238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24101420 0x7fab2410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 --2- 172.21.15.16:0/3910461238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24100a80 0x7fab24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 --2- 172.21.15.16:0/3910461238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24106a80 0x7fab24106e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 -- 172.21.15.16:0/3910461238 >> 172.21.15.16:0/3910461238 conn(0x7fab240fc7a0 msgr2=0x7fab240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:36.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 -- 172.21.15.16:0/3910461238 shutdown_connections 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.267+0000 7fab29e4d640 1 -- 172.21.15.16:0/3910461238 wait complete. 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 Processor -- start 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 -- start start 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24101420 0x7fab240fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24106a80 0x7fab240fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab24110e80 con 0x7fab24100a80 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab24110d00 con 0x7fab24106a80 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab29e4d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab24111000 con 0x7fab24101420 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45146/0 (socket says 172.21.15.16:45146) 2024-09-12T00:54:36.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.268+0000 7fab237fe640 1 -- 172.21.15.16:0/2965548779 learned_addr learned my addr 172.21.15.16:0/2965548779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab22ffd640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24101420 0x7fab240fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab23fff640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24106a80 0x7fab240fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 -- 172.21.15.16:0/2965548779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24101420 msgr2=0x7fab240fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24101420 0x7fab240fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 -- 172.21.15.16:0/2965548779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24106a80 msgr2=0x7fab240fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24106a80 0x7fab240fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 -- 172.21.15.16:0/2965548779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab241a69c0 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab237fe640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab14009230 tx=0x7fab14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1403f030 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab14003ad0 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241a6c50 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14035070 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.269+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a7130 con 0x7fab24100a80 2024-09-12T00:54:36.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.270+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab24100ee0 con 0x7fab24100a80 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.272+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fab14042030 con 0x7fab24100a80 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.272+0000 7fab20ff9640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 0x7fab0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.272+0000 7fab22ffd640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 0x7fab0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.272+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fab140ef470 con 0x7fab24100a80 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.273+0000 7fab22ffd640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 0x7fab0c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab240fe920 tx=0x7fab1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:36.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.276+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab140bbdd0 con 0x7fab24100a80 2024-09-12T00:54:36.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.501+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fab2410a000 con 0x7fab24100a80 2024-09-12T00:54:36.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.505+0000 7fab20ff9640 1 -- 172.21.15.16:0/2965548779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7fab140c00d0 con 0x7fab24100a80 2024-09-12T00:54:36.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 msgr2=0x7fab0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:36.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 0x7fab0c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab240fe920 tx=0x7fab1800b390 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 msgr2=0x7fab240fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab14009230 tx=0x7fab14004270 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 shutdown_connections 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fab0c0a2c10 0x7fab0c0a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab24106a80 0x7fab240fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab24101420 0x7fab240fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 --2- 172.21.15.16:0/2965548779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24100a80 0x7fab240fce90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 >> 172.21.15.16:0/2965548779 conn(0x7fab240fc7a0 msgr2=0x7fab2406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:36.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 shutdown_connections 2024-09-12T00:54:36.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:36.508+0000 7fab29e4d640 1 -- 172.21.15.16:0/2965548779 wait complete. 2024-09-12T00:54:37.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1272021982},{"type":"v1","addr":"172.21.15.16:6801","nonce":1272021982}]},"active_addr":"172.21.15.16:6801/1272021982","active_change":"2024-09-12T00:47:08.126680+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2952840401}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1572487203}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":90830535}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2290747564}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3944279287}]}]} 2024-09-12T00:54:37.263 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T00:54:37.263 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T00:54:37.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:54:37.523 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:37 smithi099 ceph-mon[35654]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T00:54:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:37 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2965548779' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T00:54:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:37 smithi159 ceph-mon[35566]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T00:54:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:37 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2965548779' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T00:54:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:37 smithi016 ceph-mon[30972]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-12T00:54:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:37 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2965548779' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 -- 172.21.15.16:0/1581258611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458106a80 msgr2=0x7f7458106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 --2- 172.21.15.16:0/1581258611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458106a80 0x7f7458106e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7448009930 tx=0x7f744802a420 comp rx=0 tx=0).stop 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 -- 172.21.15.16:0/1581258611 shutdown_connections 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 --2- 172.21.15.16:0/1581258611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458101420 0x7f745810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 --2- 172.21.15.16:0/1581258611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458100a80 0x7f7458100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 --2- 172.21.15.16:0/1581258611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458106a80 0x7f7458106e60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 -- 172.21.15.16:0/1581258611 >> 172.21.15.16:0/1581258611 conn(0x7f74580fc7a0 msgr2=0x7f74580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 -- 172.21.15.16:0/1581258611 shutdown_connections 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.001+0000 7f745e17d640 1 -- 172.21.15.16:0/1581258611 wait complete. 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 Processor -- start 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 -- start start 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458101420 0x7f7458199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 0x7f74581a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7458110f20 con 0x7f7458100a80 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7458110da0 con 0x7f7458106a80 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f745e17d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74581110a0 con 0x7f7458101420 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f74577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f74577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45168/0 (socket says 172.21.15.16:45168) 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f74577fe640 1 -- 172.21.15.16:0/2038151950 learned_addr learned my addr 172.21.15.16:0/2038151950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f7456ffd640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458101420 0x7f7458199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.002+0000 7f7457fff640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 0x7f74581a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 -- 172.21.15.16:0/2038151950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458101420 msgr2=0x7f7458199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458101420 0x7f7458199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 -- 172.21.15.16:0/2038151950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 msgr2=0x7f74581a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 0x7f74581a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 -- 172.21.15.16:0/2038151950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74581a4a10 con 0x7f7458100a80 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f7457fff640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 0x7f74581a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f74577fe640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7448009a50 tx=0x7f7448004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744803f030 con 0x7f7458100a80 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7448003ce0 con 0x7f7458100a80 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74581a4be0 con 0x7f7458100a80 2024-09-12T00:54:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448035070 con 0x7f7458100a80 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.003+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a5030 con 0x7f7458100a80 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.004+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7458100f60 con 0x7f7458100a80 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.005+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7448042030 con 0x7f7458100a80 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.006+0000 7f7454ff9640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 0x7f74400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.006+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f74480ef650 con 0x7f7458100a80 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.006+0000 7f7456ffd640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 0x7f74400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.006+0000 7f7456ffd640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 0x7f74400a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f745806b2e0 tx=0x7f744c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.010+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74480bc600 con 0x7f7458100a80 2024-09-12T00:54:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.185+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7458108a60 con 0x7f7458100a80 2024-09-12T00:54:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.187+0000 7f7454ff9640 1 -- 172.21.15.16:0/2038151950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22191 (secure 0 0 0) 0x7f74480bbf00 con 0x7f7458100a80 2024-09-12T00:54:38.186 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:38.186 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:54:04.465131+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 msgr2=0x7f74400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 0x7f74400a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f745806b2e0 tx=0x7f744c008040 comp rx=0 tx=0).stop 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 msgr2=0x7f7458199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7448009a50 tx=0x7f7448004270 comp rx=0 tx=0).stop 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 shutdown_connections 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74400a2c10 0x7f74400a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458106a80 0x7f74581a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7458101420 0x7f7458199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 --2- 172.21.15.16:0/2038151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458100a80 0x7f7458199450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 >> 172.21.15.16:0/2038151950 conn(0x7f74580fc7a0 msgr2=0x7f74580fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 shutdown_connections 2024-09-12T00:54:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:38.189+0000 7f745e17d640 1 -- 172.21.15.16:0/2038151950 wait complete. 2024-09-12T00:54:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:38 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2038151950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:38.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:38 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2038151950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:38.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:38 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2038151950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:39.006 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T00:54:39.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:54:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:39 smithi099 ceph-mon[35654]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-12T00:54:39.534 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:39 smithi016 ceph-mon[30972]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-12T00:54:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:39 smithi159 ceph-mon[35566]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 12 op/s 2024-09-12T00:54:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 -- 172.21.15.16:0/236261465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c106f10 msgr2=0x7fce7c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 --2- 172.21.15.16:0/236261465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c106f10 0x7fce7c10ea90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce74009ca0 tx=0x7fce7402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 -- 172.21.15.16:0/236261465 shutdown_connections 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 --2- 172.21.15.16:0/236261465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c106f10 0x7fce7c10ea90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 --2- 172.21.15.16:0/236261465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c102790 0x7fce7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 --2- 172.21.15.16:0/236261465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c101de0 0x7fce7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 -- 172.21.15.16:0/236261465 >> 172.21.15.16:0/236261465 conn(0x7fce7c0fd8c0 msgr2=0x7fce7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 -- 172.21.15.16:0/236261465 shutdown_connections 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.760+0000 7fce81d30640 1 -- 172.21.15.16:0/236261465 wait complete. 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 Processor -- start 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 -- start start 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c102790 0x7fce7c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 0x7fce7c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c111f70 con 0x7fce7c101de0 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c111df0 con 0x7fce7c106f10 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce81d30640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce7c1120f0 con 0x7fce7c102790 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58746/0 (socket says 172.21.15.16:58746) 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 -- 172.21.15.16:0/1825283013 learned_addr learned my addr 172.21.15.16:0/1825283013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce72ffd640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c102790 0x7fce7c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 -- 172.21.15.16:0/1825283013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c102790 msgr2=0x7fce7c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c102790 0x7fce7c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7bfff640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 0x7fce7c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.761+0000 7fce7b7fe640 1 -- 172.21.15.16:0/1825283013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 msgr2=0x7fce7c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce7b7fe640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 0x7fce7c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce7b7fe640 1 -- 172.21.15.16:0/1825283013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7c19f590 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce7bfff640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 0x7fce7c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce7b7fe640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00d770 tx=0x7fce6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c017070 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6c005500 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7c19f880 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004e00 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.762+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce7c1a7160 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.763+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce7c1021c0 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.764+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce6c004fa0 con 0x7fce7c101de0 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.765+0000 7fce797fa640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 0x7fce680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.765+0000 7fce72ffd640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 0x7fce680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.765+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fce6c0cefd0 con 0x7fce7c101de0 2024-09-12T00:54:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.766+0000 7fce72ffd640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 0x7fce680a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0ff700 tx=0x7fce6400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.768+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce6c09be80 con 0x7fce7c101de0 2024-09-12T00:54:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.948+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce7c1a7330 con 0x7fce7c101de0 2024-09-12T00:54:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.950+0000 7fce797fa640 1 -- 172.21.15.16:0/1825283013 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22191 (secure 0 0 0) 0x7fce6c09b780 con 0x7fce7c101de0 2024-09-12T00:54:39.950 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:54:39.950 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:54:04.465131+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 msgr2=0x7fce680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 0x7fce680a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce7c0ff700 tx=0x7fce6400b390 comp rx=0 tx=0).stop 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 msgr2=0x7fce7c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00d770 tx=0x7fce6c00dc30 comp rx=0 tx=0).stop 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 shutdown_connections 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fce680a2c10 0x7fce680a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce7c106f10 0x7fce7c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce7c102790 0x7fce7c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 --2- 172.21.15.16:0/1825283013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce7c101de0 0x7fce7c19a5e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.952+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 >> 172.21.15.16:0/1825283013 conn(0x7fce7c0fd8c0 msgr2=0x7fce7c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.953+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 shutdown_connections 2024-09-12T00:54:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:39.953+0000 7fce81d30640 1 -- 172.21.15.16:0/1825283013 wait complete. 2024-09-12T00:54:40.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:40 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1825283013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:40 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1825283013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1825283013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:54:40.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T00:54:40.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T00:54:40.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T00:54:40.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T00:54:40.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T00:54:40.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T00:54:40.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T00:54:40.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T00:54:40.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T00:54:40.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T00:54:40.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T00:54:40.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T00:54:41.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:41 smithi099 ceph-mon[35654]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T00:54:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:41 smithi159 ceph-mon[35566]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T00:54:41.623 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:41 smithi016 ceph-mon[30972]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-12T00:54:41.686 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:42.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 -- 172.21.15.16:0/4105662411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96101009d0 msgr2=0x7f9610100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 --2- 172.21.15.16:0/4105662411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96101009d0 0x7f9610100db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009900 tx=0x7f95f802a410 comp rx=0 tx=0).stop 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 -- 172.21.15.16:0/4105662411 shutdown_connections 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 --2- 172.21.15.16:0/4105662411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961010e4f0 0x7f9610110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 --2- 172.21.15.16:0/4105662411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9610101380 0x7f961010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 --2- 172.21.15.16:0/4105662411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96101009d0 0x7f9610100db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 -- 172.21.15.16:0/4105662411 >> 172.21.15.16:0/4105662411 conn(0x7f96100fc7d0 msgr2=0x7f96100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 -- 172.21.15.16:0/4105662411 shutdown_connections 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.557+0000 7f9616158640 1 -- 172.21.15.16:0/4105662411 wait complete. 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 Processor -- start 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 -- start start 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96101009d0 0x7f96101a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:42.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 0x7f96101a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961010e4f0 0x7f961019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9610114270 con 0x7f9610101380 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96101140f0 con 0x7f96101009d0 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.558+0000 7f9616158640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96101143f0 con 0x7f961010e4f0 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 0x7f96101a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 0x7f96101a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58768/0 (socket says 172.21.15.16:58768) 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 -- 172.21.15.16:0/996319723 learned_addr learned my addr 172.21.15.16:0/996319723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960ffff640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961010e4f0 0x7f961019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960f7fe640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96101009d0 0x7f96101a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 -- 172.21.15.16:0/996319723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961010e4f0 msgr2=0x7f961019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961010e4f0 0x7f961019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 -- 172.21.15.16:0/996319723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96101009d0 msgr2=0x7f96101a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96101009d0 0x7f96101a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961019acd0 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960f7fe640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96101009d0 0x7f96101a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960effd640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 0x7f96101a0920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f960000c970 tx=0x7f960000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600007bb0 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9600007d50 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961019afc0 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960000f450 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.559+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101a72b0 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.560+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9610062900 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.561+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9600004fa0 con 0x7f9610101380 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.562+0000 7f960cff9640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.562+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f96000cf2c0 con 0x7f9610101380 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.562+0000 7f960f7fe640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.562+0000 7f960cff9640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] conn(0x7f95fc0ac460 0x7f95fc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.563+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95fc0aef30 con 0x7f95fc0ac460 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.563+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f96000cf670 con 0x7f9610101380 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.563+0000 7f960ffff640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] conn(0x7f95fc0ac460 0x7f95fc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.563+0000 7f960f7fe640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005ab0 tx=0x7f95f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.563+0000 7f960ffff640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] conn(0x7f95fc0ac460 0x7f95fc0ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.564+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== osd.9 v2:172.21.15.16:6826/45245397 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95fc0aef30 con 0x7f95fc0ac460 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.577+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 --> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9610062d80 con 0x7f95fc0ac460 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f960cff9640 1 -- 172.21.15.16:0/996319723 <== osd.9 v2:172.21.15.16:6826/45245397 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9610062d80 con 0x7f95fc0ac460 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] conn(0x7f95fc0ac460 msgr2=0x7f95fc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] conn(0x7f95fc0ac460 0x7f95fc0ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95fc0a2c10 msgr2=0x7f95fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95fc0a2c10 0x7f95fc0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005ab0 tx=0x7f95f8005a20 comp rx=0 tx=0).stop 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 msgr2=0x7f96101a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 --2- 172.21.15.16:0/996319723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610101380 0x7f96101a0920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f960000c970 tx=0x7f960000ce30 comp rx=0 tx=0).stop 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f960ffff640 1 -- 172.21.15.16:0/996319723 reap_dead start 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 shutdown_connections 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 >> 172.21.15.16:0/996319723 conn(0x7f96100fc7d0 msgr2=0x7f96100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 shutdown_connections 2024-09-12T00:54:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.578+0000 7f9616158640 1 -- 172.21.15.16:0/996319723 wait complete. 2024-09-12T00:54:43.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:43 smithi016 ceph-mon[30972]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T00:54:43.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:43 smithi099 ceph-mon[35654]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T00:54:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:43 smithi159 ceph-mon[35566]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T00:54:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 -- 172.21.15.16:0/3857593739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102ef0 msgr2=0x7f0a2410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3857593739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102ef0 0x7f0a2410da40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009ca0 tx=0x7f0a1c02a950 comp rx=0 tx=0).stop 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 -- 172.21.15.16:0/3857593739 shutdown_connections 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3857593739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24102ef0 0x7f0a2410da40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3857593739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a24102570 0x7f0a241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3857593739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a24100e20 0x7f0a24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 -- 172.21.15.16:0/3857593739 >> 172.21.15.16:0/3857593739 conn(0x7f0a240fc7d0 msgr2=0x7f0a240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 -- 172.21.15.16:0/3857593739 shutdown_connections 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.454+0000 7f0a29e92640 1 -- 172.21.15.16:0/3857593739 wait complete. 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.455+0000 7f0a29e92640 1 Processor -- start 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.455+0000 7f0a29e92640 1 -- start start 2024-09-12T00:54:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.455+0000 7f0a29e92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 0x7f0a24199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a24102570 0x7f0a241998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a24102ef0 0x7f0a241a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24110ff0 con 0x7f0a24100e20 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24110e70 con 0x7f0a24102570 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24111170 con 0x7f0a24102ef0 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 0x7f0a24199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 0x7f0a24199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58756/0 (socket says 172.21.15.16:58756) 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 -- 172.21.15.16:0/3218186132 learned_addr learned my addr 172.21.15.16:0/3218186132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a22ffd640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a24102570 0x7f0a241998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a23fff640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a24102ef0 0x7f0a241a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 -- 172.21.15.16:0/3218186132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a24102ef0 msgr2=0x7f0a241a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a24102ef0 0x7f0a241a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 -- 172.21.15.16:0/3218186132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a24102570 msgr2=0x7f0a241998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a24102570 0x7f0a241998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241a4920 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a237fe640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 0x7f0a24199360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400d950 tx=0x7f0a1400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14017070 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241a4af0 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1400de80 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400f620 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.456+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a5030 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.458+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a24107260 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.459+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0a14009c30 con 0x7f0a24100e20 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a20ff9640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0a0c0a2c10 0x7f0a0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a22ffd640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0a0c0a2c10 0x7f0a0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f0a140cebe0 con 0x7f0a24100e20 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a20ff9640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] conn(0x7f0a0c0ac460 0x7f0a0c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a1401ee50 con 0x7f0a0c0ac460 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f0a140d1330 con 0x7f0a24100e20 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a23fff640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] conn(0x7f0a0c0ac460 0x7f0a0c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.460+0000 7f0a22ffd640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0a0c0a2c10 0x7f0a0c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a2406aac0 tx=0x7f0a18009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.461+0000 7f0a23fff640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] conn(0x7f0a0c0ac460 0x7f0a0c0ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.461+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== osd.0 v2:172.21.15.16:6802/4092481179 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a1401ee50 con 0x7f0a0c0ac460 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.481+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 --> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a241021b0 con 0x7f0a0c0ac460 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a20ff9640 1 -- 172.21.15.16:0/3218186132 <== osd.0 v2:172.21.15.16:6802/4092481179 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a241021b0 con 0x7f0a0c0ac460 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] conn(0x7f0a0c0ac460 msgr2=0x7f0a0c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] conn(0x7f0a0c0ac460 0x7f0a0c0ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0a0c0a2c10 msgr2=0x7f0a0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0a0c0a2c10 0x7f0a0c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a2406aac0 tx=0x7f0a18009110 comp rx=0 tx=0).stop 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 msgr2=0x7f0a24199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 --2- 172.21.15.16:0/3218186132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a24100e20 0x7f0a24199360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400d950 tx=0x7f0a1400dd50 comp rx=0 tx=0).stop 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a23fff640 1 -- 172.21.15.16:0/3218186132 reap_dead start 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 shutdown_connections 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.482+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 >> 172.21.15.16:0/3218186132 conn(0x7f0a240fc7d0 msgr2=0x7f0a24106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.483+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 shutdown_connections 2024-09-12T00:54:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:42.483+0000 7f0a29e92640 1 -- 172.21.15.16:0/3218186132 wait complete. 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.124+0000 7fda24656640 1 -- 172.21.15.16:0/3777599358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c101df0 msgr2=0x7fda1c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.124+0000 7fda24656640 1 --2- 172.21.15.16:0/3777599358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c101df0 0x7fda1c1021d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009900 tx=0x7fda0c02a410 comp rx=0 tx=0).stop 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 -- 172.21.15.16:0/3777599358 shutdown_connections 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 --2- 172.21.15.16:0/3777599358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 0x7fda1c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 --2- 172.21.15.16:0/3777599358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda1c1027a0 0x7fda1c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 --2- 172.21.15.16:0/3777599358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c101df0 0x7fda1c1021d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 -- 172.21.15.16:0/3777599358 >> 172.21.15.16:0/3777599358 conn(0x7fda1c0fd8f0 msgr2=0x7fda1c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 -- 172.21.15.16:0/3777599358 shutdown_connections 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 -- 172.21.15.16:0/3777599358 wait complete. 2024-09-12T00:54:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 Processor -- start 2024-09-12T00:54:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.125+0000 7fda24656640 1 -- start start 2024-09-12T00:54:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda1c101df0 0x7fda1c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 0x7fda1c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 0x7fda1c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c111f40 con 0x7fda1c1027a0 2024-09-12T00:54:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c111dc0 con 0x7fda1c106f20 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda24656640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c1120c0 con 0x7fda1c101df0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda21bf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 0x7fda1c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda21bf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 0x7fda1c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58790/0 (socket says 172.21.15.16:58790) 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.126+0000 7fda21bf1640 1 -- 172.21.15.16:0/4156788575 learned_addr learned my addr 172.21.15.16:0/4156788575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda22bf3640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 0x7fda1c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda223f2640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda1c101df0 0x7fda1c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 -- 172.21.15.16:0/4156788575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda1c101df0 msgr2=0x7fda1c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda1c101df0 0x7fda1c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 -- 172.21.15.16:0/4156788575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 msgr2=0x7fda1c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 0x7fda1c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c19d390 con 0x7fda1c1027a0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda22bf3640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda1c106f20 0x7fda1c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda21bf1640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 0x7fda1c198920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c920 tx=0x7fda1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18007c70 con 0x7fda1c1027a0 2024-09-12T00:54:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c19d680 con 0x7fda1c1027a0 2024-09-12T00:54:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda18007e10 con 0x7fda1c1027a0 2024-09-12T00:54:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18018890 con 0x7fda1c1027a0 2024-09-12T00:54:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.127+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a4f60 con 0x7fda1c1027a0 2024-09-12T00:54:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.128+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda1c062980 con 0x7fda1c1027a0 2024-09-12T00:54:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.130+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda18018a30 con 0x7fda1c1027a0 2024-09-12T00:54:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.130+0000 7fda137fe640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fda080a2c10 0x7fda080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.130+0000 7fda223f2640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fda080a2c10 0x7fda080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fda180cf500 con 0x7fda1c1027a0 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda137fe640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] conn(0x7fda080ac460 0x7fda080ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda080aef30 con 0x7fda080ac460 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fda180cf9e0 con 0x7fda1c1027a0 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda22bf3640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] conn(0x7fda080ac460 0x7fda080ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda223f2640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fda080a2c10 0x7fda080a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009230 tx=0x7fda0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.131+0000 7fda22bf3640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] conn(0x7fda080ac460 0x7fda080ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.132+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== osd.6 v2:172.21.15.16:6818/4235557256 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda080aef30 con 0x7fda080ac460 2024-09-12T00:54:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 --> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda1c062d80 con 0x7fda080ac460 2024-09-12T00:54:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda137fe640 1 -- 172.21.15.16:0/4156788575 <== osd.6 v2:172.21.15.16:6818/4235557256 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda1c062d80 con 0x7fda080ac460 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] conn(0x7fda080ac460 msgr2=0x7fda080ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] conn(0x7fda080ac460 0x7fda080ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fda080a2c10 msgr2=0x7fda080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fda080a2c10 0x7fda080a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009230 tx=0x7fda0c0058d0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 msgr2=0x7fda1c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 --2- 172.21.15.16:0/4156788575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda1c1027a0 0x7fda1c198920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c920 tx=0x7fda1800cde0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda22bf3640 1 -- 172.21.15.16:0/4156788575 reap_dead start 2024-09-12T00:54:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 shutdown_connections 2024-09-12T00:54:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.146+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 >> 172.21.15.16:0/4156788575 conn(0x7fda1c0fd8f0 msgr2=0x7fda1c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.147+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 shutdown_connections 2024-09-12T00:54:44.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.147+0000 7fda24656640 1 -- 172.21.15.16:0/4156788575 wait complete. 2024-09-12T00:54:44.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 -- 172.21.15.16:0/3714858739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074f40 msgr2=0x7f3e9810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/3714858739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074f40 0x7f3e9810de70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e8802a480 comp rx=0 tx=0).stop 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 -- 172.21.15.16:0/3714858739 shutdown_connections 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/3714858739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e9810e3b0 0x7f3e98110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/3714858739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074f40 0x7f3e9810de70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/3714858739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98074620 0x7f3e98074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 -- 172.21.15.16:0/3714858739 >> 172.21.15.16:0/3714858739 conn(0x7f3e980fd8b0 msgr2=0x7f3e980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 -- 172.21.15.16:0/3714858739 shutdown_connections 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.737+0000 7f3e9f54f640 1 -- 172.21.15.16:0/3714858739 wait complete. 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 Processor -- start 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 -- start start 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 0x7f3e9819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98074f40 0x7f3e9819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e9810e3b0 0x7f3e9819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e98114130 con 0x7f3e98074620 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e98113fb0 con 0x7f3e9810e3b0 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9f54f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e981142b0 con 0x7f3e98074f40 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9d2eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 0x7f3e9819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9d2eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 0x7f3e9819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58818/0 (socket says 172.21.15.16:58818) 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9d2eb640 1 -- 172.21.15.16:0/2366296330 learned_addr learned my addr 172.21.15.16:0/2366296330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9caea640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98074f40 0x7f3e9819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.738+0000 7f3e9daec640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e9810e3b0 0x7f3e9819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 -- 172.21.15.16:0/2366296330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98074f40 msgr2=0x7f3e9819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e98074f40 0x7f3e9819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 -- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e9810e3b0 msgr2=0x7f3e9819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e9810e3b0 0x7f3e9819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9819f640 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9d2eb640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 0x7f3e9819a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400d950 tx=0x7f3e9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94017030 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e94004020 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9819f930 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940047a0 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.739+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e981a7210 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.740+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e98062980 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.741+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3e940041d0 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e8e7fc640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3e840a2c90 0x7f3e840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f3e940cf140 con 0x7f3e98074620 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e9caea640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3e840a2c90 0x7f3e840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e8e7fc640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] conn(0x7f3e840ac4e0 0x7f3e840ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:44.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e840aefb0 con 0x7f3e840ac4e0 2024-09-12T00:54:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3e940cf620 con 0x7f3e98074620 2024-09-12T00:54:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.742+0000 7f3e9caea640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3e840a2c90 0x7f3e840a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e980fff20 tx=0x7f3e88005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.743+0000 7f3e9daec640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] conn(0x7f3e840ac4e0 0x7f3e840ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.743+0000 7f3e9daec640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] conn(0x7f3e840ac4e0 0x7f3e840ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:44.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.744+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== osd.4 v2:172.21.15.99:6808/2525858476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e840aefb0 con 0x7f3e840ac4e0 2024-09-12T00:54:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.758+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 --> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e98062d80 con 0x7f3e840ac4e0 2024-09-12T00:54:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e8e7fc640 1 -- 172.21.15.16:0/2366296330 <== osd.4 v2:172.21.15.99:6808/2525858476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3e98062d80 con 0x7f3e840ac4e0 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] conn(0x7f3e840ac4e0 msgr2=0x7f3e840ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] conn(0x7f3e840ac4e0 0x7f3e840ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3e840a2c90 msgr2=0x7f3e840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3e840a2c90 0x7f3e840a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e980fff20 tx=0x7f3e88005a20 comp rx=0 tx=0).stop 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 msgr2=0x7f3e9819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 --2- 172.21.15.16:0/2366296330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e98074620 0x7f3e9819a690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400d950 tx=0x7f3e9400de10 comp rx=0 tx=0).stop 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9daec640 1 -- 172.21.15.16:0/2366296330 reap_dead start 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 shutdown_connections 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 >> 172.21.15.16:0/2366296330 conn(0x7f3e980fd8b0 msgr2=0x7f3e980ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 shutdown_connections 2024-09-12T00:54:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:44.759+0000 7f3e9f54f640 1 -- 172.21.15.16:0/2366296330 wait complete. 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 -- 172.21.15.16:0/3603101446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 msgr2=0x7fbc10106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/3603101446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10106840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009930 tx=0x7fbc0002a420 comp rx=0 tx=0).stop 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 -- 172.21.15.16:0/3603101446 shutdown_connections 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/3603101446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 0x7fbc1010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/3603101446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10106840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/3603101446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc10100350 0x7fbc10100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 -- 172.21.15.16:0/3603101446 >> 172.21.15.16:0/3603101446 conn(0x7fbc10077210 msgr2=0x7fbc100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 -- 172.21.15.16:0/3603101446 shutdown_connections 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.291+0000 7fbc17fa8640 1 -- 172.21.15.16:0/3603101446 wait complete. 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 Processor -- start 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 -- start start 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc10100350 0x7fbc101984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 0x7fbc1019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10111f60 con 0x7fbc10100d00 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc10111de0 con 0x7fbc10100350 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc17fa8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101120e0 con 0x7fbc10106f50 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:45.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58832/0 (socket says 172.21.15.16:58832) 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 -- 172.21.15.16:0/1063024745 learned_addr learned my addr 172.21.15.16:0/1063024745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15d44640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc10100350 0x7fbc101984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc16545640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 0x7fbc1019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 -- 172.21.15.16:0/1063024745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 msgr2=0x7fbc1019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 0x7fbc1019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 -- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc10100350 msgr2=0x7fbc101984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc10100350 0x7fbc101984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc15543640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1019d470 con 0x7fbc10100d00 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.292+0000 7fbc16545640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc10106f50 0x7fbc1019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc15543640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10198a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0003f030 con 0x7fbc10100d00 2024-09-12T00:54:45.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1019d700 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00003ce0 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00035070 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.293+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101a5040 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.294+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc10062980 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.295+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbc00042030 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc06ffd640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbbfc0a2c10 0x7fbbfc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc15d44640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbbfc0a2c10 0x7fbbfc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fbc000efc00 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc06ffd640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] conn(0x7fbbfc0ac460 0x7fbbfc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc0003ce50 con 0x7fbbfc0ac460 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fbc000f00e0 con 0x7fbc10100d00 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc15d44640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbbfc0a2c10 0x7fbbfc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0058c0 tx=0x7fbc0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.296+0000 7fbc16545640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] conn(0x7fbbfc0ac460 0x7fbbfc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.297+0000 7fbc16545640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] conn(0x7fbbfc0ac460 0x7fbbfc0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.298+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== osd.10 v2:172.21.15.99:6824/1469352212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc0003ce50 con 0x7fbbfc0ac460 2024-09-12T00:54:45.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.311+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 --> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbc10062e00 con 0x7fbbfc0ac460 2024-09-12T00:54:45.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc06ffd640 1 -- 172.21.15.16:0/1063024745 <== osd.10 v2:172.21.15.99:6824/1469352212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbc10062e00 con 0x7fbbfc0ac460 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] conn(0x7fbbfc0ac460 msgr2=0x7fbbfc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] conn(0x7fbbfc0ac460 0x7fbbfc0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbbfc0a2c10 msgr2=0x7fbbfc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbbfc0a2c10 0x7fbbfc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0058c0 tx=0x7fbc0c00a2d0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 msgr2=0x7fbc10198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 --2- 172.21.15.16:0/1063024745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc10100d00 0x7fbc10198a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005940 tx=0x7fbc000042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc16545640 1 -- 172.21.15.16:0/1063024745 reap_dead start 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 shutdown_connections 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 >> 172.21.15.16:0/1063024745 conn(0x7fbc10077210 msgr2=0x7fbc10105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 shutdown_connections 2024-09-12T00:54:45.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:45.312+0000 7fbc17fa8640 1 -- 172.21.15.16:0/1063024745 wait complete. 2024-09-12T00:54:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:45 smithi099 ceph-mon[35654]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T00:54:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:45 smithi159 ceph-mon[35566]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T00:54:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:45 smithi016 ceph-mon[30972]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T00:54:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 -- 172.21.15.16:0/1976408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 msgr2=0x7fb4301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 --2- 172.21.15.16:0/1976408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301067e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb424009900 tx=0x7fb42402a410 comp rx=0 tx=0).stop 2024-09-12T00:54:46.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 -- 172.21.15.16:0/1976408536 shutdown_connections 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 --2- 172.21.15.16:0/1976408536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 0x7fb43010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 --2- 172.21.15.16:0/1976408536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301067e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 --2- 172.21.15.16:0/1976408536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb430101de0 0x7fb4301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 -- 172.21.15.16:0/1976408536 >> 172.21.15.16:0/1976408536 conn(0x7fb4300fd8c0 msgr2=0x7fb4300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 -- 172.21.15.16:0/1976408536 shutdown_connections 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.500+0000 7fb435ff6640 1 -- 172.21.15.16:0/1976408536 wait complete. 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 Processor -- start 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 -- start start 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb430101de0 0x7fb430198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 0x7fb43019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111f90 con 0x7fb430102790 2024-09-12T00:54:46.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111e10 con 0x7fb430106f10 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.501+0000 7fb435ff6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb430112110 con 0x7fb430101de0 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58848/0 (socket says 172.21.15.16:58848) 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 -- 172.21.15.16:0/262171854 learned_addr learned my addr 172.21.15.16:0/262171854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42f7fe640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb430101de0 0x7fb430198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42ffff640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 0x7fb43019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 -- 172.21.15.16:0/262171854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb430101de0 msgr2=0x7fb430198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb430101de0 0x7fb430198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 -- 172.21.15.16:0/262171854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 msgr2=0x7fb43019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 0x7fb43019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43019d440 con 0x7fb430102790 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42ffff640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb430106f10 0x7fb43019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:54:46.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42effd640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301989d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb4240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42403f030 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43019d6d0 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb424003ce0 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424035070 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.502+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a5010 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.503+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb430062980 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.504+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb424042030 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42cff9640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb4180a2c10 0x7fb4180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42f7fe640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb4180a2c10 0x7fb4180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fb4240efcb0 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42cff9640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] conn(0x7fb4180ac460 0x7fb4180ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb424001d30 con 0x7fb4180ac460 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.505+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fb4240f0060 con 0x7fb430102790 2024-09-12T00:54:46.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.506+0000 7fb42ffff640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] conn(0x7fb4180ac460 0x7fb4180ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.506+0000 7fb42f7fe640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb4180a2c10 0x7fb4180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb4200058c0 tx=0x7fb42000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.506+0000 7fb42ffff640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] conn(0x7fb4180ac460 0x7fb4180ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.507+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== osd.3 v2:172.21.15.16:6810/846431474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb424001d30 con 0x7fb4180ac460 2024-09-12T00:54:46.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.520+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 --> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb430062e00 con 0x7fb4180ac460 2024-09-12T00:54:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb42cff9640 1 -- 172.21.15.16:0/262171854 <== osd.3 v2:172.21.15.16:6810/846431474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb430062e00 con 0x7fb4180ac460 2024-09-12T00:54:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] conn(0x7fb4180ac460 msgr2=0x7fb4180ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] conn(0x7fb4180ac460 0x7fb4180ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb4180a2c10 msgr2=0x7fb4180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb4180a2c10 0x7fb4180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb4200058c0 tx=0x7fb42000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 msgr2=0x7fb4301989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 --2- 172.21.15.16:0/262171854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb430102790 0x7fb4301989d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb424005ab0 tx=0x7fb4240042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb42ffff640 1 -- 172.21.15.16:0/262171854 reap_dead start 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 shutdown_connections 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 >> 172.21.15.16:0/262171854 conn(0x7fb4300fd8c0 msgr2=0x7fb4300fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 shutdown_connections 2024-09-12T00:54:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:46.521+0000 7fb435ff6640 1 -- 172.21.15.16:0/262171854 wait complete. 2024-09-12T00:54:47.106 INFO:teuthology.orchestra.run.smithi016.stdout:137438953528 2024-09-12T00:54:47.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T00:54:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:47 smithi016 ceph-mon[30972]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T00:54:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/302689032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4102570 msgr2=0x7fa6b41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/302689032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4102570 0x7fa6b41029b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009900 tx=0x7fa69c02a480 comp rx=0 tx=0).stop 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/302689032 shutdown_connections 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/302689032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4102ef0 0x7fa6b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/302689032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4102570 0x7fa6b41029b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/302689032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4100e20 0x7fa6b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/302689032 >> 172.21.15.16:0/302689032 conn(0x7fa6b40fc7d0 msgr2=0x7fa6b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/302689032 shutdown_connections 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/302689032 wait complete. 2024-09-12T00:54:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 Processor -- start 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.288+0000 7fa6ba2eb640 1 -- start start 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 0x7fa6b4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4102570 0x7fa6b4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4102ef0 0x7fa6b4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4110f80 con 0x7fa6b4100e20 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4110e00 con 0x7fa6b4102ef0 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6ba2eb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4111100 con 0x7fa6b4102570 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 0x7fa6b4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 0x7fa6b4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58854/0 (socket says 172.21.15.16:58854) 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 -- 172.21.15.16:0/661573608 learned_addr learned my addr 172.21.15.16:0/661573608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b8888640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4102ef0 0x7fa6b4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6abfff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4102570 0x7fa6b4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 -- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4102570 msgr2=0x7fa6b4197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4102570 0x7fa6b4197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 -- 172.21.15.16:0/661573608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4102ef0 msgr2=0x7fa6b4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4102ef0 0x7fa6b4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b41a68b0 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b3fff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 0x7fa6b4197040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c920 tx=0x7fa6a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.289+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4007c70 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.290+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4007e10 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.290+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b41a6ba0 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.290+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4018890 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.290+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a70b0 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.291+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6b41071f0 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.292+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa6a4018a30 con 0x7fa6b4100e20 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.292+0000 7fa6b1ffb640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa6a00a2c10 0x7fa6a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6abfff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa6a00a2c10 0x7fa6a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fa6a40cf4e0 con 0x7fa6b4100e20 2024-09-12T00:54:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6b1ffb640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] conn(0x7fa6a00ac460 0x7fa6a00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6a00aef30 con 0x7fa6a00ac460 2024-09-12T00:54:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa6a40cf9c0 con 0x7fa6b4100e20 2024-09-12T00:54:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6abfff640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa6a00a2c10 0x7fa6a00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000c00 tx=0x7fa69c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.293+0000 7fa6b8888640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] conn(0x7fa6a00ac460 0x7fa6a00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.294+0000 7fa6b8888640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] conn(0x7fa6a00ac460 0x7fa6a00ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.295+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== osd.5 v2:172.21.15.159:6808/516420178 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6a00aef30 con 0x7fa6a00ac460 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.309+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 --> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa6b41021b0 con 0x7fa6a00ac460 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6b1ffb640 1 -- 172.21.15.16:0/661573608 <== osd.5 v2:172.21.15.159:6808/516420178 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa6b41021b0 con 0x7fa6a00ac460 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] conn(0x7fa6a00ac460 msgr2=0x7fa6a00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] conn(0x7fa6a00ac460 0x7fa6a00ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa6a00a2c10 msgr2=0x7fa6a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa6a00a2c10 0x7fa6a00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa69c000c00 tx=0x7fa69c0058d0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 msgr2=0x7fa6b4197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 --2- 172.21.15.16:0/661573608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b4100e20 0x7fa6b4197040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c920 tx=0x7fa6a400cde0 comp rx=0 tx=0).stop 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6b8888640 1 -- 172.21.15.16:0/661573608 reap_dead start 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 shutdown_connections 2024-09-12T00:54:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 >> 172.21.15.16:0/661573608 conn(0x7fa6b40fc7d0 msgr2=0x7fa6b41067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:47.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 shutdown_connections 2024-09-12T00:54:47.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:47.310+0000 7fa6ba2eb640 1 -- 172.21.15.16:0/661573608 wait complete. 2024-09-12T00:54:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:47 smithi099 ceph-mon[35654]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T00:54:47.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:47 smithi159 ceph-mon[35566]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T00:54:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:188978561071 2024-09-12T00:54:48.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T00:54:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 -- 172.21.15.16:0/1254398473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878102570 msgr2=0x7f18781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 --2- 172.21.15.16:0/1254398473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878102570 0x7f18781029b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f186c009900 tx=0x7f186c02a480 comp rx=0 tx=0).stop 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 -- 172.21.15.16:0/1254398473 shutdown_connections 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 --2- 172.21.15.16:0/1254398473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102ef0 0x7f187810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 --2- 172.21.15.16:0/1254398473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878102570 0x7f18781029b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 --2- 172.21.15.16:0/1254398473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1878100e20 0x7f1878101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 -- 172.21.15.16:0/1254398473 >> 172.21.15.16:0/1254398473 conn(0x7f18780fc7d0 msgr2=0x7f18780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 -- 172.21.15.16:0/1254398473 shutdown_connections 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.870+0000 7f187e704640 1 -- 172.21.15.16:0/1254398473 wait complete. 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 Processor -- start 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 -- start start 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 0x7f18781992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102570 0x7f1878199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1878102ef0 0x7f18781a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1878110f50 con 0x7f1878100e20 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1878110dd0 con 0x7f1878102570 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f187e704640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18781110d0 con 0x7f1878102ef0 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f1877fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 0x7f18781992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f1877fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 0x7f18781992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52458/0 (socket says 172.21.15.16:52458) 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f1877fff640 1 -- 172.21.15.16:0/3953328513 learned_addr learned my addr 172.21.15.16:0/3953328513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.871+0000 7f18777fe640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102570 0x7f1878199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f187cca1640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1878102ef0 0x7f18781a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 -- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1878102ef0 msgr2=0x7f18781a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1878102ef0 0x7f18781a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 -- 172.21.15.16:0/3953328513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102570 msgr2=0x7f1878199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102570 0x7f1878199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18781a4880 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f18777fe640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878102570 0x7f1878199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f1877fff640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 0x7f18781992c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868007c70 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1868007e10 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781a4a50 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868018890 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.872+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a4f90 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.873+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1878107220 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.874+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1868018a30 con 0x7f1878100e20 2024-09-12T00:54:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18757fa640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f18600a2c90 0x7f18600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18777fe640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f18600a2c90 0x7f18600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f18680cf5b0 con 0x7f1878100e20 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18757fa640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] conn(0x7f18600ac4e0 0x7f18600ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18600aefb0 con 0x7f18600ac4e0 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.875+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f18680cfa90 con 0x7f1878100e20 2024-09-12T00:54:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.876+0000 7f18777fe640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f18600a2c90 0x7f18600a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f187806aac0 tx=0x7f186c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.876+0000 7f187cca1640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] conn(0x7f18600ac4e0 0x7f18600ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.876+0000 7f187cca1640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] conn(0x7f18600ac4e0 0x7f18600ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.877+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== osd.2 v2:172.21.15.159:6800/3066904609 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18600aefb0 con 0x7f18600ac4e0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.891+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 --> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18781021b0 con 0x7f18600ac4e0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.891+0000 7f18757fa640 1 -- 172.21.15.16:0/3953328513 <== osd.2 v2:172.21.15.159:6800/3066904609 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18781021b0 con 0x7f18600ac4e0 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.891+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] conn(0x7f18600ac4e0 msgr2=0x7f18600ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.891+0000 7f187e704640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] conn(0x7f18600ac4e0 0x7f18600ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f18600a2c90 msgr2=0x7f18600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f18600a2c90 0x7f18600a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f187806aac0 tx=0x7f186c005a20 comp rx=0 tx=0).stop 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 msgr2=0x7f18781992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 --2- 172.21.15.16:0/3953328513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1878100e20 0x7f18781992c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f186800c920 tx=0x7f186800cde0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187cca1640 1 -- 172.21.15.16:0/3953328513 reap_dead start 2024-09-12T00:54:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 shutdown_connections 2024-09-12T00:54:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 >> 172.21.15.16:0/3953328513 conn(0x7f18780fc7d0 msgr2=0x7f18780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 shutdown_connections 2024-09-12T00:54:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.892+0000 7f187e704640 1 -- 172.21.15.16:0/3953328513 wait complete. 2024-09-12T00:54:48.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 -- 172.21.15.16:0/4229528847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc10c170 msgr2=0x7fe5bc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/4229528847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc10c170 0x7fe5bc10e610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009ca0 tx=0x7fe5b402a950 comp rx=0 tx=0).stop 2024-09-12T00:54:48.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 -- 172.21.15.16:0/4229528847 shutdown_connections 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/4229528847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc10c170 0x7fe5bc10e610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/4229528847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5bc103660 0x7fe5bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/4229528847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5bc102d40 0x7fe5bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 -- 172.21.15.16:0/4229528847 >> 172.21.15.16:0/4229528847 conn(0x7fe5bc0fc7a0 msgr2=0x7fe5bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 -- 172.21.15.16:0/4229528847 shutdown_connections 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.157+0000 7fe5c1a38640 1 -- 172.21.15.16:0/4229528847 wait complete. 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 Processor -- start 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 -- start start 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 0x7fe5bc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5bc103660 0x7fe5bc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5bc10c170 0x7fe5bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111f70 con 0x7fe5bc102d40 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc111df0 con 0x7fe5bc10c170 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5c1a38640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1120f0 con 0x7fe5bc103660 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 0x7fe5bc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 0x7fe5bc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58866/0 (socket says 172.21.15.16:58866) 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5baffd640 1 -- 172.21.15.16:0/2417593754 learned_addr learned my addr 172.21.15.16:0/2417593754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5bb7fe640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5bc10c170 0x7fe5bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.158+0000 7fe5ba7fc640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5bc103660 0x7fe5bc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 -- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5bc103660 msgr2=0x7fe5bc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5bc103660 0x7fe5bc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 -- 172.21.15.16:0/2417593754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5bc10c170 msgr2=0x7fe5bc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5bc10c170 0x7fe5bc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc19d3a0 con 0x7fe5bc102d40 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5baffd640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 0x7fe5bc1983f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c970 tx=0x7fe5ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac007bb0 con 0x7fe5bc102d40 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc19d690 con 0x7fe5bc102d40 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5ac007d50 con 0x7fe5bc102d40 2024-09-12T00:54:48.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00f450 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.159+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a4f70 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.160+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5bc0628c0 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.162+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe5ac004fa0 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5c0a36640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe5a40a2c10 0x7fe5a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5ba7fc640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe5a40a2c10 0x7fe5a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fe5ac0cf200 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5c0a36640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] conn(0x7fe5a40ac460 0x7fe5a40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5a40aef30 con 0x7fe5a40ac460 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.163+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe5ac0cf5b0 con 0x7fe5bc102d40 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.164+0000 7fe5ba7fc640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe5a40a2c10 0x7fe5a40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc10d4b0 tx=0x7fe5b000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.164+0000 7fe5bb7fe640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] conn(0x7fe5a40ac460 0x7fe5a40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:48.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.164+0000 7fe5bb7fe640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] conn(0x7fe5a40ac460 0x7fe5a40ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.165+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== osd.11 v2:172.21.15.159:6824/4132934366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5a40aef30 con 0x7fe5a40ac460 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.179+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 --> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5bc062d40 con 0x7fe5a40ac460 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.179+0000 7fe5c0a36640 1 -- 172.21.15.16:0/2417593754 <== osd.11 v2:172.21.15.159:6824/4132934366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe5bc062d40 con 0x7fe5a40ac460 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] conn(0x7fe5a40ac460 msgr2=0x7fe5a40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] conn(0x7fe5a40ac460 0x7fe5a40ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe5a40a2c10 msgr2=0x7fe5a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe5a40a2c10 0x7fe5a40a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc10d4b0 tx=0x7fe5b000a250 comp rx=0 tx=0).stop 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 msgr2=0x7fe5bc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 --2- 172.21.15.16:0/2417593754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5bc102d40 0x7fe5bc1983f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00c970 tx=0x7fe5ac00ce30 comp rx=0 tx=0).stop 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5bb7fe640 1 -- 172.21.15.16:0/2417593754 reap_dead start 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 shutdown_connections 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 >> 172.21.15.16:0/2417593754 conn(0x7fe5bc0fc7a0 msgr2=0x7fe5bc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 shutdown_connections 2024-09-12T00:54:49.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:48.180+0000 7fe5c1a38640 1 -- 172.21.15.16:0/2417593754 wait complete. 2024-09-12T00:54:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:49 smithi099 ceph-mon[35654]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T00:54:49.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:49 smithi159 ceph-mon[35566]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T00:54:49.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:49 smithi016 ceph-mon[30972]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T00:54:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:326417514519 2024-09-12T00:54:49.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T00:54:50.530 INFO:teuthology.orchestra.run.smithi016.stdout:107374182461 2024-09-12T00:54:50.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 -- 172.21.15.16:0/887694225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940102570 msgr2=0x7f59401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 --2- 172.21.15.16:0/887694225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940102570 0x7f59401029b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5934009900 tx=0x7f593402a480 comp rx=0 tx=0).stop 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 -- 172.21.15.16:0/887694225 shutdown_connections 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 --2- 172.21.15.16:0/887694225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940102ef0 0x7f594010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 --2- 172.21.15.16:0/887694225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940102570 0x7f59401029b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 --2- 172.21.15.16:0/887694225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5940100e20 0x7f5940101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 -- 172.21.15.16:0/887694225 >> 172.21.15.16:0/887694225 conn(0x7f59400fc7d0 msgr2=0x7f59400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 -- 172.21.15.16:0/887694225 shutdown_connections 2024-09-12T00:54:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.863+0000 7f59487a9640 1 -- 172.21.15.16:0/887694225 wait complete. 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 Processor -- start 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 -- start start 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 0x7f59401973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5940102570 0x7f5940197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940102ef0 0x7f594019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5940110f80 con 0x7f5940100e20 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5940110e00 con 0x7f5940102570 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f59487a9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5940111100 con 0x7f5940102ef0 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5946545640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 0x7f59401973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5946545640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 0x7f59401973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52466/0 (socket says 172.21.15.16:52466) 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5946545640 1 -- 172.21.15.16:0/196167103 learned_addr learned my addr 172.21.15.16:0/196167103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:50.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5946d46640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940102ef0 0x7f594019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5945d44640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5940102570 0x7f5940197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.864+0000 7f5946545640 1 -- 172.21.15.16:0/196167103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940102ef0 msgr2=0x7f594019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f5946545640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5940102ef0 0x7f594019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f5946545640 1 -- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5940102570 msgr2=0x7f5940197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f5946545640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5940102570 0x7f5940197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f5946545640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594019c390 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f5946545640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 0x7f59401973e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c890 tx=0x7f593c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c004220 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f593c0043c0 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594019c680 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c0049e0 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.865+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a3f60 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.866+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5940062940 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.867+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f593c0079b0 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.868+0000 7f59337fe640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f592c0a2c90 0x7f592c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.868+0000 7f5945d44640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f592c0a2c90 0x7f592c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f593c0cf2e0 con 0x7f5940100e20 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f59337fe640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] conn(0x7f592c0ac4e0 0x7f592c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f592c0aefb0 con 0x7f592c0ac4e0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f593c0cf690 con 0x7f5940100e20 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f5945d44640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f592c0a2c90 0x7f592c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59400fec10 tx=0x7f5934005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f5946d46640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] conn(0x7f592c0ac4e0 0x7f592c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.869+0000 7f5946d46640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] conn(0x7f592c0ac4e0 0x7f592c0ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.870+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== osd.1 v2:172.21.15.99:6800/524186263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f592c0aefb0 con 0x7f592c0ac4e0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.885+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 --> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5940062d40 con 0x7f592c0ac4e0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59337fe640 1 -- 172.21.15.16:0/196167103 <== osd.1 v2:172.21.15.99:6800/524186263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5940062d40 con 0x7f592c0ac4e0 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] conn(0x7f592c0ac4e0 msgr2=0x7f592c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] conn(0x7f592c0ac4e0 0x7f592c0ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f592c0a2c90 msgr2=0x7f592c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f592c0a2c90 0x7f592c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59400fec10 tx=0x7f5934005a20 comp rx=0 tx=0).stop 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 msgr2=0x7f59401973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 --2- 172.21.15.16:0/196167103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5940100e20 0x7f59401973e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c890 tx=0x7f593c00cd50 comp rx=0 tx=0).stop 2024-09-12T00:54:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f5946d46640 1 -- 172.21.15.16:0/196167103 reap_dead start 2024-09-12T00:54:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 shutdown_connections 2024-09-12T00:54:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 >> 172.21.15.16:0/196167103 conn(0x7f59400fc7d0 msgr2=0x7f59401068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 shutdown_connections 2024-09-12T00:54:50.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:49.886+0000 7f59487a9640 1 -- 172.21.15.16:0/196167103 wait complete. 2024-09-12T00:54:51.098 INFO:teuthology.orchestra.run.smithi016.stdout:292057776154 2024-09-12T00:54:51.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T00:54:51.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:51 smithi016 ceph-mon[30972]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:51 smithi099 ceph-mon[35654]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:51 smithi159 ceph-mon[35566]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:51.694 INFO:teuthology.orchestra.run.smithi016.stdout:158913790005 2024-09-12T00:54:51.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T00:54:52.529 INFO:teuthology.orchestra.run.smithi016.stdout:55834574922 2024-09-12T00:54:52.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T00:54:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 -- 172.21.15.16:0/2042239107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba81036c0 msgr2=0x7fbba810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 --2- 172.21.15.16:0/2042239107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba81036c0 0x7fbba810bbb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009900 tx=0x7fbb9802a480 comp rx=0 tx=0).stop 2024-09-12T00:54:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 -- 172.21.15.16:0/2042239107 shutdown_connections 2024-09-12T00:54:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 --2- 172.21.15.16:0/2042239107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbba810c180 0x7fbba810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 --2- 172.21.15.16:0/2042239107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba81036c0 0x7fbba810bbb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 --2- 172.21.15.16:0/2042239107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbba8102da0 0x7fbba8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 -- 172.21.15.16:0/2042239107 >> 172.21.15.16:0/2042239107 conn(0x7fbba80fc7a0 msgr2=0x7fbba80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 -- 172.21.15.16:0/2042239107 shutdown_connections 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.763+0000 7fbbb0339640 1 -- 172.21.15.16:0/2042239107 wait complete. 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 Processor -- start 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 -- start start 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbba8102da0 0x7fbba8100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbba81036c0 0x7fbba8100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 0x7fbba8101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8111f60 con 0x7fbba810c180 2024-09-12T00:54:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8111de0 con 0x7fbba8102da0 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbb0339640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81120e0 con 0x7fbba81036c0 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbae8d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 0x7fbba8101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbae8d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 0x7fbba8101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52484/0 (socket says 172.21.15.16:52484) 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.764+0000 7fbbae8d6640 1 -- 172.21.15.16:0/1858770435 learned_addr learned my addr 172.21.15.16:0/1858770435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbad8d4640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbba81036c0 0x7fbba8100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae0d5640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbba8102da0 0x7fbba8100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 -- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbba81036c0 msgr2=0x7fbba8100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbba81036c0 0x7fbba8100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 -- 172.21.15.16:0/1858770435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbba8102da0 msgr2=0x7fbba8100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbba8102da0 0x7fbba8100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81a6940 con 0x7fbba810c180 2024-09-12T00:54:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbae8d6640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 0x7fbba8101480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbba000eb10 tx=0x7fbba000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004b20 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81a6c30 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba0004cc0 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0010760 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.765+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a7140 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.766+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbba80628c0 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.767+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbba0005020 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbb9f7fe640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbb940a2c10 0x7fbb940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fbba00cfd20 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbbae0d5640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbb940a2c10 0x7fbb940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbb9f7fe640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] conn(0x7fbb940ac460 0x7fbb940ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb940aef30 con 0x7fbb940ac460 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.768+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fbba00d00d0 con 0x7fbba810c180 2024-09-12T00:54:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.769+0000 7fbbae0d5640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbb940a2c10 0x7fbb940a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009bd0 tx=0x7fbba4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.769+0000 7fbbad8d4640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] conn(0x7fbb940ac460 0x7fbb940ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.769+0000 7fbbad8d4640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] conn(0x7fbb940ac460 0x7fbb940ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.770+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== osd.8 v2:172.21.15.159:6816/274647051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb940aef30 con 0x7fbb940ac460 2024-09-12T00:54:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.784+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 --> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbba8062cc0 con 0x7fbb940ac460 2024-09-12T00:54:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.784+0000 7fbb9f7fe640 1 -- 172.21.15.16:0/1858770435 <== osd.8 v2:172.21.15.159:6816/274647051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbba8062cc0 con 0x7fbb940ac460 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] conn(0x7fbb940ac460 msgr2=0x7fbb940ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] conn(0x7fbb940ac460 0x7fbb940ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbb940a2c10 msgr2=0x7fbb940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbb940a2c10 0x7fbb940a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009bd0 tx=0x7fbba4005f90 comp rx=0 tx=0).stop 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 msgr2=0x7fbba8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 --2- 172.21.15.16:0/1858770435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbba810c180 0x7fbba8101480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbba000eb10 tx=0x7fbba000c2d0 comp rx=0 tx=0).stop 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbae8d6640 1 -- 172.21.15.16:0/1858770435 reap_dead start 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 shutdown_connections 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 >> 172.21.15.16:0/1858770435 conn(0x7fbba80fc7a0 msgr2=0x7fbba80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 shutdown_connections 2024-09-12T00:54:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:52.785+0000 7fbbb0339640 1 -- 172.21.15.16:0/1858770435 wait complete. 2024-09-12T00:54:53.340 INFO:teuthology.orchestra.run.smithi016.stdout:34359738445 2024-09-12T00:54:53.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T00:54:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:53 smithi099 ceph-mon[35654]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:53 smithi159 ceph-mon[35566]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:53 smithi016 ceph-mon[30972]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:54:54.235 INFO:teuthology.orchestra.run.smithi016.stdout:77309411396 2024-09-12T00:54:54.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T00:54:54.697 INFO:teuthology.orchestra.run.smithi016.stdout:240518168613 2024-09-12T00:54:54.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T00:54:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.183 INFO:teuthology.orchestra.run.smithi016.stdout:270582939679 2024-09-12T00:54:55.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T00:54:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.244 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.252 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:55.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:55 smithi016 ceph-mon[30972]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:54:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:55 smithi099 ceph-mon[35654]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:54:55.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:55 smithi159 ceph-mon[35566]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:54:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:56.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:56 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1312099757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 -- 172.21.15.16:0/4123307350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102ef0 msgr2=0x7f323010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 --2- 172.21.15.16:0/4123307350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102ef0 0x7f323010da40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f322802a950 comp rx=0 tx=0).stop 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 -- 172.21.15.16:0/4123307350 shutdown_connections 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 --2- 172.21.15.16:0/4123307350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102ef0 0x7f323010da40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 --2- 172.21.15.16:0/4123307350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230102570 0x7f32301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 --2- 172.21.15.16:0/4123307350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230100e20 0x7f3230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 -- 172.21.15.16:0/4123307350 >> 172.21.15.16:0/4123307350 conn(0x7f32300fc7d0 msgr2=0x7f32300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 -- 172.21.15.16:0/4123307350 shutdown_connections 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.544+0000 7f3235ca1640 1 -- 172.21.15.16:0/4123307350 wait complete. 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.545+0000 7f3235ca1640 1 Processor -- start 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.545+0000 7f3235ca1640 1 -- start start 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230100e20 0x7f32301972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230102ef0 0x7f323019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3230110fd0 con 0x7f3230102570 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3230110e50 con 0x7f3230100e20 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3235ca1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3230111150 con 0x7f3230102ef0 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52526/0 (socket says 172.21.15.16:52526) 2024-09-12T00:54:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 -- 172.21.15.16:0/894773509 learned_addr learned my addr 172.21.15.16:0/894773509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f322f7fe640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230100e20 0x7f32301972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f322ffff640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230102ef0 0x7f323019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 -- 172.21.15.16:0/894773509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230102ef0 msgr2=0x7f323019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230102ef0 0x7f323019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 -- 172.21.15.16:0/894773509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230100e20 msgr2=0x7f32301972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230100e20 0x7f32301972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 -- 172.21.15.16:0/894773509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019c2a0 con 0x7f3230102570 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.546+0000 7f3226ffd640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f321800d950 tx=0x7f321800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.547+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218017070 con 0x7f3230102570 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.547+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323019c590 con 0x7f3230102570 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.547+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321800de80 con 0x7f3230102570 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.547+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321800f620 con 0x7f3230102570 2024-09-12T00:54:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.547+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a3e70 con 0x7f3230102570 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.548+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3230101200 con 0x7f3230102570 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.550+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3218009c30 con 0x7f3230102570 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.550+0000 7f322d7fa640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 0x7f321c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.551+0000 7f322f7fe640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 0x7f321c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.551+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f32180ced20 con 0x7f3230102570 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.551+0000 7f322f7fe640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 0x7f321c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f322000abe0 tx=0x7f322000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:56.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.555+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f321809bbd0 con 0x7f3230102570 2024-09-12T00:54:56.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:56 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1312099757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T00:54:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- 172.21.15.16:0/2970816764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 --2- 172.21.15.16:0/2970816764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009ca0 tx=0x7f8fc802a950 comp rx=0 tx=0).stop 2024-09-12T00:54:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- 172.21.15.16:0/2970816764 shutdown_connections 2024-09-12T00:54:56.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 --2- 172.21.15.16:0/2970816764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 --2- 172.21.15.16:0/2970816764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102570 0x7f8fd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 --2- 172.21.15.16:0/2970816764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- 172.21.15.16:0/2970816764 >> 172.21.15.16:0/2970816764 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- 172.21.15.16:0/2970816764 shutdown_connections 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- 172.21.15.16:0/2970816764 wait complete. 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 Processor -- start 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.022+0000 7f8fd638a640 1 -- start start 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110f80 con 0x7f8fd0102570 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110e00 con 0x7f8fd0100e20 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd638a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0111100 con 0x7f8fd0102ef0 2024-09-12T00:54:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52494/0 (socket says 172.21.15.16:52494) 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 -- 172.21.15.16:0/1312099757 learned_addr learned my addr 172.21.15.16:0/1312099757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcffff640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fd4927640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 -- 172.21.15.16:0/1312099757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.663 INFO:teuthology.orchestra.run.smithi016.stdout:326417514520 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 -- 172.21.15.16:0/1312099757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 msgr2=0x7f8fd01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcf7fe640 1 -- 172.21.15.16:0/1312099757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019d380 con 0x7f8fd0102570 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.023+0000 7f8fcffff640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fcf7fe640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c920 tx=0x7f8fc400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4007c70 con 0x7f8fd0102570 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019d670 con 0x7f8fd0102570 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4007e10 con 0x7f8fd0102570 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4018890 con 0x7f8fd0102570 2024-09-12T00:54:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.024+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a4f50 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.025+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd0101200 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.026+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8fc4018a70 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.026+0000 7f8fcd7fa640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 0x7f8fb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.027+0000 7f8fcffff640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 0x7f8fb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.027+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f8fc40cf0c0 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.027+0000 7f8fcffff640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 0x7f8fb80a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009fc0 tx=0x7f8fc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.031+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fc4014030 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.215+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f8fd010a0a0 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.216+0000 7f8fcd7fa640 1 -- 172.21.15.16:0/1312099757 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8fc409c070 con 0x7f8fd0102570 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 msgr2=0x7f8fb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 0x7f8fb80a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009fc0 tx=0x7f8fc0005f90 comp rx=0 tx=0).stop 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 msgr2=0x7f8fd0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c920 tx=0x7f8fc400cde0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 shutdown_connections 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f8fb80a2c10 0x7f8fb80a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fd0102570 0x7f8fd0198910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 --2- 172.21.15.16:0/1312099757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.219+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 >> 172.21.15.16:0/1312099757 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.220+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 shutdown_connections 2024-09-12T00:54:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.220+0000 7f8fd638a640 1 -- 172.21.15.16:0/1312099757 wait complete. 2024-09-12T00:54:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.736+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f32301a4040 con 0x7f3230102570 2024-09-12T00:54:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.736+0000 7f322d7fa640 1 -- 172.21.15.16:0/894773509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f321809b4d0 con 0x7f3230102570 2024-09-12T00:54:56.736 INFO:teuthology.orchestra.run.smithi016.stdout:292057776156 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 msgr2=0x7f321c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 0x7f321c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f322000abe0 tx=0x7f322000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 msgr2=0x7f3230197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f321800d950 tx=0x7f321800dd50 comp rx=0 tx=0).stop 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 shutdown_connections 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f321c0a2c10 0x7f321c0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230102ef0 0x7f323019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3230102570 0x7f3230197830 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 --2- 172.21.15.16:0/894773509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3230100e20 0x7f32301972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.738+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 >> 172.21.15.16:0/894773509 conn(0x7f32300fc7d0 msgr2=0x7f32301067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.739+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 shutdown_connections 2024-09-12T00:54:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.739+0000 7f3235ca1640 1 -- 172.21.15.16:0/894773509 wait complete. 2024-09-12T00:54:56.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:56 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1312099757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T00:54:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- 172.21.15.16:0/2707771114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268102790 msgr2=0x7f02681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 --2- 172.21.15.16:0/2707771114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268102790 0x7f02681067e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0258009900 tx=0x7f025802a410 comp rx=0 tx=0).stop 2024-09-12T00:54:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- 172.21.15.16:0/2707771114 shutdown_connections 2024-09-12T00:54:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 --2- 172.21.15.16:0/2707771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268106f10 0x7f026810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 --2- 172.21.15.16:0/2707771114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268102790 0x7f02681067e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 --2- 172.21.15.16:0/2707771114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268101de0 0x7f02681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- 172.21.15.16:0/2707771114 >> 172.21.15.16:0/2707771114 conn(0x7f02680fd8c0 msgr2=0x7f02680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- 172.21.15.16:0/2707771114 shutdown_connections 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- 172.21.15.16:0/2707771114 wait complete. 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 Processor -- start 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.343+0000 7f026fa7c640 1 -- start start 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268101de0 0x7f0268198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268102790 0x7f0268198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0268111f90 con 0x7f0268106f10 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0268111e10 con 0x7f0268101de0 2024-09-12T00:54:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026fa7c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0268112110 con 0x7f0268102790 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52552/0 (socket says 172.21.15.16:52552) 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 -- 172.21.15.16:0/1482395551 learned_addr learned my addr 172.21.15.16:0/1482395551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026d818640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268101de0 0x7f0268198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026d017640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268102790 0x7f0268198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 -- 172.21.15.16:0/1482395551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268102790 msgr2=0x7f0268198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268102790 0x7f0268198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 -- 172.21.15.16:0/1482395551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268101de0 msgr2=0x7f0268198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268101de0 0x7f0268198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 -- 172.21.15.16:0/1482395551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026819d500 con 0x7f0268106f10 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f026e019640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f026000cad0 tx=0x7f026000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000ad60 con 0x7f0268106f10 2024-09-12T00:54:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.344+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0260005020 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.345+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000f6b0 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.345+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026819d7f0 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.345+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02681a50d0 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.345+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02681021c0 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.347+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0260005560 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.348+0000 7f025effd640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 0x7f02540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.348+0000 7f026d818640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 0x7f02540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.348+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f02600cf490 con 0x7f0268106f10 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.348+0000 7f026d818640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 0x7f02540a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0264009990 tx=0x7f0264009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:57.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.351+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0260014030 con 0x7f0268106f10 2024-09-12T00:54:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.529+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0268109b60 con 0x7f0268106f10 2024-09-12T00:54:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.529+0000 7f025effd640 1 -- 172.21.15.16:0/1482395551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f026009c440 con 0x7f0268106f10 2024-09-12T00:54:57.529 INFO:teuthology.orchestra.run.smithi016.stdout:34359738448 2024-09-12T00:54:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.531+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 msgr2=0x7f02540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.531+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 0x7f02540a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0264009990 tx=0x7f0264009110 comp rx=0 tx=0).stop 2024-09-12T00:54:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.531+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 msgr2=0x7f026819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.531+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f026000cad0 tx=0x7f026000ced0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:57 smithi016 ceph-mon[30972]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T00:54:57.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:57 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1621852125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T00:54:57.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:57 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/894773509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 shutdown_connections 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f02540a2c10 0x7f02540a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268106f10 0x7f026819ce20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0268102790 0x7f0268198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 --2- 172.21.15.16:0/1482395551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0268101de0 0x7f0268198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 >> 172.21.15.16:0/1482395551 conn(0x7f02680fd8c0 msgr2=0x7f02680fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 shutdown_connections 2024-09-12T00:54:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:57.532+0000 7f026fa7c640 1 -- 172.21.15.16:0/1482395551 wait complete. 2024-09-12T00:54:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:57 smithi159 ceph-mon[35566]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T00:54:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:57 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1621852125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T00:54:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:57 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/894773509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T00:54:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:57 smithi099 ceph-mon[35654]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T00:54:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:57 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1621852125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T00:54:57.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:57 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/894773509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T00:54:58.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1539148419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 msgr2=0x7fd3d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:58.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1539148419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d810d950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009cd0 tx=0x7fd3d002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1539148419 shutdown_connections 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1539148419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d810d950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1539148419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 0x7fd3d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1539148419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 0x7fd3d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1539148419 >> 172.21.15.16:0/1539148419 conn(0x7fd3d80fc7a0 msgr2=0x7fd3d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1539148419 shutdown_connections 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.424+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1539148419 wait complete. 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 Processor -- start 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 -- start start 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 0x7fd3d8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 0x7fd3d81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8110ea0 con 0x7fd3d8101420 2024-09-12T00:54:58.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8110d20 con 0x7fd3d8106a80 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3dc8c7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d8111020 con 0x7fd3d8100a80 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52500/0 (socket says 172.21.15.16:52500) 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 -- 172.21.15.16:0/1621852125 learned_addr learned my addr 172.21.15.16:0/1621852125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d659c640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 0x7fd3d8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d6d9d640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 0x7fd3d81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 -- 172.21.15.16:0/1621852125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 msgr2=0x7fd3d8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 0x7fd3d8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.384 INFO:teuthology.orchestra.run.smithi016.stdout:158913790007 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 -- 172.21.15.16:0/1621852125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 msgr2=0x7fd3d81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 0x7fd3d81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.425+0000 7fd3d5d9b640 1 -- 172.21.15.16:0/1621852125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d81a4830 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3d5d9b640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b4a0 tx=0x7fd3cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc014070 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3cc010af0 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d81a4a00 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc017d10 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.426+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d81a4f10 con 0x7fd3d8101420 2024-09-12T00:54:58.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.427+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d8100ee0 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.428+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd3cc017070 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.429+0000 7fd3c77fe640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 0x7fd3c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.429+0000 7fd3d659c640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 0x7fd3c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.429+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd3cc0ce390 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.429+0000 7fd3d659c640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 0x7fd3c00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005e10 tx=0x7fd3c8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.433+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3cc09b450 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.613+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd3d81a5200 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.614+0000 7fd3c77fe640 1 -- 172.21.15.16:0/1621852125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd3cc09ad50 con 0x7fd3d8101420 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 msgr2=0x7fd3c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 0x7fd3c00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005e10 tx=0x7fd3c8005da0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 msgr2=0x7fd3d81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00b4a0 tx=0x7fd3cc00b960 comp rx=0 tx=0).stop 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 shutdown_connections 2024-09-12T00:54:58.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fd3c00a2c10 0x7fd3c00a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3d8106a80 0x7fd3d81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8101420 0x7fd3d81997b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 --2- 172.21.15.16:0/1621852125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d8100a80 0x7fd3d8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 >> 172.21.15.16:0/1621852125 conn(0x7fd3d80fc7a0 msgr2=0x7fd3d80fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 shutdown_connections 2024-09-12T00:54:58.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:56.616+0000 7fd3dc8c7640 1 -- 172.21.15.16:0/1621852125 wait complete. 2024-09-12T00:54:58.387 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:58 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1482395551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T00:54:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:54:58.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:58 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1482395551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T00:54:58.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:58 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1482395551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T00:54:59.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790007 for osd.5 2024-09-12T00:54:59.071 DEBUG:teuthology.parallel:result is None 2024-09-12T00:54:59.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 -- 172.21.15.16:0/1889790881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 msgr2=0x7f9390103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 --2- 172.21.15.16:0/1889790881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390103120 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9380009900 tx=0x7f938002a480 comp rx=0 tx=0).stop 2024-09-12T00:54:59.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 -- 172.21.15.16:0/1889790881 shutdown_connections 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 --2- 172.21.15.16:0/1889790881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f939010c170 0x7f939010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 --2- 172.21.15.16:0/1889790881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9390103660 0x7f939010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 --2- 172.21.15.16:0/1889790881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390103120 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 -- 172.21.15.16:0/1889790881 >> 172.21.15.16:0/1889790881 conn(0x7f93900fc7a0 msgr2=0x7f93900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 -- 172.21.15.16:0/1889790881 shutdown_connections 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.246+0000 7f9395913640 1 -- 172.21.15.16:0/1889790881 wait complete. 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 Processor -- start 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 -- start start 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9390103660 0x7f9390198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939010c170 0x7f939019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9390111f70 con 0x7f9390102d40 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9390111df0 con 0x7f9390103660 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f9395913640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93901120f0 con 0x7f939010c170 2024-09-12T00:54:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f938effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f938effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33750/0 (socket says 172.21.15.16:33750) 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.247+0000 7f938effd640 1 -- 172.21.15.16:0/542077985 learned_addr learned my addr 172.21.15.16:0/542077985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938f7fe640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939010c170 0x7f939019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938e7fc640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9390103660 0x7f9390198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 -- 172.21.15.16:0/542077985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939010c170 msgr2=0x7f939019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f939010c170 0x7f939019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 -- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9390103660 msgr2=0x7f9390198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9390103660 0x7f9390198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939019d400 con 0x7f9390102d40 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f938effd640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390198450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9380006010 tx=0x7f93800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938003f030 con 0x7f9390102d40 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9380003ce0 con 0x7f9390102d40 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939019d690 con 0x7f9390102d40 2024-09-12T00:54:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380035070 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.248+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901a4fd0 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.249+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9390062980 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.250+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9380042030 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f9394911640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f93780a2c10 0x7f93780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f938e7fc640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f93780a2c10 0x7f93780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f93800efc80 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f9394911640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] conn(0x7f93780ac460 0x7f93780ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9380040e70 con 0x7f93780ac460 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.251+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f93800f0160 con 0x7f9390102d40 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.252+0000 7f938e7fc640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f93780a2c10 0x7f93780a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93840058c0 tx=0x7f9384005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.252+0000 7f938f7fe640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] conn(0x7f93780ac460 0x7f93780ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.252+0000 7f938f7fe640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] conn(0x7f93780ac460 0x7f93780ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:54:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.253+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== osd.7 v2:172.21.15.99:6816/777307661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9380040e70 con 0x7f93780ac460 2024-09-12T00:54:59.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.267+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 --> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9390062e00 con 0x7f93780ac460 2024-09-12T00:54:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.267+0000 7f9394911640 1 -- 172.21.15.16:0/542077985 <== osd.7 v2:172.21.15.99:6816/777307661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9390062e00 con 0x7f93780ac460 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] conn(0x7f93780ac460 msgr2=0x7f93780ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] conn(0x7f93780ac460 0x7f93780ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f93780a2c10 msgr2=0x7f93780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f93780a2c10 0x7f93780a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93840058c0 tx=0x7f9384005850 comp rx=0 tx=0).stop 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 msgr2=0x7f9390198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 --2- 172.21.15.16:0/542077985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9390102d40 0x7f9390198450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9380006010 tx=0x7f93800042f0 comp rx=0 tx=0).stop 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f938f7fe640 1 -- 172.21.15.16:0/542077985 reap_dead start 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 shutdown_connections 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 >> 172.21.15.16:0/542077985 conn(0x7f93900fc7a0 msgr2=0x7f93900fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 shutdown_connections 2024-09-12T00:54:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:54:59.268+0000 7f9395913640 1 -- 172.21.15.16:0/542077985 wait complete. 2024-09-12T00:54:59.591 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:54:59 smithi016 ceph-mon[30972]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T00:54:59.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:54:59 smithi159 ceph-mon[35566]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T00:54:59.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:54:59 smithi099 ceph-mon[35654]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T00:54:59.967 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514519 got 326417514520 for osd.11 2024-09-12T00:54:59.967 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:00.577 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:00 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:55:00.603 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:00 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:55:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:00 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:55:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- 172.21.15.16:0/1396728766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe69010e470 msgr2=0x7fe6901108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 --2- 172.21.15.16:0/1396728766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe69010e470 0x7fe6901108a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe68802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- 172.21.15.16:0/1396728766 shutdown_connections 2024-09-12T00:55:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 --2- 172.21.15.16:0/1396728766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe69010e470 0x7fe6901108a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 --2- 172.21.15.16:0/1396728766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe690074f50 0x7fe69010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 --2- 172.21.15.16:0/1396728766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6900745a0 0x7fe690074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- 172.21.15.16:0/1396728766 >> 172.21.15.16:0/1396728766 conn(0x7fe6900fd8d0 msgr2=0x7fe6900ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- 172.21.15.16:0/1396728766 shutdown_connections 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- 172.21.15.16:0/1396728766 wait complete. 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 Processor -- start 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.391+0000 7fe69652d640 1 -- start start 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 0x7fe69019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 0x7fe69019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114250 con 0x7fe6900745a0 2024-09-12T00:55:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901140d0 con 0x7fe69010e470 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901143d0 con 0x7fe690074f50 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33760/0 (socket says 172.21.15.16:33760) 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 -- 172.21.15.16:0/1018926372 learned_addr learned my addr 172.21.15.16:0/1018926372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe694aca640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 0x7fe69019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe687fff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 0x7fe69019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 -- 172.21.15.16:0/1018926372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 msgr2=0x7fe69019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 0x7fe69019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 -- 172.21.15.16:0/1018926372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 msgr2=0x7fe69019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stdout:240518168614 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 0x7fe69019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 -- 172.21.15.16:0/1018926372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69019f640 con 0x7fe6900745a0 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe687fff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 0x7fe69019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe694aca640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 0x7fe69019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68ffff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b4a0 tx=0x7fe68000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:00.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680014070 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe680010af0 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680017d10 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.392+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69019f930 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.393+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a7210 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.393+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690074980 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.395+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe680017070 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.395+0000 7fe68dffb640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 0x7fe67c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.395+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fe6800ce270 con 0x7fe6900745a0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.396+0000 7fe687fff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 0x7fe67c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.396+0000 7fe687fff640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 0x7fe67c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe69010f720 tx=0x7fe678005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:00.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.399+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe68009b330 con 0x7fe6900745a0 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.577+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe6901a7440 con 0x7fe6900745a0 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.578+0000 7fe68dffb640 1 -- 172.21.15.16:0/1018926372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe68009ac30 con 0x7fe6900745a0 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 msgr2=0x7fe67c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 0x7fe67c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe69010f720 tx=0x7fe678005f90 comp rx=0 tx=0).stop 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 msgr2=0x7fe69019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b4a0 tx=0x7fe68000b960 comp rx=0 tx=0).stop 2024-09-12T00:55:00.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 shutdown_connections 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe67c0a2c90 0x7fe67c0a5150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe69010e470 0x7fe69019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe690074f50 0x7fe69019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 --2- 172.21.15.16:0/1018926372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6900745a0 0x7fe69019a690 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 >> 172.21.15.16:0/1018926372 conn(0x7fe6900fd8d0 msgr2=0x7fe6900fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 shutdown_connections 2024-09-12T00:55:00.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:00.580+0000 7fe69652d640 1 -- 172.21.15.16:0/1018926372 wait complete. 2024-09-12T00:55:01.293 INFO:teuthology.orchestra.run.smithi016.stdout:210453397548 2024-09-12T00:55:01.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T00:55:01.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:01.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1018926372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T00:55:01.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.319 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:01 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1018926372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:01 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.761 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738448 for osd.0 2024-09-12T00:55:01.762 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1018926372' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:01.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:01 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:02.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776156 for osd.10 2024-09-12T00:55:02.872 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:03 smithi016 ceph-mon[30972]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:03.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 -- 172.21.15.16:0/2219469881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 msgr2=0x7ffa8010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:03.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/2219469881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8010ea90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009ca0 tx=0x7ffa7802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 -- 172.21.15.16:0/2219469881 shutdown_connections 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/2219469881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8010ea90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/2219469881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa80102790 0x7ffa801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/2219469881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80101de0 0x7ffa801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 -- 172.21.15.16:0/2219469881 >> 172.21.15.16:0/2219469881 conn(0x7ffa800fd8c0 msgr2=0x7ffa800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 -- 172.21.15.16:0/2219469881 shutdown_connections 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.511+0000 7ffa87cc3640 1 -- 172.21.15.16:0/2219469881 wait complete. 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.512+0000 7ffa87cc3640 1 Processor -- start 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.512+0000 7ffa87cc3640 1 -- start start 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa80101de0 0x7ffa801983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80102790 0x7ffa801988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111f90 con 0x7ffa80106f10 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80111e10 con 0x7ffa80102790 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa80112110 con 0x7ffa80101de0 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 -- 172.21.15.16:0/3318139 learned_addr learned my addr 172.21.15.16:0/3318139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa8525e640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80102790 0x7ffa801988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 -- 172.21.15.16:0/3318139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa80101de0 msgr2=0x7ffa801983a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa80101de0 0x7ffa801983a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 -- 172.21.15.16:0/3318139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80102790 msgr2=0x7ffa801988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80102790 0x7ffa801988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 -- 172.21.15.16:0/3318139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8019d350 con 0x7ffa80106f10 2024-09-12T00:55:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa86260640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa78031a60 tx=0x7ffa78031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78003c30 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa78003dd0 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8019d5e0 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa780047e0 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.513+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801a4f20 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.514+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa801021c0 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.516+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa7802f790 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.517+0000 7ffa76ffd640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 0x7ffa6c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.517+0000 7ffa85a5f640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 0x7ffa6c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.517+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ffa780f04f0 con 0x7ffa80106f10 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.517+0000 7ffa85a5f640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 0x7ffa6c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00a930 tx=0x7ffa7c005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.521+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa780bd5b0 con 0x7ffa80106f10 2024-09-12T00:55:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:03 smithi159 ceph-mon[35566]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.702+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffa80109b60 con 0x7ffa80106f10 2024-09-12T00:55:03.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.702+0000 7ffa76ffd640 1 -- 172.21.15.16:0/3318139 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa780bceb0 con 0x7ffa80106f10 2024-09-12T00:55:03.702 INFO:teuthology.orchestra.run.smithi016.stdout:137438953532 2024-09-12T00:55:03.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 msgr2=0x7ffa6c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 0x7ffa6c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00a930 tx=0x7ffa7c005c00 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 msgr2=0x7ffa8019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa78031a60 tx=0x7ffa78031a90 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 shutdown_connections 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7ffa6c0a2c90 0x7ffa6c0a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa80106f10 0x7ffa8019cc70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80102790 0x7ffa801988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 --2- 172.21.15.16:0/3318139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa80101de0 0x7ffa801983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 >> 172.21.15.16:0/3318139 conn(0x7ffa800fd8c0 msgr2=0x7ffa800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 shutdown_connections 2024-09-12T00:55:03.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:03.705+0000 7ffa87cc3640 1 -- 172.21.15.16:0/3318139 wait complete. 2024-09-12T00:55:03.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:03 smithi099 ceph-mon[35654]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2204963028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38102d40 msgr2=0x7f9b38103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2204963028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38102d40 0x7f9b38103120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009900 tx=0x7f9b3402a480 comp rx=0 tx=0).stop 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2204963028 shutdown_connections 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2204963028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b3810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2204963028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38103660 0x7f9b3810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2204963028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38102d40 0x7f9b38103120 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2204963028 >> 172.21.15.16:0/2204963028 conn(0x7f9b380fc7a0 msgr2=0x7f9b380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2204963028 shutdown_connections 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.265+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2204963028 wait complete. 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.266+0000 7f9b3f9e8640 1 Processor -- start 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.266+0000 7f9b3f9e8640 1 -- start start 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38102d40 0x7f9b381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b38101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38112030 con 0x7f9b38103660 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38111eb0 con 0x7f9b3810c170 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b381121b0 con 0x7f9b38102d40 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33804/0 (socket says 172.21.15.16:33804) 2024-09-12T00:55:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 -- 172.21.15.16:0/2779189312 learned_addr learned my addr 172.21.15.16:0/2779189312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3df85640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b38101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3d784640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38102d40 0x7f9b381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 -- 172.21.15.16:0/2779189312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38102d40 msgr2=0x7f9b381009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38102d40 0x7f9b381009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 -- 172.21.15.16:0/2779189312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 msgr2=0x7f9b38101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b38101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 -- 172.21.15.16:0/2779189312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a6a60 con 0x7f9b38103660 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3df85640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b38101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3cf83640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00b3f0 tx=0x7f9b2c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c015020 con 0x7f9b38103660 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2c011d10 con 0x7f9b38103660 2024-09-12T00:55:04.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a6d50 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c00f2e0 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.267+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a71e0 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.268+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b381031a0 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.270+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9b2c018020 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.270+0000 7f9b2a7fc640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 0x7f9b240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.270+0000 7f9b3d784640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 0x7f9b240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.270+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9b2c0ced90 con 0x7f9b38103660 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.271+0000 7f9b3d784640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 0x7f9b240a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b340096c0 tx=0x7f9b34000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.273+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b2c01f020 con 0x7f9b38103660 2024-09-12T00:55:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.455+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9b38101ed0 con 0x7f9b38103660 2024-09-12T00:55:04.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3318139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T00:55:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.455+0000 7f9b2a7fc640 1 -- 172.21.15.16:0/2779189312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b2c09be50 con 0x7f9b38103660 2024-09-12T00:55:04.455 INFO:teuthology.orchestra.run.smithi016.stdout:55834574925 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 msgr2=0x7f9b240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 0x7f9b240a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b340096c0 tx=0x7f9b34000bc0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 msgr2=0x7f9b38100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00b3f0 tx=0x7f9b2c00b8b0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 shutdown_connections 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b240a2c10 0x7f9b240a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b3810c170 0x7f9b38101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b38103660 0x7f9b38100ee0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 --2- 172.21.15.16:0/2779189312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b38102d40 0x7f9b381009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 >> 172.21.15.16:0/2779189312 conn(0x7f9b380fc7a0 msgr2=0x7f9b380fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 shutdown_connections 2024-09-12T00:55:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.458+0000 7f9b3f9e8640 1 -- 172.21.15.16:0/2779189312 wait complete. 2024-09-12T00:55:04.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.499+0000 7f9b63d35640 1 -- 172.21.15.16:0/2425550682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c100b20 msgr2=0x7f9b5c100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.499+0000 7f9b63d35640 1 --2- 172.21.15.16:0/2425550682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c100b20 0x7f9b5c100f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009900 tx=0x7f9b5802a410 comp rx=0 tx=0).stop 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 -- 172.21.15.16:0/2425550682 shutdown_connections 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 --2- 172.21.15.16:0/2425550682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c10e500 0x7f9b5c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 --2- 172.21.15.16:0/2425550682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 0x7f9b5c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 --2- 172.21.15.16:0/2425550682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c100b20 0x7f9b5c100f00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 -- 172.21.15.16:0/2425550682 >> 172.21.15.16:0/2425550682 conn(0x7f9b5c0fc7d0 msgr2=0x7f9b5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 -- 172.21.15.16:0/2425550682 shutdown_connections 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 -- 172.21.15.16:0/2425550682 wait complete. 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 Processor -- start 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 -- start start 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 0x7f9b5c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.500+0000 7f9b63d35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 0x7f9b5c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c114260 con 0x7f9b5c10e500 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1140e0 con 0x7f9b5c101440 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1143e0 con 0x7f9b5c100b20 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33820/0 (socket says 172.21.15.16:33820) 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 -- 172.21.15.16:0/4283194955 learned_addr learned my addr 172.21.15.16:0/4283194955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b612d0640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 0x7f9b5c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b61ad1640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 0x7f9b5c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 -- 172.21.15.16:0/4283194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 msgr2=0x7f9b5c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 0x7f9b5c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 -- 172.21.15.16:0/4283194955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 msgr2=0x7f9b5c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 0x7f9b5c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 -- 172.21.15.16:0/4283194955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c19f690 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b61ad1640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 0x7f9b5c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b622d2640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400cab0 tx=0x7f9b5400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400ae20 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c19f980 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b5400f930 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54018b80 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.501+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a7260 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.502+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c100f00 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.504+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9b5400fae0 con 0x7f9b5c10e500 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.505+0000 7f9b52ffd640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 0x7f9b480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.505+0000 7f9b61ad1640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 0x7f9b480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.505+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9b540cf920 con 0x7f9b5c10e500 2024-09-12T00:55:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.505+0000 7f9b61ad1640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 0x7f9b480a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b58000c00 tx=0x7f9b58032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.508+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b54014030 con 0x7f9b5c10e500 2024-09-12T00:55:04.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3318139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T00:55:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.693+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9b5c109600 con 0x7f9b5c10e500 2024-09-12T00:55:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.694+0000 7f9b52ffd640 1 -- 172.21.15.16:0/4283194955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b5409c960 con 0x7f9b5c10e500 2024-09-12T00:55:04.693 INFO:teuthology.orchestra.run.smithi016.stdout:77309411399 2024-09-12T00:55:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 msgr2=0x7f9b480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 0x7f9b480a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9b58000c00 tx=0x7f9b58032d50 comp rx=0 tx=0).stop 2024-09-12T00:55:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 msgr2=0x7f9b5c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400cab0 tx=0x7f9b5400cf70 comp rx=0 tx=0).stop 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 shutdown_connections 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9b480a2c10 0x7f9b480a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b5c10e500 0x7f9b5c19efb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c101440 0x7f9b5c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 --2- 172.21.15.16:0/4283194955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b5c100b20 0x7f9b5c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 >> 172.21.15.16:0/4283194955 conn(0x7f9b5c0fc7d0 msgr2=0x7f9b5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 shutdown_connections 2024-09-12T00:55:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:04.696+0000 7f9b63d35640 1 -- 172.21.15.16:0/4283194955 wait complete. 2024-09-12T00:55:04.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3318139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T00:55:05.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 -- 172.21.15.16:0/2343451995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 msgr2=0x7fb274100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 --2- 172.21.15.16:0/2343451995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb274100860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb270009900 tx=0x7fb27002a410 comp rx=0 tx=0).stop 2024-09-12T00:55:05.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 -- 172.21.15.16:0/2343451995 shutdown_connections 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 --2- 172.21.15.16:0/2343451995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2741055b0 0x7fb27410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 --2- 172.21.15.16:0/2343451995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb274100e30 0x7fb274104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 --2- 172.21.15.16:0/2343451995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb274100860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.160+0000 7fb27bb72640 1 -- 172.21.15.16:0/2343451995 >> 172.21.15.16:0/2343451995 conn(0x7fb274077210 msgr2=0x7fb2740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- 172.21.15.16:0/2343451995 shutdown_connections 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- 172.21.15.16:0/2343451995 wait complete. 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 Processor -- start 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- start start 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb274100e30 0x7fb2741a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 0x7fb27419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2741106b0 con 0x7fb274100480 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb274110530 con 0x7fb274100e30 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.161+0000 7fb27bb72640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb274110830 con 0x7fb2741055b0 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27a10f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 0x7fb27419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27910d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb274100e30 0x7fb2741a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33832/0 (socket says 172.21.15.16:33832) 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 -- 172.21.15.16:0/326619131 learned_addr learned my addr 172.21.15.16:0/326619131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:05.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 -- 172.21.15.16:0/326619131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 msgr2=0x7fb27419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 0x7fb27419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 -- 172.21.15.16:0/326619131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb274100e30 msgr2=0x7fb2741a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb274100e30 0x7fb2741a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 -- 172.21.15.16:0/326619131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419aee0 con 0x7fb274100480 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27a10f640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 0x7fb27419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb27990e640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb270005ab0 tx=0x7fb2700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.162+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27003f030 con 0x7fb274100480 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.163+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419b170 con 0x7fb274100480 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.163+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb270003ce0 con 0x7fb274100480 2024-09-12T00:55:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.163+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270035070 con 0x7fb274100480 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.163+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a7350 con 0x7fb274100480 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.164+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274101960 con 0x7fb274100480 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.165+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb270042030 con 0x7fb274100480 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.165+0000 7fb26affd640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 0x7fb2600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.166+0000 7fb27910d640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 0x7fb2600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.166+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fb2700ef4f0 con 0x7fb274100480 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.166+0000 7fb27910d640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 0x7fb2600a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb27419bec0 tx=0x7fb264009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.170+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2700bc4a0 con 0x7fb274100480 2024-09-12T00:55:05.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.349+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb2740fdcd0 con 0x7fb274100480 2024-09-12T00:55:05.349 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:05 smithi016 ceph-mon[30972]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:05.349 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2779189312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T00:55:05.349 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4283194955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T00:55:05.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.350+0000 7fb26affd640 1 -- 172.21.15.16:0/326619131 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb2700bbda0 con 0x7fb274100480 2024-09-12T00:55:05.350 INFO:teuthology.orchestra.run.smithi016.stdout:188978561075 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 msgr2=0x7fb2600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 0x7fb2600a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb27419bec0 tx=0x7fb264009110 comp rx=0 tx=0).stop 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 msgr2=0x7fb2741a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb270005ab0 tx=0x7fb2700042f0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 shutdown_connections 2024-09-12T00:55:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb2600a2c10 0x7fb2600a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2741055b0 0x7fb27419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb274100e30 0x7fb2741a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 --2- 172.21.15.16:0/326619131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb274100480 0x7fb2741a05a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.352+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 >> 172.21.15.16:0/326619131 conn(0x7fb274077210 msgr2=0x7fb274102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.353+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 shutdown_connections 2024-09-12T00:55:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.353+0000 7fb27bb72640 1 -- 172.21.15.16:0/326619131 wait complete. 2024-09-12T00:55:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:05 smithi159 ceph-mon[35566]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2779189312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T00:55:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4283194955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T00:55:05.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574925 for osd.1 2024-09-12T00:55:05.635 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:05.750 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168614 for osd.8 2024-09-12T00:55:05.750 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:05.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:05 smithi099 ceph-mon[35654]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2779189312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T00:55:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4283194955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 -- 172.21.15.16:0/3496748786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 msgr2=0x7f791c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3496748786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c10bb50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f790c009900 tx=0x7f790c02a480 comp rx=0 tx=0).stop 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 -- 172.21.15.16:0/3496748786 shutdown_connections 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3496748786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f791c10c170 0x7f791c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3496748786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c10bb50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3496748786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c102d40 0x7f791c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.923+0000 7f7923ef9640 1 -- 172.21.15.16:0/3496748786 >> 172.21.15.16:0/3496748786 conn(0x7f791c0fc7a0 msgr2=0x7f791c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- 172.21.15.16:0/3496748786 shutdown_connections 2024-09-12T00:55:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- 172.21.15.16:0/3496748786 wait complete. 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 Processor -- start 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- start start 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f791c102d40 0x7f791c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 0x7f791c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111f70 con 0x7f791c103660 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f791c111df0 con 0x7f791c10c170 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7923ef9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f791c1120f0 con 0x7f791c102d40 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.924+0000 7f7921494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33862/0 (socket says 172.21.15.16:33862) 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 -- 172.21.15.16:0/3750669238 learned_addr learned my addr 172.21.15.16:0/3750669238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7922496640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 0x7f791c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 -- 172.21.15.16:0/3750669238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f791c102d40 msgr2=0x7f791c1009a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f791c102d40 0x7f791c1009a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 -- 172.21.15.16:0/3750669238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 msgr2=0x7f791c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 0x7f791c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 -- 172.21.15.16:0/3750669238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c1a69a0 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7922496640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 0x7f791c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7921494640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f790c006010 tx=0x7f790c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c03f030 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c1a6c30 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790c003ce0 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c035070 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.925+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1a7110 con 0x7f791c103660 2024-09-12T00:55:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.926+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c103120 con 0x7f791c103660 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.927+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f790c042030 con 0x7f791c103660 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.928+0000 7f7912ffd640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.928+0000 7f7921c95640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.928+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f790c0efae0 con 0x7f791c103660 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.929+0000 7f7921c95640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 0x7f79080a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f791800b990 tx=0x7f791800b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:05.931+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f790c0bca50 con 0x7f791c103660 2024-09-12T00:55:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.108+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f791c101ed0 con 0x7f791c103660 2024-09-12T00:55:06.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.108+0000 7f7912ffd640 1 -- 172.21.15.16:0/3750669238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f790c0bc350 con 0x7f791c103660 2024-09-12T00:55:06.108 INFO:teuthology.orchestra.run.smithi016.stdout:107374182465 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 msgr2=0x7f79080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 0x7f79080a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f791800b990 tx=0x7f791800b370 comp rx=0 tx=0).stop 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 msgr2=0x7f791c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f790c006010 tx=0x7f790c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 shutdown_connections 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f79080a2c10 0x7f79080a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f791c10c170 0x7f791c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:06.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f791c103660 0x7f791c100ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 --2- 172.21.15.16:0/3750669238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f791c102d40 0x7f791c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.110+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 >> 172.21.15.16:0/3750669238 conn(0x7f791c0fc7a0 msgr2=0x7f791c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.111+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 shutdown_connections 2024-09-12T00:55:06.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:06.111+0000 7f7923ef9640 1 -- 172.21.15.16:0/3750669238 wait complete. 2024-09-12T00:55:06.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953528 got 137438953532 for osd.4 2024-09-12T00:55:06.421 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:06 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/326619131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T00:55:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:06 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3750669238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T00:55:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:06 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/326619131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T00:55:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:06 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3750669238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T00:55:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:06 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/326619131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T00:55:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:06 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3750669238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T00:55:06.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411399 for osd.2 2024-09-12T00:55:06.886 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:07.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:07 smithi016 ceph-mon[30972]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:07.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561075 for osd.6 2024-09-12T00:55:07.592 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:07 smithi159 ceph-mon[35566]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:07.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:07 smithi099 ceph-mon[35654]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:07.926 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:08.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182465 for osd.3 2024-09-12T00:55:08.362 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T00:55:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-12T00:55:08.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:08 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-12T00:55:08.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T00:55:08.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-12T00:55:08.619 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T00:55:08.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T00:55:08.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T00:55:08.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T00:55:08.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-12T00:55:08.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:08 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-09-12T00:55:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:08 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-09-12T00:55:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 -- 172.21.15.16:0/443841752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798101420 msgr2=0x7fb79810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 --2- 172.21.15.16:0/443841752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798101420 0x7fb79810d950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb790009cd0 tx=0x7fb79002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 -- 172.21.15.16:0/443841752 shutdown_connections 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 --2- 172.21.15.16:0/443841752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798101420 0x7fb79810d950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 --2- 172.21.15.16:0/443841752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798100a80 0x7fb798100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 --2- 172.21.15.16:0/443841752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798106a80 0x7fb798106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 -- 172.21.15.16:0/443841752 >> 172.21.15.16:0/443841752 conn(0x7fb7980fc7a0 msgr2=0x7fb7980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 -- 172.21.15.16:0/443841752 shutdown_connections 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.867+0000 7fb79fc06640 1 -- 172.21.15.16:0/443841752 wait complete. 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 Processor -- start 2024-09-12T00:55:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 -- start start 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798101420 0x7fb79819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798106a80 0x7fb79819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb798110ef0 con 0x7fb798100a80 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb798110d70 con 0x7fb798101420 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79fc06640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb798111070 con 0x7fb798106a80 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57102/0 (socket says 172.21.15.16:57102) 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 -- 172.21.15.16:0/1052129776 learned_addr learned my addr 172.21.15.16:0/1052129776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d1a1640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798101420 0x7fb79819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79e1a3640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798106a80 0x7fb79819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 -- 172.21.15.16:0/1052129776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798106a80 msgr2=0x7fb79819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798106a80 0x7fb79819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 -- 172.21.15.16:0/1052129776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798101420 msgr2=0x7fb79819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798101420 0x7fb79819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.868+0000 7fb79d9a2640 1 -- 172.21.15.16:0/1052129776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79819f630 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb79d9a2640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c890 tx=0x7fb79400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794004220 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79819f920 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7940043c0 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794004a00 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.869+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981a7200 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.870+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb798100ee0 con 0x7fb798100a80 2024-09-12T00:55:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.872+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7940079b0 con 0x7fb798100a80 2024-09-12T00:55:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.872+0000 7fb78effd640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 0x7fb7840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.873+0000 7fb79d1a1640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 0x7fb7840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.873+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8219+0+0 (secure 0 0 0) 0x7fb7940cf2d0 con 0x7fb798100a80 2024-09-12T00:55:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.873+0000 7fb79d1a1640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 0x7fb7840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb79810dd70 tx=0x7fb788009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:08.876+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb794014030 con 0x7fb798100a80 2024-09-12T00:55:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.056+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb79819b9e0 con 0x7fb798100a80 2024-09-12T00:55:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.057+0000 7fb78effd640 1 -- 172.21.15.16:0/1052129776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb79819b9e0 con 0x7fb798100a80 2024-09-12T00:55:09.057 INFO:teuthology.orchestra.run.smithi016.stdout:270582939684 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 msgr2=0x7fb7840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 0x7fb7840a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb79810dd70 tx=0x7fb788009110 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 msgr2=0x7fb79819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb79400c890 tx=0x7fb79400cc90 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 shutdown_connections 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7840a2c10 0x7fb7840a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb798106a80 0x7fb79819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb798101420 0x7fb79819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 --2- 172.21.15.16:0/1052129776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb798100a80 0x7fb79819a680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 >> 172.21.15.16:0/1052129776 conn(0x7fb7980fc7a0 msgr2=0x7fb79806a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:09.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 shutdown_connections 2024-09-12T00:55:09.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:09.059+0000 7fb79fc06640 1 -- 172.21.15.16:0/1052129776 wait complete. 2024-09-12T00:55:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T00:55:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-12T00:55:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T00:55:09.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:09 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1052129776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T00:55:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T00:55:09.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-12T00:55:09.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-12T00:55:09.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T00:55:09.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:09 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1052129776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11, 2, 1]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-12T00:55:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-12T00:55:09.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T00:55:09.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:09 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1052129776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T00:55:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:10 smithi016 ceph-mon[30972]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T00:55:10.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939679 got 270582939684 for osd.9 2024-09-12T00:55:10.524 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:10.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:10 smithi159 ceph-mon[35566]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T00:55:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 -- 172.21.15.16:0/1846963103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac40103660 msgr2=0x7fac4010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 --2- 172.21.15.16:0/1846963103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac40103660 0x7fac4010bb50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009900 tx=0x7fac2c02a480 comp rx=0 tx=0).stop 2024-09-12T00:55:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 -- 172.21.15.16:0/1846963103 shutdown_connections 2024-09-12T00:55:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 --2- 172.21.15.16:0/1846963103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac4010c170 0x7fac4010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 --2- 172.21.15.16:0/1846963103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac40103660 0x7fac4010bb50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 --2- 172.21.15.16:0/1846963103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40102d40 0x7fac40103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 -- 172.21.15.16:0/1846963103 >> 172.21.15.16:0/1846963103 conn(0x7fac400fc7a0 msgr2=0x7fac400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 -- 172.21.15.16:0/1846963103 shutdown_connections 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.739+0000 7fac46c0d640 1 -- 172.21.15.16:0/1846963103 wait complete. 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 Processor -- start 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 -- start start 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40102d40 0x7fac4019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 0x7fac4019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac40112000 con 0x7fac4010c170 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac40111e80 con 0x7fac40103660 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac46c0d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac40112180 con 0x7fac40102d40 2024-09-12T00:55:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57108/0 (socket says 172.21.15.16:57108) 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 -- 172.21.15.16:0/130317283 learned_addr learned my addr 172.21.15.16:0/130317283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac449a9640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40102d40 0x7fac4019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac3ffff640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 0x7fac4019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 -- 172.21.15.16:0/130317283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40102d40 msgr2=0x7fac4019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40102d40 0x7fac4019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 -- 172.21.15.16:0/130317283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 msgr2=0x7fac4019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.740+0000 7fac451aa640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 0x7fac4019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac451aa640 1 -- 172.21.15.16:0/130317283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4019f5f0 con 0x7fac4010c170 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac3ffff640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 0x7fac4019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac451aa640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cce0 tx=0x7fac3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800f450 con 0x7fac4010c170 2024-09-12T00:55:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4019f8e0 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac3800fad0 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38018b60 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.741+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac401a71c0 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.742+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac40103120 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.743+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac3800fc80 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.744+0000 7fac3dffb640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 0x7fac300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.744+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fac380cf6f0 con 0x7fac4010c170 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.744+0000 7fac449a9640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 0x7fac300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.744+0000 7fac449a9640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 0x7fac300a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac34009990 tx=0x7fac34009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.748+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac38014030 con 0x7fac4010c170 2024-09-12T00:55:10.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:10 smithi099 ceph-mon[35654]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T00:55:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.936+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fac401a7460 con 0x7fac4010c170 2024-09-12T00:55:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.936+0000 7fac3dffb640 1 -- 172.21.15.16:0/130317283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fac3809c500 con 0x7fac4010c170 2024-09-12T00:55:10.936 INFO:teuthology.orchestra.run.smithi016.stdout:210453397551 2024-09-12T00:55:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 msgr2=0x7fac300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 0x7fac300a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac34009990 tx=0x7fac34009110 comp rx=0 tx=0).stop 2024-09-12T00:55:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 msgr2=0x7fac4019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cce0 tx=0x7fac3800a620 comp rx=0 tx=0).stop 2024-09-12T00:55:10.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 shutdown_connections 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fac300a2c10 0x7fac300a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4010c170 0x7fac4019ef10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac40103660 0x7fac4019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 --2- 172.21.15.16:0/130317283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40102d40 0x7fac4019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 >> 172.21.15.16:0/130317283 conn(0x7fac400fc7a0 msgr2=0x7fac4010d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 shutdown_connections 2024-09-12T00:55:10.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:10.938+0000 7fac46c0d640 1 -- 172.21.15.16:0/130317283 wait complete. 2024-09-12T00:55:11.592 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:11 smithi016 ceph-mon[30972]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T00:55:11.592 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:11 smithi016 ceph-mon[30972]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-12T00:55:11.592 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/130317283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T00:55:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:11 smithi159 ceph-mon[35566]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T00:55:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:11 smithi159 ceph-mon[35566]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-12T00:55:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/130317283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T00:55:11.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397548 got 210453397551 for osd.7 2024-09-12T00:55:11.624 DEBUG:teuthology.parallel:result is None 2024-09-12T00:55:11.625 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T00:55:11.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:55:11.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:11 smithi099 ceph-mon[35654]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T00:55:11.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:11 smithi099 ceph-mon[35654]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-12T00:55:11.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/130317283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T00:55:11.921 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 -- 172.21.15.16:0/3274441900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881036e0 msgr2=0x7f7488107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 --2- 172.21.15.16:0/3274441900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881036e0 0x7f7488107b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7470009930 tx=0x7f747002a420 comp rx=0 tx=0).stop 2024-09-12T00:55:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 -- 172.21.15.16:0/3274441900 shutdown_connections 2024-09-12T00:55:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 --2- 172.21.15.16:0/3274441900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881082a0 0x7f748810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 --2- 172.21.15.16:0/3274441900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881036e0 0x7f7488107b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 --2- 172.21.15.16:0/3274441900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 0x7f7488103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.478+0000 7f748d34e640 1 -- 172.21.15.16:0/3274441900 >> 172.21.15.16:0/3274441900 conn(0x7f74880fc7d0 msgr2=0x7f74880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.479+0000 7f748d34e640 1 -- 172.21.15.16:0/3274441900 shutdown_connections 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.479+0000 7f748d34e640 1 -- 172.21.15.16:0/3274441900 wait complete. 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.479+0000 7f748d34e640 1 Processor -- start 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 -- start start 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 0x7f74881a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881036e0 0x7f74881a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74881120a0 con 0x7f74881082a0 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111f20 con 0x7f74881036e0 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f748d34e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7488112220 con 0x7f7488102d30 2024-09-12T00:55:12.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57120/0 (socket says 172.21.15.16:57120) 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 -- 172.21.15.16:0/3273488662 learned_addr learned my addr 172.21.15.16:0/3273488662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f7486ffd640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 0x7f74881a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74867fc640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881036e0 0x7f74881a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 -- 172.21.15.16:0/3273488662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 msgr2=0x7f74881a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 0x7f74881a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 -- 172.21.15.16:0/3273488662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881036e0 msgr2=0x7f74881a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881036e0 0x7f74881a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.480+0000 7f74877fe640 1 -- 172.21.15.16:0/3273488662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748819acd0 con 0x7f74881082a0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f74877fe640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f748000c970 tx=0x7f748000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480007bb0 con 0x7f74881082a0 2024-09-12T00:55:12.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748819afc0 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7480007d50 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480018800 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.481+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a72b0 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.482+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488103110 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.483+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f748000fa50 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.484+0000 7f747ffff640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 0x7f74740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.484+0000 7f7486ffd640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 0x7f74740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.484+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f74800cf170 con 0x7f74881082a0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.484+0000 7f7486ffd640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 0x7f74740a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7478005fd0 tx=0x7f7478009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:12.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.488+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7480014030 con 0x7f74881082a0 2024-09-12T00:55:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.663+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7488062630 con 0x7f74740a2c10 2024-09-12T00:55:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.669+0000 7f747ffff640 1 -- 172.21.15.16:0/3273488662 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258719 (secure 0 0 0) 0x7f7488062630 con 0x7f74740a2c10 2024-09-12T00:55:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:12.672 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:55:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 msgr2=0x7f74740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:12.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 0x7f74740a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7478005fd0 tx=0x7f7478009680 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 msgr2=0x7f748819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f748000c970 tx=0x7f748000ce30 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 shutdown_connections 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f74740a2c10 0x7f74740a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74881082a0 0x7f748819a5a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74881036e0 0x7f74881a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 --2- 172.21.15.16:0/3273488662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7488102d30 0x7f74881a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.672+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 >> 172.21.15.16:0/3273488662 conn(0x7f74880fc7d0 msgr2=0x7f74880fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.673+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 shutdown_connections 2024-09-12T00:55:12.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:12.673+0000 7f748d34e640 1 -- 172.21.15.16:0/3273488662 wait complete. 2024-09-12T00:55:13.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:55:13.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:55:13.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:12 smithi016 ceph-mon[30972]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:55:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:55:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:12 smithi099 ceph-mon[35654]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:55:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:55:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:12 smithi159 ceph-mon[35566]: from='mgr.14150 172.21.15.16:0/1789168142' entity='mgr.x' 2024-09-12T00:55:13.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-12T00:55:12.224885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657021,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":702,"num_read_kb":470,"num_write":422,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372852,"kb_used_data":11092,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568972,"statfs":{"total":1151940427776,"available":1151558627328,"internally_reserved":0,"allocated":11358208,"data_stored":7108950,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002188"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755743+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:54:04.755743+0000","last_peered":"2024-09-12T00:54:04.755743+0000","last_clean":"2024-09-12T00:54:04.755743+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:54:04.755743+0000","last_undegraded":"2024-09-12T00:54:04.755743+0000","last_fullsized":"2024-09-12T00:54:04.755743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.326856+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:55:08.326856+0000","last_peered":"2024-09-12T00:55:08.326856+0000","last_clean":"2024-09-12T00:55:08.326856+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:55:08.326856+0000","last_undegraded":"2024-09-12T00:55:08.326856+0000","last_fullsized":"2024-09-12T00:55:08.326856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.326906+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:55:08.326906+0000","last_peered":"2024-09-12T00:55:08.326906+0000","last_clean":"2024-09-12T00:55:08.326906+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:55:08.326906+0000","last_undegraded":"2024-09-12T00:55:08.326906+0000","last_fullsized":"2024-09-12T00:55:08.326906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903828+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:55:09.903828+0000","last_peered":"2024-09-12T00:55:09.903828+0000","last_clean":"2024-09-12T00:55:09.903828+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:55:09.903828+0000","last_undegraded":"2024-09-12T00:55:09.903828+0000","last_fullsized":"2024-09-12T00:55:09.903828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332440+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:55:09.332440+0000","last_peered":"2024-09-12T00:55:09.332440+0000","last_clean":"2024-09-12T00:55:09.332440+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:55:09.332440+0000","last_undegraded":"2024-09-12T00:55:09.332440+0000","last_fullsized":"2024-09-12T00:55:09.332440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.767888+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:54:04.767888+0000","last_peered":"2024-09-12T00:54:04.767888+0000","last_clean":"2024-09-12T00:54:04.767888+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:54:04.767888+0000","last_undegraded":"2024-09-12T00:54:04.767888+0000","last_fullsized":"2024-09-12T00:54:04.767888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336115+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:55:09.336115+0000","last_peered":"2024-09-12T00:55:09.336115+0000","last_clean":"2024-09-12T00:55:09.336115+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:55:09.336115+0000","last_undegraded":"2024-09-12T00:55:09.336115+0000","last_fullsized":"2024-09-12T00:55:09.336115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-12T00:54:04.770258+0000","last_change":"2024-09-12T00:55:08.337560+0000","last_active":"2024-09-12T00:55:08.337560+0000","last_peered":"2024-09-12T00:54:04.770258+0000","last_clean":"2024-09-12T00:54:04.770258+0000","last_became_active":"2024-09-12T00:54:03.754586+0000","last_became_peered":"2024-09-12T00:54:03.754586+0000","last_unstale":"2024-09-12T00:54:04.770258+0000","last_undegraded":"2024-09-12T00:54:04.770258+0000","last_fullsized":"2024-09-12T00:54:04.770258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770626+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:54:04.770626+0000","last_peered":"2024-09-12T00:54:04.770626+0000","last_clean":"2024-09-12T00:54:04.770626+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:54:04.770626+0000","last_undegraded":"2024-09-12T00:54:04.770626+0000","last_fullsized":"2024-09-12T00:54:04.770626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768153+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:54:04.768153+0000","last_peered":"2024-09-12T00:54:04.768153+0000","last_clean":"2024-09-12T00:54:04.768153+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:54:04.768153+0000","last_undegraded":"2024-09-12T00:54:04.768153+0000","last_fullsized":"2024-09-12T00:54:04.768153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336529+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:55:09.336529+0000","last_peered":"2024-09-12T00:55:09.336529+0000","last_clean":"2024-09-12T00:55:09.336529+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:55:09.336529+0000","last_undegraded":"2024-09-12T00:55:09.336529+0000","last_fullsized":"2024-09-12T00:55:09.336529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903741+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:55:09.903741+0000","last_peered":"2024-09-12T00:55:09.903741+0000","last_clean":"2024-09-12T00:55:09.903741+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:55:09.903741+0000","last_undegraded":"2024-09-12T00:55:09.903741+0000","last_fullsized":"2024-09-12T00:55:09.903741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327339+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:55:09.327339+0000","last_peered":"2024-09-12T00:55:09.327339+0000","last_clean":"2024-09-12T00:55:09.327339+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:55:09.327339+0000","last_undegraded":"2024-09-12T00:55:09.327339+0000","last_fullsized":"2024-09-12T00:55:09.327339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.326876+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:55:08.326876+0000","last_peered":"2024-09-12T00:55:08.326876+0000","last_clean":"2024-09-12T00:55:08.326876+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:55:08.326876+0000","last_undegraded":"2024-09-12T00:55:08.326876+0000","last_fullsized":"2024-09-12T00:55:08.326876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.326575+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:55:08.326575+0000","last_peered":"2024-09-12T00:55:08.326575+0000","last_clean":"2024-09-12T00:55:08.326575+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:55:08.326575+0000","last_undegraded":"2024-09-12T00:55:08.326575+0000","last_fullsized":"2024-09-12T00:55:08.326575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770578+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:54:04.770578+0000","last_peered":"2024-09-12T00:54:04.770578+0000","last_clean":"2024-09-12T00:54:04.770578+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:54:04.770578+0000","last_undegraded":"2024-09-12T00:54:04.770578+0000","last_fullsized":"2024-09-12T00:54:04.770578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755788+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:54:04.755788+0000","last_peered":"2024-09-12T00:54:04.755788+0000","last_clean":"2024-09-12T00:54:04.755788+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:54:04.755788+0000","last_undegraded":"2024-09-12T00:54:04.755788+0000","last_fullsized":"2024-09-12T00:54:04.755788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.320548+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:55:08.320548+0000","last_peered":"2024-09-12T00:55:08.320548+0000","last_clean":"2024-09-12T00:55:08.320548+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:55:08.320548+0000","last_undegraded":"2024-09-12T00:55:08.320548+0000","last_fullsized":"2024-09-12T00:55:08.320548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333047+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:55:09.333047+0000","last_peered":"2024-09-12T00:55:09.333047+0000","last_clean":"2024-09-12T00:55:09.333047+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:55:09.333047+0000","last_undegraded":"2024-09-12T00:55:09.333047+0000","last_fullsized":"2024-09-12T00:55:09.333047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755469+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:54:04.755469+0000","last_peered":"2024-09-12T00:54:04.755469+0000","last_clean":"2024-09-12T00:54:04.755469+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:54:04.755469+0000","last_undegraded":"2024-09-12T00:54:04.755469+0000","last_fullsized":"2024-09-12T00:54:04.755469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327337+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:55:09.327337+0000","last_peered":"2024-09-12T00:55:09.327337+0000","last_clean":"2024-09-12T00:55:09.327337+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:55:09.327337+0000","last_undegraded":"2024-09-12T00:55:09.327337+0000","last_fullsized":"2024-09-12T00:55:09.327337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770710+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:54:04.770710+0000","last_peered":"2024-09-12T00:54:04.770710+0000","last_clean":"2024-09-12T00:54:04.770710+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:54:04.770710+0000","last_undegraded":"2024-09-12T00:54:04.770710+0000","last_fullsized":"2024-09-12T00:54:04.770710+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768471+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:54:04.768471+0000","last_peered":"2024-09-12T00:54:04.768471+0000","last_clean":"2024-09-12T00:54:04.768471+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:54:04.768471+0000","last_undegraded":"2024-09-12T00:54:04.768471+0000","last_fullsized":"2024-09-12T00:54:04.768471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330690+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:55:09.330690+0000","last_peered":"2024-09-12T00:55:09.330690+0000","last_clean":"2024-09-12T00:55:09.330690+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:55:09.330690+0000","last_undegraded":"2024-09-12T00:55:09.330690+0000","last_fullsized":"2024-09-12T00:55:09.330690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769177+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:54:04.769177+0000","last_peered":"2024-09-12T00:54:04.769177+0000","last_clean":"2024-09-12T00:54:04.769177+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:54:04.769177+0000","last_undegraded":"2024-09-12T00:54:04.769177+0000","last_fullsized":"2024-09-12T00:54:04.769177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333367+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:55:09.333367+0000","last_peered":"2024-09-12T00:55:09.333367+0000","last_clean":"2024-09-12T00:55:09.333367+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:55:09.333367+0000","last_undegraded":"2024-09-12T00:55:09.333367+0000","last_fullsized":"2024-09-12T00:55:09.333367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755640+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:54:04.755640+0000","last_peered":"2024-09-12T00:54:04.755640+0000","last_clean":"2024-09-12T00:54:04.755640+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:54:04.755640+0000","last_undegraded":"2024-09-12T00:54:04.755640+0000","last_fullsized":"2024-09-12T00:54:04.755640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327315+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:55:09.327315+0000","last_peered":"2024-09-12T00:55:09.327315+0000","last_clean":"2024-09-12T00:55:09.327315+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:55:09.327315+0000","last_undegraded":"2024-09-12T00:55:09.327315+0000","last_fullsized":"2024-09-12T00:55:09.327315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.327068+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:55:08.327068+0000","last_peered":"2024-09-12T00:55:08.327068+0000","last_clean":"2024-09-12T00:55:08.327068+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:55:08.327068+0000","last_undegraded":"2024-09-12T00:55:08.327068+0000","last_fullsized":"2024-09-12T00:55:08.327068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332630+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:55:09.332630+0000","last_peered":"2024-09-12T00:55:09.332630+0000","last_clean":"2024-09-12T00:55:09.332630+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:55:09.332630+0000","last_undegraded":"2024-09-12T00:55:09.332630+0000","last_fullsized":"2024-09-12T00:55:09.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332505+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:55:09.332505+0000","last_peered":"2024-09-12T00:55:09.332505+0000","last_clean":"2024-09-12T00:55:09.332505+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:55:09.332505+0000","last_undegraded":"2024-09-12T00:55:09.332505+0000","last_fullsized":"2024-09-12T00:55:09.332505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755330+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:54:04.755330+0000","last_peered":"2024-09-12T00:54:04.755330+0000","last_clean":"2024-09-12T00:54:04.755330+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:54:04.755330+0000","last_undegraded":"2024-09-12T00:54:04.755330+0000","last_fullsized":"2024-09-12T00:54:04.755330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332270+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:55:09.332270+0000","last_peered":"2024-09-12T00:55:09.332270+0000","last_clean":"2024-09-12T00:55:09.332270+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:55:09.332270+0000","last_undegraded":"2024-09-12T00:55:09.332270+0000","last_fullsized":"2024-09-12T00:55:09.332270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330666+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:55:09.330666+0000","last_peered":"2024-09-12T00:55:09.330666+0000","last_clean":"2024-09-12T00:55:09.330666+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:55:09.330666+0000","last_undegraded":"2024-09-12T00:55:09.330666+0000","last_fullsized":"2024-09-12T00:55:09.330666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330552+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:55:09.330552+0000","last_peered":"2024-09-12T00:55:09.330552+0000","last_clean":"2024-09-12T00:55:09.330552+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:55:09.330552+0000","last_undegraded":"2024-09-12T00:55:09.330552+0000","last_fullsized":"2024-09-12T00:55:09.330552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:14.817789+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:54:14.817789+0000","last_peered":"2024-09-12T00:54:14.817789+0000","last_clean":"2024-09-12T00:54:14.817789+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:54:14.817789+0000","last_undegraded":"2024-09-12T00:54:14.817789+0000","last_fullsized":"2024-09-12T00:54:14.817789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327066+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:55:09.327066+0000","last_peered":"2024-09-12T00:55:09.327066+0000","last_clean":"2024-09-12T00:55:09.327066+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:55:09.327066+0000","last_undegraded":"2024-09-12T00:55:09.327066+0000","last_fullsized":"2024-09-12T00:55:09.327066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770308+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:54:04.770308+0000","last_peered":"2024-09-12T00:54:04.770308+0000","last_clean":"2024-09-12T00:54:04.770308+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:54:04.770308+0000","last_undegraded":"2024-09-12T00:54:04.770308+0000","last_fullsized":"2024-09-12T00:54:04.770308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330996+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:55:09.330996+0000","last_peered":"2024-09-12T00:55:09.330996+0000","last_clean":"2024-09-12T00:55:09.330996+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:55:09.330996+0000","last_undegraded":"2024-09-12T00:55:09.330996+0000","last_fullsized":"2024-09-12T00:55:09.330996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.320343+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:55:08.320343+0000","last_peered":"2024-09-12T00:55:08.320343+0000","last_clean":"2024-09-12T00:55:08.320343+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:55:08.320343+0000","last_undegraded":"2024-09-12T00:55:08.320343+0000","last_fullsized":"2024-09-12T00:55:08.320343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:55:06.893528+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:55:06.893528+0000","last_peered":"2024-09-12T00:55:06.893528+0000","last_clean":"2024-09-12T00:55:06.893528+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:55:06.893528+0000","last_undegraded":"2024-09-12T00:55:06.893528+0000","last_fullsized":"2024-09-12T00:55:06.893528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770497+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:54:04.770497+0000","last_peered":"2024-09-12T00:54:04.770497+0000","last_clean":"2024-09-12T00:54:04.770497+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:54:04.770497+0000","last_undegraded":"2024-09-12T00:54:04.770497+0000","last_fullsized":"2024-09-12T00:54:04.770497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336488+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:55:09.336488+0000","last_peered":"2024-09-12T00:55:09.336488+0000","last_clean":"2024-09-12T00:55:09.336488+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:55:09.336488+0000","last_undegraded":"2024-09-12T00:55:09.336488+0000","last_fullsized":"2024-09-12T00:55:09.336488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.327269+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:55:08.327269+0000","last_peered":"2024-09-12T00:55:08.327269+0000","last_clean":"2024-09-12T00:55:08.327269+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:55:08.327269+0000","last_undegraded":"2024-09-12T00:55:08.327269+0000","last_fullsized":"2024-09-12T00:55:08.327269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331035+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:55:09.331035+0000","last_peered":"2024-09-12T00:55:09.331035+0000","last_clean":"2024-09-12T00:55:09.331035+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:55:09.331035+0000","last_undegraded":"2024-09-12T00:55:09.331035+0000","last_fullsized":"2024-09-12T00:55:09.331035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770432+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:54:04.770432+0000","last_peered":"2024-09-12T00:54:04.770432+0000","last_clean":"2024-09-12T00:54:04.770432+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:54:04.770432+0000","last_undegraded":"2024-09-12T00:54:04.770432+0000","last_fullsized":"2024-09-12T00:54:04.770432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":155,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:55:06.897411+0000","last_change":"2024-09-12T00:54:01.669518+0000","last_active":"2024-09-12T00:55:06.897411+0000","last_peered":"2024-09-12T00:55:06.897411+0000","last_clean":"2024-09-12T00:55:06.897411+0000","last_became_active":"2024-09-12T00:54:01.669145+0000","last_became_peered":"2024-09-12T00:54:01.669145+0000","last_unstale":"2024-09-12T00:55:06.897411+0000","last_undegraded":"2024-09-12T00:55:06.897411+0000","last_fullsized":"2024-09-12T00:55:06.897411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:48.866081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":111,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332899+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:55:09.332899+0000","last_peered":"2024-09-12T00:55:09.332899+0000","last_clean":"2024-09-12T00:55:09.332899+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:55:09.332899+0000","last_undegraded":"2024-09-12T00:55:09.332899+0000","last_fullsized":"2024-09-12T00:55:09.332899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333258+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:55:09.333258+0000","last_peered":"2024-09-12T00:55:09.333258+0000","last_clean":"2024-09-12T00:55:09.333258+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:55:09.333258+0000","last_undegraded":"2024-09-12T00:55:09.333258+0000","last_fullsized":"2024-09-12T00:55:09.333258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327141+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:55:09.327141+0000","last_peered":"2024-09-12T00:55:09.327141+0000","last_clean":"2024-09-12T00:55:09.327141+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:55:09.327141+0000","last_undegraded":"2024-09-12T00:55:09.327141+0000","last_fullsized":"2024-09-12T00:55:09.327141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331294+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:55:09.331294+0000","last_peered":"2024-09-12T00:55:09.331294+0000","last_clean":"2024-09-12T00:55:09.331294+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:55:09.331294+0000","last_undegraded":"2024-09-12T00:55:09.331294+0000","last_fullsized":"2024-09-12T00:55:09.331294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336386+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:55:09.336386+0000","last_peered":"2024-09-12T00:55:09.336386+0000","last_clean":"2024-09-12T00:55:09.336386+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:55:09.336386+0000","last_undegraded":"2024-09-12T00:55:09.336386+0000","last_fullsized":"2024-09-12T00:55:09.336386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":85,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.902902+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:55:09.902902+0000","last_peered":"2024-09-12T00:55:09.902902+0000","last_clean":"2024-09-12T00:55:09.902902+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:55:09.902902+0000","last_undegraded":"2024-09-12T00:55:09.902902+0000","last_fullsized":"2024-09-12T00:55:09.902902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336035+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:55:09.336035+0000","last_peered":"2024-09-12T00:55:09.336035+0000","last_clean":"2024-09-12T00:55:09.336035+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:55:09.336035+0000","last_undegraded":"2024-09-12T00:55:09.336035+0000","last_fullsized":"2024-09-12T00:55:09.336035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331724+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:55:09.331724+0000","last_peered":"2024-09-12T00:55:09.331724+0000","last_clean":"2024-09-12T00:55:09.331724+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:55:09.331724+0000","last_undegraded":"2024-09-12T00:55:09.331724+0000","last_fullsized":"2024-09-12T00:55:09.331724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.335750+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:55:09.335750+0000","last_peered":"2024-09-12T00:55:09.335750+0000","last_clean":"2024-09-12T00:55:09.335750+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:55:09.335750+0000","last_undegraded":"2024-09-12T00:55:09.335750+0000","last_fullsized":"2024-09-12T00:55:09.335750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333036+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:55:09.333036+0000","last_peered":"2024-09-12T00:55:09.333036+0000","last_clean":"2024-09-12T00:55:09.333036+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:55:09.333036+0000","last_undegraded":"2024-09-12T00:55:09.333036+0000","last_fullsized":"2024-09-12T00:55:09.333036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755554+0000","last_change":"2024-09-12T00:54:03.762778+0000","last_active":"2024-09-12T00:54:04.755554+0000","last_peered":"2024-09-12T00:54:04.755554+0000","last_clean":"2024-09-12T00:54:04.755554+0000","last_became_active":"2024-09-12T00:54:03.762469+0000","last_became_peered":"2024-09-12T00:54:03.762469+0000","last_unstale":"2024-09-12T00:54:04.755554+0000","last_undegraded":"2024-09-12T00:54:04.755554+0000","last_fullsized":"2024-09-12T00:54:04.755554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:01.749510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327476+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:55:09.327476+0000","last_peered":"2024-09-12T00:55:09.327476+0000","last_clean":"2024-09-12T00:55:09.327476+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:55:09.327476+0000","last_undegraded":"2024-09-12T00:55:09.327476+0000","last_fullsized":"2024-09-12T00:55:09.327476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336543+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:55:09.336543+0000","last_peered":"2024-09-12T00:55:09.336543+0000","last_clean":"2024-09-12T00:55:09.336543+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:55:09.336543+0000","last_undegraded":"2024-09-12T00:55:09.336543+0000","last_fullsized":"2024-09-12T00:55:09.336543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331655+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:55:09.331655+0000","last_peered":"2024-09-12T00:55:09.331655+0000","last_clean":"2024-09-12T00:55:09.331655+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:55:09.331655+0000","last_undegraded":"2024-09-12T00:55:09.331655+0000","last_fullsized":"2024-09-12T00:55:09.331655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336441+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:55:09.336441+0000","last_peered":"2024-09-12T00:55:09.336441+0000","last_clean":"2024-09-12T00:55:09.336441+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:55:09.336441+0000","last_undegraded":"2024-09-12T00:55:09.336441+0000","last_fullsized":"2024-09-12T00:55:09.336441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:10.902872+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:55:10.902872+0000","last_peered":"2024-09-12T00:55:10.902872+0000","last_clean":"2024-09-12T00:55:10.902872+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:55:10.902872+0000","last_undegraded":"2024-09-12T00:55:10.902872+0000","last_fullsized":"2024-09-12T00:55:10.902872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336672+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:55:09.336672+0000","last_peered":"2024-09-12T00:55:09.336672+0000","last_clean":"2024-09-12T00:55:09.336672+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:55:09.336672+0000","last_undegraded":"2024-09-12T00:55:09.336672+0000","last_fullsized":"2024-09-12T00:55:09.336672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336173+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:55:09.336173+0000","last_peered":"2024-09-12T00:55:09.336173+0000","last_clean":"2024-09-12T00:55:09.336173+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:55:09.336173+0000","last_undegraded":"2024-09-12T00:55:09.336173+0000","last_fullsized":"2024-09-12T00:55:09.336173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903073+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:55:09.903073+0000","last_peered":"2024-09-12T00:55:09.903073+0000","last_clean":"2024-09-12T00:55:09.903073+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:55:09.903073+0000","last_undegraded":"2024-09-12T00:55:09.903073+0000","last_fullsized":"2024-09-12T00:55:09.903073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768811+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:54:04.768811+0000","last_peered":"2024-09-12T00:54:04.768811+0000","last_clean":"2024-09-12T00:54:04.768811+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:54:04.768811+0000","last_undegraded":"2024-09-12T00:54:04.768811+0000","last_fullsized":"2024-09-12T00:54:04.768811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336181+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:55:09.336181+0000","last_peered":"2024-09-12T00:55:09.336181+0000","last_clean":"2024-09-12T00:55:09.336181+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:55:09.336181+0000","last_undegraded":"2024-09-12T00:55:09.336181+0000","last_fullsized":"2024-09-12T00:55:09.336181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327057+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:55:09.327057+0000","last_peered":"2024-09-12T00:55:09.327057+0000","last_clean":"2024-09-12T00:55:09.327057+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:55:09.327057+0000","last_undegraded":"2024-09-12T00:55:09.327057+0000","last_fullsized":"2024-09-12T00:55:09.327057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327276+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:55:09.327276+0000","last_peered":"2024-09-12T00:55:09.327276+0000","last_clean":"2024-09-12T00:55:09.327276+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:55:09.327276+0000","last_undegraded":"2024-09-12T00:55:09.327276+0000","last_fullsized":"2024-09-12T00:55:09.327276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330809+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:55:09.330809+0000","last_peered":"2024-09-12T00:55:09.330809+0000","last_clean":"2024-09-12T00:55:09.330809+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:55:09.330809+0000","last_undegraded":"2024-09-12T00:55:09.330809+0000","last_fullsized":"2024-09-12T00:55:09.330809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336329+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:55:09.336329+0000","last_peered":"2024-09-12T00:55:09.336329+0000","last_clean":"2024-09-12T00:55:09.336329+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:55:09.336329+0000","last_undegraded":"2024-09-12T00:55:09.336329+0000","last_fullsized":"2024-09-12T00:55:09.336329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768097+0000","last_change":"2024-09-12T00:54:03.761993+0000","last_active":"2024-09-12T00:54:04.768097+0000","last_peered":"2024-09-12T00:54:04.768097+0000","last_clean":"2024-09-12T00:54:04.768097+0000","last_became_active":"2024-09-12T00:54:03.761620+0000","last_became_peered":"2024-09-12T00:54:03.761620+0000","last_unstale":"2024-09-12T00:54:04.768097+0000","last_undegraded":"2024-09-12T00:54:04.768097+0000","last_fullsized":"2024-09-12T00:54:04.768097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:54:35.657480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.320616+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:55:08.320616+0000","last_peered":"2024-09-12T00:55:08.320616+0000","last_clean":"2024-09-12T00:55:08.320616+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:55:08.320616+0000","last_undegraded":"2024-09-12T00:55:08.320616+0000","last_fullsized":"2024-09-12T00:55:08.320616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:07.386806+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:54:07.386806+0000","last_peered":"2024-09-12T00:54:07.386806+0000","last_clean":"2024-09-12T00:54:07.386806+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:54:07.386806+0000","last_undegraded":"2024-09-12T00:54:07.386806+0000","last_fullsized":"2024-09-12T00:54:07.386806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333224+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:55:09.333224+0000","last_peered":"2024-09-12T00:55:09.333224+0000","last_clean":"2024-09-12T00:55:09.333224+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:55:09.333224+0000","last_undegraded":"2024-09-12T00:55:09.333224+0000","last_fullsized":"2024-09-12T00:55:09.333224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327341+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:55:09.327341+0000","last_peered":"2024-09-12T00:55:09.327341+0000","last_clean":"2024-09-12T00:55:09.327341+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:55:09.327341+0000","last_undegraded":"2024-09-12T00:55:09.327341+0000","last_fullsized":"2024-09-12T00:55:09.327341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":33,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-12T00:54:31.771740+0000","last_change":"2024-09-12T00:55:08.321265+0000","last_active":"2024-09-12T00:55:08.321265+0000","last_peered":"2024-09-12T00:54:31.771740+0000","last_clean":"2024-09-12T00:54:31.771740+0000","last_became_active":"2024-09-12T00:54:01.666917+0000","last_became_peered":"2024-09-12T00:54:01.666917+0000","last_unstale":"2024-09-12T00:54:31.771740+0000","last_undegraded":"2024-09-12T00:54:31.771740+0000","last_fullsized":"2024-09-12T00:54:31.771740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,8,11],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769200+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:54:04.769200+0000","last_peered":"2024-09-12T00:54:04.769200+0000","last_clean":"2024-09-12T00:54:04.769200+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:54:04.769200+0000","last_undegraded":"2024-09-12T00:54:04.769200+0000","last_fullsized":"2024-09-12T00:54:04.769200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903305+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:55:09.903305+0000","last_peered":"2024-09-12T00:55:09.903305+0000","last_clean":"2024-09-12T00:55:09.903305+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:55:09.903305+0000","last_undegraded":"2024-09-12T00:55:09.903305+0000","last_fullsized":"2024-09-12T00:55:09.903305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.327040+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:55:08.327040+0000","last_peered":"2024-09-12T00:55:08.327040+0000","last_clean":"2024-09-12T00:55:08.327040+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:55:08.327040+0000","last_undegraded":"2024-09-12T00:55:08.327040+0000","last_fullsized":"2024-09-12T00:55:08.327040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.799867+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:54:12.799867+0000","last_peered":"2024-09-12T00:54:12.799867+0000","last_clean":"2024-09-12T00:54:12.799867+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:54:12.799867+0000","last_undegraded":"2024-09-12T00:54:12.799867+0000","last_fullsized":"2024-09-12T00:54:12.799867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332584+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:55:09.332584+0000","last_peered":"2024-09-12T00:55:09.332584+0000","last_clean":"2024-09-12T00:55:09.332584+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:55:09.332584+0000","last_undegraded":"2024-09-12T00:55:09.332584+0000","last_fullsized":"2024-09-12T00:55:09.332584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:15.823344+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:54:15.823344+0000","last_peered":"2024-09-12T00:54:15.823344+0000","last_clean":"2024-09-12T00:54:15.823344+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:54:15.823344+0000","last_undegraded":"2024-09-12T00:54:15.823344+0000","last_fullsized":"2024-09-12T00:54:15.823344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.755557+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:54:04.755557+0000","last_peered":"2024-09-12T00:54:04.755557+0000","last_clean":"2024-09-12T00:54:04.755557+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:54:04.755557+0000","last_undegraded":"2024-09-12T00:54:04.755557+0000","last_fullsized":"2024-09-12T00:54:04.755557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332735+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:55:09.332735+0000","last_peered":"2024-09-12T00:55:09.332735+0000","last_clean":"2024-09-12T00:55:09.332735+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:55:09.332735+0000","last_undegraded":"2024-09-12T00:55:09.332735+0000","last_fullsized":"2024-09-12T00:55:09.332735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.768828+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:54:04.768828+0000","last_peered":"2024-09-12T00:54:04.768828+0000","last_clean":"2024-09-12T00:54:04.768828+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:54:04.768828+0000","last_undegraded":"2024-09-12T00:54:04.768828+0000","last_fullsized":"2024-09-12T00:54:04.768828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:12.800746+0000","last_change":"2024-09-12T00:54:01.670253+0000","last_active":"2024-09-12T00:54:12.800746+0000","last_peered":"2024-09-12T00:54:12.800746+0000","last_clean":"2024-09-12T00:54:12.800746+0000","last_became_active":"2024-09-12T00:54:01.669438+0000","last_became_peered":"2024-09-12T00:54:01.669438+0000","last_unstale":"2024-09-12T00:54:12.800746+0000","last_undegraded":"2024-09-12T00:54:12.800746+0000","last_fullsized":"2024-09-12T00:54:12.800746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:46.323570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336201+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:55:09.336201+0000","last_peered":"2024-09-12T00:55:09.336201+0000","last_clean":"2024-09-12T00:55:09.336201+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:55:09.336201+0000","last_undegraded":"2024-09-12T00:55:09.336201+0000","last_fullsized":"2024-09-12T00:55:09.336201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":2,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-12T00:54:03.742667+0000","last_change":"2024-09-12T00:55:08.337602+0000","last_active":"2024-09-12T00:54:02.731909+0000","last_peered":"2024-09-12T00:54:02.731909+0000","last_clean":"2024-09-12T00:54:02.731909+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T00:54:03.742667+0000","last_undegraded":"2024-09-12T00:54:03.742667+0000","last_fullsized":"2024-09-12T00:54:03.742667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332897+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:55:09.332897+0000","last_peered":"2024-09-12T00:55:09.332897+0000","last_clean":"2024-09-12T00:55:09.332897+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:55:09.332897+0000","last_undegraded":"2024-09-12T00:55:09.332897+0000","last_fullsized":"2024-09-12T00:55:09.332897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770106+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:54:04.770106+0000","last_peered":"2024-09-12T00:54:04.770106+0000","last_clean":"2024-09-12T00:54:04.770106+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:54:04.770106+0000","last_undegraded":"2024-09-12T00:54:04.770106+0000","last_fullsized":"2024-09-12T00:54:04.770106+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333106+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:55:09.333106+0000","last_peered":"2024-09-12T00:55:09.333106+0000","last_clean":"2024-09-12T00:55:09.333106+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:55:09.333106+0000","last_undegraded":"2024-09-12T00:55:09.333106+0000","last_fullsized":"2024-09-12T00:55:09.333106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.327184+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:55:08.327184+0000","last_peered":"2024-09-12T00:55:08.327184+0000","last_clean":"2024-09-12T00:55:08.327184+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:55:08.327184+0000","last_undegraded":"2024-09-12T00:55:08.327184+0000","last_fullsized":"2024-09-12T00:55:08.327184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.770949+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:54:04.770949+0000","last_peered":"2024-09-12T00:54:04.770949+0000","last_clean":"2024-09-12T00:54:04.770949+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:54:04.770949+0000","last_undegraded":"2024-09-12T00:54:04.770949+0000","last_fullsized":"2024-09-12T00:54:04.770949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T00:54:04.769067+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:54:04.769067+0000","last_peered":"2024-09-12T00:54:04.769067+0000","last_clean":"2024-09-12T00:54:04.769067+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:54:04.769067+0000","last_undegraded":"2024-09-12T00:54:04.769067+0000","last_fullsized":"2024-09-12T00:54:04.769067+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T00:55:08.320843+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:55:08.320843+0000","last_peered":"2024-09-12T00:55:08.320843+0000","last_clean":"2024-09-12T00:55:08.320843+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:55:08.320843+0000","last_undegraded":"2024-09-12T00:55:08.320843+0000","last_fullsized":"2024-09-12T00:55:08.320843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66419,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":566,"num_read_kb":351,"num_write":239,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":199257,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214598,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":427530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1077770,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":427067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":444555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":991232,"data_stored":1037075,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":443894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":1027909,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":435697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":426609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":445420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":486908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":428516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":60304,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:55:13.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:55:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:14.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:13 smithi016 ceph-mon[30972]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T00:55:14.062 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:13 smithi016 ceph-mon[30972]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 -- 172.21.15.16:0/1590352667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102790 msgr2=0x7f5e901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 --2- 172.21.15.16:0/1590352667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102790 0x7f5e901067e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009900 tx=0x7f5e8002a410 comp rx=0 tx=0).stop 2024-09-12T00:55:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 -- 172.21.15.16:0/1590352667 shutdown_connections 2024-09-12T00:55:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 --2- 172.21.15.16:0/1590352667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90106f10 0x7f5e9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 --2- 172.21.15.16:0/1590352667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90102790 0x7f5e901067e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 --2- 172.21.15.16:0/1590352667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 0x7f5e901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 -- 172.21.15.16:0/1590352667 >> 172.21.15.16:0/1590352667 conn(0x7f5e900fd8c0 msgr2=0x7f5e900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 -- 172.21.15.16:0/1590352667 shutdown_connections 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.145+0000 7f5e97f77640 1 -- 172.21.15.16:0/1590352667 wait complete. 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 Processor -- start 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 -- start start 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 0x7f5e90198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90102790 0x7f5e901989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90111f90 con 0x7f5e90106f10 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90111e10 con 0x7f5e90101de0 2024-09-12T00:55:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e97f77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90112110 con 0x7f5e90102790 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e96514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e96514640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57140/0 (socket says 172.21.15.16:57140) 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e96514640 1 -- 172.21.15.16:0/3905485606 learned_addr learned my addr 172.21.15.16:0/3905485606 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e95d13640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 0x7f5e90198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.146+0000 7f5e95512640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90102790 0x7f5e901989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 -- 172.21.15.16:0/3905485606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90102790 msgr2=0x7f5e901989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90102790 0x7f5e901989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 -- 172.21.15.16:0/3905485606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 msgr2=0x7f5e90198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 0x7f5e90198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 -- 172.21.15.16:0/3905485606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9019d420 con 0x7f5e90106f10 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e96514640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800b970 tx=0x7f5e8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:14.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88019070 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9019d710 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e8800cb40 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88007610 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.147+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e901a4ff0 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.148+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e901021c0 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.149+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e8801c070 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.150+0000 7f5e86ffd640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 0x7f5e7c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.150+0000 7f5e95d13640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 0x7f5e7c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.150+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f5e880ce620 con 0x7f5e90106f10 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.150+0000 7f5e95d13640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 0x7f5e7c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00bbe0 tx=0x7f5e8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.154+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e8809aef0 con 0x7f5e90106f10 2024-09-12T00:55:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:13 smithi099 ceph-mon[35654]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T00:55:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:13 smithi099 ceph-mon[35654]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.326+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e9010a160 con 0x7f5e7c0a2c10 2024-09-12T00:55:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.333+0000 7f5e86ffd640 1 -- 172.21.15.16:0/3905485606 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f5e8c010c30 con 0x7f5e7c0a2c10 2024-09-12T00:55:14.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:55:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 msgr2=0x7f5e7c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 0x7f5e7c0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c00bbe0 tx=0x7f5e8c00b390 comp rx=0 tx=0).stop 2024-09-12T00:55:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 msgr2=0x7f5e9019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800b970 tx=0x7f5e8800be30 comp rx=0 tx=0).stop 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 shutdown_connections 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5e7c0a2c10 0x7f5e7c0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e90106f10 0x7f5e9019cd40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e90102790 0x7f5e901989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 --2- 172.21.15.16:0/3905485606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e90101de0 0x7f5e90198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 >> 172.21.15.16:0/3905485606 conn(0x7f5e900fd8c0 msgr2=0x7f5e900fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 shutdown_connections 2024-09-12T00:55:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:14.337+0000 7f5e97f77640 1 -- 172.21.15.16:0/3905485606 wait complete. 2024-09-12T00:55:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:13 smithi159 ceph-mon[35566]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T00:55:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:13 smithi159 ceph-mon[35566]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:15.057 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-12T00:55:14.225340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657021,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":717,"num_read_kb":484,"num_write":422,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373036,"kb_used_data":11236,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568788,"statfs":{"total":1151940427776,"available":1151558438912,"internally_reserved":0,"allocated":11505664,"data_stored":7211610,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":39,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002121"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.335291+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:55:09.335291+0000","last_peered":"2024-09-12T00:55:09.335291+0000","last_clean":"2024-09-12T00:55:09.335291+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:55:09.335291+0000","last_undegraded":"2024-09-12T00:55:09.335291+0000","last_fullsized":"2024-09-12T00:55:09.335291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852819+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:55:09.852819+0000","last_peered":"2024-09-12T00:55:09.852819+0000","last_clean":"2024-09-12T00:55:09.852819+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:55:09.852819+0000","last_undegraded":"2024-09-12T00:55:09.852819+0000","last_fullsized":"2024-09-12T00:55:09.852819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852717+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:55:09.852717+0000","last_peered":"2024-09-12T00:55:09.852717+0000","last_clean":"2024-09-12T00:55:09.852717+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:55:09.852717+0000","last_undegraded":"2024-09-12T00:55:09.852717+0000","last_fullsized":"2024-09-12T00:55:09.852717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903828+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:55:09.903828+0000","last_peered":"2024-09-12T00:55:09.903828+0000","last_clean":"2024-09-12T00:55:09.903828+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:55:09.903828+0000","last_undegraded":"2024-09-12T00:55:09.903828+0000","last_fullsized":"2024-09-12T00:55:09.903828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332440+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:55:09.332440+0000","last_peered":"2024-09-12T00:55:09.332440+0000","last_clean":"2024-09-12T00:55:09.332440+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:55:09.332440+0000","last_undegraded":"2024-09-12T00:55:09.332440+0000","last_fullsized":"2024-09-12T00:55:09.332440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.328324+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:55:09.328324+0000","last_peered":"2024-09-12T00:55:09.328324+0000","last_clean":"2024-09-12T00:55:09.328324+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:55:09.328324+0000","last_undegraded":"2024-09-12T00:55:09.328324+0000","last_fullsized":"2024-09-12T00:55:09.328324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336115+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:55:09.336115+0000","last_peered":"2024-09-12T00:55:09.336115+0000","last_clean":"2024-09-12T00:55:09.336115+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:55:09.336115+0000","last_undegraded":"2024-09-12T00:55:09.336115+0000","last_fullsized":"2024-09-12T00:55:09.336115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.334005+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:55:09.334005+0000","last_peered":"2024-09-12T00:55:09.334005+0000","last_clean":"2024-09-12T00:55:09.334005+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:55:09.334005+0000","last_undegraded":"2024-09-12T00:55:09.334005+0000","last_fullsized":"2024-09-12T00:55:09.334005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331285+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:55:09.331285+0000","last_peered":"2024-09-12T00:55:09.331285+0000","last_clean":"2024-09-12T00:55:09.331285+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:55:09.331285+0000","last_undegraded":"2024-09-12T00:55:09.331285+0000","last_fullsized":"2024-09-12T00:55:09.331285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.328907+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:55:09.328907+0000","last_peered":"2024-09-12T00:55:09.328907+0000","last_clean":"2024-09-12T00:55:09.328907+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:55:09.328907+0000","last_undegraded":"2024-09-12T00:55:09.328907+0000","last_fullsized":"2024-09-12T00:55:09.328907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336529+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:55:09.336529+0000","last_peered":"2024-09-12T00:55:09.336529+0000","last_clean":"2024-09-12T00:55:09.336529+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:55:09.336529+0000","last_undegraded":"2024-09-12T00:55:09.336529+0000","last_fullsized":"2024-09-12T00:55:09.336529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903741+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:55:09.903741+0000","last_peered":"2024-09-12T00:55:09.903741+0000","last_clean":"2024-09-12T00:55:09.903741+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:55:09.903741+0000","last_undegraded":"2024-09-12T00:55:09.903741+0000","last_fullsized":"2024-09-12T00:55:09.903741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327339+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:55:09.327339+0000","last_peered":"2024-09-12T00:55:09.327339+0000","last_clean":"2024-09-12T00:55:09.327339+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:55:09.327339+0000","last_undegraded":"2024-09-12T00:55:09.327339+0000","last_fullsized":"2024-09-12T00:55:09.327339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852645+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:55:09.852645+0000","last_peered":"2024-09-12T00:55:09.852645+0000","last_clean":"2024-09-12T00:55:09.852645+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:55:09.852645+0000","last_undegraded":"2024-09-12T00:55:09.852645+0000","last_fullsized":"2024-09-12T00:55:09.852645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852605+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:55:09.852605+0000","last_peered":"2024-09-12T00:55:09.852605+0000","last_clean":"2024-09-12T00:55:09.852605+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:55:09.852605+0000","last_undegraded":"2024-09-12T00:55:09.852605+0000","last_fullsized":"2024-09-12T00:55:09.852605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330864+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:55:09.330864+0000","last_peered":"2024-09-12T00:55:09.330864+0000","last_clean":"2024-09-12T00:55:09.330864+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:55:09.330864+0000","last_undegraded":"2024-09-12T00:55:09.330864+0000","last_fullsized":"2024-09-12T00:55:09.330864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.335393+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:55:09.335393+0000","last_peered":"2024-09-12T00:55:09.335393+0000","last_clean":"2024-09-12T00:55:09.335393+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:55:09.335393+0000","last_undegraded":"2024-09-12T00:55:09.335393+0000","last_fullsized":"2024-09-12T00:55:09.335393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.329703+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:55:09.329703+0000","last_peered":"2024-09-12T00:55:09.329703+0000","last_clean":"2024-09-12T00:55:09.329703+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:55:09.329703+0000","last_undegraded":"2024-09-12T00:55:09.329703+0000","last_fullsized":"2024-09-12T00:55:09.329703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333047+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:55:09.333047+0000","last_peered":"2024-09-12T00:55:09.333047+0000","last_clean":"2024-09-12T00:55:09.333047+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:55:09.333047+0000","last_undegraded":"2024-09-12T00:55:09.333047+0000","last_fullsized":"2024-09-12T00:55:09.333047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.334973+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:55:09.334973+0000","last_peered":"2024-09-12T00:55:09.334973+0000","last_clean":"2024-09-12T00:55:09.334973+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:55:09.334973+0000","last_undegraded":"2024-09-12T00:55:09.334973+0000","last_fullsized":"2024-09-12T00:55:09.334973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327337+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:55:09.327337+0000","last_peered":"2024-09-12T00:55:09.327337+0000","last_clean":"2024-09-12T00:55:09.327337+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:55:09.327337+0000","last_undegraded":"2024-09-12T00:55:09.327337+0000","last_fullsized":"2024-09-12T00:55:09.327337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331075+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:55:09.331075+0000","last_peered":"2024-09-12T00:55:09.331075+0000","last_clean":"2024-09-12T00:55:09.331075+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:55:09.331075+0000","last_undegraded":"2024-09-12T00:55:09.331075+0000","last_fullsized":"2024-09-12T00:55:09.331075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331257+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:55:09.331257+0000","last_peered":"2024-09-12T00:55:09.331257+0000","last_clean":"2024-09-12T00:55:09.331257+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:55:09.331257+0000","last_undegraded":"2024-09-12T00:55:09.331257+0000","last_fullsized":"2024-09-12T00:55:09.331257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330690+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:55:09.330690+0000","last_peered":"2024-09-12T00:55:09.330690+0000","last_clean":"2024-09-12T00:55:09.330690+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:55:09.330690+0000","last_undegraded":"2024-09-12T00:55:09.330690+0000","last_fullsized":"2024-09-12T00:55:09.330690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331657+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:55:09.331657+0000","last_peered":"2024-09-12T00:55:09.331657+0000","last_clean":"2024-09-12T00:55:09.331657+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:55:09.331657+0000","last_undegraded":"2024-09-12T00:55:09.331657+0000","last_fullsized":"2024-09-12T00:55:09.331657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333367+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:55:09.333367+0000","last_peered":"2024-09-12T00:55:09.333367+0000","last_clean":"2024-09-12T00:55:09.333367+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:55:09.333367+0000","last_undegraded":"2024-09-12T00:55:09.333367+0000","last_fullsized":"2024-09-12T00:55:09.333367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.335196+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:55:09.335196+0000","last_peered":"2024-09-12T00:55:09.335196+0000","last_clean":"2024-09-12T00:55:09.335196+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:55:09.335196+0000","last_undegraded":"2024-09-12T00:55:09.335196+0000","last_fullsized":"2024-09-12T00:55:09.335196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327315+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:55:09.327315+0000","last_peered":"2024-09-12T00:55:09.327315+0000","last_clean":"2024-09-12T00:55:09.327315+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:55:09.327315+0000","last_undegraded":"2024-09-12T00:55:09.327315+0000","last_fullsized":"2024-09-12T00:55:09.327315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852974+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:55:09.852974+0000","last_peered":"2024-09-12T00:55:09.852974+0000","last_clean":"2024-09-12T00:55:09.852974+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:55:09.852974+0000","last_undegraded":"2024-09-12T00:55:09.852974+0000","last_fullsized":"2024-09-12T00:55:09.852974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332630+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:55:09.332630+0000","last_peered":"2024-09-12T00:55:09.332630+0000","last_clean":"2024-09-12T00:55:09.332630+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:55:09.332630+0000","last_undegraded":"2024-09-12T00:55:09.332630+0000","last_fullsized":"2024-09-12T00:55:09.332630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332505+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:55:09.332505+0000","last_peered":"2024-09-12T00:55:09.332505+0000","last_clean":"2024-09-12T00:55:09.332505+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:55:09.332505+0000","last_undegraded":"2024-09-12T00:55:09.332505+0000","last_fullsized":"2024-09-12T00:55:09.332505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.334097+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:55:09.334097+0000","last_peered":"2024-09-12T00:55:09.334097+0000","last_clean":"2024-09-12T00:55:09.334097+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:55:09.334097+0000","last_undegraded":"2024-09-12T00:55:09.334097+0000","last_fullsized":"2024-09-12T00:55:09.334097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332270+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:55:09.332270+0000","last_peered":"2024-09-12T00:55:09.332270+0000","last_clean":"2024-09-12T00:55:09.332270+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:55:09.332270+0000","last_undegraded":"2024-09-12T00:55:09.332270+0000","last_fullsized":"2024-09-12T00:55:09.332270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330666+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:55:09.330666+0000","last_peered":"2024-09-12T00:55:09.330666+0000","last_clean":"2024-09-12T00:55:09.330666+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:55:09.330666+0000","last_undegraded":"2024-09-12T00:55:09.330666+0000","last_fullsized":"2024-09-12T00:55:09.330666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330552+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:55:09.330552+0000","last_peered":"2024-09-12T00:55:09.330552+0000","last_clean":"2024-09-12T00:55:09.330552+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:55:09.330552+0000","last_undegraded":"2024-09-12T00:55:09.330552+0000","last_fullsized":"2024-09-12T00:55:09.330552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.328419+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:55:09.328419+0000","last_peered":"2024-09-12T00:55:09.328419+0000","last_clean":"2024-09-12T00:55:09.328419+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:55:09.328419+0000","last_undegraded":"2024-09-12T00:55:09.328419+0000","last_fullsized":"2024-09-12T00:55:09.328419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327066+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:55:09.327066+0000","last_peered":"2024-09-12T00:55:09.327066+0000","last_clean":"2024-09-12T00:55:09.327066+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:55:09.327066+0000","last_undegraded":"2024-09-12T00:55:09.327066+0000","last_fullsized":"2024-09-12T00:55:09.327066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331257+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:55:09.331257+0000","last_peered":"2024-09-12T00:55:09.331257+0000","last_clean":"2024-09-12T00:55:09.331257+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:55:09.331257+0000","last_undegraded":"2024-09-12T00:55:09.331257+0000","last_fullsized":"2024-09-12T00:55:09.331257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330996+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:55:09.330996+0000","last_peered":"2024-09-12T00:55:09.330996+0000","last_clean":"2024-09-12T00:55:09.330996+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:55:09.330996+0000","last_undegraded":"2024-09-12T00:55:09.330996+0000","last_fullsized":"2024-09-12T00:55:09.330996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.329094+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:55:09.329094+0000","last_peered":"2024-09-12T00:55:09.329094+0000","last_clean":"2024-09-12T00:55:09.329094+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:55:09.329094+0000","last_undegraded":"2024-09-12T00:55:09.329094+0000","last_fullsized":"2024-09-12T00:55:09.329094+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:11.900266+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:55:11.900266+0000","last_peered":"2024-09-12T00:55:11.900266+0000","last_clean":"2024-09-12T00:55:11.900266+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:55:11.900266+0000","last_undegraded":"2024-09-12T00:55:11.900266+0000","last_fullsized":"2024-09-12T00:55:11.900266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331184+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:55:09.331184+0000","last_peered":"2024-09-12T00:55:09.331184+0000","last_clean":"2024-09-12T00:55:09.331184+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:55:09.331184+0000","last_undegraded":"2024-09-12T00:55:09.331184+0000","last_fullsized":"2024-09-12T00:55:09.331184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336488+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:55:09.336488+0000","last_peered":"2024-09-12T00:55:09.336488+0000","last_clean":"2024-09-12T00:55:09.336488+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:55:09.336488+0000","last_undegraded":"2024-09-12T00:55:09.336488+0000","last_fullsized":"2024-09-12T00:55:09.336488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.853030+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:55:09.853030+0000","last_peered":"2024-09-12T00:55:09.853030+0000","last_clean":"2024-09-12T00:55:09.853030+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:55:09.853030+0000","last_undegraded":"2024-09-12T00:55:09.853030+0000","last_fullsized":"2024-09-12T00:55:09.853030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331035+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:55:09.331035+0000","last_peered":"2024-09-12T00:55:09.331035+0000","last_clean":"2024-09-12T00:55:09.331035+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:55:09.331035+0000","last_undegraded":"2024-09-12T00:55:09.331035+0000","last_fullsized":"2024-09-12T00:55:09.331035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331083+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:55:09.331083+0000","last_peered":"2024-09-12T00:55:09.331083+0000","last_clean":"2024-09-12T00:55:09.331083+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:55:09.331083+0000","last_undegraded":"2024-09-12T00:55:09.331083+0000","last_fullsized":"2024-09-12T00:55:09.331083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":187,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:11.910208+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:55:11.910208+0000","last_peered":"2024-09-12T00:55:11.910208+0000","last_clean":"2024-09-12T00:55:11.910208+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:55:11.910208+0000","last_undegraded":"2024-09-12T00:55:11.910208+0000","last_fullsized":"2024-09-12T00:55:11.910208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":112,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332899+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:55:09.332899+0000","last_peered":"2024-09-12T00:55:09.332899+0000","last_clean":"2024-09-12T00:55:09.332899+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:55:09.332899+0000","last_undegraded":"2024-09-12T00:55:09.332899+0000","last_fullsized":"2024-09-12T00:55:09.332899+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333258+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:55:09.333258+0000","last_peered":"2024-09-12T00:55:09.333258+0000","last_clean":"2024-09-12T00:55:09.333258+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:55:09.333258+0000","last_undegraded":"2024-09-12T00:55:09.333258+0000","last_fullsized":"2024-09-12T00:55:09.333258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327141+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:55:09.327141+0000","last_peered":"2024-09-12T00:55:09.327141+0000","last_clean":"2024-09-12T00:55:09.327141+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:55:09.327141+0000","last_undegraded":"2024-09-12T00:55:09.327141+0000","last_fullsized":"2024-09-12T00:55:09.327141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331294+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:55:09.331294+0000","last_peered":"2024-09-12T00:55:09.331294+0000","last_clean":"2024-09-12T00:55:09.331294+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:55:09.331294+0000","last_undegraded":"2024-09-12T00:55:09.331294+0000","last_fullsized":"2024-09-12T00:55:09.331294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336386+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:55:09.336386+0000","last_peered":"2024-09-12T00:55:09.336386+0000","last_clean":"2024-09-12T00:55:09.336386+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:55:09.336386+0000","last_undegraded":"2024-09-12T00:55:09.336386+0000","last_fullsized":"2024-09-12T00:55:09.336386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'6","reported_seq":85,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.902902+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:55:09.902902+0000","last_peered":"2024-09-12T00:55:09.902902+0000","last_clean":"2024-09-12T00:55:09.902902+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:55:09.902902+0000","last_undegraded":"2024-09-12T00:55:09.902902+0000","last_fullsized":"2024-09-12T00:55:09.902902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336035+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:55:09.336035+0000","last_peered":"2024-09-12T00:55:09.336035+0000","last_clean":"2024-09-12T00:55:09.336035+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:55:09.336035+0000","last_undegraded":"2024-09-12T00:55:09.336035+0000","last_fullsized":"2024-09-12T00:55:09.336035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331724+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:55:09.331724+0000","last_peered":"2024-09-12T00:55:09.331724+0000","last_clean":"2024-09-12T00:55:09.331724+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:55:09.331724+0000","last_undegraded":"2024-09-12T00:55:09.331724+0000","last_fullsized":"2024-09-12T00:55:09.331724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.335750+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:55:09.335750+0000","last_peered":"2024-09-12T00:55:09.335750+0000","last_clean":"2024-09-12T00:55:09.335750+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:55:09.335750+0000","last_undegraded":"2024-09-12T00:55:09.335750+0000","last_fullsized":"2024-09-12T00:55:09.335750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333036+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:55:09.333036+0000","last_peered":"2024-09-12T00:55:09.333036+0000","last_clean":"2024-09-12T00:55:09.333036+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:55:09.333036+0000","last_undegraded":"2024-09-12T00:55:09.333036+0000","last_fullsized":"2024-09-12T00:55:09.333036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.337263+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:55:09.337263+0000","last_peered":"2024-09-12T00:55:09.337263+0000","last_clean":"2024-09-12T00:55:09.337263+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:55:09.337263+0000","last_undegraded":"2024-09-12T00:55:09.337263+0000","last_fullsized":"2024-09-12T00:55:09.337263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327476+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:55:09.327476+0000","last_peered":"2024-09-12T00:55:09.327476+0000","last_clean":"2024-09-12T00:55:09.327476+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:55:09.327476+0000","last_undegraded":"2024-09-12T00:55:09.327476+0000","last_fullsized":"2024-09-12T00:55:09.327476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336543+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:55:09.336543+0000","last_peered":"2024-09-12T00:55:09.336543+0000","last_clean":"2024-09-12T00:55:09.336543+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:55:09.336543+0000","last_undegraded":"2024-09-12T00:55:09.336543+0000","last_fullsized":"2024-09-12T00:55:09.336543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331655+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:55:09.331655+0000","last_peered":"2024-09-12T00:55:09.331655+0000","last_clean":"2024-09-12T00:55:09.331655+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:55:09.331655+0000","last_undegraded":"2024-09-12T00:55:09.331655+0000","last_fullsized":"2024-09-12T00:55:09.331655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336441+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:55:09.336441+0000","last_peered":"2024-09-12T00:55:09.336441+0000","last_clean":"2024-09-12T00:55:09.336441+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:55:09.336441+0000","last_undegraded":"2024-09-12T00:55:09.336441+0000","last_fullsized":"2024-09-12T00:55:09.336441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'6","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:10.902872+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:55:10.902872+0000","last_peered":"2024-09-12T00:55:10.902872+0000","last_clean":"2024-09-12T00:55:10.902872+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:55:10.902872+0000","last_undegraded":"2024-09-12T00:55:10.902872+0000","last_fullsized":"2024-09-12T00:55:10.902872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336672+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:55:09.336672+0000","last_peered":"2024-09-12T00:55:09.336672+0000","last_clean":"2024-09-12T00:55:09.336672+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:55:09.336672+0000","last_undegraded":"2024-09-12T00:55:09.336672+0000","last_fullsized":"2024-09-12T00:55:09.336672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336173+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:55:09.336173+0000","last_peered":"2024-09-12T00:55:09.336173+0000","last_clean":"2024-09-12T00:55:09.336173+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:55:09.336173+0000","last_undegraded":"2024-09-12T00:55:09.336173+0000","last_fullsized":"2024-09-12T00:55:09.336173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903073+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:55:09.903073+0000","last_peered":"2024-09-12T00:55:09.903073+0000","last_clean":"2024-09-12T00:55:09.903073+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:55:09.903073+0000","last_undegraded":"2024-09-12T00:55:09.903073+0000","last_fullsized":"2024-09-12T00:55:09.903073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331402+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:55:09.331402+0000","last_peered":"2024-09-12T00:55:09.331402+0000","last_clean":"2024-09-12T00:55:09.331402+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:55:09.331402+0000","last_undegraded":"2024-09-12T00:55:09.331402+0000","last_fullsized":"2024-09-12T00:55:09.331402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336181+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:55:09.336181+0000","last_peered":"2024-09-12T00:55:09.336181+0000","last_clean":"2024-09-12T00:55:09.336181+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:55:09.336181+0000","last_undegraded":"2024-09-12T00:55:09.336181+0000","last_fullsized":"2024-09-12T00:55:09.336181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327057+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:55:09.327057+0000","last_peered":"2024-09-12T00:55:09.327057+0000","last_clean":"2024-09-12T00:55:09.327057+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:55:09.327057+0000","last_undegraded":"2024-09-12T00:55:09.327057+0000","last_fullsized":"2024-09-12T00:55:09.327057+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327276+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:55:09.327276+0000","last_peered":"2024-09-12T00:55:09.327276+0000","last_clean":"2024-09-12T00:55:09.327276+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:55:09.327276+0000","last_undegraded":"2024-09-12T00:55:09.327276+0000","last_fullsized":"2024-09-12T00:55:09.327276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330809+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:55:09.330809+0000","last_peered":"2024-09-12T00:55:09.330809+0000","last_clean":"2024-09-12T00:55:09.330809+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:55:09.330809+0000","last_undegraded":"2024-09-12T00:55:09.330809+0000","last_fullsized":"2024-09-12T00:55:09.330809+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336329+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:55:09.336329+0000","last_peered":"2024-09-12T00:55:09.336329+0000","last_clean":"2024-09-12T00:55:09.336329+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:55:09.336329+0000","last_undegraded":"2024-09-12T00:55:09.336329+0000","last_fullsized":"2024-09-12T00:55:09.336329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.853667+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:55:09.853667+0000","last_peered":"2024-09-12T00:55:09.853667+0000","last_clean":"2024-09-12T00:55:09.853667+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:55:09.853667+0000","last_undegraded":"2024-09-12T00:55:09.853667+0000","last_fullsized":"2024-09-12T00:55:09.853667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.329219+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:55:09.329219+0000","last_peered":"2024-09-12T00:55:09.329219+0000","last_clean":"2024-09-12T00:55:09.329219+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:55:09.329219+0000","last_undegraded":"2024-09-12T00:55:09.329219+0000","last_fullsized":"2024-09-12T00:55:09.329219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.328620+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:55:09.328620+0000","last_peered":"2024-09-12T00:55:09.328620+0000","last_clean":"2024-09-12T00:55:09.328620+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:55:09.328620+0000","last_undegraded":"2024-09-12T00:55:09.328620+0000","last_fullsized":"2024-09-12T00:55:09.328620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333224+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:55:09.333224+0000","last_peered":"2024-09-12T00:55:09.333224+0000","last_clean":"2024-09-12T00:55:09.333224+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:55:09.333224+0000","last_undegraded":"2024-09-12T00:55:09.333224+0000","last_fullsized":"2024-09-12T00:55:09.333224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.327341+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:55:09.327341+0000","last_peered":"2024-09-12T00:55:09.327341+0000","last_clean":"2024-09-12T00:55:09.327341+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:55:09.327341+0000","last_undegraded":"2024-09-12T00:55:09.327341+0000","last_fullsized":"2024-09-12T00:55:09.327341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'14","reported_seq":52,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.342679+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:55:09.342679+0000","last_peered":"2024-09-12T00:55:09.342679+0000","last_clean":"2024-09-12T00:55:09.342679+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:55:09.342679+0000","last_undegraded":"2024-09-12T00:55:09.342679+0000","last_fullsized":"2024-09-12T00:55:09.342679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331695+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:55:09.331695+0000","last_peered":"2024-09-12T00:55:09.331695+0000","last_clean":"2024-09-12T00:55:09.331695+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:55:09.331695+0000","last_undegraded":"2024-09-12T00:55:09.331695+0000","last_fullsized":"2024-09-12T00:55:09.331695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903305+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:55:09.903305+0000","last_peered":"2024-09-12T00:55:09.903305+0000","last_clean":"2024-09-12T00:55:09.903305+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:55:09.903305+0000","last_undegraded":"2024-09-12T00:55:09.903305+0000","last_fullsized":"2024-09-12T00:55:09.903305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.852824+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:55:09.852824+0000","last_peered":"2024-09-12T00:55:09.852824+0000","last_clean":"2024-09-12T00:55:09.852824+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:55:09.852824+0000","last_undegraded":"2024-09-12T00:55:09.852824+0000","last_fullsized":"2024-09-12T00:55:09.852824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330961+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:55:09.330961+0000","last_peered":"2024-09-12T00:55:09.330961+0000","last_clean":"2024-09-12T00:55:09.330961+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:55:09.330961+0000","last_undegraded":"2024-09-12T00:55:09.330961+0000","last_fullsized":"2024-09-12T00:55:09.330961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332584+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:55:09.332584+0000","last_peered":"2024-09-12T00:55:09.332584+0000","last_clean":"2024-09-12T00:55:09.332584+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:55:09.332584+0000","last_undegraded":"2024-09-12T00:55:09.332584+0000","last_fullsized":"2024-09-12T00:55:09.332584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331401+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:55:09.331401+0000","last_peered":"2024-09-12T00:55:09.331401+0000","last_clean":"2024-09-12T00:55:09.331401+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:55:09.331401+0000","last_undegraded":"2024-09-12T00:55:09.331401+0000","last_fullsized":"2024-09-12T00:55:09.331401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.334763+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:55:09.334763+0000","last_peered":"2024-09-12T00:55:09.334763+0000","last_clean":"2024-09-12T00:55:09.334763+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:55:09.334763+0000","last_undegraded":"2024-09-12T00:55:09.334763+0000","last_fullsized":"2024-09-12T00:55:09.334763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332735+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:55:09.332735+0000","last_peered":"2024-09-12T00:55:09.332735+0000","last_clean":"2024-09-12T00:55:09.332735+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:55:09.332735+0000","last_undegraded":"2024-09-12T00:55:09.332735+0000","last_fullsized":"2024-09-12T00:55:09.332735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331415+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:55:09.331415+0000","last_peered":"2024-09-12T00:55:09.331415+0000","last_clean":"2024-09-12T00:55:09.331415+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:55:09.331415+0000","last_undegraded":"2024-09-12T00:55:09.331415+0000","last_fullsized":"2024-09-12T00:55:09.331415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.343499+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:55:09.343499+0000","last_peered":"2024-09-12T00:55:09.343499+0000","last_clean":"2024-09-12T00:55:09.343499+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:55:09.343499+0000","last_undegraded":"2024-09-12T00:55:09.343499+0000","last_fullsized":"2024-09-12T00:55:09.343499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.336201+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:55:09.336201+0000","last_peered":"2024-09-12T00:55:09.336201+0000","last_clean":"2024-09-12T00:55:09.336201+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:55:09.336201+0000","last_undegraded":"2024-09-12T00:55:09.336201+0000","last_fullsized":"2024-09-12T00:55:09.336201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.903676+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:55:09.903676+0000","last_peered":"2024-09-12T00:55:09.903676+0000","last_clean":"2024-09-12T00:55:09.903676+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:55:09.903676+0000","last_undegraded":"2024-09-12T00:55:09.903676+0000","last_fullsized":"2024-09-12T00:55:09.903676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.332897+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:55:09.332897+0000","last_peered":"2024-09-12T00:55:09.332897+0000","last_clean":"2024-09-12T00:55:09.332897+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:55:09.332897+0000","last_undegraded":"2024-09-12T00:55:09.332897+0000","last_fullsized":"2024-09-12T00:55:09.332897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.330927+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:55:09.330927+0000","last_peered":"2024-09-12T00:55:09.330927+0000","last_clean":"2024-09-12T00:55:09.330927+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:55:09.330927+0000","last_undegraded":"2024-09-12T00:55:09.330927+0000","last_fullsized":"2024-09-12T00:55:09.330927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.333106+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:55:09.333106+0000","last_peered":"2024-09-12T00:55:09.333106+0000","last_clean":"2024-09-12T00:55:09.333106+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:55:09.333106+0000","last_undegraded":"2024-09-12T00:55:09.333106+0000","last_fullsized":"2024-09-12T00:55:09.333106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.853029+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:55:09.853029+0000","last_peered":"2024-09-12T00:55:09.853029+0000","last_clean":"2024-09-12T00:55:09.853029+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:55:09.853029+0000","last_undegraded":"2024-09-12T00:55:09.853029+0000","last_fullsized":"2024-09-12T00:55:09.853029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331786+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:55:09.331786+0000","last_peered":"2024-09-12T00:55:09.331786+0000","last_clean":"2024-09-12T00:55:09.331786+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:55:09.331786+0000","last_undegraded":"2024-09-12T00:55:09.331786+0000","last_fullsized":"2024-09-12T00:55:09.331786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:09.331681+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:55:09.331681+0000","last_peered":"2024-09-12T00:55:09.331681+0000","last_clean":"2024-09-12T00:55:09.331681+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:55:09.331681+0000","last_undegraded":"2024-09-12T00:55:09.331681+0000","last_fullsized":"2024-09-12T00:55:09.331681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":93,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T00:55:11.899560+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:55:11.899560+0000","last_peered":"2024-09-12T00:55:11.899560+0000","last_clean":"2024-09-12T00:55:11.899560+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:55:11.899560+0000","last_undegraded":"2024-09-12T00:55:11.899560+0000","last_fullsized":"2024-09-12T00:55:11.899560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66419,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":581,"num_read_kb":365,"num_write":239,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":199257,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":3542208,"data_compressed":214598,"data_compressed_allocated":270336,"data_compressed_original":3506176,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514524,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":917504,"data_stored":444640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1073152,"data_stored":1094880,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939686,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":942080,"data_stored":444177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":444555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":991232,"data_stored":1037075,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":443894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":1027909,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":435697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":443719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":937984,"data_stored":445420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":987136,"data_stored":504018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":445626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34675,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":60304,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:55:15.061 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T00:55:15.061 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T00:55:15.061 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T00:55:15.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T00:55:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:15 smithi099 ceph-mon[35654]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-12T00:55:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:15 smithi099 ceph-mon[35654]: Cluster is now healthy 2024-09-12T00:55:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:15 smithi159 ceph-mon[35566]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-12T00:55:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:15 smithi159 ceph-mon[35566]: Cluster is now healthy 2024-09-12T00:55:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:15 smithi016 ceph-mon[30972]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-12T00:55:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:15 smithi016 ceph-mon[30972]: Cluster is now healthy 2024-09-12T00:55:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1461705382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc103700 msgr2=0x7fb7bc10bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1461705382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc103700 0x7fb7bc10bbf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009900 tx=0x7fb7b002a480 comp rx=0 tx=0).stop 2024-09-12T00:55:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1461705382 shutdown_connections 2024-09-12T00:55:15.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1461705382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 0x7fb7bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1461705382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc103700 0x7fb7bc10bbf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1461705382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1461705382 >> 172.21.15.16:0/1461705382 conn(0x7fb7bc0fc7a0 msgr2=0x7fb7bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1461705382 shutdown_connections 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.820+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1461705382 wait complete. 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 Processor -- start 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 -- start start 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 0x7fb7bc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:15.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 0x7fb7bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc111f30 con 0x7fb7bc102de0 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc111db0 con 0x7fb7bc10c180 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7c1b1e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1120b0 con 0x7fb7bc103700 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57168/0 (socket says 172.21.15.16:57168) 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/1163613190 learned_addr learned my addr 172.21.15.16:0/1163613190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.821+0000 7fb7baffd640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 0x7fb7bc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bbfff640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 0x7fb7bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/1163613190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 msgr2=0x7fb7bc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 0x7fb7bc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:15.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/1163613190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 msgr2=0x7fb7bc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 0x7fb7bc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 -- 172.21.15.16:0/1163613190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19d3a0 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7baffd640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 0x7fb7bc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7bb7fe640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c920 tx=0x7fb7ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac007c70 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac007e10 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac018890 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc19d690 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.822+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a4f70 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.823+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc1031c0 con 0x7fb7bc102de0 2024-09-12T00:55:15.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.824+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7ac018a70 con 0x7fb7bc102de0 2024-09-12T00:55:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.825+0000 7fb7b8ff9640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 0x7fb7a40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.825+0000 7fb7baffd640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 0x7fb7a40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.825+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb7ac0cf390 con 0x7fb7bc102de0 2024-09-12T00:55:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.825+0000 7fb7baffd640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 0x7fb7a40a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0feda0 tx=0x7fb7b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:15.829+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7ac014030 con 0x7fb7bc102de0 2024-09-12T00:55:16.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.045+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb7bc107200 con 0x7fb7bc102de0 2024-09-12T00:55:16.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.046+0000 7fb7b8ff9640 1 -- 172.21.15.16:0/1163613190 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb7ac09c1a0 con 0x7fb7bc102de0 2024-09-12T00:55:16.046 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:16.046 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T00:55:16.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 msgr2=0x7fb7a40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 0x7fb7a40a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0feda0 tx=0x7fb7b0005a20 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 msgr2=0x7fb7bc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c920 tx=0x7fb7ac00cde0 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 shutdown_connections 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb7a40a2c90 0x7fb7a40a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc10c180 0x7fb7bc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7bc103700 0x7fb7bc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 --2- 172.21.15.16:0/1163613190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7bc102de0 0x7fb7bc1983f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 >> 172.21.15.16:0/1163613190 conn(0x7fb7bc0fc7a0 msgr2=0x7fb7bc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 shutdown_connections 2024-09-12T00:55:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:16.048+0000 7fb7c1b1e640 1 -- 172.21.15.16:0/1163613190 wait complete. 2024-09-12T00:55:16.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:16 smithi099 ceph-mon[35654]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:16.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:16 smithi099 ceph-mon[35654]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:16 smithi159 ceph-mon[35566]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:16 smithi159 ceph-mon[35566]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:16 smithi016 ceph-mon[30972]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:16 smithi016 ceph-mon[30972]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:55:16.758 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T00:55:16.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:55:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:17.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:17 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1163613190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T00:55:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:17 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1163613190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T00:55:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:17 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1163613190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- 172.21.15.16:0/1203307790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c106a70 msgr2=0x7f704c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 --2- 172.21.15.16:0/1203307790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c106a70 0x7f704c106e50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7048009930 tx=0x7f704802a420 comp rx=0 tx=0).stop 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- 172.21.15.16:0/1203307790 shutdown_connections 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 --2- 172.21.15.16:0/1203307790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c101410 0x7f704c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 --2- 172.21.15.16:0/1203307790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c100a70 0x7f704c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 --2- 172.21.15.16:0/1203307790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c106a70 0x7f704c106e50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- 172.21.15.16:0/1203307790 >> 172.21.15.16:0/1203307790 conn(0x7f704c0fc790 msgr2=0x7f704c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- 172.21.15.16:0/1203307790 shutdown_connections 2024-09-12T00:55:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- 172.21.15.16:0/1203307790 wait complete. 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 Processor -- start 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.495+0000 7f7053a6e640 1 -- start start 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c100a70 0x7f704c0fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 0x7f704c0fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f704c110e90 con 0x7f704c101410 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f704c110d10 con 0x7f704c106a70 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f704c111010 con 0x7f704c100a70 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57184/0 (socket says 172.21.15.16:57184) 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 -- 172.21.15.16:0/157853218 learned_addr learned my addr 172.21.15.16:0/157853218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f705200b640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 0x7f704c0fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f705180a640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c100a70 0x7f704c0fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 -- 172.21.15.16:0/157853218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c100a70 msgr2=0x7f704c0fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c100a70 0x7f704c0fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 -- 172.21.15.16:0/157853218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 msgr2=0x7f704c0fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 0x7f704c0fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 -- 172.21.15.16:0/157853218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a6920 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f705200b640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 0x7f704c0fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7051009640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4a0 tx=0x7f703c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c014070 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c010af0 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a6c10 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c017d10 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.496+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a71a0 con 0x7f704c101410 2024-09-12T00:55:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.497+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c100ed0 con 0x7f704c101410 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.499+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f703c017070 con 0x7f704c101410 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.499+0000 7f7042ffd640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 0x7f70380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.499+0000 7f705180a640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 0x7f70380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.499+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f703c0ce830 con 0x7f704c101410 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.500+0000 7f705180a640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 0x7f70380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7048000c00 tx=0x7f7048031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.503+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f703c09b750 con 0x7f704c101410 2024-09-12T00:55:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.715+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f704c0fe340 con 0x7f704c101410 2024-09-12T00:55:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.715+0000 7f7042ffd640 1 -- 172.21.15.16:0/157853218 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7f704c0fe340 con 0x7f704c101410 2024-09-12T00:55:17.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:17.715 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:55:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 msgr2=0x7f70380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 0x7f70380a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7048000c00 tx=0x7f7048031490 comp rx=0 tx=0).stop 2024-09-12T00:55:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 msgr2=0x7f704c0fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4a0 tx=0x7f703c00b960 comp rx=0 tx=0).stop 2024-09-12T00:55:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 shutdown_connections 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f70380a2c90 0x7f70380a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c106a70 0x7f704c0fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c101410 0x7f704c0fd350 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 --2- 172.21.15.16:0/157853218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f704c100a70 0x7f704c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 >> 172.21.15.16:0/157853218 conn(0x7f704c0fc790 msgr2=0x7f704c06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 shutdown_connections 2024-09-12T00:55:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:17.718+0000 7f7053a6e640 1 -- 172.21.15.16:0/157853218 wait complete. 2024-09-12T00:55:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:18 smithi099 ceph-mon[35654]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:18 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/157853218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:18 smithi159 ceph-mon[35566]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:18 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/157853218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:18.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:18 smithi016 ceph-mon[30972]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:18.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:18 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/157853218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:18.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:55:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 -- 172.21.15.16:0/1329903277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c102790 msgr2=0x7fbe5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1329903277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c102790 0x7fbe5c1067e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009900 tx=0x7fbe5002a410 comp rx=0 tx=0).stop 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 -- 172.21.15.16:0/1329903277 shutdown_connections 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1329903277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c106f10 0x7fbe5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1329903277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c102790 0x7fbe5c1067e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1329903277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 0x7fbe5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 -- 172.21.15.16:0/1329903277 >> 172.21.15.16:0/1329903277 conn(0x7fbe5c0fd8c0 msgr2=0x7fbe5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 -- 172.21.15.16:0/1329903277 shutdown_connections 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.336+0000 7fbe647c7640 1 -- 172.21.15.16:0/1329903277 wait complete. 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 Processor -- start 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 -- start start 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 0x7fbe5c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c102790 0x7fbe5c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c111f70 con 0x7fbe5c106f10 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c111df0 con 0x7fbe5c101de0 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe647c7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c1120f0 con 0x7fbe5c102790 2024-09-12T00:55:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45400/0 (socket says 172.21.15.16:45400) 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 -- 172.21.15.16:0/1180605556 learned_addr learned my addr 172.21.15.16:0/1180605556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62563640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 0x7fbe5c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe61d62640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c102790 0x7fbe5c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 -- 172.21.15.16:0/1180605556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c102790 msgr2=0x7fbe5c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c102790 0x7fbe5c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 -- 172.21.15.16:0/1180605556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 msgr2=0x7fbe5c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 0x7fbe5c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.337+0000 7fbe62d64640 1 -- 172.21.15.16:0/1180605556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c19d440 con 0x7fbe5c106f10 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe62d64640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d750 tx=0x7fbe5400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54017070 con 0x7fbe5c106f10 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe54005500 con 0x7fbe5c106f10 2024-09-12T00:55:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c19d730 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54004d90 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.338+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a5010 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.339+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c1021c0 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.340+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe54004f30 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.341+0000 7fbe5b7fe640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 0x7fbe440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.341+0000 7fbe62563640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 0x7fbe440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.341+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fbe540ceb10 con 0x7fbe5c106f10 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.341+0000 7fbe62563640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 0x7fbe440a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005fd0 tx=0x7fbe4c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.345+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe5409b920 con 0x7fbe5c106f10 2024-09-12T00:55:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.557+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe5c10a150 con 0x7fbe5c106f10 2024-09-12T00:55:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.558+0000 7fbe5b7fe640 1 -- 172.21.15.16:0/1180605556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7fbe5409b220 con 0x7fbe5c106f10 2024-09-12T00:55:19.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:19.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:55:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 msgr2=0x7fbe440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 0x7fbe440a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005fd0 tx=0x7fbe4c009680 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 msgr2=0x7fbe5c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d750 tx=0x7fbe5400dc10 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 shutdown_connections 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fbe440a2c10 0x7fbe440a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5c106f10 0x7fbe5c19cd60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe5c102790 0x7fbe5c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 --2- 172.21.15.16:0/1180605556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe5c101de0 0x7fbe5c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.560+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 >> 172.21.15.16:0/1180605556 conn(0x7fbe5c0fd8c0 msgr2=0x7fbe5c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.561+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 shutdown_connections 2024-09-12T00:55:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:19.561+0000 7fbe647c7640 1 -- 172.21.15.16:0/1180605556 wait complete. 2024-09-12T00:55:20.249 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:20 smithi016 ceph-mon[30972]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:20.250 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:20 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1180605556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:20 smithi099 ceph-mon[35654]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:20 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1180605556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:20.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:55:20.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:20 smithi159 ceph-mon[35566]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 5.8 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:20.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:20 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1180605556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 -- 172.21.15.16:0/3294624798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54106f10 msgr2=0x7fad5410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 --2- 172.21.15.16:0/3294624798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54106f10 0x7fad5410ea90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad44009ca0 tx=0x7fad4402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 -- 172.21.15.16:0/3294624798 shutdown_connections 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 --2- 172.21.15.16:0/3294624798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54106f10 0x7fad5410ea90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 --2- 172.21.15.16:0/3294624798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54102790 0x7fad541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 --2- 172.21.15.16:0/3294624798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54101de0 0x7fad541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 -- 172.21.15.16:0/3294624798 >> 172.21.15.16:0/3294624798 conn(0x7fad540fd8c0 msgr2=0x7fad540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 -- 172.21.15.16:0/3294624798 shutdown_connections 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 -- 172.21.15.16:0/3294624798 wait complete. 2024-09-12T00:55:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.052+0000 7fad598b7640 1 Processor -- start 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 -- start start 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54101de0 0x7fad541982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54106f10 0x7fad5419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad54111f70 con 0x7fad54102790 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fad54111df0 con 0x7fad54106f10 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad598b7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad541120f0 con 0x7fad54101de0 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45414/0 (socket says 172.21.15.16:45414) 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 -- 172.21.15.16:0/1584242279 learned_addr learned my addr 172.21.15.16:0/1584242279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad52ffd640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54101de0 0x7fad541982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad537fe640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54106f10 0x7fad5419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 -- 172.21.15.16:0/1584242279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54101de0 msgr2=0x7fad541982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54101de0 0x7fad541982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 -- 172.21.15.16:0/1584242279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54106f10 msgr2=0x7fad5419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54106f10 0x7fad5419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 -- 172.21.15.16:0/1584242279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5419d2a0 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.053+0000 7fad527fc640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00d770 tx=0x7fad3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.054+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c017070 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.054+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5419d590 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.054+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad3c005500 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.054+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c004e00 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.054+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541a4e70 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.055+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad541021c0 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.056+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad3c004fa0 con 0x7fad54102790 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.057+0000 7fad588b5640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 0x7fad400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:21.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.057+0000 7fad52ffd640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 0x7fad400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.057+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fad3c0cec90 con 0x7fad54102790 2024-09-12T00:55:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.057+0000 7fad52ffd640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 0x7fad400a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005e10 tx=0x7fad4c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:21.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.061+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad3c0d4020 con 0x7fad54102790 2024-09-12T00:55:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.240+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad5410a120 con 0x7fad54102790 2024-09-12T00:55:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.242+0000 7fad588b5640 1 -- 172.21.15.16:0/1584242279 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22641 (secure 0 0 0) 0x7fad3c09baa0 con 0x7fad54102790 2024-09-12T00:55:21.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:21.242 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:55:09.318256+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 msgr2=0x7fad400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 0x7fad400a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005e10 tx=0x7fad4c005da0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 msgr2=0x7fad54198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00d770 tx=0x7fad3c00dc30 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 shutdown_connections 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fad400a2c10 0x7fad400a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fad54106f10 0x7fad5419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad54102790 0x7fad54198830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 --2- 172.21.15.16:0/1584242279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad54101de0 0x7fad541982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 >> 172.21.15.16:0/1584242279 conn(0x7fad540fd8c0 msgr2=0x7fad540febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 shutdown_connections 2024-09-12T00:55:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:21.244+0000 7fad598b7640 1 -- 172.21.15.16:0/1584242279 wait complete. 2024-09-12T00:55:21.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:55:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:22.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:22 smithi099 ceph-mon[35654]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:22.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:22 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1584242279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:22.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:22 smithi016 ceph-mon[30972]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:22.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:22 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1584242279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:22.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:22 smithi159 ceph-mon[35566]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-12T00:55:22.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:22 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1584242279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 -- 172.21.15.16:0/991050380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64481082a0 msgr2=0x7f644810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 --2- 172.21.15.16:0/991050380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64481082a0 0x7f644810ec40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6440009cd0 tx=0x7f644002a910 comp rx=0 tx=0).stop 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 -- 172.21.15.16:0/991050380 shutdown_connections 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 --2- 172.21.15.16:0/991050380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64481082a0 0x7f644810ec40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 --2- 172.21.15.16:0/991050380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64481036e0 0x7f6448107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 --2- 172.21.15.16:0/991050380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6448102d30 0x7f6448103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 -- 172.21.15.16:0/991050380 >> 172.21.15.16:0/991050380 conn(0x7f64480fc7d0 msgr2=0x7f64480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 -- 172.21.15.16:0/991050380 shutdown_connections 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.746+0000 7f644dfed640 1 -- 172.21.15.16:0/991050380 wait complete. 2024-09-12T00:55:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 Processor -- start 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 -- start start 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64481036e0 0x7f64481a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64481082a0 0x7f644819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6448112100 con 0x7f6448102d30 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6448111f80 con 0x7f64481082a0 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f644dfed640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6448112280 con 0x7f64481036e0 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45430/0 (socket says 172.21.15.16:45430) 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 -- 172.21.15.16:0/2390011018 learned_addr learned my addr 172.21.15.16:0/2390011018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f643ffff640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64481036e0 0x7f64481a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 -- 172.21.15.16:0/2390011018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64481036e0 msgr2=0x7f64481a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64481036e0 0x7f64481a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 -- 172.21.15.16:0/2390011018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64481082a0 msgr2=0x7f644819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64481082a0 0x7f644819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 -- 172.21.15.16:0/2390011018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644819ad20 con 0x7f6448102d30 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.747+0000 7f64477fe640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f643800d770 tx=0x7f643800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.748+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438017070 con 0x7f6448102d30 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.748+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644819b010 con 0x7f6448102d30 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.748+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6438005500 con 0x7f6448102d30 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.748+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438004e00 con 0x7f6448102d30 2024-09-12T00:55:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.748+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64481a7300 con 0x7f6448102d30 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.749+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6448103110 con 0x7f6448102d30 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.750+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6438004fa0 con 0x7f6448102d30 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.751+0000 7f64457fa640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 0x7f64340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.751+0000 7f643ffff640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 0x7f64340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.751+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f64380ced00 con 0x7f6448102d30 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.751+0000 7f643ffff640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 0x7f64340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6448100a90 tx=0x7f6430008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.755+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f643809bab0 con 0x7f6448102d30 2024-09-12T00:55:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.967+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f644819bb20 con 0x7f6448102d30 2024-09-12T00:55:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.967+0000 7f64457fa640 1 -- 172.21.15.16:0/2390011018 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7f643809b3b0 con 0x7f6448102d30 2024-09-12T00:55:22.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:22.968 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:55:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.969+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 msgr2=0x7f64340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.969+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 0x7f64340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6448100a90 tx=0x7f6430008040 comp rx=0 tx=0).stop 2024-09-12T00:55:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.969+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 msgr2=0x7f64481a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.969+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f643800d770 tx=0x7f643800dc30 comp rx=0 tx=0).stop 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 shutdown_connections 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f64340a2c10 0x7f64340a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64481082a0 0x7f644819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64481036e0 0x7f64481a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 --2- 172.21.15.16:0/2390011018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6448102d30 0x7f64481a0430 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 >> 172.21.15.16:0/2390011018 conn(0x7f64480fc7d0 msgr2=0x7f6448101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 shutdown_connections 2024-09-12T00:55:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:22.970+0000 7f644dfed640 1 -- 172.21.15.16:0/2390011018 wait complete. 2024-09-12T00:55:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:23 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2390011018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:23.341 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:23 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2390011018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:23 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2390011018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:23.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T00:54:04.465326+0000', 'modified': '2024-09-12T00:54:16.816930+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24505, 'mds_1': 24487, 'mds_2': 24482, 'mds_3': 24481, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3482588678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3482588678}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3482588678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/2826352912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 2826352912}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 2826352912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/4113169911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 4113169911}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 4113169911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/3000703552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 3000703552}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 3000703552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/310857906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 310857906}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 310857906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/1045510613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 1045510613}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 1045510613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1983976622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1983976622}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1983976622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/850851643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 850851643}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 850851643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.159:6837/247659815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 247659815}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 247659815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.159:6839/1957754994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 1957754994}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 1957754994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24505, 'qdb_cluster': [24505, 24487, 24482, 24481, 24452]} 2024-09-12T00:55:23.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T00:55:23.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T00:55:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:24.329 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:24 smithi016 ceph-mon[30972]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:24.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:24 smithi159 ceph-mon[35566]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 -- 172.21.15.16:0/447399206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074f40 msgr2=0x7fb1a410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/447399206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074f40 0x7fb1a410de80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009900 tx=0x7fb18c02a480 comp rx=0 tx=0).stop 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 -- 172.21.15.16:0/447399206 shutdown_connections 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/447399206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 0x7fb1a4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/447399206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074f40 0x7fb1a410de80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/447399206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074620 0x7fb1a4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 -- 172.21.15.16:0/447399206 >> 172.21.15.16:0/447399206 conn(0x7fb1a40fd8c0 msgr2=0x7fb1a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 -- 172.21.15.16:0/447399206 shutdown_connections 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.411+0000 7fb1a8b29640 1 -- 172.21.15.16:0/447399206 wait complete. 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 Processor -- start 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 -- start start 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 0x7fb1a41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 0x7fb1a419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4114230 con 0x7fb1a4074620 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41140b0 con 0x7fb1a4074f40 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a8b29640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41143b0 con 0x7fb1a410e3c0 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45446/0 (socket says 172.21.15.16:45446) 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 -- 172.21.15.16:0/3100594279 learned_addr learned my addr 172.21.15.16:0/3100594279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 0x7fb1a419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb199d9b640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 0x7fb1a41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 -- 172.21.15.16:0/3100594279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 msgr2=0x7fb1a419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 0x7fb1a419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 -- 172.21.15.16:0/3100594279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 msgr2=0x7fb1a41a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 0x7fb1a41a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb1a259c640 1 -- 172.21.15.16:0/3100594279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a419ad00 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.412+0000 7fb199d9b640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 0x7fb1a41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb1a259c640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb19400b4a0 tx=0x7fb19400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb19bfff640 1 -- 172.21.15.16:0/3100594279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194014070 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a419aff0 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb19bfff640 1 -- 172.21.15.16:0/3100594279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb194010af0 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb19bfff640 1 -- 172.21.15.16:0/3100594279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194017d10 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.413+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a72e0 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.415+0000 7fb19bfff640 1 -- 172.21.15.16:0/3100594279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb194017070 con 0x7fb1a4074620 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.415+0000 7fb1aaf02740 1 Processor -- start 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 -- start start 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7d2f0 0x56544de87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de87bb0 0x56544de89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56544dde3090 con 0x56544de7c9d0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56544dd8aa70 con 0x56544de87bb0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56544dddf050 con 0x56544de7d2f0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45452/0 (socket says 172.21.15.16:45452) 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/1774499997 learned_addr learned my addr 172.21.15.16:0/1774499997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb199d9b640 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de87bb0 0x56544de89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a259c640 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7d2f0 0x56544de87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/1774499997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7d2f0 msgr2=0x56544de87670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7d2f0 0x56544de87670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/1774499997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de87bb0 msgr2=0x56544de89fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de87bb0 0x56544de89fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/1774499997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544dd97130 con 0x56544de7c9d0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.416+0000 7fb19bfff640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 0x7fb1900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00a430 tx=0x7fb19c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd655ae69cd34b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb1a259c640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 0x7fb1900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb198d99640 1 -- 172.21.15.16:0/1774499997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c013790 con 0x56544de7c9d0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb19bfff640 1 -- 172.21.15.16:0/3100594279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb1940ce480 con 0x7fb1a4074620 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb198d99640 1 -- 172.21.15.16:0/1774499997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c019070 con 0x56544de7c9d0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb198d99640 1 -- 172.21.15.16:0/1774499997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c014de0 con 0x56544de7c9d0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1774499997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 msgr2=0x56544de7cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb1a259c640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 0x7fb1900a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x56544de79dc0 tx=0x7fb19400bf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.417+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00a430 tx=0x7fb19c009650 comp rx=0 tx=0).stop 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1774499997 shutdown_connections 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de87bb0 0x56544de89fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7d2f0 0x56544de87670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1774499997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7c9d0 0x56544de7cdb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1774499997 >> 172.21.15.16:0/1774499997 conn(0x56544de78f60 msgr2=0x56544de79340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1774499997 shutdown_connections 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1774499997 wait complete. 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- start start 2024-09-12T00:55:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 Processor -- start 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544de874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de84d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56544de8a780 con 0x56544de7d2f0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56544dd9df20 con 0x56544de7c9d0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56544de8a900 con 0x56544de87bb0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb1aaf02740 10 client.0 _ll_register_callbacks cb 0x56544dd67030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.418+0000 7fb199d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45466/0 (socket says 172.21.15.16:45466) 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 -- 172.21.15.16:0/1444928277 learned_addr learned my addr 172.21.15.16:0/1444928277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de84d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544de874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 -- 172.21.15.16:0/1444928277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 msgr2=0x56544de84d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de84d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 -- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 msgr2=0x56544de874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544de874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 -- 172.21.15.16:0/1444928277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544db542b0 con 0x56544de7d2f0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb199d9b640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb18c031550 tx=0x7fb18c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840a7969072a2656 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb189ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c035070 con 0x56544de7d2f0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb18c005220 con 0x56544de7d2f0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19c019070 con 0x56544de7d2f0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb18c005d50 con 0x56544de7d2f0 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb1aaf02740 10 client.14961 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.419+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56544de960f0 con 0x56544de7d2f0 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c019070 con 0x56544de7d2f0 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb18c003890 con 0x56544de7d2f0 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 10 client.14961 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 10 client.14961 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 20 client.14961 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544de98720 0x56544de99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 4 client.14961 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56544ddf0600 con 0x56544de98720 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.420+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544de98720 0x56544de99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544de98720 0x56544de99b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=459575f218903bed server_cookie=b64921689d3ee9e0 in_seq=0 out_seq=0 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb189ffb640 10 client.14961 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb189ffb640 1 -- 172.21.15.16:0/1444928277 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56544ddf0600 con 0x56544de98720 2024-09-12T00:55:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb189ffb640 10 client.14961 handle_command_reply: tid=0 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1aaf02740 1 client.14961 shutdown 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 msgr2=0x56544de847f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb18c031550 tx=0x7fb18c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 shutdown_connections 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.421+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544de98720 0x56544de99b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de84d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de847f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb189ffb640 0 client.14961 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/1444928277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544de874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 >> 172.21.15.16:0/1444928277 conn(0x56544de78f60 msgr2=0x56544de7ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 shutdown_connections 2024-09-12T00:55:24.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 1 -- 172.21.15.16:0/1444928277 wait complete. 2024-09-12T00:55:24.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.422+0000 7fb1aaf02740 20 client.14961 trim_cache size 0 max 16384 2024-09-12T00:55:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 Processor -- start 2024-09-12T00:55:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 -- start start 2024-09-12T00:55:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7c9d0 0x56544de86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7d2f0 0x56544de81ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56544de97ed0 con 0x56544de87bb0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56544de985a0 con 0x56544de7d2f0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56544de98c70 con 0x56544de7c9d0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45470/0 (socket says 172.21.15.16:45470) 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/2055612984 learned_addr learned my addr 172.21.15.16:0/2055612984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7c9d0 0x56544de86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.440+0000 7fb199d9b640 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7d2f0 0x56544de81ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/2055612984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7c9d0 msgr2=0x56544de86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7c9d0 0x56544de86650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/2055612984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7d2f0 msgr2=0x56544de81ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7d2f0 0x56544de81ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 -- 172.21.15.16:0/2055612984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544de82d80 con 0x56544de87bb0 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00a400 tx=0x7fb19c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f18a2431866a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb18bfff640 1 -- 172.21.15.16:0/2055612984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c009b30 con 0x56544de87bb0 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb18bfff640 1 -- 172.21.15.16:0/2055612984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c019070 con 0x56544de87bb0 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb18bfff640 1 -- 172.21.15.16:0/2055612984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c02c790 con 0x56544de87bb0 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1aaf02740 1 -- 172.21.15.16:0/2055612984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 msgr2=0x56544de81ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.441+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00a400 tx=0x7fb19c004b50 comp rx=0 tx=0).stop 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- 172.21.15.16:0/2055612984 shutdown_connections 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de87bb0 0x56544de81ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7d2f0 0x56544de81ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/2055612984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de7c9d0 0x56544de86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- 172.21.15.16:0/2055612984 >> 172.21.15.16:0/2055612984 conn(0x56544de78f60 msgr2=0x56544de85f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- 172.21.15.16:0/2055612984 shutdown_connections 2024-09-12T00:55:24.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- 172.21.15.16:0/2055612984 wait complete. 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- start start 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 Processor -- start 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544dea2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56544de99a10 con 0x56544de7d2f0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56544de99340 con 0x56544de7c9d0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56544de9a0e0 con 0x56544de87bb0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1aaf02740 10 client.0 _ll_register_callbacks cb 0x7fb18c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb199d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb199d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45476/0 (socket says 172.21.15.16:45476) 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb199d9b640 1 -- 172.21.15.16:0/3328560470 learned_addr learned my addr 172.21.15.16:0/3328560470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.442+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544dea2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 -- 172.21.15.16:0/3328560470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 msgr2=0x56544de81f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de81f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 -- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 msgr2=0x56544dea2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544dea2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 -- 172.21.15.16:0/3328560470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544dea0210 con 0x56544de7d2f0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de81f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb199d9b640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003c60 tx=0x7fb18c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7099e7cb4e43e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb189ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c035050 con 0x56544de7d2f0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb18c031950 con 0x56544de7d2f0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb19c019070 con 0x56544de7d2f0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb18c0044e0 con 0x56544de7d2f0 2024-09-12T00:55:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb1aaf02740 10 client.14967 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56544df02fb0 con 0x56544de7d2f0 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.443+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19c019070 con 0x56544de7d2f0 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb18c0447a0 con 0x56544de7d2f0 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 10 client.14967 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 20 client.14967 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544df07a90 0x56544df067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 4 client.14967 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "status"}) -- 0x56544ddeab10 con 0x56544df07a90 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544df07a90 0x56544df067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb1a2d9d640 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544df07a90 0x56544df067a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df31b6b4f4e0e4c0 server_cookie=b0e9afaab873c971 in_seq=0 out_seq=0 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.444+0000 7fb189ffb640 10 client.14967 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb189ffb640 1 -- 172.21.15.16:0/3328560470 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56544ddeab10 con 0x56544df07a90 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb189ffb640 10 client.14967 handle_command_reply: tid=0 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 client.14967 shutdown 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 msgr2=0x56544de81770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb18c003c60 tx=0x7fb18c003f60 comp rx=0 tx=0).stop 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 shutdown_connections 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56544df07a90 0x56544df067a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56544de87bb0 0x56544de81f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb189ffb640 0 client.14967 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56544de7d2f0 0x56544de81770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 --2- 172.21.15.16:0/3328560470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56544de7c9d0 0x56544dea2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 >> 172.21.15.16:0/3328560470 conn(0x56544de78f60 msgr2=0x56544de7ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 shutdown_connections 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 1 -- 172.21.15.16:0/3328560470 wait complete. 2024-09-12T00:55:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1aaf02740 20 client.14967 trim_cache size 0 max 16384 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24452, 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 68.633067393000005, 2024-09-12T00:55:24.459 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:24.460 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:24.460 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:24.460 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 102.601785022 2024-09-12T00:55:24.460 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 msgr2=0x7fb1900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.445+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 0x7fb1900a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x56544de79dc0 tx=0x7fb19400bf90 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 msgr2=0x7fb1a41a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb19400b4a0 tx=0x7fb19400b960 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 shutdown_connections 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb1900a2c10 0x7fb1900a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1a410e3c0 0x7fb1a419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a4074f40 0x7fb1a41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 --2- 172.21.15.16:0/3100594279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1a4074620 0x7fb1a41a0410 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 >> 172.21.15.16:0/3100594279 conn(0x7fb1a40fd8c0 msgr2=0x7fb1a40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 shutdown_connections 2024-09-12T00:55:24.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:24.446+0000 7fb1a8b29640 1 -- 172.21.15.16:0/3100594279 wait complete. 2024-09-12T00:55:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:24 smithi099 ceph-mon[35654]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:25.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T00:55:25.384 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 -- 172.21.15.16:0/1101420358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 msgr2=0x7fe1ac103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/1101420358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac103120 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009900 tx=0x7fe1a802a480 comp rx=0 tx=0).stop 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 -- 172.21.15.16:0/1101420358 shutdown_connections 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/1101420358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/1101420358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 0x7fe1ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/1101420358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac103120 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 -- 172.21.15.16:0/1101420358 >> 172.21.15.16:0/1101420358 conn(0x7fe1ac0fc7a0 msgr2=0x7fe1ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 -- 172.21.15.16:0/1101420358 shutdown_connections 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.844+0000 7fe1b47d5640 1 -- 172.21.15.16:0/1101420358 wait complete. 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 Processor -- start 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 -- start start 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 0x7fe1ac1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac111fb0 con 0x7fe1ac102d40 2024-09-12T00:55:25.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac111e30 con 0x7fe1ac10c170 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b47d5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac112130 con 0x7fe1ac103660 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45484/0 (socket says 172.21.15.16:45484) 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.845+0000 7fe1b2571640 1 -- 172.21.15.16:0/3555995964 learned_addr learned my addr 172.21.15.16:0/3555995964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 0x7fe1ac1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 -- 172.21.15.16:0/3555995964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 msgr2=0x7fe1ac1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 0x7fe1ac1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 -- 172.21.15.16:0/3555995964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 msgr2=0x7fe1ac19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 -- 172.21.15.16:0/3555995964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac19d460 con 0x7fe1ac102d40 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b2571640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006010 tx=0x7fe1a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1a37fe640 1 -- 172.21.15.16:0/3555995964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a803f030 con 0x7fe1ac102d40 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1a37fe640 1 -- 172.21.15.16:0/3555995964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a8003ce0 con 0x7fe1ac102d40 2024-09-12T00:55:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac19d6f0 con 0x7fe1ac102d40 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1a37fe640 1 -- 172.21.15.16:0/3555995964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8035070 con 0x7fe1ac102d40 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.846+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a5030 con 0x7fe1ac102d40 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1a37fe640 1 -- 172.21.15.16:0/3555995964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe1a8042030 con 0x7fe1ac102d40 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 Processor -- start 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 -- start start 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cd4b3f090 con 0x556cd4bd89d0 2024-09-12T00:55:25.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd4ae6a70 con 0x556cd4bd92f0 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cd4b3b050 con 0x556cd4be3bb0 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b2d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b2d72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45494/0 (socket says 172.21.15.16:45494) 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b2d72640 1 -- 172.21.15.16:0/1602968575 learned_addr learned my addr 172.21.15.16:0/1602968575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1a37fe640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 0x7fe1980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.849+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 -- 172.21.15.16:0/1602968575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 msgr2=0x556cd4be5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 -- 172.21.15.16:0/1602968575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 msgr2=0x556cd4be3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 -- 172.21.15.16:0/1602968575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd4af3130 con 0x556cd4bd89d0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1a37fe640 1 -- 172.21.15.16:0/3555995964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fe1a80ef770 con 0x7fe1ac102d40 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 0x7fe1980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4010540 tx=0x7fe1a4010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c109a93dc87b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1a0ff9640 1 -- 172.21.15.16:0/1602968575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a401d400 con 0x556cd4bd89d0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1a0ff9640 1 -- 172.21.15.16:0/1602968575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a40134c0 con 0x556cd4bd89d0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 0x7fe1980a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4021000 tx=0x7fe1a401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.850+0000 7fe1a0ff9640 1 -- 172.21.15.16:0/1602968575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4011760 con 0x556cd4bd89d0 2024-09-12T00:55:25.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1602968575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 msgr2=0x556cd4bd8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4010540 tx=0x7fe1a4010a00 comp rx=0 tx=0).stop 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1602968575 shutdown_connections 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1602968575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bd8db0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1602968575 >> 172.21.15.16:0/1602968575 conn(0x556cd4bd4f60 msgr2=0x556cd4bd5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1602968575 shutdown_connections 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.851+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1602968575 wait complete. 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 -- start start 2024-09-12T00:55:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 Processor -- start 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cd4be6780 con 0x556cd4bd89d0 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd4af9f20 con 0x556cd4bd92f0 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cd4be6900 con 0x556cd4be3bb0 2024-09-12T00:55:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b6bae740 10 client.0 _ll_register_callbacks cb 0x556cd4ac3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45496/0 (socket says 172.21.15.16:45496) 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 -- 172.21.15.16:0/2250039644 learned_addr learned my addr 172.21.15.16:0/2250039644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 -- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 msgr2=0x556cd4be0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 -- 172.21.15.16:0/2250039644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 msgr2=0x556cd4be07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.852+0000 7fe1b2571640 1 -- 172.21.15.16:0/2250039644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd48b02b0 con 0x556cd4bd89d0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b2571640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80c4680 tx=0x7fe1a808af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea907b2ebcd029ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8049070 con 0x556cd4bd89d0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fe1a80b85b0 con 0x556cd4bd89d0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a40134c0 con 0x556cd4bd89d0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1a80bba80 con 0x556cd4bd89d0 2024-09-12T00:55:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 10 client.14979 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556cd4bf2040 con 0x556cd4bd89d0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1a40134c0 con 0x556cd4bd89d0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1a80bf020 con 0x556cd4bd89d0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 10 client.14979 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 10 client.14979 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.853+0000 7fe1b6bae740 20 client.14979 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4bf46f0 0x556cd4bf5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe1b6bae740 4 client.14979 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd4b4c600 con 0x556cd4bf46f0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4bf46f0 0x556cd4bf5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4bf46f0 0x556cd4bf5b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bee574dc19236e3e server_cookie=257d63990036befb in_seq=0 out_seq=0 2024-09-12T00:55:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.854+0000 7fe195ffb640 10 client.14979 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe195ffb640 1 -- 172.21.15.16:0/2250039644 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cd4b4c600 con 0x556cd4bf46f0 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe195ffb640 10 client.14979 handle_command_reply: tid=0 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 client.14979 shutdown 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 msgr2=0x556cd4be3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80c4680 tx=0x7fe1a808af80 comp rx=0 tx=0).stop 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 shutdown_connections 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4be0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4be07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4bf46f0 0x556cd4bf5b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/2250039644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4be3480 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 >> 172.21.15.16:0/2250039644 conn(0x556cd4bd4f60 msgr2=0x556cd4bdac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.855+0000 7fe195ffb640 0 client.14979 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.856+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 shutdown_connections 2024-09-12T00:55:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.856+0000 7fe1b6bae740 1 -- 172.21.15.16:0/2250039644 wait complete. 2024-09-12T00:55:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.856+0000 7fe1b6bae740 20 client.14979 trim_cache size 0 max 16384 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.873+0000 7fe1b6bae740 1 Processor -- start 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.873+0000 7fe1b6bae740 1 -- start start 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 0x556cd4bf5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bdf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf3ea0 con 0x556cd4bd92f0 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf4570 con 0x556cd4bd89d0 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf4c40 con 0x556cd4be3bb0 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45508/0 (socket says 172.21.15.16:45508) 2024-09-12T00:55:25.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 -- 172.21.15.16:0/1240694399 learned_addr learned my addr 172.21.15.16:0/1240694399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 0x556cd4bf5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 -- 172.21.15.16:0/1240694399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 msgr2=0x556cd4bdf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bdf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bdf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 -- 172.21.15.16:0/1240694399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 msgr2=0x556cd4bf5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 0x556cd4bf5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 -- 172.21.15.16:0/1240694399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd4bd8800 con 0x556cd4bd92f0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 0x556cd4bf5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.874+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x556cd4be0060 tx=0x7fe19c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd9e6fa6231c3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe197fff640 1 -- 172.21.15.16:0/1240694399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c01d250 con 0x556cd4bd92f0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe197fff640 1 -- 172.21.15.16:0/1240694399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19c01d8d0 con 0x556cd4bd92f0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe197fff640 1 -- 172.21.15.16:0/1240694399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c015950 con 0x556cd4bd92f0 2024-09-12T00:55:25.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1240694399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 msgr2=0x556cd4be0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x556cd4be0060 tx=0x7fe19c008040 comp rx=0 tx=0).stop 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1240694399 shutdown_connections 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bdf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd92f0 0x556cd4be0090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1240694399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd89d0 0x556cd4bf5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1240694399 >> 172.21.15.16:0/1240694399 conn(0x556cd4bd4f60 msgr2=0x556cd4bf3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1240694399 shutdown_connections 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.875+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1240694399 wait complete. 2024-09-12T00:55:25.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 -- start start 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 Processor -- start 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4bf9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bf98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf59e0 con 0x556cd4bd89d0 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf5310 con 0x556cd4bd92f0 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cd4bf60b0 con 0x556cd4be3bb0 2024-09-12T00:55:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b6bae740 10 client.0 _ll_register_callbacks cb 0x7fe1a803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45524/0 (socket says 172.21.15.16:45524) 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 -- 172.21.15.16:0/1268642004 learned_addr learned my addr 172.21.15.16:0/1268642004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2d72640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bf98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4bf9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 -- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 msgr2=0x556cd4bf98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bf98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 -- 172.21.15.16:0/1268642004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 msgr2=0x556cd4bf9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4bf9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.876+0000 7fe1b2571640 1 -- 172.21.15.16:0/1268642004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd4c5d0c0 con 0x556cd4bd89d0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b2571640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8004650 tx=0x7fe1a80bc4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105eb345b11cb508 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8049070 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fe1a80efb50 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19c01d8d0 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1a803bcb0 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 10 client.14985 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556cd4c5e3c0 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19c01d8d0 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1a80bf070 con 0x556cd4bd89d0 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 10 client.14985 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 10 client.14985 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 20 client.14985 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4c62d90 0x556cd4c61aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 4 client.14985 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.877+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "status"}) -- 0x556cd4b46b10 con 0x556cd4c62d90 2024-09-12T00:55:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4c62d90 0x556cd4c61aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe1b1d70640 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4c62d90 0x556cd4c61aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f44926da0c0a95ba server_cookie=959ebc1d7a27de0b in_seq=0 out_seq=0 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe195ffb640 10 client.14985 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe195ffb640 1 -- 172.21.15.16:0/1268642004 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556cd4b46b10 con 0x556cd4c62d90 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe195ffb640 10 client.14985 handle_command_reply: tid=0 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe1b6bae740 1 client.14985 shutdown 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 msgr2=0x556cd4bf8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.878+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8004650 tx=0x7fe1a80bc4a0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 shutdown_connections 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cd4be3bb0 0x556cd4bf98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd4bd92f0 0x556cd4bf9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556cd4c62d90 0x556cd4c61aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 --2- 172.21.15.16:0/1268642004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556cd4bd89d0 0x556cd4bf8e30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 >> 172.21.15.16:0/1268642004 conn(0x556cd4bd4f60 msgr2=0x556cd4bd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe195ffb640 0 client.14985 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 shutdown_connections 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 1 -- 172.21.15.16:0/1268642004 wait complete. 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b6bae740 20 client.14985 trim_cache size 0 max 16384 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-12T00:55:25.896 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 71.072212923999999, 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 109.263697093 2024-09-12T00:55:25.897 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 msgr2=0x7fe1980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 0x7fe1980a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4021000 tx=0x7fe1a401e000 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 msgr2=0x7fe1ac1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8006010 tx=0x7fe1a80042f0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 shutdown_connections 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe1980a2c90 0x7fe1980a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1ac10c170 0x7fe1ac19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ac103660 0x7fe1ac1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 --2- 172.21.15.16:0/3555995964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1ac102d40 0x7fe1ac1984b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 >> 172.21.15.16:0/3555995964 conn(0x7fe1ac0fc7a0 msgr2=0x7fe1ac0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 shutdown_connections 2024-09-12T00:55:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:25.879+0000 7fe1b47d5640 1 -- 172.21.15.16:0/3555995964 wait complete. 2024-09-12T00:55:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:26 smithi159 ceph-mon[35566]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:26.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:26 smithi016 ceph-mon[30972]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:26.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:26 smithi099 ceph-mon[35654]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-12T00:55:26.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T00:55:26.978 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 -- 172.21.15.16:0/3582934356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58106f10 msgr2=0x7f4d5810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 --2- 172.21.15.16:0/3582934356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58106f10 0x7f4d5810ea90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4d50009ca0 tx=0x7f4d5002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 -- 172.21.15.16:0/3582934356 shutdown_connections 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 --2- 172.21.15.16:0/3582934356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58106f10 0x7f4d5810ea90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 --2- 172.21.15.16:0/3582934356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 0x7f4d581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 --2- 172.21.15.16:0/3582934356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58101de0 0x7f4d581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 -- 172.21.15.16:0/3582934356 >> 172.21.15.16:0/3582934356 conn(0x7f4d580fd8c0 msgr2=0x7f4d580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 -- 172.21.15.16:0/3582934356 shutdown_connections 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.479+0000 7f4d5db39640 1 -- 172.21.15.16:0/3582934356 wait complete. 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 Processor -- start 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 -- start start 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 0x7f4d58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58106f10 0x7f4d58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d58111fc0 con 0x7f4d58101de0 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d58111e40 con 0x7f4d58106f10 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d5db39640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d58112140 con 0x7f4d58102790 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45540/0 (socket says 172.21.15.16:45540) 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d577fe640 1 -- 172.21.15.16:0/2437295268 learned_addr learned my addr 172.21.15.16:0/2437295268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d57fff640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58106f10 0x7f4d58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.480+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 0x7f4d58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 -- 172.21.15.16:0/2437295268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 msgr2=0x7f4d58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 0x7f4d58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 -- 172.21.15.16:0/2437295268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58106f10 msgr2=0x7f4d58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58106f10 0x7f4d58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 -- 172.21.15.16:0/2437295268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d581a6850 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d577fe640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800d950 tx=0x7f4d4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d54ff9640 1 -- 172.21.15.16:0/2437295268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48017070 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d54ff9640 1 -- 172.21.15.16:0/2437295268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d48009c30 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d581a6b40 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d54ff9640 1 -- 172.21.15.16:0/2437295268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800f6b0 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.481+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d581a7080 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.483+0000 7f4d54ff9640 1 -- 172.21.15.16:0/2437295268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4d480109d0 con 0x7f4d58101de0 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 Processor -- start 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 -- start start 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546cdf9d0 0x55d546cdfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 0x55d546cecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d546c46090 con 0x55d546ce02f0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d546beda70 con 0x55d546ceabb0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d546c42050 con 0x55d546cdf9d0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45542/0 (socket says 172.21.15.16:45542) 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d577fe640 1 -- 172.21.15.16:0/702240975 learned_addr learned my addr 172.21.15.16:0/702240975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d54ff9640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 0x7f4d400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d57fff640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546cdf9d0 0x55d546cdfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.484+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 0x55d546cecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d57fff640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 0x7f4d400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 -- 172.21.15.16:0/702240975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546cdf9d0 msgr2=0x55d546cdfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546cdf9d0 0x55d546cdfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 -- 172.21.15.16:0/702240975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 msgr2=0x55d546cecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 0x55d546cecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 -- 172.21.15.16:0/702240975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d546bfa130 con 0x55d546ce02f0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 0x55d546cecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d54ff9640 1 -- 172.21.15.16:0/2437295268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f4d480cee10 con 0x7f4d58101de0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d577fe640 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d4809b220 tx=0x7f4d4800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda6825746d7e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d57fff640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 0x7f4d400a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d5002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d3dffb640 1 -- 172.21.15.16:0/702240975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48020030 con 0x55d546ce02f0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d3dffb640 1 -- 172.21.15.16:0/702240975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4809bc20 con 0x55d546ce02f0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.485+0000 7f4d3dffb640 1 -- 172.21.15.16:0/702240975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d480cca20 con 0x55d546ce02f0 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- 172.21.15.16:0/702240975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 msgr2=0x55d546cea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4d4809b220 tx=0x7f4d4800bf60 comp rx=0 tx=0).stop 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- 172.21.15.16:0/702240975 shutdown_connections 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ceabb0 0x55d546cecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cea670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/702240975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546cdf9d0 0x55d546cdfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- 172.21.15.16:0/702240975 >> 172.21.15.16:0/702240975 conn(0x55d546cdbf60 msgr2=0x55d546cdc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- 172.21.15.16:0/702240975 shutdown_connections 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- 172.21.15.16:0/702240975 wait complete. 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 -- start start 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.486+0000 7f4d5ff12740 1 Processor -- start 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ce02f0 0x55d546ce7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d546ced720 con 0x55d546ceabb0 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d546c00f20 con 0x55d546cdf9d0 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d546ced8a0 con 0x55d546ce02f0 2024-09-12T00:55:27.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d5ff12740 10 client.0 _ll_register_callbacks cb 0x55d546bca030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45556/0 (socket says 172.21.15.16:45556) 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 -- 172.21.15.16:0/3569587327 learned_addr learned my addr 172.21.15.16:0/3569587327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d577fe640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ce02f0 0x55d546ce7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 -- 172.21.15.16:0/3569587327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ce02f0 msgr2=0x55d546ce7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ce02f0 0x55d546ce7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 -- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 msgr2=0x55d546cea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 -- 172.21.15.16:0/3569587327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5469b72b0 con 0x55d546ceabb0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d57fff640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4d50035000 tx=0x7f4d500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916827c2d476a810 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:27.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.487+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004450 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f4d500045f0 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4809bc20 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d500319d0 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 10 client.14997 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d546cf9090 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4809bc20 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d500306e0 con 0x55d546ceabb0 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 10 client.14997 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 20 client.14997 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546cfb7d0 0x55d546cfcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 4 client.14997 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d546c53600 con 0x55d546cfb7d0 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.488+0000 7f4d577fe640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546cfb7d0 0x55d546cfcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.489+0000 7f4d36ffd640 10 client.14997 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.489+0000 7f4d577fe640 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546cfb7d0 0x55d546cfcc40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51cbe86dfce2d44c server_cookie=13a2a27fe62acecb in_seq=0 out_seq=0 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d36ffd640 1 -- 172.21.15.16:0/3569587327 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d546c53600 con 0x55d546cfb7d0 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d36ffd640 10 client.14997 handle_command_reply: tid=0 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 client.14997 shutdown 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 msgr2=0x55d546ce7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4d50035000 tx=0x7f4d500095c0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 shutdown_connections 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546cfb7d0 0x55d546cfcc40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ceabb0 0x55d546ce7cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ce02f0 0x55d546ce7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d36ffd640 0 client.14997 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/3569587327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 >> 172.21.15.16:0/3569587327 conn(0x55d546cdbf60 msgr2=0x55d546ce1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 shutdown_connections 2024-09-12T00:55:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 1 -- 172.21.15.16:0/3569587327 wait complete. 2024-09-12T00:55:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.490+0000 7f4d5ff12740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 Processor -- start 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 -- start start 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ce02f0 0x55d546cfcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546ce7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfaf80 con 0x55d546cdf9d0 2024-09-12T00:55:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfb650 con 0x55d546ce02f0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfbd20 con 0x55d546ceabb0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45568/0 (socket says 172.21.15.16:45568) 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 -- 172.21.15.16:0/116665206 learned_addr learned my addr 172.21.15.16:0/116665206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d57fff640 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546ce7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 -- 172.21.15.16:0/116665206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 msgr2=0x55d546ce7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546ce7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 -- 172.21.15.16:0/116665206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ce02f0 msgr2=0x55d546cfcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ce02f0 0x55d546cfcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ce02f0 0x55d546cfcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 -- 172.21.15.16:0/116665206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d546ce0120 con 0x55d546cdf9d0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d577fe640 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d4809b220 tx=0x7f4d48010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5733ec794705e7ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d3d7fa640 1 -- 172.21.15.16:0/116665206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48020040 con 0x55d546cdf9d0 2024-09-12T00:55:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d3d7fa640 1 -- 172.21.15.16:0/116665206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4809bc20 con 0x55d546cdf9d0 2024-09-12T00:55:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d3d7fa640 1 -- 172.21.15.16:0/116665206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4809abd0 con 0x55d546cdf9d0 2024-09-12T00:55:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 -- 172.21.15.16:0/116665206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 msgr2=0x55d546cfb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.509+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d4809b220 tx=0x7f4d48010d70 comp rx=0 tx=0).stop 2024-09-12T00:55:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 -- 172.21.15.16:0/116665206 shutdown_connections 2024-09-12T00:55:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546ce7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546ce02f0 0x55d546cfcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/116665206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546cdf9d0 0x55d546cfb100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 -- 172.21.15.16:0/116665206 >> 172.21.15.16:0/116665206 conn(0x55d546cdbf60 msgr2=0x55d546ce7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 -- 172.21.15.16:0/116665206 shutdown_connections 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 -- 172.21.15.16:0/116665206 wait complete. 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 -- start start 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 Processor -- start 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cfdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.510+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546cfe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfcac0 con 0x55d546ce02f0 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfc3f0 con 0x55d546cdf9d0 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d546cfd190 con 0x55d546ceabb0 2024-09-12T00:55:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 10 client.0 _ll_register_callbacks cb 0x7f4d500098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45582/0 (socket says 172.21.15.16:45582) 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 -- 172.21.15.16:0/4273718846 learned_addr learned my addr 172.21.15.16:0/4273718846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d577fe640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cfdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d57fff640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546cfe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 -- 172.21.15.16:0/4273718846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 msgr2=0x55d546cfe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546cfe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 -- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 msgr2=0x55d546cfdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cfdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 -- 172.21.15.16:0/4273718846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d546d63ed0 con 0x55d546ce02f0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d57fff640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546cfe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d56ffd640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d58075f80 tx=0x7f4d4c012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7374966308a9e04a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01f060 con 0x55d546ce02f0 2024-09-12T00:55:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f4d4c00dd00 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.511+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d4809bc20 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d4c01a430 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d546d65130 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d4809bc20 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4d4c025630 con 0x55d546ce02f0 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 20 client.15003 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546d69d10 0x55d546d68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 4 client.15003 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "status"}) -- 0x55d546c4db10 con 0x55d546d69d10 2024-09-12T00:55:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.512+0000 7f4d577fe640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546d69d10 0x55d546d68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d577fe640 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546d69d10 0x55d546d68a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7eef2ab7143b887 server_cookie=81cee779e6446650 in_seq=0 out_seq=0 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d36ffd640 10 client.15003 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d36ffd640 1 -- 172.21.15.16:0/4273718846 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d546c4db10 con 0x55d546d69d10 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d36ffd640 10 client.15003 handle_command_reply: tid=0 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 client.15003 shutdown 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 msgr2=0x55d546cfe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d58075f80 tx=0x7f4d4c012be0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 shutdown_connections 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55d546d69d10 0x55d546d68a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d546ceabb0 0x55d546cfe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d546ce02f0 0x55d546cfe290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 --2- 172.21.15.16:0/4273718846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d546cdf9d0 0x55d546cfdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 >> 172.21.15.16:0/4273718846 conn(0x55d546cdbf60 msgr2=0x55d546cdc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d36ffd640 0 client.15003 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 shutdown_connections 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 1 -- 172.21.15.16:0/4273718846 wait complete. 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.513+0000 7f4d5ff12740 20 client.15003 trim_cache size 0 max 16384 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:27.518 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 73.716531571999994, 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 94.012323910000006 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:27.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 msgr2=0x7f4d400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 0x7f4d400a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d50006010 tx=0x7f4d5002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 msgr2=0x7f4d580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800d950 tx=0x7f4d4800de10 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 shutdown_connections 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f4d400a2c10 0x7f4d400a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d58106f10 0x7f4d58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d58102790 0x7f4d58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 --2- 172.21.15.16:0/2437295268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d58101de0 0x7f4d580744f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 >> 172.21.15.16:0/2437295268 conn(0x7f4d580fd8c0 msgr2=0x7f4d580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 shutdown_connections 2024-09-12T00:55:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:27.514+0000 7f4d5db39640 1 -- 172.21.15.16:0/2437295268 wait complete. 2024-09-12T00:55:28.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:28 smithi016 ceph-mon[30972]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:28.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T00:55:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:28 smithi159 ceph-mon[35566]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:28 smithi099 ceph-mon[35654]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:28.590 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 -- 172.21.15.16:0/2887891118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 msgr2=0x7f9934106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 --2- 172.21.15.16:0/2887891118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f9934106e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9924009930 tx=0x7f992402a420 comp rx=0 tx=0).stop 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 -- 172.21.15.16:0/2887891118 shutdown_connections 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 --2- 172.21.15.16:0/2887891118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934101420 0x7f993410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 --2- 172.21.15.16:0/2887891118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934100a80 0x7f9934100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 --2- 172.21.15.16:0/2887891118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f9934106e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 -- 172.21.15.16:0/2887891118 >> 172.21.15.16:0/2887891118 conn(0x7f99340fc7a0 msgr2=0x7f99340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 -- 172.21.15.16:0/2887891118 shutdown_connections 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.094+0000 7f993a0af640 1 -- 172.21.15.16:0/2887891118 wait complete. 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 Processor -- start 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 -- start start 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 0x7f9934102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 0x7f9934102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9934110ea0 con 0x7f9934106a80 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9934110d20 con 0x7f9934101420 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.095+0000 7f993a0af640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9934111020 con 0x7f9934100a80 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49740/0 (socket says 172.21.15.16:49740) 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 -- 172.21.15.16:0/792219991 learned_addr learned my addr 172.21.15.16:0/792219991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9932ffd640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 0x7f9934102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f99337fe640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 0x7f9934102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 -- 172.21.15.16:0/792219991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 msgr2=0x7f9934102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 0x7f9934102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 -- 172.21.15.16:0/792219991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 msgr2=0x7f9934102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 0x7f9934102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 -- 172.21.15.16:0/792219991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341a6990 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f99337fe640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 0x7f9934102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9932ffd640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 0x7f9934102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9933fff640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b970 tx=0x7f992c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9930ff9640 1 -- 172.21.15.16:0/792219991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c019070 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9930ff9640 1 -- 172.21.15.16:0/792219991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c00cb40 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a6c80 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f9930ff9640 1 -- 172.21.15.16:0/792219991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c007650 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.096+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a71c0 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f9930ff9640 1 -- 172.21.15.16:0/792219991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f992c01c070 con 0x7f9934106a80 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 Processor -- start 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 -- start start 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cbd9090 con 0x55ae7cc729d0 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cb80a70 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f993c488740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cbd5050 con 0x55ae7cc732f0 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.099+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49746/0 (socket says 172.21.15.16:49746) 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 -- 172.21.15.16:0/2300363768 learned_addr learned my addr 172.21.15.16:0/2300363768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9930ff9640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 0x7f991c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f99337fe640 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 -- 172.21.15.16:0/2300363768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 msgr2=0x55ae7cc7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 -- 172.21.15.16:0/2300363768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 msgr2=0x55ae7cc7ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 -- 172.21.15.16:0/2300363768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae7cb8d130 con 0x55ae7cc729d0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9933fff640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 0x7f991c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9930ff9640 1 -- 172.21.15.16:0/792219991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f992c0ce6f0 con 0x7f9934106a80 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f992800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b925ff3cf69ac9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9919ffb640 1 -- 172.21.15.16:0/2300363768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928019030 con 0x55ae7cc729d0 2024-09-12T00:55:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9919ffb640 1 -- 172.21.15.16:0/2300363768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9928011a20 con 0x55ae7cc729d0 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.100+0000 7f9919ffb640 1 -- 172.21.15.16:0/2300363768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992801d950 con 0x55ae7cc729d0 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- 172.21.15.16:0/2300363768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 msgr2=0x55ae7cc72db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f9933fff640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 0x7f991c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b940 tx=0x7f992c014d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f992800b730 comp rx=0 tx=0).stop 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- 172.21.15.16:0/2300363768 shutdown_connections 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 --2- 172.21.15.16:0/2300363768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc72db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- 172.21.15.16:0/2300363768 >> 172.21.15.16:0/2300363768 conn(0x55ae7cc6ef60 msgr2=0x55ae7cc6f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- 172.21.15.16:0/2300363768 shutdown_connections 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- 172.21.15.16:0/2300363768 wait complete. 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.101+0000 7f993c488740 1 -- start start 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 Processor -- start 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc80780 con 0x55ae7cc729d0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cb93f20 con 0x55ae7cc732f0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc80900 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f993c488740 10 client.0 _ll_register_callbacks cb 0x55ae7cb5d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49760/0 (socket says 172.21.15.16:49760) 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 -- 172.21.15.16:0/1088658583 learned_addr learned my addr 172.21.15.16:0/1088658583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f99337fe640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 -- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc7dbb0 msgr2=0x55ae7cc7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 -- 172.21.15.16:0/1088658583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 msgr2=0x55ae7cc7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 -- 172.21.15.16:0/1088658583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae7c94a2b0 con 0x55ae7cc729d0 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9928005bc0 tx=0x7f9928015f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11cfed628cc826f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.102+0000 7f9912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928026070 con 0x55ae7cc729d0 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f9928011800 con 0x55ae7cc729d0 2024-09-12T00:55:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9928011a20 con 0x55ae7cc729d0 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9928019040 con 0x55ae7cc729d0 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 10 client.15015 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae7cc8c0a0 con 0x55ae7cc729d0 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9928011a20 con 0x55ae7cc729d0 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9928029030 con 0x55ae7cc729d0 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 10 client.15015 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 10 client.15015 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 20 client.15015 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:29.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7cc8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 4 client.15015 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.103+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae7cbe6600 con 0x55ae7cc8e340 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.104+0000 7f99337fe640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7cc8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.104+0000 7f99337fe640 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7cc8f7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1711f43edb2d3087 server_cookie=96576b03b10a60ea in_seq=0 out_seq=0 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.104+0000 7f9912ffd640 10 client.15015 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f9912ffd640 1 -- 172.21.15.16:0/1088658583 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae7cbe6600 con 0x55ae7cc8e340 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f9912ffd640 10 client.15015 handle_command_reply: tid=0 2024-09-12T00:55:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 client.15015 shutdown 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 msgr2=0x55ae7cc7d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9928005bc0 tx=0x7f9928015f80 comp rx=0 tx=0).stop 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 shutdown_connections 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7cc8f7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f9912ffd640 0 client.15015 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 --2- 172.21.15.16:0/1088658583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc7d410 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 >> 172.21.15.16:0/1088658583 conn(0x55ae7cc6ef60 msgr2=0x55ae7cc74c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 shutdown_connections 2024-09-12T00:55:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 1 -- 172.21.15.16:0/1088658583 wait complete. 2024-09-12T00:55:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.105+0000 7f993c488740 20 client.15015 trim_cache size 0 max 16384 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 Processor -- start 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 -- start start 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8daf0 con 0x55ae7cc729d0 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8e1c0 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f993c488740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8e890 con 0x55ae7cc732f0 2024-09-12T00:55:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49772/0 (socket says 172.21.15.16:49772) 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.123+0000 7f9932ffd640 1 -- 172.21.15.16:0/1792013665 learned_addr learned my addr 172.21.15.16:0/1792013665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f99337fe640 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 -- 172.21.15.16:0/1792013665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 msgr2=0x55ae7cc8f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc8f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 -- 172.21.15.16:0/1792013665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 msgr2=0x55ae7cc79070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 -- 172.21.15.16:0/1792013665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae7cc7d780 con 0x55ae7cc729d0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f9932ffd640 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f9928011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71848423a0161aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f99197fa640 1 -- 172.21.15.16:0/1792013665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928022040 con 0x55ae7cc729d0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f99197fa640 1 -- 172.21.15.16:0/1792013665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992801d4f0 con 0x55ae7cc729d0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.124+0000 7f99197fa640 1 -- 172.21.15.16:0/1792013665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928026040 con 0x55ae7cc729d0 2024-09-12T00:55:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- 172.21.15.16:0/1792013665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 msgr2=0x55ae7cc8dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99280058c0 tx=0x7f9928011fd0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- 172.21.15.16:0/1792013665 shutdown_connections 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc7dbb0 0x55ae7cc79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc732f0 0x55ae7cc8f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 --2- 172.21.15.16:0/1792013665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc729d0 0x55ae7cc8dc70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- 172.21.15.16:0/1792013665 >> 172.21.15.16:0/1792013665 conn(0x55ae7cc6ef60 msgr2=0x55ae7cc69ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- 172.21.15.16:0/1792013665 shutdown_connections 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- 172.21.15.16:0/1792013665 wait complete. 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 -- start start 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.125+0000 7f993c488740 1 Processor -- start 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc729d0 0x55ae7cc92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8f630 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8ef60 con 0x55ae7cc732f0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae7cc8fd00 con 0x55ae7cc729d0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f993c488740 10 client.0 _ll_register_callbacks cb 0x7f9928013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49776/0 (socket says 172.21.15.16:49776) 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 -- 172.21.15.16:0/2205359360 learned_addr learned my addr 172.21.15.16:0/2205359360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f99337fe640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc729d0 0x55ae7cc92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 -- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc729d0 msgr2=0x55ae7cc92c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc729d0 0x55ae7cc92c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 -- 172.21.15.16:0/2205359360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 msgr2=0x55ae7cc931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 -- 172.21.15.16:0/2205359360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae7cc93aa0 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9932ffd640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f992800b600 tx=0x7f992800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6d44972bb12332 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.126+0000 7f9912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928026040 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992801d4f0 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f99280117e0 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f992800bb80 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae7ccf8050 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992801d4f0 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9928029070 con 0x55ae7cc7dbb0 2024-09-12T00:55:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 10 client.15021 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 20 client.15021 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7ccfb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 4 client.15021 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.127+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "status"}) -- 0x55ae7cbe0b10 con 0x55ae7cc8e340 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f99337fe640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7ccfb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f9912ffd640 10 client.15021 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f99337fe640 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7ccfb620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6dd6a8089844997 server_cookie=aff88f9c0f5918c9 in_seq=0 out_seq=0 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f9912ffd640 1 -- 172.21.15.16:0/2205359360 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ae7cbe0b10 con 0x55ae7cc8e340 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f9912ffd640 10 client.15021 handle_command_reply: tid=0 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f993c488740 1 client.15021 shutdown 2024-09-12T00:55:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 msgr2=0x55ae7ccf6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f992800b600 tx=0x7f992800b3a0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.128+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 shutdown_connections 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae7cc7dbb0 0x55ae7ccf6a40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ae7cc8e340 0x55ae7ccfb620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ae7cc732f0 0x55ae7cc931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 --2- 172.21.15.16:0/2205359360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae7cc729d0 0x55ae7cc92c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 >> 172.21.15.16:0/2205359360 conn(0x55ae7cc6ef60 msgr2=0x55ae7cc6f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f9912ffd640 0 client.15021 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 shutdown_connections 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 1 -- 172.21.15.16:0/2205359360 wait complete. 2024-09-12T00:55:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993c488740 20 client.15021 trim_cache size 0 max 16384 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24487, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 76.337360360999995, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:29.135 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 109.981288452 2024-09-12T00:55:29.136 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 msgr2=0x7f991c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 0x7f991c0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b940 tx=0x7f992c014d00 comp rx=0 tx=0).stop 2024-09-12T00:55:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 msgr2=0x7f99341030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b970 tx=0x7f992c00be30 comp rx=0 tx=0).stop 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 shutdown_connections 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f991c0a2c10 0x7f991c0a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9934106a80 0x7f99341030d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9934101420 0x7f9934102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 --2- 172.21.15.16:0/792219991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9934100a80 0x7f9934102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 >> 172.21.15.16:0/792219991 conn(0x7f99340fc7a0 msgr2=0x7f993406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.129+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 shutdown_connections 2024-09-12T00:55:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:29.130+0000 7f993a0af640 1 -- 172.21.15.16:0/792219991 wait complete. 2024-09-12T00:55:29.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T00:55:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:30.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:30 smithi016 ceph-mon[30972]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:30.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:30 smithi159 ceph-mon[35566]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:30.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:30 smithi099 ceph-mon[35654]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:30.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.638+0000 7f3f24e12640 1 -- 172.21.15.16:0/1702950268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20106f10 msgr2=0x7f3f2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.638+0000 7f3f24e12640 1 --2- 172.21.15.16:0/1702950268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20106f10 0x7f3f2010ea90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009ca0 tx=0x7f3f1802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.638+0000 7f3f24e12640 1 -- 172.21.15.16:0/1702950268 shutdown_connections 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.638+0000 7f3f24e12640 1 --2- 172.21.15.16:0/1702950268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20106f10 0x7f3f2010ea90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.638+0000 7f3f24e12640 1 --2- 172.21.15.16:0/1702950268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20102790 0x7f3f201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 --2- 172.21.15.16:0/1702950268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 -- 172.21.15.16:0/1702950268 >> 172.21.15.16:0/1702950268 conn(0x7f3f200fd8c0 msgr2=0x7f3f200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 -- 172.21.15.16:0/1702950268 shutdown_connections 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 -- 172.21.15.16:0/1702950268 wait complete. 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 Processor -- start 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.639+0000 7f3f24e12640 1 -- start start 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f20198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20106f10 0x7f3f2019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20111f70 con 0x7f3f20102790 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20111df0 con 0x7f3f20106f10 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f24e12640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201120f0 con 0x7f3f20101de0 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49788/0 (socket says 172.21.15.16:49788) 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/764039627 learned_addr learned my addr 172.21.15.16:0/764039627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20106f10 0x7f3f2019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f20198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/764039627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 msgr2=0x7f3f20198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f20198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/764039627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20106f10 msgr2=0x7f3f2019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20106f10 0x7f3f2019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/764039627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2019d340 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.640+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f20198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f0f7fe640 1 -- 172.21.15.16:0/764039627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14017070 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f0f7fe640 1 -- 172.21.15.16:0/764039627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f14009c30 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f0f7fe640 1 -- 172.21.15.16:0/764039627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400f6b0 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2019d630 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.641+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201a4f10 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.643+0000 7f3f0f7fe640 1 -- 172.21.15.16:0/764039627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f140109d0 con 0x7f3f20102790 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.643+0000 7f3f271eb740 1 Processor -- start 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 -- start start 2024-09-12T00:55:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7ad8ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 0x560e7ada03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e7acf5090 con 0x560e7ad8f1f0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e7ac9ca70 con 0x560e7ad9dfc0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f271eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e7acf1050 con 0x560e7ad8e8d0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49798/0 (socket says 172.21.15.16:49798) 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 -- 172.21.15.16:0/4014686072 learned_addr learned my addr 172.21.15.16:0/4014686072 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7ad8ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 0x560e7ada03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 -- 172.21.15.16:0/4014686072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 msgr2=0x560e7ad8ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7ad8ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 -- 172.21.15.16:0/4014686072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 msgr2=0x560e7ada03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f0f7fe640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 0x7f3f080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 0x560e7ada03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1e59c640 1 -- 172.21.15.16:0/4014686072 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7aca9130 con 0x560e7ad8f1f0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7ad8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.644+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 0x560e7ada03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 0x7f3f080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f0f7fe640 1 -- 172.21.15.16:0/764039627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f3f140ced10 con 0x7f3f20102790 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000bc20 tx=0x7f3f1000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0fd46ac048f70d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f0cff9640 1 -- 172.21.15.16:0/4014686072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10017620 con 0x560e7ad8f1f0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f0cff9640 1 -- 172.21.15.16:0/4014686072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f10017ca0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.645+0000 7f3f0cff9640 1 -- 172.21.15.16:0/4014686072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000fce0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- 172.21.15.16:0/4014686072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 msgr2=0x560e7ad9da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000bc20 tx=0x7f3f1000d710 comp rx=0 tx=0).stop 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 0x7f3f080a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f18004af0 tx=0x7f3f18004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- 172.21.15.16:0/4014686072 shutdown_connections 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad9dfc0 0x560e7ada03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9da80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- 172.21.15.16:0/4014686072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7ad8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- 172.21.15.16:0/4014686072 >> 172.21.15.16:0/4014686072 conn(0x560e7ad8af60 msgr2=0x560e7ad8b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- 172.21.15.16:0/4014686072 shutdown_connections 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- 172.21.15.16:0/4014686072 wait complete. 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 -- start start 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 Processor -- start 2024-09-12T00:55:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8e8d0 0x560e7ad9d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.646+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 0x560e7ad9b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e7ada0bf0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e7acaff20 con 0x560e7ad8e8d0 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e7ada0d70 con 0x560e7ad9dfc0 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 10 client.0 _ll_register_callbacks cb 0x560e7ac79030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49810/0 (socket says 172.21.15.16:49810) 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/1128468610 learned_addr learned my addr 172.21.15.16:0/1128468610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:30.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8e8d0 0x560e7ad9d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 0x560e7ad9b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 msgr2=0x560e7ad9b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 0x560e7ad9b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/1128468610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8e8d0 msgr2=0x560e7ad9d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8e8d0 0x560e7ad9d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 -- 172.21.15.16:0/1128468610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7aa662b0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 0x560e7ad9b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3f1409b3b0 tx=0x7f3f1409b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac4fe588b1c3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14020070 con 0x560e7ad8f1f0 2024-09-12T00:55:30.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f3f140cc920 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f10017ca0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f1409b420 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 10 client.15033 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.647+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e7adac400 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f10017ca0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f140ccce0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 10 client.15033 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 10 client.15033 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 20 client.15033 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7adaea50 0x560e7adafec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 4 client.15033 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e7ad02600 con 0x560e7adaea50 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.648+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7adaea50 0x560e7adafec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.649+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7adaea50 0x560e7adafec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b744130f4f713a28 server_cookie=d7127aaea0222f0a in_seq=0 out_seq=0 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.649+0000 7f3f05ffb640 10 client.15033 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f05ffb640 1 -- 172.21.15.16:0/1128468610 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e7ad02600 con 0x560e7adaea50 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f05ffb640 10 client.15033 handle_command_reply: tid=0 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 client.15033 shutdown 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 msgr2=0x560e7ad9aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3f1409b3b0 tx=0x7f3f1409b020 comp rx=0 tx=0).stop 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 shutdown_connections 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7adaea50 0x560e7adafec0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad9dfc0 0x560e7ad9b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad8f1f0 0x560e7ad9aac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1128468610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8e8d0 0x560e7ad9d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 >> 172.21.15.16:0/1128468610 conn(0x560e7ad8af60 msgr2=0x560e7ad90cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f05ffb640 0 client.15033 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 shutdown_connections 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 1 -- 172.21.15.16:0/1128468610 wait complete. 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.650+0000 7f3f271eb740 20 client.15033 trim_cache size 0 max 16384 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 Processor -- start 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 -- start start 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adaf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adaf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e7adae200 con 0x560e7ad9dfc0 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e7adae8d0 con 0x560e7ad8f1f0 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f271eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e7adaefa0 con 0x560e7ad8e8d0 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49820/0 (socket says 172.21.15.16:49820) 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.668+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/1864025878 learned_addr learned my addr 172.21.15.16:0/1864025878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adaf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adaf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/1864025878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 msgr2=0x560e7adaf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adaf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/1864025878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 msgr2=0x560e7adaf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adaf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/1864025878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7ad8f020 con 0x560e7ad9dfc0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adaf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3f18035000 tx=0x7f3f18041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2ed0fd067c49d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f07fff640 1 -- 172.21.15.16:0/1864025878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004040 con 0x560e7ad9dfc0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f07fff640 1 -- 172.21.15.16:0/1864025878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f18036040 con 0x560e7ad9dfc0 2024-09-12T00:55:30.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.669+0000 7f3f07fff640 1 -- 172.21.15.16:0/1864025878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1802fd70 con 0x560e7ad9dfc0 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- 172.21.15.16:0/1864025878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 msgr2=0x560e7adae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3f18035000 tx=0x7f3f18041000 comp rx=0 tx=0).stop 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- 172.21.15.16:0/1864025878 shutdown_connections 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adae380 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adaf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 --2- 172.21.15.16:0/1864025878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adaf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- 172.21.15.16:0/1864025878 >> 172.21.15.16:0/1864025878 conn(0x560e7ad8af60 msgr2=0x560e7ad8b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- 172.21.15.16:0/1864025878 shutdown_connections 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- 172.21.15.16:0/1864025878 wait complete. 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 -- start start 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.670+0000 7f3f271eb740 1 Processor -- start 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adb1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adb1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e7adafd40 con 0x560e7ad9dfc0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560e7adaf670 con 0x560e7ad8f1f0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e7adb0410 con 0x560e7ad8e8d0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 10 client.0 _ll_register_callbacks cb 0x7f3f1409b1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49822/0 (socket says 172.21.15.16:49822) 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/3323435368 learned_addr learned my addr 172.21.15.16:0/3323435368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adb1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adb1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 msgr2=0x560e7adb1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adb1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/3323435368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 msgr2=0x560e7adb1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adb1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 -- 172.21.15.16:0/3323435368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7ae17320 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1dd9b640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adb1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f1ed9d640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009bf0 tx=0x7f3f18031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ea3b3030026cec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1802fe80 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f3f18003f90 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.671+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f18036040 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f18004210 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 10 client.15039 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e7ae18590 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f18036040 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f18030cc0 con 0x560e7ad9dfc0 2024-09-12T00:55:30.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 10 client.15039 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 10 client.15039 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 20 client.15039 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7ae1cde0 0x560e7ae1baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 4 client.15039 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "status"}) -- 0x560e7acfcb10 con 0x560e7ae1cde0 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.672+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7ae1cde0 0x560e7ae1baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f1e59c640 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7ae1cde0 0x560e7ae1baf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81058f82240d5177 server_cookie=61ed970562f6e323 in_seq=0 out_seq=0 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f05ffb640 10 client.15039 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f05ffb640 1 -- 172.21.15.16:0/3323435368 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560e7acfcb10 con 0x560e7ae1cde0 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f05ffb640 10 client.15039 handle_command_reply: tid=0 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 client.15039 shutdown 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 msgr2=0x560e7adb1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009bf0 tx=0x7f3f18031f20 comp rx=0 tx=0).stop 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 shutdown_connections 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560e7ae1cde0 0x560e7ae1baf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e7ad9dfc0 0x560e7adb1bb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560e7ad8f1f0 0x560e7adb1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 --2- 172.21.15.16:0/3323435368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e7ad8e8d0 0x560e7adb1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f05ffb640 0 client.15039 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 >> 172.21.15.16:0/3323435368 conn(0x560e7ad8af60 msgr2=0x560e7ad85ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 shutdown_connections 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 1 -- 172.21.15.16:0/3323435368 wait complete. 2024-09-12T00:55:30.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.673+0000 7f3f271eb740 20 client.15039 trim_cache size 0 max 16384 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24505, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 85.911283025000003, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:30.690 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 106.246216903 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 msgr2=0x7f3f080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 0x7f3f080a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f18004af0 tx=0x7f3f18004a80 comp rx=0 tx=0).stop 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 msgr2=0x7f3f201988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400de10 comp rx=0 tx=0).stop 2024-09-12T00:55:30.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 shutdown_connections 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f3f080a2c90 0x7f3f080a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f20106f10 0x7f3f2019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f20102790 0x7f3f201988d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 --2- 172.21.15.16:0/764039627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f20101de0 0x7f3f20198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 >> 172.21.15.16:0/764039627 conn(0x7f3f200fd8c0 msgr2=0x7f3f200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 shutdown_connections 2024-09-12T00:55:30.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:30.674+0000 7f3f24e12640 1 -- 172.21.15.16:0/764039627 wait complete. 2024-09-12T00:55:31.424 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T00:55:31.425 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T00:55:31.436 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:55:31.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T00:55:31.692 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:32.106 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:32 smithi016 ceph-mon[30972]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:32.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 -- 172.21.15.16:0/4280874439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020107960 msgr2=0x7fb020109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 --2- 172.21.15.16:0/4280874439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020107960 0x7fb020109d50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb018009ca0 tx=0x7fb01802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 -- 172.21.15.16:0/4280874439 shutdown_connections 2024-09-12T00:55:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 --2- 172.21.15.16:0/4280874439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020107960 0x7fb020109d50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 --2- 172.21.15.16:0/4280874439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 --2- 172.21.15.16:0/4280874439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb020102700 0x7fb020104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 -- 172.21.15.16:0/4280874439 >> 172.21.15.16:0/4280874439 conn(0x7fb0200fc280 msgr2=0x7fb0200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 -- 172.21.15.16:0/4280874439 shutdown_connections 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.192+0000 7fb027c88640 1 -- 172.21.15.16:0/4280874439 wait complete. 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 Processor -- start 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 -- start start 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb020107960 0x7fb02019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:32.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb02010d860 con 0x7fb020102700 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02010d6e0 con 0x7fb020105030 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb027c88640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb02010d9e0 con 0x7fb020107960 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49840/0 (socket says 172.21.15.16:49840) 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 -- 172.21.15.16:0/1433213333 learned_addr learned my addr 172.21.15.16:0/1433213333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 -- 172.21.15.16:0/1433213333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb020107960 msgr2=0x7fb02019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb020107960 0x7fb02019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025223640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 -- 172.21.15.16:0/1433213333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 msgr2=0x7fb020198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025a24640 1 -- 172.21.15.16:0/1433213333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02019fb10 con 0x7fb020102700 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.193+0000 7fb025223640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb025a24640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d950 tx=0x7fb01c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c017070 con 0x7fb020102700 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01c009c30 con 0x7fb020102700 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02019fe00 con 0x7fb020102700 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00f6b0 con 0x7fb020102700 2024-09-12T00:55:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.194+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a02b0 con 0x7fb020102700 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.195+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0200fdd70 con 0x7fb020102700 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.196+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb01c0109d0 con 0x7fb020102700 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.197+0000 7fb016ffd640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 0x7fb00c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.197+0000 7fb025223640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 0x7fb00c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.197+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb01c0cee30 con 0x7fb020102700 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.197+0000 7fb025223640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 0x7fb00c0a5180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb020199400 tx=0x7fb010005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.200+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb01c09bc60 con 0x7fb020102700 2024-09-12T00:55:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:32 smithi159 ceph-mon[35566]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:32.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.392+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb020062630 con 0x7fb00c0a2cc0 2024-09-12T00:55:32.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.394+0000 7fb016ffd640 1 -- 172.21.15.16:0/1433213333 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb020062630 con 0x7fb00c0a2cc0 2024-09-12T00:55:32.392 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-12T00:55:32.392 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-12T00:55:32.392 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-12T00:55:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 msgr2=0x7fb00c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 0x7fb00c0a5180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb020199400 tx=0x7fb010005d30 comp rx=0 tx=0).stop 2024-09-12T00:55:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 msgr2=0x7fb020197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:32.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d950 tx=0x7fb01c00de10 comp rx=0 tx=0).stop 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 shutdown_connections 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb00c0a2cc0 0x7fb00c0a5180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb020107960 0x7fb02019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb020105030 0x7fb020198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.396+0000 7fb027c88640 1 --2- 172.21.15.16:0/1433213333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb020102700 0x7fb020197ee0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.397+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 >> 172.21.15.16:0/1433213333 conn(0x7fb0200fc280 msgr2=0x7fb020103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.397+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 shutdown_connections 2024-09-12T00:55:32.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:32.397+0000 7fb027c88640 1 -- 172.21.15.16:0/1433213333 wait complete. 2024-09-12T00:55:32.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:32 smithi099 ceph-mon[35654]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:33.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T00:55:33.403 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 -- 172.21.15.16:0/665053377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 msgr2=0x7f41c81086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/665053377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c81086e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009ca0 tx=0x7f41c002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 -- 172.21.15.16:0/665053377 shutdown_connections 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/665053377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c81086e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/665053377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 0x7f41c8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/665053377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c8100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 -- 172.21.15.16:0/665053377 >> 172.21.15.16:0/665053377 conn(0x7f41c80fba40 msgr2=0x7f41c80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 -- 172.21.15.16:0/665053377 shutdown_connections 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.900+0000 7f41cd6bf640 1 -- 172.21.15.16:0/665053377 wait complete. 2024-09-12T00:55:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 Processor -- start 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 -- start start 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c80fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 0x7f41c819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810bf30 con 0x7f41c8101e50 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810bdb0 con 0x7f41c8101490 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41cd6bf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41c810c0b0 con 0x7f41c8100290 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49870/0 (socket says 172.21.15.16:49870) 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 -- 172.21.15.16:0/3637341786 learned_addr learned my addr 172.21.15.16:0/3637341786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c6ffd640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c80fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41bffff640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 0x7f41c819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 -- 172.21.15.16:0/3637341786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 msgr2=0x7f41c80fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c80fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 -- 172.21.15.16:0/3637341786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 msgr2=0x7f41c819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.901+0000 7f41c77fe640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 0x7f41c819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c77fe640 1 -- 172.21.15.16:0/3637341786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c819fbb0 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c6ffd640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c80fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c77fe640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f41c0031ee0 tx=0x7f41c0031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0003c30 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c0003dd0 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c819fde0 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c002fc10 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.902+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c81a02f0 con 0x7f41c8101e50 2024-09-12T00:55:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.903+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8107d10 con 0x7f41c8101e50 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.904+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f41c00046c0 con 0x7f41c8101e50 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.905+0000 7f41c4ff9640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 0x7f41b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.905+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f41c00f0660 con 0x7f41c8101e50 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.905+0000 7f41c6ffd640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 0x7f41b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.905+0000 7f41c6ffd640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 0x7f41b40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41b80058c0 tx=0x7f41b8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:33.908+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41c00bd520 con 0x7f41c8101e50 2024-09-12T00:55:34.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.093+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f41c8062630 con 0x7f41b40a2c10 2024-09-12T00:55:34.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:34 smithi016 ceph-mon[30972]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:34.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:34 smithi016 ceph-mon[30972]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.105+0000 7f41c4ff9640 1 -- 172.21.15.16:0/3637341786 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f41c8062630 con 0x7f41b40a2c10 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (103s) 22s ago 103s 17.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 1cbd98c44bc0 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi099 running (100s) 32s ago 100s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 01164f74bce2 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi159 running (115s) 32s ago 114s 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 5133935a71e8 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (2m) 22s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 dec9a3a84c03 2024-09-12T00:55:34.104 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi099 running (2m) 32s ago 119s 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 bff36bc65523 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi159 running (117s) 32s ago 117s 20.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 06f3cd5eaea5 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (107s) 22s ago 106s 16.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 a2a5f5cc7f0d 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi099 running (112s) 32s ago 112s 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 0b433f505ac7 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi159 running (110s) 32s ago 109s 29.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 6b85d3bd1373 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (2m) 22s ago 2m 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 bdbbc108db0e 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi099 running (2m) 32s ago 2m 16.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 3a2b00e8e300 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi159 running (2m) 32s ago 2m 18.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 1da01add056a 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (9m) 22s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 ba43bc913f44 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi099 *:8443,8765 running (7m) 32s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4498af33367 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi159 *:8443,8765 running (7m) 32s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 2e923c6eaf7b 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (9m) 22s ago 9m 51.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 77905c72892b 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi099 running (7m) 32s ago 7m 40.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 1e0b261d1f25 2024-09-12T00:55:34.105 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi159 running (7m) 32s ago 7m 38.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 125230a9ff3b 2024-09-12T00:55:34.106 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 22s ago 7m 45.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 66d25cbddf64 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi099 running (6m) 32s ago 6m 46.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8a09476fc8e2 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi159 running (6m) 32s ago 6m 47.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d67678fb3499 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (5m) 22s ago 5m 43.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 944262d8f090 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi099 running (5m) 32s ago 5m 42.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 11b0f006d48a 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi159 running (5m) 32s ago 5m 56.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 223c3d2c7449 2024-09-12T00:55:34.108 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 22s ago 4m 55.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 518c4ba08e62 2024-09-12T00:55:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi099 running (4m) 32s ago 4m 55.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2ff099bec90c 2024-09-12T00:55:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi159 running (3m) 32s ago 3m 42.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 838089491a20 2024-09-12T00:55:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 22s ago 3m 51.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 29ef8aa76d98 2024-09-12T00:55:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi099 running (2m) 32s ago 2m 55.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 84e5e32db179 2024-09-12T00:55:34.109 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi159 running (2m) 32s ago 2m 48.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 33f966b4fc00 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 msgr2=0x7f41b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 0x7f41b40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41b80058c0 tx=0x7f41b8005d50 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 msgr2=0x7f41c819f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f41c0031ee0 tx=0x7f41c0031f10 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 shutdown_connections 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f41b40a2c10 0x7f41b40a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41c8101e50 0x7f41c819f2a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41c8101490 0x7f41c819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 --2- 172.21.15.16:0/3637341786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c8100290 0x7f41c80fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:34.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 >> 172.21.15.16:0/3637341786 conn(0x7f41c80fba40 msgr2=0x7f41c8104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:34.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 shutdown_connections 2024-09-12T00:55:34.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:34.108+0000 7f41cd6bf640 1 -- 172.21.15.16:0/3637341786 wait complete. 2024-09-12T00:55:34.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:34 smithi159 ceph-mon[35566]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:34.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:34 smithi159 ceph-mon[35566]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:34.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:34 smithi099 ceph-mon[35654]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:34.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:34 smithi099 ceph-mon[35654]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:34.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T00:55:35.108 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:35.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:35 smithi016 ceph-mon[30972]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:35 smithi159 ceph-mon[35566]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:35 smithi099 ceph-mon[35654]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 -- 172.21.15.16:0/1533045222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20801037c0 msgr2=0x7f208010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 --2- 172.21.15.16:0/1533045222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20801037c0 0x7f208010a050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2078009ca0 tx=0x7f207802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 -- 172.21.15.16:0/1533045222 shutdown_connections 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 --2- 172.21.15.16:0/1533045222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20801037c0 0x7f208010a050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 --2- 172.21.15.16:0/1533045222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f2080103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 --2- 172.21.15.16:0/1533045222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080101c00 0x7f2080102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 -- 172.21.15.16:0/1533045222 >> 172.21.15.16:0/1533045222 conn(0x7f20800fd390 msgr2=0x7f20800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 -- 172.21.15.16:0/1533045222 shutdown_connections 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.609+0000 7f2086d60640 1 -- 172.21.15.16:0/1533045222 wait complete. 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 Processor -- start 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 -- start start 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f208019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20801037c0 0x7f20801a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f208010d980 con 0x7f2080101c00 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f208010d800 con 0x7f2080102e00 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2086d60640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f208010db00 con 0x7f20801037c0 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49888/0 (socket says 172.21.15.16:49888) 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 -- 172.21.15.16:0/2856994497 learned_addr learned my addr 172.21.15.16:0/2856994497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f207ffff640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f208019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 -- 172.21.15.16:0/2856994497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20801037c0 msgr2=0x7f20801a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20801037c0 0x7f20801a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 -- 172.21.15.16:0/2856994497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 msgr2=0x7f208019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f208019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 -- 172.21.15.16:0/2856994497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a1e60 con 0x7f2080101c00 2024-09-12T00:55:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f207ffff640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f208019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.610+0000 7f2084afc640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f207400b450 tx=0x7f207400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.611+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074015020 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.611+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a2150 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.611+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2074011d70 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.611+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207400f340 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.611+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a26c0 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.612+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20801a2980 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.613+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2074018020 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.614+0000 7f207dffb640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 0x7f20700a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.614+0000 7f207ffff640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 0x7f20700a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.614+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f20740cec80 con 0x7f2080101c00 2024-09-12T00:55:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.614+0000 7f207ffff640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 0x7f20700a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.618+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f207401f020 con 0x7f2080101c00 2024-09-12T00:55:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.801+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f20801a2b50 con 0x7f20700a2c40 2024-09-12T00:55:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.808+0000 7f207dffb640 1 -- 172.21.15.16:0/2856994497 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f20801a2b50 con 0x7f20700a2c40 2024-09-12T00:55:35.807 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T00:55:35.807 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 34s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-12T00:55:35.807 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 34s ago 7m smithi016=x;smithi099=y;smithi159=z;count:3 2024-09-12T00:55:35.807 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 34s ago 7m smithi016:172.21.15.16=a;smithi099:172.21.15.99=b;smithi159:172.21.15.159=c;count:3 2024-09-12T00:55:35.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 34s ago - 2024-09-12T00:55:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.810+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 msgr2=0x7f20700a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.810+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 0x7f20700a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).stop 2024-09-12T00:55:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.810+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 msgr2=0x7f208019a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.810+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f207400b450 tx=0x7f207400b910 comp rx=0 tx=0).stop 2024-09-12T00:55:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 shutdown_connections 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f20700a2c40 0x7f20700a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20801037c0 0x7f20801a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2080102e00 0x7f208019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 --2- 172.21.15.16:0/2856994497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2080101c00 0x7f208019a230 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 >> 172.21.15.16:0/2856994497 conn(0x7f20800fd390 msgr2=0x7f2080108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 shutdown_connections 2024-09-12T00:55:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:35.811+0000 7f2086d60640 1 -- 172.21.15.16:0/2856994497 wait complete. 2024-09-12T00:55:36.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:36 smithi016 ceph-mon[30972]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:36.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:36 smithi159 ceph-mon[35566]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:36.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:36 smithi099 ceph-mon[35654]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T00:55:36.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T00:55:36.846 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:37.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:37 smithi016 ceph-mon[30972]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:37 smithi159 ceph-mon[35566]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1544222302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b841037c0 msgr2=0x7f2b8410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1544222302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b841037c0 0x7f2b8410a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009ca0 tx=0x7f2b7c02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1544222302 shutdown_connections 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1544222302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b841037c0 0x7f2b8410a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1544222302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b84103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1544222302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b84101c00 0x7f2b84102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1544222302 >> 172.21.15.16:0/1544222302 conn(0x7f2b840fd390 msgr2=0x7f2b840ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1544222302 shutdown_connections 2024-09-12T00:55:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1544222302 wait complete. 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 Processor -- start 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.371+0000 7f2b8a6cb640 1 -- start start 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b840745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 0x7f2b84074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410d8e0 con 0x7f2b84101c00 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410d760 con 0x7f2b84102e00 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b8a6cb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8410da60 con 0x7f2b841037c0 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49914/0 (socket says 172.21.15.16:49914) 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 -- 172.21.15.16:0/1548906957 learned_addr learned my addr 172.21.15.16:0/1548906957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b88c68640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 0x7f2b84074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:37.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b837fe640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b840745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 -- 172.21.15.16:0/1548906957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 msgr2=0x7f2b84074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 0x7f2b84074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 -- 172.21.15.16:0/1548906957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 msgr2=0x7f2b840745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b840745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 -- 172.21.15.16:0/1548906957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b840753e0 con 0x7f2b84101c00 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b837fe640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b840745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b88c68640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 0x7f2b84074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.372+0000 7f2b83fff640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d950 tx=0x7f2b7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.373+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74017020 con 0x7f2b84101c00 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.373+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b841a2070 con 0x7f2b84101c00 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.373+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b74009c30 con 0x7f2b84101c00 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.373+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400f6b0 con 0x7f2b84101c00 2024-09-12T00:55:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.373+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b841a25d0 con 0x7f2b84101c00 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.374+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84108f80 con 0x7f2b84101c00 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.375+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2b7401a020 con 0x7f2b84101c00 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.376+0000 7f2b817fa640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 0x7f2b6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.376+0000 7f2b837fe640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 0x7f2b6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.376+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f2b740cf580 con 0x7f2b84101c00 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.376+0000 7f2b837fe640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 0x7f2b6c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b78006fd0 tx=0x7f2b78008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.379+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b74021020 con 0x7f2b84101c00 2024-09-12T00:55:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:37 smithi099 ceph-mon[35654]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.562+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2b84062630 con 0x7f2b6c0a2c10 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.563+0000 7f2b817fa640 1 -- 172.21.15.16:0/1548906957 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2b84062630 con 0x7f2b6c0a2c10 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stdout:smithi099 172.21.15.99 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stdout:smithi159 172.21.15.159 2024-09-12T00:55:37.562 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-12T00:55:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.565+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 msgr2=0x7f2b6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.565+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 0x7f2b6c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2b78006fd0 tx=0x7f2b78008040 comp rx=0 tx=0).stop 2024-09-12T00:55:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.565+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 msgr2=0x7f2b84075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.565+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d950 tx=0x7f2b7400de10 comp rx=0 tx=0).stop 2024-09-12T00:55:37.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 shutdown_connections 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2b6c0a2c10 0x7f2b6c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b841037c0 0x7f2b84074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b84102e00 0x7f2b840745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 --2- 172.21.15.16:0/1548906957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b84101c00 0x7f2b84075f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 >> 172.21.15.16:0/1548906957 conn(0x7f2b840fd390 msgr2=0x7f2b84106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 shutdown_connections 2024-09-12T00:55:37.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:37.566+0000 7f2b8a6cb640 1 -- 172.21.15.16:0/1548906957 wait complete. 2024-09-12T00:55:38.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:38 smithi016 ceph-mon[30972]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:38.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:38 smithi159 ceph-mon[35566]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:38.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T00:55:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:38 smithi099 ceph-mon[35654]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 -- 172.21.15.16:0/1152296653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a81026d0 msgr2=0x7f42a8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 --2- 172.21.15.16:0/1152296653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a81026d0 0x7f42a8108f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009900 tx=0x7f42a002a410 comp rx=0 tx=0).stop 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 -- 172.21.15.16:0/1152296653 shutdown_connections 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 --2- 172.21.15.16:0/1152296653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a81026d0 0x7f42a8108f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 --2- 172.21.15.16:0/1152296653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a8101d10 0x7f42a8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 --2- 172.21.15.16:0/1152296653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8100b10 0x7f42a8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 -- 172.21.15.16:0/1152296653 >> 172.21.15.16:0/1152296653 conn(0x7f42a80fc2a0 msgr2=0x7f42a80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 -- 172.21.15.16:0/1152296653 shutdown_connections 2024-09-12T00:55:39.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.127+0000 7f42acc10640 1 -- 172.21.15.16:0/1152296653 wait complete. 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 Processor -- start 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 -- start start 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8101d10 0x7f42a81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a81026d0 0x7f42a819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c770 con 0x7f42a8100b10 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c5f0 con 0x7f42a8101d10 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42acc10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42a810c8f0 con 0x7f42a81026d0 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38588/0 (socket says 172.21.15.16:38588) 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42a659c640 1 -- 172.21.15.16:0/214646683 learned_addr learned my addr 172.21.15.16:0/214646683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42a5d9b640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8101d10 0x7f42a81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.128+0000 7f42a6d9d640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a81026d0 0x7f42a819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 -- 172.21.15.16:0/214646683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a81026d0 msgr2=0x7f42a819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a81026d0 0x7f42a819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 -- 172.21.15.16:0/214646683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8101d10 msgr2=0x7f42a81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8101d10 0x7f42a81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 -- 172.21.15.16:0/214646683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a819fac0 con 0x7f42a8100b10 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42a659c640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f429800ccb0 tx=0x7f429800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298003990 con 0x7f42a8100b10 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4298003b30 con 0x7f42a8100b10 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a819fdb0 con 0x7f42a8100b10 2024-09-12T00:55:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429800f7b0 con 0x7f42a8100b10 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.129+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a81a0260 con 0x7f42a8100b10 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.130+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a81a04f0 con 0x7f42a8100b10 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.131+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4298004a40 con 0x7f42a8100b10 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.132+0000 7f42977fe640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 0x7f42900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.132+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f42980cf360 con 0x7f42a8100b10 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.132+0000 7f42a5d9b640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 0x7f42900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.132+0000 7f42a5d9b640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 0x7f42900a50f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42a81993b0 tx=0x7f429c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.135+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4298014030 con 0x7f42a8100b10 2024-09-12T00:55:39.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.318+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f42a8062630 con 0x7f42900a2c30 2024-09-12T00:55:39.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:39 smithi016 ceph-mon[30972]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.320+0000 7f42977fe640 1 -- 172.21.15.16:0/214646683 <== mgr.14150 v2:172.21.15.16:6800/1272021982 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f42a8062630 con 0x7f42900a2c30 2024-09-12T00:55:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T00:55:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-12T00:55:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T00:55:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T00:55:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 msgr2=0x7f42900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 0x7f42900a50f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f42a81993b0 tx=0x7f429c00b410 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 msgr2=0x7f42a8197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f429800ccb0 tx=0x7f429800a620 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 shutdown_connections 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f42900a2c30 0x7f42900a50f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a81026d0 0x7f42a819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42a8101d10 0x7f42a81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 --2- 172.21.15.16:0/214646683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a8100b10 0x7f42a8197e90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 >> 172.21.15.16:0/214646683 conn(0x7f42a80fc2a0 msgr2=0x7f42a80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 shutdown_connections 2024-09-12T00:55:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:39.322+0000 7f42acc10640 1 -- 172.21.15.16:0/214646683 wait complete. 2024-09-12T00:55:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:39 smithi159 ceph-mon[35566]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:39 smithi099 ceph-mon[35654]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:40.144 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T00:55:40.153 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:55:40.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T00:55:40.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:40 smithi159 ceph-mon[35566]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:40.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:40 smithi159 ceph-mon[35566]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T00:55:40.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:40 smithi016 ceph-mon[30972]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:40.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:40 smithi016 ceph-mon[30972]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T00:55:40.414 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:40 smithi099 ceph-mon[35654]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T00:55:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:40 smithi099 ceph-mon[35654]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 -- 172.21.15.16:0/2482193055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc102e00 msgr2=0x7fe4cc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/2482193055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc102e00 0x7fe4cc103280 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009900 tx=0x7fe4bc02a410 comp rx=0 tx=0).stop 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 -- 172.21.15.16:0/2482193055 shutdown_connections 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/2482193055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/2482193055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc102e00 0x7fe4cc103280 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/2482193055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc101c00 0x7fe4cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:40.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 -- 172.21.15.16:0/2482193055 >> 172.21.15.16:0/2482193055 conn(0x7fe4cc0fd390 msgr2=0x7fe4cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 -- 172.21.15.16:0/2482193055 shutdown_connections 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.898+0000 7fe4d3d13640 1 -- 172.21.15.16:0/2482193055 wait complete. 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 Processor -- start 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 -- start start 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc101c00 0x7fe4cc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 0x7fe4cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10d880 con 0x7fe4cc1037c0 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10d700 con 0x7fe4cc102e00 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d3d13640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10da00 con 0x7fe4cc101c00 2024-09-12T00:55:40.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38608/0 (socket says 172.21.15.16:38608) 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 -- 172.21.15.16:0/1111313366 learned_addr learned my addr 172.21.15.16:0/1111313366 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d1aaf640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc101c00 0x7fe4cc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d12ae640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 0x7fe4cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 -- 172.21.15.16:0/1111313366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc101c00 msgr2=0x7fe4cc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc101c00 0x7fe4cc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 -- 172.21.15.16:0/1111313366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 msgr2=0x7fe4cc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 0x7fe4cc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d22b0640 1 -- 172.21.15.16:0/1111313366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19fb00 con 0x7fe4cc1037c0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.899+0000 7fe4d12ae640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 0x7fe4cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4d22b0640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eaa0 tx=0x7fe4c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c400cdb0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4005020 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40116b0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc19fdf0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.900+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a02f0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.901+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc108fd0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.902+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe4c4005560 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.903+0000 7fe4c2ffd640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 0x7fe4b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.903+0000 7fe4d1aaf640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 0x7fe4b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.903+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fe4c40cf1f0 con 0x7fe4cc1037c0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.903+0000 7fe4d1aaf640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 0x7fe4b80a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc102c60 tx=0x7fe4c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:40.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:40.907+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4c4016030 con 0x7fe4cc1037c0 2024-09-12T00:55:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.122+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe4cc02d7c0 con 0x7fe4cc1037c0 2024-09-12T00:55:41.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.122+0000 7fe4c2ffd640 1 -- 172.21.15.16:0/1111313366 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3125 (secure 0 0 0) 0x7fe4c409c100 con 0x7fe4cc1037c0 2024-09-12T00:55:41.121 INFO:teuthology.orchestra.run.smithi016.stdout:e27 2024-09-12T00:55:41.121 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-12T00:54:16:816953+0000 2024-09-12T00:55:41.121 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 27 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-12T00:54:04.465326+0000 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-12T00:54:16.816930+0000 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-12T00:55:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 5 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2,3,4 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24505,1=24487,2=24482,3=24481,4=24452} 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-09-12T00:55:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24505 members: 24505,24487,24482,24481,24452 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{0:24505} state up:active seq 7 addr [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/310857906,v1:172.21.15.159:6833/310857906] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{1:24487} state up:active seq 10 addr [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/3000703552,v1:172.21.15.99:6835/3000703552] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{2:24482} state up:active seq 7 addr [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{2:24410} state up:standby-replay seq 1 addr [v2:172.21.15.99:6832/4113169911,v1:172.21.15.99:6833/4113169911] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{3:24481} state up:active seq 11 addr [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{3:14601} state up:standby-replay seq 1 addr [v2:172.21.15.16:6840/2826352912,v1:172.21.15.16:6841/2826352912] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{4:24452} state up:active seq 10 addr [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{4:14589} state up:standby-replay seq 1 addr [v2:172.21.15.16:6838/3482588678,v1:172.21.15.16:6839/3482588678] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.16:6834/2008522291,v1:172.21.15.16:6835/2008522291] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.125 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.16:6836/2601027197,v1:172.21.15.16:6837/2601027197] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 msgr2=0x7fe4b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 0x7fe4b80a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc102c60 tx=0x7fe4c800b390 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 msgr2=0x7fe4cc19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eaa0 tx=0x7fe4c400ef60 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 shutdown_connections 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe4b80a2c30 0x7fe4b80a50f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4cc1037c0 0x7fe4cc19f490 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4cc102e00 0x7fe4cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 --2- 172.21.15.16:0/1111313366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4cc101c00 0x7fe4cc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:41.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.125+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 >> 172.21.15.16:0/1111313366 conn(0x7fe4cc0fd390 msgr2=0x7fe4cc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.126+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 shutdown_connections 2024-09-12T00:55:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:41.126+0000 7fe4d3d13640 1 -- 172.21.15.16:0/1111313366 wait complete. 2024-09-12T00:55:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:41 smithi159 ceph-mon[35566]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:41.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:41 smithi016 ceph-mon[30972]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:41 smithi099 ceph-mon[35654]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:55:41.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T00:55:42.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:42 smithi016 ceph-mon[30972]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:42.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:42 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1111313366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T00:55:42.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:42 smithi159 ceph-mon[35566]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:42.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:42 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1111313366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T00:55:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:42 smithi099 ceph-mon[35654]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:42.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:42 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1111313366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 -- 172.21.15.16:0/969331803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 msgr2=0x7fe634104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 --2- 172.21.15.16:0/969331803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634104a90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe630009900 tx=0x7fe63002a410 comp rx=0 tx=0).stop 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 -- 172.21.15.16:0/969331803 shutdown_connections 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 --2- 172.21.15.16:0/969331803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe634109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 --2- 172.21.15.16:0/969331803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe634104fd0 0x7fe6341073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 --2- 172.21.15.16:0/969331803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634104a90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.658+0000 7fe63c294640 1 -- 172.21.15.16:0/969331803 >> 172.21.15.16:0/969331803 conn(0x7fe6340fc2a0 msgr2=0x7fe6340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.659+0000 7fe63c294640 1 -- 172.21.15.16:0/969331803 shutdown_connections 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.659+0000 7fe63c294640 1 -- 172.21.15.16:0/969331803 wait complete. 2024-09-12T00:55:42.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.659+0000 7fe63c294640 1 Processor -- start 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.659+0000 7fe63c294640 1 -- start start 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe634104fd0 0x7fe6341985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe63419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410d990 con 0x7fe6341026a0 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410d810 con 0x7fe634107900 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe63410db10 con 0x7fe634104fd0 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38622/0 (socket says 172.21.15.16:38622) 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 -- 172.21.15.16:0/3214435760 learned_addr learned my addr 172.21.15.16:0/3214435760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a831640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe63419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 -- 172.21.15.16:0/3214435760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe634104fd0 msgr2=0x7fe6341985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe634104fd0 0x7fe6341985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 -- 172.21.15.16:0/3214435760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 msgr2=0x7fe63419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe63419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 -- 172.21.15.16:0/3214435760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63419fca0 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a831640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe63419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63a030640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe630009230 tx=0x7fe630004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63003f030 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63419ff30 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe630003ce0 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630035070 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.660+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341a0410 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.661+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6340fdd90 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.663+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe630042030 con 0x7fe6341026a0 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.663+0000 7fe62b7fe640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 0x7fe6200a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.663+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fe6300ef820 con 0x7fe6341026a0 2024-09-12T00:55:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.663+0000 7fe63982f640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 0x7fe6200a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.664+0000 7fe63982f640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 0x7fe6200a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe634199590 tx=0x7fe624006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.667+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6300bc790 con 0x7fe6341026a0 2024-09-12T00:55:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.843+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe63402d9e0 con 0x7fe6341026a0 2024-09-12T00:55:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.844+0000 7fe62b7fe640 1 -- 172.21.15.16:0/3214435760 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5250 (secure 0 0 0) 0x7fe6300bc090 con 0x7fe6341026a0 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 84 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-12T00:46:20.927223+0000 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-12T00:55:09.318256+0000 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-09-12T00:55:42.843 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/4092481179,v1:172.21.15.16:6803/4092481179] [v2:172.21.15.16:6804/4092481179,v1:172.21.15.16:6805/4092481179] exists,up 717d4d96-381a-4246-826b-f3ca949a2f46 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/524186263,v1:172.21.15.99:6801/524186263] [v2:172.21.15.99:6802/524186263,v1:172.21.15.99:6803/524186263] exists,up 5541ef54-29b2-4099-bc7b-3b758b6f0a6d 2024-09-12T00:55:42.844 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/3066904609,v1:172.21.15.159:6801/3066904609] [v2:172.21.15.159:6802/3066904609,v1:172.21.15.159:6803/3066904609] exists,up f46058bc-91f3-4062-ab02-30e93a02b1d4 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/846431474,v1:172.21.15.16:6811/846431474] [v2:172.21.15.16:6812/846431474,v1:172.21.15.16:6813/846431474] exists,up ae799129-d588-4925-a79d-869205ffc0d8 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 32 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/2525858476,v1:172.21.15.99:6809/2525858476] [v2:172.21.15.99:6810/2525858476,v1:172.21.15.99:6811/2525858476] exists,up 70a24b6e-0414-4620-a0ec-89f48dedbef7 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/516420178,v1:172.21.15.159:6809/516420178] [v2:172.21.15.159:6810/516420178,v1:172.21.15.159:6811/516420178] exists,up fff6ef9d-4e03-4343-8e20-fa384843bc9b 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 44 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/4235557256,v1:172.21.15.16:6819/4235557256] [v2:172.21.15.16:6820/4235557256,v1:172.21.15.16:6821/4235557256] exists,up 9c365b94-3cbc-4aec-b6a6-d503fd51fd0e 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 49 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/777307661,v1:172.21.15.99:6817/777307661] [v2:172.21.15.99:6818/777307661,v1:172.21.15.99:6819/777307661] exists,up 6e1b90b2-de0e-4bd2-9cab-01595a0ec06e 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 56 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/274647051,v1:172.21.15.159:6817/274647051] [v2:172.21.15.159:6818/274647051,v1:172.21.15.159:6819/274647051] exists,up 2c93a157-0688-48d0-a978-1cf9ef2cd1ba 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 63 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/45245397,v1:172.21.15.16:6827/45245397] [v2:172.21.15.16:6828/45245397,v1:172.21.15.16:6829/45245397] exists,up fd2b63bf-d329-4318-87c5-945c8d0b12fb 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 68 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/1469352212,v1:172.21.15.99:6825/1469352212] [v2:172.21.15.99:6826/1469352212,v1:172.21.15.99:6827/1469352212] exists,up df53f8db-e889-404c-a107-c3fde59b80b5 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 76 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/4132934366,v1:172.21.15.159:6825/4132934366] [v2:172.21.15.159:6826/4132934366,v1:172.21.15.159:6827/4132934366] exists,up a6545934-6ceb-4f96-aecf-d9ded7ceefde 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.6 [8,11] 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.b [8,11,2,1] 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.14 [2,4] 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.3 [4,9] 2024-09-12T00:55:42.845 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.e [4,6] 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.11 [4,8] 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1d [5,9] 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.38 [3,9] 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/1538480573 expires 2024-09-13T00:47:08.126530+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2087944877 expires 2024-09-13T00:47:08.126530+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1789090922 expires 2024-09-13T00:47:08.126530+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2269637621 expires 2024-09-13T00:46:43.483931+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/1538480573 expires 2024-09-13T00:47:08.126530+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3359074253 expires 2024-09-13T00:46:43.483931+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1286739529 expires 2024-09-13T00:47:08.126530+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1067528076 expires 2024-09-13T00:46:43.483931+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/7504959 expires 2024-09-13T00:46:43.483931+0000 2024-09-12T00:55:42.846 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/7504959 expires 2024-09-13T00:46:43.483931+0000 2024-09-12T00:55:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 msgr2=0x7fe6200a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 0x7fe6200a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe634199590 tx=0x7fe624006c90 comp rx=0 tx=0).stop 2024-09-12T00:55:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 msgr2=0x7fe634198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe630009230 tx=0x7fe630004430 comp rx=0 tx=0).stop 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 shutdown_connections 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fe6200a2bb0 0x7fe6200a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe634107900 0x7fe63419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe634104fd0 0x7fe6341985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 --2- 172.21.15.16:0/3214435760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6341026a0 0x7fe634198070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 >> 172.21.15.16:0/3214435760 conn(0x7fe6340fc2a0 msgr2=0x7fe6341030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 shutdown_connections 2024-09-12T00:55:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:42.847+0000 7fe63c294640 1 -- 172.21.15.16:0/3214435760 wait complete. 2024-09-12T00:55:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:43 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3214435760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T00:55:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:43 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3214435760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T00:55:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:43 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3214435760' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T00:55:43.550 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T00:55:43.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:55:43.823 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:44.261 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:44 smithi016 ceph-mon[30972]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T00:55:44.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:44 smithi159 ceph-mon[35566]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 -- 172.21.15.16:0/3703466337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774100e20 msgr2=0x7fb774101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3703466337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774100e20 0x7fb774101200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb764009900 tx=0x7fb76402a480 comp rx=0 tx=0).stop 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 -- 172.21.15.16:0/3703466337 shutdown_connections 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3703466337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 0x7fb77410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3703466337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774102570 0x7fb7741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3703466337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774100e20 0x7fb774101200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 -- 172.21.15.16:0/3703466337 >> 172.21.15.16:0/3703466337 conn(0x7fb7740fc7d0 msgr2=0x7fb7740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 -- 172.21.15.16:0/3703466337 shutdown_connections 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.345+0000 7fb7795c9640 1 -- 172.21.15.16:0/3703466337 wait complete. 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 Processor -- start 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 -- start start 2024-09-12T00:55:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 0x7fb774199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 0x7fb774199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb774110ff0 con 0x7fb774102570 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb774110e70 con 0x7fb774100e20 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7795c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb774111170 con 0x7fb774102ef0 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7727fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7727fc640 1 -- 172.21.15.16:0/3688645382 learned_addr learned my addr 172.21.15.16:0/3688645382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb7737fe640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 0x7fb774199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.346+0000 7fb772ffd640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 0x7fb774199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 -- 172.21.15.16:0/3688645382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 msgr2=0x7fb774199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 0x7fb774199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 -- 172.21.15.16:0/3688645382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 msgr2=0x7fb774199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 0x7fb774199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 -- 172.21.15.16:0/3688645382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741a4880 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb772ffd640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 0x7fb774199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7727fc640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c890 tx=0x7fb76800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768004220 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7680043c0 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741a4b70 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680049e0 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.347+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741a50b0 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.348+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774101200 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.349+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7680079b0 con 0x7fb774102570 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.350+0000 7fb75bfff640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 0x7fb75c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.350+0000 7fb772ffd640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 0x7fb75c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.350+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb7680cf090 con 0x7fb774102570 2024-09-12T00:55:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.350+0000 7fb772ffd640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 0x7fb75c0a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb764000c00 tx=0x7fb764005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.353+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb768014030 con 0x7fb774102570 2024-09-12T00:55:44.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:44 smithi099 ceph-mon[35654]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T00:55:44.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.563+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb774109090 con 0x7fb774102570 2024-09-12T00:55:44.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.564+0000 7fb75bfff640 1 -- 172.21.15.16:0/3688645382 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7fb7680afe20 con 0x7fb774102570 2024-09-12T00:55:44.563 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:44.563 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:55:44.563 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 msgr2=0x7fb75c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 0x7fb75c0a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb764000c00 tx=0x7fb764005a20 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 msgr2=0x7fb7741999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb76800c890 tx=0x7fb76800cd50 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 shutdown_connections 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb75c0a2b80 0x7fb75c0a5040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb774102ef0 0x7fb774199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb774102570 0x7fb7741999c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 --2- 172.21.15.16:0/3688645382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb774100e20 0x7fb774199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.566+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 >> 172.21.15.16:0/3688645382 conn(0x7fb7740fc7d0 msgr2=0x7fb7741068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.567+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 shutdown_connections 2024-09-12T00:55:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:44.567+0000 7fb7795c9640 1 -- 172.21.15.16:0/3688645382 wait complete. 2024-09-12T00:55:45.217 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:45 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3688645382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:45.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:55:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:45 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3688645382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:45.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:45 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3688645382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.993+0000 7f9e6de3a640 1 -- 172.21.15.16:0/2647782869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 msgr2=0x7f9e6810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.993+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/2647782869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e6810eaf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009cd0 tx=0x7f9e6002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 -- 172.21.15.16:0/2647782869 shutdown_connections 2024-09-12T00:55:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/2647782869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e6810eaf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/2647782869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e68100ca0 0x7f9e68106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/2647782869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e681002f0 0x7f9e681006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 -- 172.21.15.16:0/2647782869 >> 172.21.15.16:0/2647782869 conn(0x7f9e68077210 msgr2=0x7f9e680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 -- 172.21.15.16:0/2647782869 shutdown_connections 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 -- 172.21.15.16:0/2647782869 wait complete. 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 Processor -- start 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.994+0000 7f9e6de3a640 1 -- start start 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e681002f0 0x7f9e68101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e68100ca0 0x7f9e681017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68111fa0 con 0x7f9e68106f50 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68111e20 con 0x7f9e68100ca0 2024-09-12T00:55:46.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68112120 con 0x7f9e681002f0 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38664/0 (socket says 172.21.15.16:38664) 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 -- 172.21.15.16:0/3219410039 learned_addr learned my addr 172.21.15.16:0/3219410039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e66ffd640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e68100ca0 0x7f9e681017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e677fe640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e681002f0 0x7f9e68101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 -- 172.21.15.16:0/3219410039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e681002f0 msgr2=0x7f9e68101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e681002f0 0x7f9e68101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 -- 172.21.15.16:0/3219410039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e68100ca0 msgr2=0x7f9e681017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e68100ca0 0x7f9e681017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 -- 172.21.15.16:0/3219410039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e681a6960 con 0x7f9e68106f50 2024-09-12T00:55:46.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e67fff640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005cc0 tx=0x7f9e60003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60004040 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e681a6bf0 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.995+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e600046c0 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.996+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60046480 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.996+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e681a7180 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.996+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e681006d0 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.998+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e600305a0 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.998+0000 7f9e64ff9640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 0x7f9e500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.998+0000 7f9e677fe640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 0x7f9e500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.998+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f9e6002e030 con 0x7f9e68106f50 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:45.999+0000 7f9e677fe640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 0x7f9e500a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e58005fd0 tx=0x7f9e58005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.002+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e6004b060 con 0x7f9e68106f50 2024-09-12T00:55:46.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.220+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e6810a120 con 0x7f9e68106f50 2024-09-12T00:55:46.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:46 smithi016 ceph-mon[30972]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:46.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.221+0000 7f9e64ff9640 1 -- 172.21.15.16:0/3219410039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11968 (secure 0 0 0) 0x7f9e600bd590 con 0x7f9e68106f50 2024-09-12T00:55:46.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:46.221 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":27,"btime":"2024-09-12T00:54:16:816953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:54:16.816930+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24505,24487,24482,24481,24452]},"id":1}]} 2024-09-12T00:55:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 27 2024-09-12T00:55:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 msgr2=0x7f9e500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 0x7f9e500a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e58005fd0 tx=0x7f9e58005850 comp rx=0 tx=0).stop 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 msgr2=0x7f9e68101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e60005cc0 tx=0x7f9e60003970 comp rx=0 tx=0).stop 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 shutdown_connections 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f9e500a2b80 0x7f9e500a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e68106f50 0x7f9e68101cf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e68100ca0 0x7f9e681017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 --2- 172.21.15.16:0/3219410039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e681002f0 0x7f9e68101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 >> 172.21.15.16:0/3219410039 conn(0x7f9e68077210 msgr2=0x7f9e68105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 shutdown_connections 2024-09-12T00:55:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:46.223+0000 7f9e6de3a640 1 -- 172.21.15.16:0/3219410039 wait complete. 2024-09-12T00:55:46.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:46 smithi159 ceph-mon[35566]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:46.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:46 smithi099 ceph-mon[35654]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:47.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:55:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3219410039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3219410039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3219410039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2803935752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb010c170 msgr2=0x7f0eb010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2803935752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb010c170 0x7f0eb010e610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009ca0 tx=0x7f0ea802a950 comp rx=0 tx=0).stop 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2803935752 shutdown_connections 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2803935752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb010c170 0x7f0eb010e610 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2803935752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0103660 0x7f0eb010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2803935752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb0102d40 0x7f0eb0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2803935752 >> 172.21.15.16:0/2803935752 conn(0x7f0eb00fc7a0 msgr2=0x7f0eb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2803935752 shutdown_connections 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.824+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2803935752 wait complete. 2024-09-12T00:55:47.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 Processor -- start 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 -- start start 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0102d40 0x7f0eb0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb010c170 0x7f0eb019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0111f70 con 0x7f0eb0103660 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb0111df0 con 0x7f0eb0102d40 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eb61f6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb01120f0 con 0x7f0eb010c170 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38674/0 (socket says 172.21.15.16:38674) 2024-09-12T00:55:47.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 -- 172.21.15.16:0/2608006434 learned_addr learned my addr 172.21.15.16:0/2608006434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaf7fe640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0102d40 0x7f0eb0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaffff640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb010c170 0x7f0eb019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 -- 172.21.15.16:0/2608006434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb010c170 msgr2=0x7f0eb019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb010c170 0x7f0eb019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 -- 172.21.15.16:0/2608006434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0102d40 msgr2=0x7f0eb0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0102d40 0x7f0eb0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.825+0000 7f0eaeffd640 1 -- 172.21.15.16:0/2608006434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb019d340 con 0x7f0eb0103660 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eaeffd640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c970 tx=0x7f0ea400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4007bb0 con 0x7f0eb0103660 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb019d630 con 0x7f0eb0103660 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4007d50 con 0x7f0eb0103660 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4018820 con 0x7f0eb0103660 2024-09-12T00:55:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.826+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb01a4f10 con 0x7f0eb0103660 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.827+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb0103120 con 0x7f0eb0103660 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.828+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ea40189c0 con 0x7f0eb0103660 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.829+0000 7f0eacff9640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.829+0000 7f0eaf7fe640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.829+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f0ea40cefb0 con 0x7f0eb0103660 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.829+0000 7f0eaf7fe640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 0x7f0e980a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005fd0 tx=0x7f0ea000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:47.833+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ea4014030 con 0x7f0eb0103660 2024-09-12T00:55:48.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.010+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eb01a5250 con 0x7f0eb0103660 2024-09-12T00:55:48.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.012+0000 7f0eacff9640 1 -- 172.21.15.16:0/2608006434 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22641 (secure 0 0 0) 0x7f0ea409bf20 con 0x7f0eb0103660 2024-09-12T00:55:48.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:55:48.011 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":84,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:55:09.318256+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 msgr2=0x7f0e980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 0x7f0e980a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005fd0 tx=0x7f0ea000b390 comp rx=0 tx=0).stop 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 msgr2=0x7f0eb01988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c970 tx=0x7f0ea400ce30 comp rx=0 tx=0).stop 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 shutdown_connections 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb010c170 0x7f0eb019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb0103660 0x7f0eb01988d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 --2- 172.21.15.16:0/2608006434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0eb0102d40 0x7f0eb0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:48.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.014+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 >> 172.21.15.16:0/2608006434 conn(0x7f0eb00fc7a0 msgr2=0x7f0eb00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:48.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.015+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 shutdown_connections 2024-09-12T00:55:48.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:48.015+0000 7f0eb61f6640 1 -- 172.21.15.16:0/2608006434 wait complete. 2024-09-12T00:55:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:48 smithi159 ceph-mon[35566]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:48 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2608006434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:48 smithi016 ceph-mon[30972]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:48.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:48 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2608006434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:48.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:48 smithi099 ceph-mon[35654]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:48.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:48 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2608006434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:55:48.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T00:54:04.465326+0000', 'modified': '2024-09-12T00:54:16.816930+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24505, 'mds_1': 24487, 'mds_2': 24482, 'mds_3': 24481, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3482588678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3482588678}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3482588678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/2826352912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 2826352912}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 2826352912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/4113169911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 4113169911}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 4113169911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/3000703552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 3000703552}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 3000703552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/310857906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 310857906}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 310857906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/1045510613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 1045510613}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 1045510613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1983976622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1983976622}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1983976622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/850851643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 850851643}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 850851643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.159:6837/247659815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 247659815}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 247659815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.159:6839/1957754994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 1957754994}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 1957754994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24505, 'qdb_cluster': [24505, 24487, 24482, 24481, 24452]} 2024-09-12T00:55:48.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T00:55:48.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T00:55:49.203 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 -- 172.21.15.16:0/3895440637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 msgr2=0x7f2c70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 --2- 172.21.15.16:0/3895440637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c70101200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c009900 tx=0x7f2c6c02a480 comp rx=0 tx=0).stop 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 -- 172.21.15.16:0/3895440637 shutdown_connections 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 --2- 172.21.15.16:0/3895440637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102ef0 0x7f2c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 --2- 172.21.15.16:0/3895440637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102570 0x7f2c701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 --2- 172.21.15.16:0/3895440637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c70101200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 -- 172.21.15.16:0/3895440637 >> 172.21.15.16:0/3895440637 conn(0x7f2c700fc7d0 msgr2=0x7f2c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 -- 172.21.15.16:0/3895440637 shutdown_connections 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.683+0000 7f2c7726a640 1 -- 172.21.15.16:0/3895440637 wait complete. 2024-09-12T00:55:49.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 Processor -- start 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 -- start start 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 0x7f2c700fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102ef0 0x7f2c700fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70110fa0 con 0x7f2c70100e20 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70110e20 con 0x7f2c70102570 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c7726a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70111120 con 0x7f2c70102ef0 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57680/0 (socket says 172.21.15.16:57680) 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 -- 172.21.15.16:0/2888783215 learned_addr learned my addr 172.21.15.16:0/2888783215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75807640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102ef0 0x7f2c700fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c74805640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 0x7f2c700fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 -- 172.21.15.16:0/2888783215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102ef0 msgr2=0x7f2c700fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102ef0 0x7f2c700fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 -- 172.21.15.16:0/2888783215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 msgr2=0x7f2c700fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 0x7f2c700fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c75006640 1 -- 172.21.15.16:0/2888783215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c701a6990 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.684+0000 7f2c74805640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 0x7f2c700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c75006640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006010 tx=0x7f2c6c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c667fc640 1 -- 172.21.15.16:0/2888783215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c03f070 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c701a6c20 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c667fc640 1 -- 172.21.15.16:0/2888783215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6c031e30 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c667fc640 1 -- 172.21.15.16:0/2888783215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c035030 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.685+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c701a7150 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.687+0000 7f2c667fc640 1 -- 172.21.15.16:0/2888783215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c6c042070 con 0x7f2c70100e20 2024-09-12T00:55:49.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.687+0000 7f2c79643740 1 Processor -- start 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 -- start start 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a742f0 0x555fd3a7e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a80fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd39da090 con 0x555fd3a739d0 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555fd3981a70 con 0x555fd3a742f0 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c79643740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fd39d6050 con 0x555fd3a7ebb0 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c667fc640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 0x7f2c5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57690/0 (socket says 172.21.15.16:57690) 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 -- 172.21.15.16:0/1817956882 learned_addr learned my addr 172.21.15.16:0/1817956882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75006640 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a742f0 0x555fd3a7e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 0x7f2c5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c667fc640 1 -- 172.21.15.16:0/2888783215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f2c6c0efa70 con 0x7f2c70100e20 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c74805640 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a80fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 -- 172.21.15.16:0/1817956882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 msgr2=0x555fd3a80fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a80fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 -- 172.21.15.16:0/1817956882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a742f0 msgr2=0x555fd3a7e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a742f0 0x555fd3a7e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.688+0000 7f2c75807640 1 -- 172.21.15.16:0/1817956882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd398e130 con 0x555fd3a739d0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.689+0000 7f2c75807640 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800edd0 tx=0x7f2c68007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04b88ea7ce80340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.689+0000 7f2c75807640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 0x7f2c5c0a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c68007b80 tx=0x7f2c6801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.689+0000 7f2c5b7fe640 1 -- 172.21.15.16:0/1817956882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68007e50 con 0x555fd3a739d0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.689+0000 7f2c5b7fe640 1 -- 172.21.15.16:0/1817956882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6801ab30 con 0x555fd3a739d0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.689+0000 7f2c5b7fe640 1 -- 172.21.15.16:0/1817956882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801bb20 con 0x555fd3a739d0 2024-09-12T00:55:49.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- 172.21.15.16:0/1817956882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 msgr2=0x555fd3a73db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800edd0 tx=0x7f2c68007680 comp rx=0 tx=0).stop 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- 172.21.15.16:0/1817956882 shutdown_connections 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a80fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a742f0 0x555fd3a7e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- 172.21.15.16:0/1817956882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a739d0 0x555fd3a73db0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- 172.21.15.16:0/1817956882 >> 172.21.15.16:0/1817956882 conn(0x555fd3a6ff60 msgr2=0x555fd3a70340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- 172.21.15.16:0/1817956882 shutdown_connections 2024-09-12T00:55:49.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- 172.21.15.16:0/1817956882 wait complete. 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- start start 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 Processor -- start 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a81780 con 0x555fd3a7ebb0 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555fd3994f20 con 0x555fd3a739d0 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.690+0000 7f2c79643740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a81900 con 0x555fd3a742f0 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c79643740 10 client.0 _ll_register_callbacks cb 0x555fd395e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57694/0 (socket says 172.21.15.16:57694) 2024-09-12T00:55:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 -- 172.21.15.16:0/4255912066 learned_addr learned my addr 172.21.15.16:0/4255912066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75006640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 -- 172.21.15.16:0/4255912066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 msgr2=0x555fd3a7b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 -- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 msgr2=0x555fd3a7e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a7e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 -- 172.21.15.16:0/4255912066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd374b2b0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75006640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c75807640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009a90 tx=0x7f2c680097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4ac162a2951489 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800a810 con 0x555fd3a7ebb0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f2c6800a9b0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c6801ab30 con 0x555fd3a7ebb0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c68007e50 con 0x555fd3a7ebb0 2024-09-12T00:55:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c79643740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.691+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fd3a8d0f0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6801ab30 con 0x555fd3a7ebb0 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2c6802dcd0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 20 client.15111 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3a8f630 0x555fd3a90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 4 client.15111 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd39e7600 con 0x555fd3a8f630 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.692+0000 7f2c75006640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3a8f630 0x555fd3a90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c75006640 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3a8f630 0x555fd3a90aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59121bba036d93ba server_cookie=4389c9ab0808184e in_seq=0 out_seq=0 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c58ff9640 10 client.15111 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:49.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c58ff9640 1 -- 172.21.15.16:0/4255912066 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fd39e7600 con 0x555fd3a8f630 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c58ff9640 10 client.15111 handle_command_reply: tid=0 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c79643740 1 client.15111 shutdown 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 msgr2=0x555fd3a7bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.693+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009a90 tx=0x7f2c680097c0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 shutdown_connections 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3a8f630 0x555fd3a90aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3a7bd40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a7b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c58ff9640 0 client.15111 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 --2- 172.21.15.16:0/4255912066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 >> 172.21.15.16:0/4255912066 conn(0x555fd3a6ff60 msgr2=0x555fd3a75c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 shutdown_connections 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 1 -- 172.21.15.16:0/4255912066 wait complete. 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.694+0000 7f2c79643740 20 client.15111 trim_cache size 0 max 16384 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 Processor -- start 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 -- start start 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a90aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a8ede0 con 0x555fd3a742f0 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a8f4b0 con 0x555fd3a739d0 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c79643740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a8fb80 con 0x555fd3a7ebb0 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57708/0 (socket says 172.21.15.16:57708) 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 -- 172.21.15.16:0/1281711766 learned_addr learned my addr 172.21.15.16:0/1281711766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c75006640 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a90aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 -- 172.21.15.16:0/1281711766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 msgr2=0x555fd3a992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 -- 172.21.15.16:0/1281711766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 msgr2=0x555fd3a90aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a90aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 -- 172.21.15.16:0/1281711766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd3a7c340 con 0x555fd3a742f0 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.712+0000 7f2c74805640 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c700fe710 tx=0x7f2c6000df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191db201f4acd5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c5affd640 1 -- 172.21.15.16:0/1281711766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60005020 con 0x555fd3a742f0 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c5affd640 1 -- 172.21.15.16:0/1281711766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c600047a0 con 0x555fd3a742f0 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c5affd640 1 -- 172.21.15.16:0/1281711766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6001a790 con 0x555fd3a742f0 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 -- 172.21.15.16:0/1281711766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 msgr2=0x555fd3a98d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2c700fe710 tx=0x7f2c6000df30 comp rx=0 tx=0).stop 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 -- 172.21.15.16:0/1281711766 shutdown_connections 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a7ebb0 0x555fd3a992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a742f0 0x555fd3a98d80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 --2- 172.21.15.16:0/1281711766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a90aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 -- 172.21.15.16:0/1281711766 >> 172.21.15.16:0/1281711766 conn(0x555fd3a6ff60 msgr2=0x555fd3a7a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 -- 172.21.15.16:0/1281711766 shutdown_connections 2024-09-12T00:55:49.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.713+0000 7f2c79643740 1 -- 172.21.15.16:0/1281711766 wait complete. 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 -- start start 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 Processor -- start 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a90920 con 0x555fd3a7ebb0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a90250 con 0x555fd3a739d0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fd3a90ff0 con 0x555fd3a742f0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c79643740 10 client.0 _ll_register_callbacks cb 0x7f2c6c003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c75807640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57720/0 (socket says 172.21.15.16:57720) 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c75807640 1 -- 172.21.15.16:0/3003007613 learned_addr learned my addr 172.21.15.16:0/3003007613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c75006640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.714+0000 7f2c74805640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 -- 172.21.15.16:0/3003007613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 msgr2=0x555fd3a94970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a94970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 -- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 msgr2=0x555fd3a94430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a94430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 -- 172.21.15.16:0/3003007613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd3af8340 con 0x555fd3a7ebb0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c75807640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f80 tx=0x7f2c6800e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb6c658525659eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801bca0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f2c6800a610 con 0x555fd3a7ebb0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c600047a0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c6801be40 con 0x555fd3a7ebb0 2024-09-12T00:55:49.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555fd3af9600 con 0x555fd3a7ebb0 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c600047a0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2c6802d4a0 con 0x555fd3a7ebb0 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.715+0000 7f2c79643740 20 client.15117 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3afe160 0x555fd3afce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c79643740 4 client.15117 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "status"}) -- 0x555fd39e1b10 con 0x555fd3afe160 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c75006640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3afe160 0x555fd3afce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c75006640 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3afe160 0x555fd3afce70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8bac7e04245ad3f8 server_cookie=fd3b838927873823 in_seq=0 out_seq=0 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c58ff9640 10 client.15117 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:49.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c58ff9640 1 -- 172.21.15.16:0/3003007613 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555fd39e1b10 con 0x555fd3afe160 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c58ff9640 10 client.15117 handle_command_reply: tid=0 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.716+0000 7f2c79643740 1 client.15117 shutdown 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 msgr2=0x555fd3af7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f80 tx=0x7f2c6800e430 comp rx=0 tx=0).stop 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 shutdown_connections 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555fd3afe160 0x555fd3afce70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fd3a7ebb0 0x555fd3af7d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fd3a742f0 0x555fd3a94970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 --2- 172.21.15.16:0/3003007613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555fd3a739d0 0x555fd3a94430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 >> 172.21.15.16:0/3003007613 conn(0x555fd3a6ff60 msgr2=0x555fd3a6e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c58ff9640 0 client.15117 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 shutdown_connections 2024-09-12T00:55:49.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 1 -- 172.21.15.16:0/3003007613 wait complete. 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.717+0000 7f2c79643740 20 client.15117 trim_cache size 0 max 16384 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24452, 2024-09-12T00:55:49.734 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 93.903786249000007, 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 127.872503526 2024-09-12T00:55:49.735 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 msgr2=0x7f2c5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 0x7f2c5c0a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c68007b80 tx=0x7f2c6801b000 comp rx=0 tx=0).stop 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 msgr2=0x7f2c700fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c006010 tx=0x7f2c6c003c80 comp rx=0 tx=0).stop 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 shutdown_connections 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f2c5c0a2b80 0x7f2c5c0a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70102ef0 0x7f2c700fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c70102570 0x7f2c700fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 --2- 172.21.15.16:0/2888783215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c70100e20 0x7f2c700fcc80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 >> 172.21.15.16:0/2888783215 conn(0x7f2c700fc7d0 msgr2=0x7f2c7006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 shutdown_connections 2024-09-12T00:55:49.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:49.718+0000 7f2c7726a640 1 -- 172.21.15.16:0/2888783215 wait complete. 2024-09-12T00:55:50.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:50 smithi159 ceph-mon[35566]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:50.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:50 smithi016 ceph-mon[30972]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:50.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:50 smithi099 ceph-mon[35654]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T00:55:50.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T00:55:50.814 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:51.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 -- 172.21.15.16:0/1515255874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010106f10 msgr2=0x7f601010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 --2- 172.21.15.16:0/1515255874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010106f10 0x7f601010ea90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6008009ca0 tx=0x7f600802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 -- 172.21.15.16:0/1515255874 shutdown_connections 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 --2- 172.21.15.16:0/1515255874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010106f10 0x7f601010ea90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 --2- 172.21.15.16:0/1515255874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010102790 0x7f60101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 --2- 172.21.15.16:0/1515255874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 -- 172.21.15.16:0/1515255874 >> 172.21.15.16:0/1515255874 conn(0x7f60100fd8c0 msgr2=0x7f60100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 -- 172.21.15.16:0/1515255874 shutdown_connections 2024-09-12T00:55:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.325+0000 7f6017bad640 1 -- 172.21.15.16:0/1515255874 wait complete. 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 Processor -- start 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 -- start start 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010106f10 0x7f6010074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111fc0 con 0x7f6010102790 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111e40 con 0x7f6010101de0 2024-09-12T00:55:51.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.326+0000 7f6017bad640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6010112140 con 0x7f6010106f10 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57740/0 (socket says 172.21.15.16:57740) 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 -- 172.21.15.16:0/2273793235 learned_addr learned my addr 172.21.15.16:0/2273793235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f601614a640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010106f10 0x7f6010074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015949640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 -- 172.21.15.16:0/2273793235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010106f10 msgr2=0x7f6010074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010106f10 0x7f6010074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 -- 172.21.15.16:0/2273793235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 msgr2=0x7f60100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 -- 172.21.15.16:0/2273793235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60101a6860 con 0x7f6010102790 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015949640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6015148640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f600000d770 tx=0x7f600000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6006ffd640 1 -- 172.21.15.16:0/2273793235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000017070 con 0x7f6010102790 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6006ffd640 1 -- 172.21.15.16:0/2273793235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6000005500 con 0x7f6010102790 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6006ffd640 1 -- 172.21.15.16:0/2273793235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000004e00 con 0x7f6010102790 2024-09-12T00:55:51.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60101a6b50 con 0x7f6010102790 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.327+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a7090 con 0x7f6010102790 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6006ffd640 1 -- 172.21.15.16:0/2273793235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6000004fa0 con 0x7f6010102790 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 Processor -- start 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 -- start start 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0de2f0 0x558eab0e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558eab044090 con 0x558eab0e8bb0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558eaafeba70 con 0x558eab0dd9d0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6019f86740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558eab040050 con 0x558eab0de2f0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.330+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57752/0 (socket says 172.21.15.16:57752) 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 -- 172.21.15.16:0/345289110 learned_addr learned my addr 172.21.15.16:0/345289110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015949640 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0de2f0 0x558eab0e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f601614a640 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6006ffd640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 0x7f5ffc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 -- 172.21.15.16:0/345289110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0de2f0 msgr2=0x558eab0e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0de2f0 0x558eab0e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 -- 172.21.15.16:0/345289110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 msgr2=0x558eab0dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0dddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f601614a640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 0x7f5ffc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 -- 172.21.15.16:0/345289110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eaaff8130 con 0x558eab0e8bb0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6006ffd640 1 -- 172.21.15.16:0/2273793235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f60000ce990 con 0x7f6010102790 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f6015148640 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6000008be0 tx=0x7f6000022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f3493acae5028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f601614a640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 0x7f5ffc0a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6008006010 tx=0x7f600802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f5ffbfff640 1 -- 172.21.15.16:0/345289110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009ad40 con 0x558eab0e8bb0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f5ffbfff640 1 -- 172.21.15.16:0/345289110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60000046c0 con 0x558eab0e8bb0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.331+0000 7f5ffbfff640 1 -- 172.21.15.16:0/345289110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000008c10 con 0x558eab0e8bb0 2024-09-12T00:55:51.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 -- 172.21.15.16:0/345289110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 msgr2=0x558eab0eafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6000008be0 tx=0x7f6000022000 comp rx=0 tx=0).stop 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 -- 172.21.15.16:0/345289110 shutdown_connections 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0eafa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0de2f0 0x558eab0e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 --2- 172.21.15.16:0/345289110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 -- 172.21.15.16:0/345289110 >> 172.21.15.16:0/345289110 conn(0x558eab0d9f60 msgr2=0x558eab0da340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 -- 172.21.15.16:0/345289110 shutdown_connections 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.332+0000 7f6019f86740 1 -- 172.21.15.16:0/345289110 wait complete. 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 -- start start 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 Processor -- start 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 0x558eab0e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558eab0eb720 con 0x558eab0de2f0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558eaaffef20 con 0x558eab0dd9d0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558eab0eb8a0 con 0x558eab0e8bb0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6019f86740 10 client.0 _ll_register_callbacks cb 0x558eaafc8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57766/0 (socket says 172.21.15.16:57766) 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 -- 172.21.15.16:0/980070623 learned_addr learned my addr 172.21.15.16:0/980070623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f601614a640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 0x558eab0e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015949640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 -- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 msgr2=0x558eab0e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 0x558eab0e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.333+0000 7f6015148640 1 -- 172.21.15.16:0/980070623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 msgr2=0x558eab0e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6015148640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6015148640 1 -- 172.21.15.16:0/980070623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eaadb52b0 con 0x558eab0de2f0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f601614a640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 0x558eab0e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6015949640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6015148640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60000cc810 tx=0x7f600009b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315144a4579ebaf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000008e10 con 0x558eab0de2f0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f6000005290 con 0x558eab0de2f0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558eab051600 con 0x558eab0de2f0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6000020030 con 0x558eab0de2f0 2024-09-12T00:55:51.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6019f86740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558eab0f6fd0 con 0x558eab0de2f0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558eab051600 con 0x558eab0de2f0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6000097870 con 0x558eab0de2f0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.334+0000 7f6019f86740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6019f86740 10 client.15129 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6019f86740 20 client.15129 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab0f91e0 0x558eab0fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6019f86740 4 client.15129 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eab04bb10 con 0x558eab0f91e0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6015949640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab0f91e0 0x558eab0fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f6015949640 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab0f91e0 0x558eab0fa650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e92cb64e483a5ddc server_cookie=e01fb79f84256aa3 in_seq=0 out_seq=0 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.335+0000 7f5ff97fa640 10 client.15129 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f5ff97fa640 1 -- 172.21.15.16:0/980070623 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558eab04bb10 con 0x558eab0f91e0 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f5ff97fa640 10 client.15129 handle_command_reply: tid=0 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 client.15129 shutdown 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 msgr2=0x558eab0e5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60000cc810 tx=0x7f600009b2e0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 shutdown_connections 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0e8bb0 0x558eab0e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab0f91e0 0x558eab0fa650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0e5720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 --2- 172.21.15.16:0/980070623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0dd9d0 0x558eab0e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f5ff97fa640 0 client.15129 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 >> 172.21.15.16:0/980070623 conn(0x558eab0d9f60 msgr2=0x558eab0dfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 shutdown_connections 2024-09-12T00:55:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 1 -- 172.21.15.16:0/980070623 wait complete. 2024-09-12T00:55:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.336+0000 7f6019f86740 20 client.15129 trim_cache size 0 max 16384 2024-09-12T00:55:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.354+0000 7f6019f86740 1 Processor -- start 2024-09-12T00:55:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.354+0000 7f6019f86740 1 -- start start 2024-09-12T00:55:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab0f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 0x558eab0da340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558eab0f8990 con 0x558eab0de2f0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558eab0f9060 con 0x558eab0e8bb0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6019f86740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558eab0f9730 con 0x558eab0dd9d0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57770/0 (socket says 172.21.15.16:57770) 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 -- 172.21.15.16:0/470297044 learned_addr learned my addr 172.21.15.16:0/470297044 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f601614a640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 0x558eab0da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015949640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab0f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 -- 172.21.15.16:0/470297044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 msgr2=0x558eab0f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab0f8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 -- 172.21.15.16:0/470297044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 msgr2=0x558eab0da340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 0x558eab0da340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 -- 172.21.15.16:0/470297044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eab0d9d90 con 0x558eab0de2f0 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f601614a640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 0x558eab0da340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f6015148640 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6000097660 tx=0x7f600009b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9ffb6763e66a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.355+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/470297044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000009010 con 0x558eab0de2f0 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/470297044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60000a3e70 con 0x558eab0de2f0 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/470297044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600007b440 con 0x558eab0de2f0 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 -- 172.21.15.16:0/470297044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 msgr2=0x558eab0f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6000097660 tx=0x7f600009b370 comp rx=0 tx=0).stop 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 -- 172.21.15.16:0/470297044 shutdown_connections 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0e8bb0 0x558eab0da340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0de2f0 0x558eab0f98b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 --2- 172.21.15.16:0/470297044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab0f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 -- 172.21.15.16:0/470297044 >> 172.21.15.16:0/470297044 conn(0x558eab0d9f60 msgr2=0x558eab0e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 -- 172.21.15.16:0/470297044 shutdown_connections 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.356+0000 7f6019f86740 1 -- 172.21.15.16:0/470297044 wait complete. 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 -- start start 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 Processor -- start 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab102e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 0x558eab0fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558eab0fa4d0 con 0x558eab0e8bb0 2024-09-12T00:55:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558eab0f9e00 con 0x558eab0de2f0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558eab0faba0 con 0x558eab0dd9d0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6019f86740 10 client.0 _ll_register_callbacks cb 0x7f600c00c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f601614a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f601614a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57778/0 (socket says 172.21.15.16:57778) 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f6015949640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.357+0000 7f601614a640 1 -- 172.21.15.16:0/3513546392 learned_addr learned my addr 172.21.15.16:0/3513546392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f6015148640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 0x558eab0fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 -- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 msgr2=0x558eab102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 -- 172.21.15.16:0/3513546392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 msgr2=0x558eab0fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 0x558eab0fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 -- 172.21.15.16:0/3513546392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eab0da880 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f6015148640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 0x558eab0fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f601614a640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6008009ca0 tx=0x7f60080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8de191fb719b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f5ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008051070 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f6008004790 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60000a3e70 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6008031e60 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f6019f86740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558eab101510 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.358+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60000a3e70 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f60080308c0 con 0x558eab0e8bb0 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 10 client.15135 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 20 client.15135 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab168110 0x558eab166e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 4 client.15135 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "status"}) -- 0x558eaafc80e0 con 0x558eab168110 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6015949640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab168110 0x558eab166e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f6015949640 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab168110 0x558eab166e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d934d0bcd0187cec server_cookie=9a91ec945a40ca85 in_seq=0 out_seq=0 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.359+0000 7f5ff97fa640 10 client.15135 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f5ff97fa640 1 -- 172.21.15.16:0/3513546392 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558eaafc80e0 con 0x558eab168110 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f5ff97fa640 10 client.15135 handle_command_reply: tid=0 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 client.15135 shutdown 2024-09-12T00:55:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 msgr2=0x558eab0fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6008009ca0 tx=0x7f60080095c0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 shutdown_connections 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558eab168110 0x558eab166e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558eab0e8bb0 0x558eab0fc680 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558eab0de2f0 0x558eab0fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 --2- 172.21.15.16:0/3513546392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558eab0dd9d0 0x558eab102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f5ff97fa640 0 client.15135 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 >> 172.21.15.16:0/3513546392 conn(0x558eab0d9f60 msgr2=0x558eab0e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 shutdown_connections 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 1 -- 172.21.15.16:0/3513546392 wait complete. 2024-09-12T00:55:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.360+0000 7f6019f86740 20 client.15135 trim_cache size 0 max 16384 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 96.552552566000003, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:51.370 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 134.744037526 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 msgr2=0x7f5ffc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 0x7f5ffc0a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6008006010 tx=0x7f600802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 msgr2=0x7f6010074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f600000d770 tx=0x7f600000dc30 comp rx=0 tx=0).stop 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 shutdown_connections 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f5ffc0a2c00 0x7f5ffc0a50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6010106f10 0x7f6010074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6010102790 0x7f6010074a30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 --2- 172.21.15.16:0/2273793235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 >> 172.21.15.16:0/2273793235 conn(0x7f60100fd8c0 msgr2=0x7f60100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 shutdown_connections 2024-09-12T00:55:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:51.361+0000 7f6017bad640 1 -- 172.21.15.16:0/2273793235 wait complete. 2024-09-12T00:55:52.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:52 smithi016 ceph-mon[30972]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:52.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T00:55:52.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:52 smithi159 ceph-mon[35566]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:52.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:52 smithi099 ceph-mon[35654]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 -- 172.21.15.16:0/2491427312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 msgr2=0x7fb0a810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/2491427312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a810e610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009ca0 tx=0x7fb0a002a950 comp rx=0 tx=0).stop 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 -- 172.21.15.16:0/2491427312 shutdown_connections 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/2491427312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a810e610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/2491427312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8103660 0x7fb0a810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/2491427312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8102d40 0x7fb0a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 -- 172.21.15.16:0/2491427312 >> 172.21.15.16:0/2491427312 conn(0x7fb0a80fc7a0 msgr2=0x7fb0a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 -- 172.21.15.16:0/2491427312 shutdown_connections 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.027+0000 7fb0adcc1640 1 -- 172.21.15.16:0/2491427312 wait complete. 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 Processor -- start 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 -- start start 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 0x7fb0a8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8103660 0x7fb0a81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111fc0 con 0x7fb0a810c170 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111e40 con 0x7fb0a8102d40 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0adcc1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8112140 con 0x7fb0a8103660 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.028+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57790/0 (socket says 172.21.15.16:57790) 2024-09-12T00:55:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 -- 172.21.15.16:0/1791559173 learned_addr learned my addr 172.21.15.16:0/1791559173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 0x7fb0a8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8103660 0x7fb0a81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 -- 172.21.15.16:0/1791559173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8103660 msgr2=0x7fb0a81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8103660 0x7fb0a81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 -- 172.21.15.16:0/1791559173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 msgr2=0x7fb0a8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 0x7fb0a8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 -- 172.21.15.16:0/1791559173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a819d330 con 0x7fb0a810c170 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 0x7fb0a8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a7fff640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009dc0 tx=0x7fb0a0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a4ff9640 1 -- 172.21.15.16:0/1791559173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0034040 con 0x7fb0a810c170 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a819d5c0 con 0x7fb0a810c170 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a4ff9640 1 -- 172.21.15.16:0/1791559173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a002fe20 con 0x7fb0a810c170 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0a4ff9640 1 -- 172.21.15.16:0/1791559173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00047f0 con 0x7fb0a810c170 2024-09-12T00:55:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.029+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a4f00 con 0x7fb0a810c170 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0a4ff9640 1 -- 172.21.15.16:0/1791559173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb0a0037020 con 0x7fb0a810c170 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 Processor -- start 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 -- start start 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x562530081db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x56253008c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56252ffe8090 con 0x56253008cbb0 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56252ff8fa70 con 0x5625300819d0 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0b009a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56252ffe4050 con 0x5625300822f0 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57800/0 (socket says 172.21.15.16:57800) 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1263318230 learned_addr learned my addr 172.21.15.16:0/1263318230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.032+0000 7fb0a4ff9640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 0x7fb0900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x56253008c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x562530081db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a7fff640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 0x7fb0900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1263318230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 msgr2=0x56253008c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x56253008c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1263318230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 msgr2=0x562530081db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x562530081db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1263318230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252ff9c130 con 0x56253008cbb0 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a4ff9640 1 -- 172.21.15.16:0/1791559173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb0a00efff0 con 0x7fb0a810c170 2024-09-12T00:55:53.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0103a0 tx=0x7fb09c0084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d52bd0690a701d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb08dffb640 1 -- 172.21.15.16:0/1263318230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c018030 con 0x56253008cbb0 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb08dffb640 1 -- 172.21.15.16:0/1263318230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c005500 con 0x56253008cbb0 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.033+0000 7fb08dffb640 1 -- 172.21.15.16:0/1263318230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c004b30 con 0x56253008cbb0 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- 172.21.15.16:0/1263318230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 msgr2=0x56253008efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0103a0 tx=0x7fb09c0084f0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0a7fff640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 0x7fb0900a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00bc7f0 tx=0x7fb0a008e870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- 172.21.15.16:0/1263318230 shutdown_connections 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x56253008efa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x56253008c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1263318230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x562530081db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- 172.21.15.16:0/1263318230 >> 172.21.15.16:0/1263318230 conn(0x56253007df60 msgr2=0x56253007e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- 172.21.15.16:0/1263318230 shutdown_connections 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- 172.21.15.16:0/1263318230 wait complete. 2024-09-12T00:55:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 -- start start 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.034+0000 7fb0b009a740 1 Processor -- start 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 0x56253008c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x562530089c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56253008f780 con 0x5625300822f0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56252ffa2f20 con 0x56253008cbb0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56253008f900 con 0x5625300819d0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 10 client.0 _ll_register_callbacks cb 0x56252ff6c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57810/0 (socket says 172.21.15.16:57810) 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 -- 172.21.15.16:0/1364800125 learned_addr learned my addr 172.21.15.16:0/1364800125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 0x56253008c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x562530089c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 -- 172.21.15.16:0/1364800125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 msgr2=0x56253008c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 0x56253008c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 -- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 msgr2=0x562530089c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x562530089c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 -- 172.21.15.16:0/1364800125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252fd592b0 con 0x5625300822f0 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 0x56253008c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb09800bfb0 tx=0x7fb0980128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9aaeba6c205067 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0980035f0 con 0x5625300822f0 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09c005500 con 0x5625300822f0 2024-09-12T00:55:53.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.035+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb098003790 con 0x5625300822f0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb098003990 con 0x5625300822f0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56253009b0d0 con 0x5625300822f0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c005500 con 0x5625300822f0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb098019070 con 0x5625300822f0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 20 client.15147 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253009d720 0x56253009eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 4 client.15147 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252fff5600 con 0x56253009d720 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.036+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253009d720 0x56253009eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.037+0000 7fb086ffd640 10 client.15147 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.037+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253009d720 0x56253009eb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2a1025466bf62e1 server_cookie=8e88b9dbaaa369b2 in_seq=0 out_seq=0 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.037+0000 7fb086ffd640 1 -- 172.21.15.16:0/1364800125 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56252fff5600 con 0x56253009d720 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.037+0000 7fb086ffd640 10 client.15147 handle_command_reply: tid=0 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 client.15147 shutdown 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 msgr2=0x562530089740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb09800bfb0 tx=0x7fb0980128e0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 shutdown_connections 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253009d720 0x56253009eb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x562530089c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300822f0 0x562530089740 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb086ffd640 0 client.15147 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1364800125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300819d0 0x56253008c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 >> 172.21.15.16:0/1364800125 conn(0x56253007df60 msgr2=0x562530083c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 shutdown_connections 2024-09-12T00:55:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 1 -- 172.21.15.16:0/1364800125 wait complete. 2024-09-12T00:55:53.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.038+0000 7fb0b009a740 20 client.15147 trim_cache size 0 max 16384 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 Processor -- start 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 -- start start 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x56253009d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x562530084610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56253009ced0 con 0x56253008cbb0 2024-09-12T00:55:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56253009d5a0 con 0x5625300819d0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0b009a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56253009dc70 con 0x5625300822f0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57824/0 (socket says 172.21.15.16:57824) 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1393318939 learned_addr learned my addr 172.21.15.16:0/1393318939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x56253009d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x562530084610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1393318939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 msgr2=0x562530084610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x562530084610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1393318939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 msgr2=0x56253009d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x56253009d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1393318939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562530082120 con 0x56253008cbb0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x562530084610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.056+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005e10 tx=0x7fb09c008450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5794ad9a65d36a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb08d7fa640 1 -- 172.21.15.16:0/1393318939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0035f0 con 0x56253008cbb0 2024-09-12T00:55:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb08d7fa640 1 -- 172.21.15.16:0/1393318939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c026030 con 0x56253008cbb0 2024-09-12T00:55:53.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb08d7fa640 1 -- 172.21.15.16:0/1393318939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c012a30 con 0x56253008cbb0 2024-09-12T00:55:53.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 -- 172.21.15.16:0/1393318939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 msgr2=0x562530084b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005e10 tx=0x7fb09c008450 comp rx=0 tx=0).stop 2024-09-12T00:55:53.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 -- 172.21.15.16:0/1393318939 shutdown_connections 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56253008cbb0 0x562530084b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x562530084610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1393318939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625300819d0 0x56253009d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 -- 172.21.15.16:0/1393318939 >> 172.21.15.16:0/1393318939 conn(0x56253007df60 msgr2=0x56253007e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 -- 172.21.15.16:0/1393318939 shutdown_connections 2024-09-12T00:55:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.057+0000 7fb0b009a740 1 -- 172.21.15.16:0/1393318939 wait complete. 2024-09-12T00:55:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 -- start start 2024-09-12T00:55:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 Processor -- start 2024-09-12T00:55:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x5625300a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x5625300a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56253009ea10 con 0x5625300819d0 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56253009e340 con 0x56253008cbb0 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56253009f0e0 con 0x5625300822f0 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0b009a740 10 client.0 _ll_register_callbacks cb 0x7fb09800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57828/0 (socket says 172.21.15.16:57828) 2024-09-12T00:55:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1195549001 learned_addr learned my addr 172.21.15.16:0/1195549001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.058+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x5625300a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1195549001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 msgr2=0x5625300a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x5625300a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 msgr2=0x5625300a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x5625300a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 -- 172.21.15.16:0/1195549001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625300a11a0 con 0x5625300819d0 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0a6ffd640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009010 tx=0x7fb09c008d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196cb2bef8e3c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c005100 con 0x5625300819d0 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fb09c0052a0 con 0x5625300819d0 2024-09-12T00:55:53.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09c026030 con 0x5625300819d0 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb09c0054e0 con 0x5625300819d0 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0b009a740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625300a5c60 con 0x5625300819d0 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.059+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c026030 con 0x5625300819d0 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb09c004980 con 0x5625300819d0 2024-09-12T00:55:53.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:55:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 20 client.15153 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253010c110 0x56253010ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 4 client.15153 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "status"}) -- 0x56252ffefb10 con 0x56253010c110 2024-09-12T00:55:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253010c110 0x56253010ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb0a77fe640 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253010c110 0x56253010ae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcc9efd1681a55e6 server_cookie=18e3756196d12ddd in_seq=0 out_seq=0 2024-09-12T00:55:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.060+0000 7fb086ffd640 10 client.15153 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb086ffd640 1 -- 172.21.15.16:0/1195549001 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56252ffefb10 con 0x56253010c110 2024-09-12T00:55:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb086ffd640 10 client.15153 handle_command_reply: tid=0 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 client.15153 shutdown 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 msgr2=0x5625300a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009010 tx=0x7fb09c008d00 comp rx=0 tx=0).stop 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 shutdown_connections 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x56253010c110 0x56253010ae20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56253008cbb0 0x5625300a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625300822f0 0x5625300a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 --2- 172.21.15.16:0/1195549001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625300819d0 0x5625300a7370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 >> 172.21.15.16:0/1195549001 conn(0x56253007df60 msgr2=0x56253008adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb086ffd640 0 client.15153 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 shutdown_connections 2024-09-12T00:55:53.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 1 -- 172.21.15.16:0/1195549001 wait complete. 2024-09-12T00:55:53.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.061+0000 7fb0b009a740 20 client.15153 trim_cache size 0 max 16384 2024-09-12T00:55:53.073 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:53.073 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:53.073 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 99.263259969000003, 2024-09-12T00:55:53.074 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:53.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:53.075 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:53.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 119.55905157700001 2024-09-12T00:55:53.075 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 msgr2=0x7fb0900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 0x7fb0900a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00bc7f0 tx=0x7fb0a008e870 comp rx=0 tx=0).stop 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 msgr2=0x7fb0a819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009dc0 tx=0x7fb0a0003f10 comp rx=0 tx=0).stop 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 shutdown_connections 2024-09-12T00:55:53.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fb0900a2b80 0x7fb0900a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0a810c170 0x7fb0a819cc50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0a8103660 0x7fb0a81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 --2- 172.21.15.16:0/1791559173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0a8102d40 0x7fb0a8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 >> 172.21.15.16:0/1791559173 conn(0x7fb0a80fc7a0 msgr2=0x7fb0a80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:53.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 shutdown_connections 2024-09-12T00:55:53.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:53.062+0000 7fb0adcc1640 1 -- 172.21.15.16:0/1791559173 wait complete. 2024-09-12T00:55:53.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T00:55:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:54.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:54 smithi016 ceph-mon[30972]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:54.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:54 smithi159 ceph-mon[35566]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:54.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:54 smithi099 ceph-mon[35654]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 -- 172.21.15.16:0/1105716850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4100a80 msgr2=0x7f08c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 --2- 172.21.15.16:0/1105716850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4100a80 0x7f08c4100ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009930 tx=0x7f08b402a420 comp rx=0 tx=0).stop 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 -- 172.21.15.16:0/1105716850 shutdown_connections 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 --2- 172.21.15.16:0/1105716850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4101420 0x7f08c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 --2- 172.21.15.16:0/1105716850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4100a80 0x7f08c4100ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 --2- 172.21.15.16:0/1105716850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 0x7f08c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 -- 172.21.15.16:0/1105716850 >> 172.21.15.16:0/1105716850 conn(0x7f08c40fc7a0 msgr2=0x7f08c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 -- 172.21.15.16:0/1105716850 shutdown_connections 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.684+0000 7f08cc006640 1 -- 172.21.15.16:0/1105716850 wait complete. 2024-09-12T00:55:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 Processor -- start 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 -- start start 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4100a80 0x7f08c41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 0x7f08c41a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4110ea0 con 0x7f08c4101420 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4110d20 con 0x7f08c4100a80 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08cc006640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4111020 con 0x7f08c4106a80 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08c95a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.685+0000 7f08c95a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57834/0 (socket says 172.21.15.16:57834) 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 -- 172.21.15.16:0/406983765 learned_addr learned my addr 172.21.15.16:0/406983765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 0x7f08c41a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c9da2640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4100a80 0x7f08c41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 -- 172.21.15.16:0/406983765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 msgr2=0x7f08c41a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 0x7f08c41a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 -- 172.21.15.16:0/406983765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4100a80 msgr2=0x7f08c41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4100a80 0x7f08c41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 -- 172.21.15.16:0/406983765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c41a4870 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08c95a1640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a50 tx=0x7f08b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08baffd640 1 -- 172.21.15.16:0/406983765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b403f030 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41a4a40 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08baffd640 1 -- 172.21.15.16:0/406983765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b4003ce0 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08baffd640 1 -- 172.21.15.16:0/406983765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4035070 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.686+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41a4e90 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08baffd640 1 -- 172.21.15.16:0/406983765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08b4042030 con 0x7f08c4101420 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 Processor -- start 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 -- start start 2024-09-12T00:55:54.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a33c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 0x55e196a453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e19699a090 con 0x55e196a341d0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e196941a70 con 0x55e196a42fc0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ce3df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e196996050 con 0x55e196a338b0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08c9da2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08ca5a3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a33c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08c9da2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57846/0 (socket says 172.21.15.16:57846) 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.689+0000 7f08c9da2640 1 -- 172.21.15.16:0/1574682393 learned_addr learned my addr 172.21.15.16:0/1574682393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c95a1640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 0x55e196a453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08baffd640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 0x7f08b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 -- 172.21.15.16:0/1574682393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 msgr2=0x55e196a33c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a33c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 -- 172.21.15.16:0/1574682393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 msgr2=0x55e196a453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 0x55e196a453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 -- 172.21.15.16:0/1574682393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e19694e130 con 0x55e196a341d0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c95a1640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 0x55e196a453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 0x7f08b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08baffd640 1 -- 172.21.15.16:0/406983765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f08b40ef270 con 0x7f08c4101420 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08c9da2640 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f08c000cc60 tx=0x7f08c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f672c4c1fa0d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08affff640 1 -- 172.21.15.16:0/1574682393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0014070 con 0x55e196a341d0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 0x7f08b00a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00a640 tx=0x7f08bc004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08affff640 1 -- 172.21.15.16:0/1574682393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c00047a0 con 0x55e196a341d0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.690+0000 7f08affff640 1 -- 172.21.15.16:0/1574682393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0010760 con 0x55e196a341d0 2024-09-12T00:55:54.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 -- 172.21.15.16:0/1574682393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 msgr2=0x55e196a42a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f08c000cc60 tx=0x7f08c000e720 comp rx=0 tx=0).stop 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 -- 172.21.15.16:0/1574682393 shutdown_connections 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a42fc0 0x55e196a453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a42a80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 --2- 172.21.15.16:0/1574682393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a33c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 -- 172.21.15.16:0/1574682393 >> 172.21.15.16:0/1574682393 conn(0x55e196a2ff60 msgr2=0x55e196a30340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 -- 172.21.15.16:0/1574682393 shutdown_connections 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.691+0000 7f08ce3df740 1 -- 172.21.15.16:0/1574682393 wait complete. 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 -- start start 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 Processor -- start 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a341d0 0x55e196a3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e196a45b90 con 0x55e196a42fc0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e196954f20 con 0x55e196a338b0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e196a45d10 con 0x55e196a341d0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ce3df740 10 client.0 _ll_register_callbacks cb 0x55e19691e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57860/0 (socket says 172.21.15.16:57860) 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 -- 172.21.15.16:0/776439281 learned_addr learned my addr 172.21.15.16:0/776439281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08c95a1640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a341d0 0x55e196a3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08c9da2640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 -- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a341d0 msgr2=0x55e196a3fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a341d0 0x55e196a3fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 -- 172.21.15.16:0/776439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 msgr2=0x55e196a42840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a42840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.692+0000 7f08ca5a3640 1 -- 172.21.15.16:0/776439281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e19670b2b0 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007b80 tx=0x7f08bc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682b4d2dc5044abf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc026020 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f08bc017a40 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08c00047a0 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08bc017c40 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 10 client.15165 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e196a51590 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c00047a0 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f08bc021430 con 0x55e196a42fc0 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 10 client.15165 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 10 client.15165 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 20 client.15165 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:54.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196a53bf0 0x55e196a55060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.693+0000 7f08ce3df740 4 client.15165 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.694+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1969a7600 con 0x55e196a53bf0 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.694+0000 7f08c9da2640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196a53bf0 0x55e196a55060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.694+0000 7f08ad7fa640 10 client.15165 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.694+0000 7f08c9da2640 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196a53bf0 0x55e196a55060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61b3de26c8ad9717 server_cookie=9a882d51da9079c2 in_seq=0 out_seq=0 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ad7fa640 1 -- 172.21.15.16:0/776439281 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1969a7600 con 0x55e196a53bf0 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ad7fa640 10 client.15165 handle_command_reply: tid=0 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 client.15165 shutdown 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 msgr2=0x55e196a400d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007b80 tx=0x7f08bc00d040 comp rx=0 tx=0).stop 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 shutdown_connections 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a400d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a341d0 0x55e196a3fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196a53bf0 0x55e196a55060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 --2- 172.21.15.16:0/776439281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a42840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ad7fa640 0 client.15165 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 >> 172.21.15.16:0/776439281 conn(0x55e196a2ff60 msgr2=0x55e196a35cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 shutdown_connections 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 1 -- 172.21.15.16:0/776439281 wait complete. 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.695+0000 7f08ce3df740 20 client.15165 trim_cache size 0 max 16384 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 Processor -- start 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 -- start start 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a55060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 0x55e196a5d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e196a533a0 con 0x55e196a42fc0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e196a53a70 con 0x55e196a341d0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ce3df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e196a54140 con 0x55e196a338b0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ca5a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ca5a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57868/0 (socket says 172.21.15.16:57868) 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08ca5a3640 1 -- 172.21.15.16:0/2346823362 learned_addr learned my addr 172.21.15.16:0/2346823362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.714+0000 7f08c9da2640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a55060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 -- 172.21.15.16:0/2346823362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 msgr2=0x55e196a55060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a55060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08c95a1640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 0x55e196a5d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 -- 172.21.15.16:0/2346823362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 msgr2=0x55e196a5d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 0x55e196a5d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 -- 172.21.15.16:0/2346823362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e196a34000 con 0x55e196a42fc0 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08c95a1640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 0x55e196a5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:54.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00af70 tx=0x7f08bc00a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5354c313f654fbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08af7fe640 1 -- 172.21.15.16:0/2346823362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc026040 con 0x55e196a42fc0 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08af7fe640 1 -- 172.21.15.16:0/2346823362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc01b030 con 0x55e196a42fc0 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.715+0000 7f08af7fe640 1 -- 172.21.15.16:0/2346823362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc022920 con 0x55e196a42fc0 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- 172.21.15.16:0/2346823362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 msgr2=0x55e196a5d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00af70 tx=0x7f08bc00a510 comp rx=0 tx=0).stop 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- 172.21.15.16:0/2346823362 shutdown_connections 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a42fc0 0x55e196a5d880 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a341d0 0x55e196a5d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2346823362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a338b0 0x55e196a55060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- 172.21.15.16:0/2346823362 >> 172.21.15.16:0/2346823362 conn(0x55e196a2ff60 msgr2=0x55e196a3e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- 172.21.15.16:0/2346823362 shutdown_connections 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- 172.21.15.16:0/2346823362 wait complete. 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 -- start start 2024-09-12T00:55:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.716+0000 7f08ce3df740 1 Processor -- start 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a5d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a42fc0 0x55e196a56900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e196a54ee0 con 0x55e196a341d0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e196a54810 con 0x55e196a338b0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e196a555b0 con 0x55e196a42fc0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ce3df740 10 client.0 _ll_register_callbacks cb 0x7f08bc00a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57882/0 (socket says 172.21.15.16:57882) 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 -- 172.21.15.16:0/2721253818 learned_addr learned my addr 172.21.15.16:0/2721253818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c9da2640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a5d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ca5a3640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a42fc0 0x55e196a56900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 -- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a42fc0 msgr2=0x55e196a56900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a42fc0 0x55e196a56900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 -- 172.21.15.16:0/2721253818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 msgr2=0x55e196a5d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a5d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 -- 172.21.15.16:0/2721253818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e196a57130 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08c95a1640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ecd80 tx=0x7f08b40bb6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bf995e6ce07455 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.717+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4049070 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f08b40b8170 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08bc01b030 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08b40bbae0 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e196a5bb80 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc01b030 con 0x55e196a341d0 2024-09-12T00:55:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f08b40d4ce0 con 0x55e196a341d0 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 10 client.15171 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 20 client.15171 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196ac2930 0x55e196ac1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 4 client.15171 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "status"}) -- 0x55e1969a1b10 con 0x55e196ac2930 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.718+0000 7f08c9da2640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196ac2930 0x55e196ac1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08c9da2640 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196ac2930 0x55e196ac1640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=460f0e0bf48b81ee server_cookie=a8c640d33c7d704d in_seq=0 out_seq=0 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ad7fa640 10 client.15171 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ad7fa640 1 -- 172.21.15.16:0/2721253818 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e1969a1b10 con 0x55e196ac2930 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ad7fa640 10 client.15171 handle_command_reply: tid=0 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ce3df740 1 client.15171 shutdown 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 msgr2=0x55e196a5daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.719+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ecd80 tx=0x7f08b40bb6e0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 shutdown_connections 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e196a42fc0 0x55e196a56900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e196a341d0 0x55e196a5daf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e196ac2930 0x55e196ac1640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 --2- 172.21.15.16:0/2721253818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e196a338b0 0x55e196a5d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 >> 172.21.15.16:0/2721253818 conn(0x55e196a2ff60 msgr2=0x55e196a3aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ad7fa640 0 client.15171 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 shutdown_connections 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 1 -- 172.21.15.16:0/2721253818 wait complete. 2024-09-12T00:55:54.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08ce3df740 20 client.15171 trim_cache size 0 max 16384 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24487, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 101.92823678000001, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:54.735 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 135.57216363399999 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 msgr2=0x7f08b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 0x7f08b00a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00a640 tx=0x7f08bc004b50 comp rx=0 tx=0).stop 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 msgr2=0x7f08c41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009a50 tx=0x7f08b4004270 comp rx=0 tx=0).stop 2024-09-12T00:55:54.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.720+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 shutdown_connections 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f08b00a2b80 0x7f08b00a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c4106a80 0x7f08c41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c4101420 0x7f08c41997f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 --2- 172.21.15.16:0/406983765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08c4100a80 0x7f08c41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 >> 172.21.15.16:0/406983765 conn(0x7f08c40fc7a0 msgr2=0x7f08c40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 shutdown_connections 2024-09-12T00:55:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:54.721+0000 7f08cc006640 1 -- 172.21.15.16:0/406983765 wait complete. 2024-09-12T00:55:55.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T00:55:55.797 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:56.192 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:56 smithi016 ceph-mon[30972]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 -- 172.21.15.16:0/1242381171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0102d40 msgr2=0x7facf0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 --2- 172.21.15.16:0/1242381171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0102d40 0x7facf0103120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facec009900 tx=0x7facec02a480 comp rx=0 tx=0).stop 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 -- 172.21.15.16:0/1242381171 shutdown_connections 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 --2- 172.21.15.16:0/1242381171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 0x7facf010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 --2- 172.21.15.16:0/1242381171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0103660 0x7facf010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 --2- 172.21.15.16:0/1242381171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0102d40 0x7facf0103120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 -- 172.21.15.16:0/1242381171 >> 172.21.15.16:0/1242381171 conn(0x7facf00fc7a0 msgr2=0x7facf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 -- 172.21.15.16:0/1242381171 shutdown_connections 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.276+0000 7facf87e3640 1 -- 172.21.15.16:0/1242381171 wait complete. 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 Processor -- start 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 -- start start 2024-09-12T00:55:56.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0102d40 0x7facf01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 0x7facf019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facf0112030 con 0x7facf0103660 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7facf0111eb0 con 0x7facf0102d40 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf87e3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7facf01121b0 con 0x7facf010c170 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf5d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf5d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57896/0 (socket says 172.21.15.16:57896) 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf5d7e640 1 -- 172.21.15.16:0/1531715094 learned_addr learned my addr 172.21.15.16:0/1531715094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf6d80640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 0x7facf019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.277+0000 7facf657f640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0102d40 0x7facf01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 -- 172.21.15.16:0/1531715094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 msgr2=0x7facf019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 0x7facf019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 -- 172.21.15.16:0/1531715094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0102d40 msgr2=0x7facf01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0102d40 0x7facf01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 -- 172.21.15.16:0/1531715094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf019d550 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf5d7e640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7face000b3f0 tx=0x7face000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7face77fe640 1 -- 172.21.15.16:0/1531715094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0015020 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7face77fe640 1 -- 172.21.15.16:0/1531715094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face0011d10 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.278+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf019d840 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.279+0000 7face77fe640 1 -- 172.21.15.16:0/1531715094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face000f2e0 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.279+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01a5120 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.281+0000 7face77fe640 1 -- 172.21.15.16:0/1531715094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7face0018020 con 0x7facf0103660 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 Processor -- start 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 -- start start 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7face77fe640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 0x7facdc0a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 0x55cbd357edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facf6d80640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 0x7facdc0a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357f2f0 0x55cbd3589670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7face77fe640 1 -- 172.21.15.16:0/1531715094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7face00ce520 con 0x7facf0103660 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbd34e5090 con 0x55cbd3589bb0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cbd348ca70 con 0x55cbd357f2f0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facfabbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbd34e1050 con 0x55cbd357e9d0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57908/0 (socket says 172.21.15.16:57908) 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.282+0000 7facf6d80640 1 -- 172.21.15.16:0/4166114098 learned_addr learned my addr 172.21.15.16:0/4166114098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf657f640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 0x55cbd357edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf5d7e640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357f2f0 0x55cbd3589670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 0x7facdc0a4f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7face800a860 tx=0x7face800ac60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 -- 172.21.15.16:0/4166114098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 msgr2=0x55cbd357edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 0x55cbd357edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 -- 172.21.15.16:0/4166114098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357f2f0 msgr2=0x55cbd3589670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357f2f0 0x55cbd3589670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 -- 172.21.15.16:0/4166114098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd3499130 con 0x55cbd3589bb0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf657f640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 0x55cbd357edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7facf6d80640 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7face8009a60 tx=0x7face8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3cb194766baa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7face4ff9640 1 -- 172.21.15.16:0/4166114098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face802d070 con 0x55cbd3589bb0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7face4ff9640 1 -- 172.21.15.16:0/4166114098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.283+0000 7face4ff9640 1 -- 172.21.15.16:0/4166114098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8028d50 con 0x55cbd3589bb0 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- 172.21.15.16:0/4166114098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 msgr2=0x55cbd358bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7face8009a60 tx=0x7face8021000 comp rx=0 tx=0).stop 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- 172.21.15.16:0/4166114098 shutdown_connections 2024-09-12T00:55:56.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd358bfa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357f2f0 0x55cbd3589670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 --2- 172.21.15.16:0/4166114098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357e9d0 0x55cbd357edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- 172.21.15.16:0/4166114098 >> 172.21.15.16:0/4166114098 conn(0x55cbd357af60 msgr2=0x55cbd357b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- 172.21.15.16:0/4166114098 shutdown_connections 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- 172.21.15.16:0/4166114098 wait complete. 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.284+0000 7facfabbc740 1 -- start start 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 Processor -- start 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd3589480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd35867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbd358c780 con 0x55cbd3589bb0 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cbd349ff20 con 0x55cbd357e9d0 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbd358c900 con 0x55cbd357f2f0 2024-09-12T00:55:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facfabbc740 10 client.0 _ll_register_callbacks cb 0x55cbd3469030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57924/0 (socket says 172.21.15.16:57924) 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 -- 172.21.15.16:0/4265934186 learned_addr learned my addr 172.21.15.16:0/4265934186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf657f640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd3589480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf5d7e640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd35867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 -- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 msgr2=0x55cbd35867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd35867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 -- 172.21.15.16:0/4265934186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 msgr2=0x55cbd3589480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd3589480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 -- 172.21.15.16:0/4265934186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd32562b0 con 0x55cbd3589bb0 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf5d7e640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd35867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:55:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facf6d80640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7face8000f80 tx=0x7face8028c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e18de8ecd1a3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.285+0000 7facd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face802d040 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7face8013040 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7face8015e80 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 10 client.15183 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbd35980d0 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7face8030030 con 0x55cbd3589bb0 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 10 client.15183 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 10 client.15183 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 20 client.15183 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd359a3e0 0x55cbd359b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 4 client.15183 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.286+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd34f2600 con 0x55cbd359a3e0 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.287+0000 7facf657f640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd359a3e0 0x55cbd359b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.287+0000 7facf657f640 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd359a3e0 0x55cbd359b850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c98a49b00849ea server_cookie=bd6e46cdb01eb10c in_seq=0 out_seq=0 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.287+0000 7facd9ffb640 10 client.15183 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facd9ffb640 1 -- 172.21.15.16:0/4265934186 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbd34f2600 con 0x55cbd359a3e0 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facd9ffb640 10 client.15183 handle_command_reply: tid=0 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 client.15183 shutdown 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 msgr2=0x55cbd3586d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7face8000f80 tx=0x7face8028c80 comp rx=0 tx=0).stop 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 shutdown_connections 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd359a3e0 0x55cbd359b850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3586d10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd35867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 --2- 172.21.15.16:0/4265934186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd3589480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facd9ffb640 0 client.15183 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 >> 172.21.15.16:0/4265934186 conn(0x55cbd357af60 msgr2=0x55cbd3580c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 shutdown_connections 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 1 -- 172.21.15.16:0/4265934186 wait complete. 2024-09-12T00:55:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.288+0000 7facfabbc740 20 client.15183 trim_cache size 0 max 16384 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.306+0000 7facfabbc740 1 Processor -- start 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 -- start start 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd359b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbd3599b90 con 0x55cbd3589bb0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cbd359a260 con 0x55cbd357e9d0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facfabbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbd359a930 con 0x55cbd357f2f0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57926/0 (socket says 172.21.15.16:57926) 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 -- 172.21.15.16:0/1639986588 learned_addr learned my addr 172.21.15.16:0/1639986588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf657f640 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd359b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf5d7e640 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 -- 172.21.15.16:0/1639986588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 msgr2=0x55cbd3583a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 -- 172.21.15.16:0/1639986588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 msgr2=0x55cbd359b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd359b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 -- 172.21.15.16:0/1639986588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd35897f0 con 0x55cbd3589bb0 2024-09-12T00:55:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.307+0000 7facf6d80640 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7face8004b90 tx=0x7face8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bbcf3617564772 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facdbfff640 1 -- 172.21.15.16:0/1639986588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face802d040 con 0x55cbd3589bb0 2024-09-12T00:55:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facdbfff640 1 -- 172.21.15.16:0/1639986588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facdbfff640 1 -- 172.21.15.16:0/1639986588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face801a990 con 0x55cbd3589bb0 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 -- 172.21.15.16:0/1639986588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 msgr2=0x55cbd3583f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7face8004b90 tx=0x7face8009390 comp rx=0 tx=0).stop 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 -- 172.21.15.16:0/1639986588 shutdown_connections 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583f90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 --2- 172.21.15.16:0/1639986588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd359b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 -- 172.21.15.16:0/1639986588 >> 172.21.15.16:0/1639986588 conn(0x55cbd357af60 msgr2=0x55cbd3581de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.308+0000 7facfabbc740 1 -- 172.21.15.16:0/1639986588 shutdown_connections 2024-09-12T00:55:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 -- 172.21.15.16:0/1639986588 wait complete. 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 -- start start 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 Processor -- start 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd35a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cbd359b6d0 con 0x55cbd3589bb0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cbd359b000 con 0x55cbd357e9d0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbd359bda0 con 0x55cbd357f2f0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facfabbc740 10 client.0 _ll_register_callbacks cb 0x7face802b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facf6d80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57928/0 (socket says 172.21.15.16:57928) 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facf6d80640 1 -- 172.21.15.16:0/4052716938 learned_addr learned my addr 172.21.15.16:0/4052716938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facf5d7e640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.309+0000 7facf657f640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd35a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 -- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 msgr2=0x55cbd3583310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 -- 172.21.15.16:0/4052716938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 msgr2=0x55cbd35a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd35a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 -- 172.21.15.16:0/4052716938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd3583d90 con 0x55cbd3589bb0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facf6d80640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7face8004a80 tx=0x7face8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89550fb0b9a032ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face804c060 con 0x55cbd3589bb0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7face8033650 con 0x55cbd3589bb0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7face802d040 con 0x55cbd3589bb0 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facfabbc740 10 client.15189 fetch_fsmap learned FSMap version 27 2024-09-12T00:55:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbd35a27b0 con 0x55cbd3589bb0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8024070 con 0x55cbd3589bb0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.310+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7face8030070 con 0x55cbd3589bb0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 10 client.15189 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 20 client.15189 populate_metadata read hostname 'smithi016' 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd36090c0 0x55cbd3607dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 4 client.15189 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "status"}) -- 0x55cbd34ecb10 con 0x55cbd36090c0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facf657f640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd36090c0 0x55cbd3607dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facf657f640 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd36090c0 0x55cbd3607dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485f05a80b5deb65 server_cookie=d757a2301756df40 in_seq=0 out_seq=0 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.311+0000 7facd9ffb640 10 client.15189 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facd9ffb640 1 -- 172.21.15.16:0/4052716938 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbd34ecb10 con 0x55cbd36090c0 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facd9ffb640 10 client.15189 handle_command_reply: tid=0 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 client.15189 shutdown 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 msgr2=0x55cbd3583850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7face8004a80 tx=0x7face8009ee0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 shutdown_connections 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55cbd36090c0 0x55cbd3607dd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cbd3589bb0 0x55cbd3583850 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbd357f2f0 0x55cbd3583310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 --2- 172.21.15.16:0/4052716938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cbd357e9d0 0x55cbd35a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 >> 172.21.15.16:0/4052716938 conn(0x55cbd357af60 msgr2=0x55cbd3587eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facd9ffb640 0 client.15189 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 shutdown_connections 2024-09-12T00:55:56.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 1 -- 172.21.15.16:0/4052716938 wait complete. 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.312+0000 7facfabbc740 20 client.15189 trim_cache size 0 max 16384 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24505, 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 111.551032263, 2024-09-12T00:55:56.318 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 27, 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 84, 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 131.88596670800001 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 msgr2=0x7facdc0a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 0x7facdc0a4f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7face800a860 tx=0x7face800ac60 comp rx=0 tx=0).stop 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 msgr2=0x7facf0198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7face000b3f0 tx=0x7face000b8b0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 shutdown_connections 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7facdc0a2a50 0x7facdc0a4f10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7facf010c170 0x7facf019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0103660 0x7facf0198ae0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 --2- 172.21.15.16:0/1531715094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf0102d40 0x7facf01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 >> 172.21.15.16:0/1531715094 conn(0x7facf00fc7a0 msgr2=0x7facf00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 shutdown_connections 2024-09-12T00:55:56.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:56.313+0000 7facf87e3640 1 -- 172.21.15.16:0/1531715094 wait complete. 2024-09-12T00:55:56.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:56 smithi159 ceph-mon[35566]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T00:55:56.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:56 smithi099 ceph-mon[35654]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T00:55:56.984 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T00:55:56.996 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T00:55:56.997 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T00:55:56.997 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T00:55:56.997 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T00:55:56.998 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T00:55:56.998 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:55:56.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T00:55:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 -- 172.21.15.16:0/1568369246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c102600 msgr2=0x7f954c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 --2- 172.21.15.16:0/1568369246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c102600 0x7f954c108e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9544009ca0 tx=0x7f954402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 -- 172.21.15.16:0/1568369246 shutdown_connections 2024-09-12T00:55:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 --2- 172.21.15.16:0/1568369246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c102600 0x7f954c108e90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 --2- 172.21.15.16:0/1568369246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c101c40 0x7f954c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 --2- 172.21.15.16:0/1568369246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c100a50 0x7f954c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 -- 172.21.15.16:0/1568369246 >> 172.21.15.16:0/1568369246 conn(0x7f954c0fc1b0 msgr2=0x7f954c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 -- 172.21.15.16:0/1568369246 shutdown_connections 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.103+0000 7f95545e8640 1 -- 172.21.15.16:0/1568369246 wait complete. 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 Processor -- start 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 -- start start 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c100a50 0x7f954c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 0x7f954c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d7e0 con 0x7f954c101c40 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d660 con 0x7f954c100a50 2024-09-12T00:55:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95545e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f954c10d960 con 0x7f954c102600 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57936/0 (socket says 172.21.15.16:57936) 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 -- 172.21.15.16:0/1431479784 learned_addr learned my addr 172.21.15.16:0/1431479784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9552384640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c100a50 0x7f954c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9552b85640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 0x7f954c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 -- 172.21.15.16:0/1431479784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 msgr2=0x7f954c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 0x7f954c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 -- 172.21.15.16:0/1431479784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c100a50 msgr2=0x7f954c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c100a50 0x7f954c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 -- 172.21.15.16:0/1431479784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954c19f9b0 con 0x7f954c101c40 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9552b85640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 0x7f954c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:55:57.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f9551b83640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d950 tx=0x7f953c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.104+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c017070 con 0x7f954c101c40 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.105+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c009c30 con 0x7f954c101c40 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.105+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954c19fca0 con 0x7f954c101c40 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.105+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c00f6b0 con 0x7f954c101c40 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.105+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954c1a01a0 con 0x7f954c101c40 2024-09-12T00:55:57.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.105+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954c107d70 con 0x7f954c101c40 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.107+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f953c0109d0 con 0x7f954c101c40 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.107+0000 7f95437fe640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 0x7f95380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.107+0000 7f9552384640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 0x7f95380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.108+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f953c0ced00 con 0x7f954c101c40 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.108+0000 7f9552384640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 0x7f95380a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f954c199270 tx=0x7f954800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:57.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.110+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f953c009de0 con 0x7f954c101c40 2024-09-12T00:55:57.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:57.335+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f954c02da50 con 0x7f954c101c40 2024-09-12T00:55:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.117+0000 7f95437fe640 1 -- 172.21.15.16:0/1431479784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f953c0bb070 con 0x7f954c101c40 2024-09-12T00:55:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.120+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 msgr2=0x7f95380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.120+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 0x7f95380a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f954c199270 tx=0x7f954800b390 comp rx=0 tx=0).stop 2024-09-12T00:55:58.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.120+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 msgr2=0x7f954c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.120+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d950 tx=0x7f953c00de10 comp rx=0 tx=0).stop 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 shutdown_connections 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f95380a2c30 0x7f95380a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f954c102600 0x7f954c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f954c101c40 0x7f954c1982e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 --2- 172.21.15.16:0/1431479784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f954c100a50 0x7f954c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 >> 172.21.15.16:0/1431479784 conn(0x7f954c0fc1b0 msgr2=0x7f954c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 shutdown_connections 2024-09-12T00:55:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:58.121+0000 7f95545e8640 1 -- 172.21.15.16:0/1431479784 wait complete. 2024-09-12T00:55:58.139 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T00:55:58.610 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T00:55:58.610 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T00:55:58.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:55:58.610 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T00:55:58.611 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-mon[35566]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:58.612 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T00:55:58.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-mon[35654]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:58.612 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T00:55:58.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-mon[30972]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T00:55:58.612 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: ignoring --setuser ceph since I am not root 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: ignoring --setgroup ceph since I am not root 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-mgr[37378]: -- 172.21.15.159:0/3892181035 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632608830e0 con 0x563260860800 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.275+0000 7f2bb09e0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.332+0000 7f2bb09e0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.481+0000 7f2bb09e0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: ignoring --setuser ceph since I am not root 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: ignoring --setgroup ceph since I am not root 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-mgr[37019]: -- 172.21.15.99:0/1427813589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629c0ba30e0 con 0x5629c0b81400 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.273+0000 7f06e588a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:55:58.613 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.330+0000 7f06e588a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:55:58.614 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.476+0000 7f06e588a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:55:58.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setuser ceph since I am not root 2024-09-12T00:55:58.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ignoring --setgroup ceph since I am not root 2024-09-12T00:55:58.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.278+0000 7f1ddb17e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T00:55:58.614 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.338+0000 7f1ddb17e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T00:55:58.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:55:58.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T00:55:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-12T00:55:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T00:55:58.703 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-12T00:55:58.703 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T00:55:58.703 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T00:55:58.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T00:55:58.768 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.493+0000 7f1ddb17e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T00:55:58.768 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.562+0000 7f1ddb17e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:55:58.768 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.721+0000 7f1ddb17e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:55:58.773 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.543+0000 7f06e588a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:55:58.773 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.687+0000 7f06e588a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:55:58.837 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.547+0000 7f2bb09e0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T00:55:58.837 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.695+0000 7f2bb09e0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:55:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:55:59.116 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:58.838+0000 7f2bb09e0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:55:59.133 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:58.828+0000 7f06e588a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:55:59.133 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:58.873+0000 7f1ddb17e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T00:55:59.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:59 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T00:55:59.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:55:59 smithi016 ceph-mon[30972]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T00:55:59.393 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:59.220+0000 7f1ddb17e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:55:59.393 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:59.286+0000 7f1ddb17e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:55:59.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:59 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T00:55:59.404 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:55:59 smithi159 ceph-mon[35566]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T00:55:59.404 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:59.155+0000 7f2bb09e0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:55:59.404 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:59.217+0000 7f2bb09e0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:55:59.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/2647967197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102ef0 msgr2=0x7fc2a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/2647967197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102ef0 0x7fc2a810da40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009ca0 tx=0x7fc2a002a950 comp rx=0 tx=0).stop 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/2647967197 shutdown_connections 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/2647967197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102ef0 0x7fc2a810da40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/2647967197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8102570 0x7fc2a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/2647967197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8100e20 0x7fc2a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/2647967197 >> 172.21.15.16:0/2647967197 conn(0x7fc2a80fc7d0 msgr2=0x7fc2a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/2647967197 shutdown_connections 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.477+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/2647967197 wait complete. 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 Processor -- start 2024-09-12T00:55:59.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 -- start start 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8100e20 0x7fc2a8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8102ef0 0x7fc2a819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8110f80 con 0x7fc2a8102570 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8110e00 con 0x7fc2a8100e20 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc2ae4f3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a8111100 con 0x7fc2a8102ef0 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc29ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40598/0 (socket says 172.21.15.16:40598) 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.478+0000 7fc29ffff640 1 -- 172.21.15.16:0/434009167 learned_addr learned my addr 172.21.15.16:0/434009167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2a7fff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8100e20 0x7fc2a8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2aca90640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8102ef0 0x7fc2a819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8102ef0 msgr2=0x7fc2a819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8102ef0 0x7fc2a819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8100e20 msgr2=0x7fc2a8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8100e20 0x7fc2a8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 -- 172.21.15.16:0/434009167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a819d3e0 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc29ffff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc29000d950 tx=0x7fc29000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290017070 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc290009c30 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a819d6d0 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000f6b0 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.479+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a4fb0 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.481+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8101200 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.482+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fc2900109d0 con 0x7fc2a8102570 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.482+0000 7fc2a5ffb640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 0x7fc2940a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.482+0000 7fc2a7fff640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 msgr2=0x7fc2940a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:55:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.482+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fc2900cebd0 con 0x7fc2a8102570 2024-09-12T00:55:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.482+0000 7fc2a7fff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 0x7fc2940a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:55:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.487+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc290009de0 con 0x7fc2a8102570 2024-09-12T00:55:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:59 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1431479784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T00:55:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:55:59 smithi099 ceph-mon[35654]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T00:55:59.524 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:59.139+0000 7f06e588a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T00:55:59.524 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:59.199+0000 7f06e588a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T00:55:59.524 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:55:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:55:59.383+0000 7f06e588a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:55:59.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:55:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:55:59.486+0000 7f1ddb17e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:55:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.683+0000 7fc2a7fff640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 msgr2=0x7fc2940a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:55:59.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.683+0000 7fc2a7fff640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 0x7fc2940a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:55:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.718+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc2a810a0a0 con 0x7fc2a8102570 2024-09-12T00:55:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.719+0000 7fc2a5ffb640 1 -- 172.21.15.16:0/434009167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fc29009b4f0 con 0x7fc2a8102570 2024-09-12T00:55:59.718 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 msgr2=0x7fc2940a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 0x7fc2940a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 msgr2=0x7fc2a8198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc29000d950 tx=0x7fc29000de10 comp rx=0 tx=0).stop 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 shutdown_connections 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fc2940a2c50 0x7fc2940a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2a8102ef0 0x7fc2a819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a8102570 0x7fc2a8198970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 --2- 172.21.15.16:0/434009167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2a8100e20 0x7fc2a8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:55:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 >> 172.21.15.16:0/434009167 conn(0x7fc2a80fc7d0 msgr2=0x7fc2a806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:55:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.722+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 shutdown_connections 2024-09-12T00:55:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:55:59.723+0000 7fc2ae4f3640 1 -- 172.21.15.16:0/434009167 wait complete. 2024-09-12T00:55:59.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:55:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:55:59.405+0000 7f2bb09e0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T00:56:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:00 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/434009167' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T00:56:00.501 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T00:56:00.502 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T00:56:00.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:00.502 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-12T00:56:00.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:00 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/434009167' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T00:56:00.524 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:00.259+0000 7f06e588a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:56:00.524 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:00.325+0000 7f06e588a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:56:00.559 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T00:56:00.559 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-12T00:56:00.560 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-12T00:56:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-12T00:56:00.562 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T00:56:00.562 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T00:56:00.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:00 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/434009167' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T00:56:00.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:00.294+0000 7f2bb09e0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:56:00.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:00.361+0000 7f2bb09e0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:56:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:00.731 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:00.449+0000 7f1ddb17e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T00:56:00.732 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:00.522+0000 7f1ddb17e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T00:56:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:00.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:00.788 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T00:56:00.816 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-12T00:56:00.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:00.819 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-12T00:56:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-12T00:56:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T00:56:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T00:56:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T00:56:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T00:56:00.888 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T00:56:00.889 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T00:56:00.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T00:56:00.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T00:56:00.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T00:56:00.889 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T00:56:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:01.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:01.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:01.193 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T00:56:01.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:01.210 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-12T00:56:01.267 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T00:56:01.267 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T00:56:01.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T00:56:01.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T00:56:01.267 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T00:56:01.282 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.028+0000 7f06e588a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:56:01.282 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.142+0000 7f06e588a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:56:01.339 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.081+0000 7f2bb09e0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:56:01.339 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.197+0000 7f2bb09e0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:56:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:01.366 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.303+0000 7f1ddb17e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T00:56:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:01.389 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T00:56:01.390 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T00:56:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:01.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:01.525 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T00:56:01.526 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-12T00:56:01.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-12T00:56:01.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T00:56:01.526 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-12T00:56:01.558 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.283+0000 7f06e588a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:56:01.558 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.344+0000 7f06e588a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:56:01.558 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.491+0000 7f06e588a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:56:01.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:01.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:01.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T00:56:01.608 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:01.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.340+0000 7f2bb09e0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:56:01.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.401+0000 7f2bb09e0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:56:01.616 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.550+0000 7f2bb09e0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:56:01.635 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T00:56:01.635 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T00:56:01.635 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:01.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.430+0000 7f1ddb17e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T00:56:01.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.588+0000 7f1ddb17e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T00:56:01.658 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T00:56:01.658 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T00:56:01.658 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/eddbff54-14e7-4eaf-b025-9f9443c2e321 2024-09-12T00:56:01.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:01.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/eddbff54-14e7-4eaf-b025-9f9443c2e321 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-12T00:56:01.761 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-12T00:56:01.761 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-12T00:56:01.840 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.559+0000 7f06e588a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:56:01.840 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.619+0000 7f06e588a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:56:01.841 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.777+0000 7f06e588a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:56:01.884 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-12T00:56:01.885 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-12T00:56:01.885 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.99:3300,172.21.15.159:3300" 2024-09-12T00:56:01.885 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@434eab66-70a0-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/eddbff54-14e7-4eaf-b025-9f9443c2e321 2024-09-12T00:56:01.885 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.16:3300/172.21.15.99:3300/172.21.15.159:3300" will pass to kernel 2024-09-12T00:56:01.886 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T00:56:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-12T00:56:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T00:56:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T00:56:01.886 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T00:56:01.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T00:56:01.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T00:56:01.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:01.907 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.618+0000 7f2bb09e0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:56:01.907 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.680+0000 7f2bb09e0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:56:01.908 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.841+0000 7f2bb09e0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:56:01.908 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.908+0000 7f2bb09e0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:56:01.910 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.655+0000 7f1ddb17e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T00:56:01.910 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.817+0000 7f1ddb17e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T00:56:01.910 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.892+0000 7f1ddb17e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T00:56:01.915 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T00:56:02.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T00:56:02.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:02.014 INFO:teuthology.orchestra.run.smithi016.stdout:client15222 2024-09-12T00:56:02.014 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:56:02.015 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T00:56:02.032 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T00:56:02.107 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.841+0000 7f06e588a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:56:02.108 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.910+0000 7f06e588a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:56:02.108 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:01.992+0000 7f06e588a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:56:02.108 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:02.051+0000 7f06e588a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:56:02.108 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:02.109+0000 7f06e588a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:56:02.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T00:56:02.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:02.131 INFO:teuthology.orchestra.run.smithi016.stdout:client15222 2024-09-12T00:56:02.131 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T00:56:02.179 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:01.977+0000 7f2bb09e0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:56:02.179 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:02.060+0000 7f2bb09e0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:56:02.179 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:02.120+0000 7f2bb09e0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:56:02.180 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:02.179+0000 7f2bb09e0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:56:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T00:56:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:02.191 INFO:teuthology.orchestra.run.smithi016.stdout:client15222 2024-09-12T00:56:02.191 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/434eab66-70a0-11ef-bcea-c7b262605968.client15222/status') 2024-09-12T00:56:02.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:01.957+0000 7f1ddb17e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T00:56:02.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.129+0000 7f1ddb17e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T00:56:02.202 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.202+0000 7f1ddb17e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T00:56:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T00:56:02.255 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.15222 (3)192.168.0.1:0/2478500940 2024-09-12T00:56:02.255 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-12T00:56:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-12T00:56:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-12T00:56:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.5774e-05 s, 2.7 MB/s 2024-09-12T00:56:02.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T00:56:02.263 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T00:56:02.273 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T00:56:02.273 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:56:02.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T00:56:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 -- 172.21.15.16:0/1332616131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 msgr2=0x7f8124109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 --2- 172.21.15.16:0/1332616131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124109c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f811c009ca0 tx=0x7f811c02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 -- 172.21.15.16:0/1332616131 shutdown_connections 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 --2- 172.21.15.16:0/1332616131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124109c90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 --2- 172.21.15.16:0/1332616131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 0x7f8124107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 --2- 172.21.15.16:0/1332616131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 -- 172.21.15.16:0/1332616131 >> 172.21.15.16:0/1332616131 conn(0x7f81240fc1e0 msgr2=0x7f81240fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 -- 172.21.15.16:0/1332616131 shutdown_connections 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 -- 172.21.15.16:0/1332616131 wait complete. 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.411+0000 7f812a053640 1 Processor -- start 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 -- start start 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 0x7f81241005d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f812410d800 con 0x7f8124107880 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f812410d680 con 0x7f8124102620 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f812a053640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f812410d980 con 0x7f8124104f50 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f8123fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:38170/0 (socket says 172.21.15.16:38170) 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 -- 172.21.15.16:0/3254174234 learned_addr learned my addr 172.21.15.16:0/3254174234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f811bfff640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 0x7f81241005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 msgr2=0x7f81241005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 0x7f81241005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 msgr2=0x7f8124100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124100b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 -- 172.21.15.16:0/3254174234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81241013a0 con 0x7f8124102620 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.412+0000 7f81237fe640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f811400e9f0 tx=0x7f811400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.413+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811400cd80 con 0x7f8124102620 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.413+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114010910 con 0x7f8124102620 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.413+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241a1f40 con 0x7f8124102620 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.413+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8114018950 con 0x7f8124102620 2024-09-12T00:56:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.413+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241a2350 con 0x7f8124102620 2024-09-12T00:56:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.414+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81240fdcd0 con 0x7f8124102620 2024-09-12T00:56:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.415+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f8114010ac0 con 0x7f8124102620 2024-09-12T00:56:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.415+0000 7f81217fa640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 0x7f81100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.416+0000 7f811bfff640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 msgr2=0x7f81100a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:56:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.416+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7f81140cf930 con 0x7f8124102620 2024-09-12T00:56:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.416+0000 7f811bfff640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 0x7f81100a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:56:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.419+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f811409c1f0 con 0x7f8124102620 2024-09-12T00:56:02.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.275+0000 7f1ddb17e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T00:56:02.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.365+0000 7f1ddb17e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T00:56:02.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.431+0000 7f1ddb17e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T00:56:02.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:02.494+0000 7f1ddb17e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T00:56:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.617+0000 7f811bfff640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 msgr2=0x7f81100a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:56:02.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.617+0000 7f811bfff640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 0x7f81100a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:56:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.633+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f812402d7c0 con 0x7f8124102620 2024-09-12T00:56:02.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.689+0000 7f81217fa640 1 -- 172.21.15.16:0/3254174234 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f81140a04f0 con 0x7f8124102620 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 msgr2=0x7f81100a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 0x7f81100a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 msgr2=0x7f8124101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f811400e9f0 tx=0x7f811400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 shutdown_connections 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7f81100a2c80 0x7f81100a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8124107880 0x7f8124100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8124104f50 0x7f81241005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 --2- 172.21.15.16:0/3254174234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8124102620 0x7f8124101f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 >> 172.21.15.16:0/3254174234 conn(0x7f81240fc1e0 msgr2=0x7f8124102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 shutdown_connections 2024-09-12T00:56:02.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.691+0000 7f812a053640 1 -- 172.21.15.16:0/3254174234 wait complete. 2024-09-12T00:56:02.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T00:56:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 -- 172.21.15.16:0/3313676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 msgr2=0x7fa8a81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/3313676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a81020b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009900 tx=0x7fa89c02a410 comp rx=0 tx=0).stop 2024-09-12T00:56:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 -- 172.21.15.16:0/3313676067 shutdown_connections 2024-09-12T00:56:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/3313676067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8a81025f0 0x7fa8a8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/3313676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a81020b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/3313676067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 -- 172.21.15.16:0/3313676067 >> 172.21.15.16:0/3313676067 conn(0x7fa8a80fc1e0 msgr2=0x7fa8a80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 -- 172.21.15.16:0/3313676067 shutdown_connections 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.810+0000 7fa8ae55f640 1 -- 172.21.15.16:0/3313676067 wait complete. 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 Processor -- start 2024-09-12T00:56:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 -- start start 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8a81025f0 0x7fa8a819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810c710 con 0x7fa8a8101c30 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810c590 con 0x7fa8a81025f0 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8ae55f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810c890 con 0x7fa8a8100a40 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8a7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8a7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a806a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:41994/0 (socket says 172.21.15.16:41994) 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8a7fff640 1 -- 172.21.15.16:0/1780001291 learned_addr learned my addr 172.21.15.16:0/1780001291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8acafc640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8a81025f0 0x7fa8a819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.811+0000 7fa8a77fe640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40630/0 (socket says 172.21.15.16:40630) 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8a81025f0 msgr2=0x7fa8a819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8a81025f0 0x7fa8a819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 msgr2=0x7fa8a819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a8101c30 0x7fa8a819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a819fa80 con 0x7fa8a8100a40 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a7fff640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a806a410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa89800e9c0 tx=0x7fa89800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89800cd80 con 0x7fa8a8100a40 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8980035f0 con 0x7fa8a8100a40 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a819fd10 con 0x7fa8a8100a40 2024-09-12T00:56:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898010690 con 0x7fa8a8100a40 2024-09-12T00:56:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.812+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a0180 con 0x7fa8a8100a40 2024-09-12T00:56:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.813+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8107db0 con 0x7fa8a8100a40 2024-09-12T00:56:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.814+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fa8980037a0 con 0x7fa8a8100a40 2024-09-12T00:56:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.815+0000 7fa8a57fa640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 0x7fa8900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.815+0000 7fa8a77fe640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 msgr2=0x7fa8900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:56:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.815+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8219+0+0 (secure 0 0 0) 0x7fa8980cf3e0 con 0x7fa8a8100a40 2024-09-12T00:56:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.815+0000 7fa8a77fe640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 0x7fa8900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T00:56:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:02.818+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa89809c350 con 0x7fa8a8100a40 2024-09-12T00:56:02.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3254174234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:02.961 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:02 smithi016 ceph-mon[30972]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:02.977 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3254174234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:02.978 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-mon[35566]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.016+0000 7fa8a77fe640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 msgr2=0x7fa8900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/1272021982 2024-09-12T00:56:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.016+0000 7fa8a77fe640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 0x7fa8900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T00:56:03.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.023+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa8a8062790 con 0x7fa8900a2c80 2024-09-12T00:56:03.023 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 00:56:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y[37015]: 2024-09-12T00:56:02.893+0000 7f06e588a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:56:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:02 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3254174234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:02 smithi099 ceph-mon[35654]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T00:56:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.355+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7fa8980982c0 con 0x7fa8a8100a40 2024-09-12T00:56:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.355+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 msgr2=0x7fa8900a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T00:56:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:03.355+0000 7fa8a57fa640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1272021982,v1:172.21.15.16:6801/1272021982] conn(0x7fa8900a2c80 0x7fa8900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:03.366 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 00:56:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z[37374]: 2024-09-12T00:56:02.978+0000 7f2bb09e0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:56:03.617 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.337+0000 7f1ddb17e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T00:56:03.617 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d45756640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.617 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d45756640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.617 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d45756640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.884 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d45756640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.884 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d45756640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d42750640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d42750640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d42750640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d42750640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:03.617+0000 7f1d42750640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:56:03.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Standby manager daemon y restarted 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Standby manager daemon y started 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Standby manager daemon z restarted 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Standby manager daemon z started 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Active manager daemon x restarted 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Activating manager daemon x 2024-09-12T00:56:03.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: mgrmap e20: x(active, starting, since 0.0120648s), standbys: y, z 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:56:03.887 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:56:03.888 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: Manager daemon x is now available 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:56:03.889 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:03 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:56:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T00:56:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:56:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Standby manager daemon y restarted 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Standby manager daemon y started 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Standby manager daemon z restarted 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Standby manager daemon z started 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:04.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Active manager daemon x restarted 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Activating manager daemon x 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: mgrmap e20: x(active, starting, since 0.0120648s), standbys: y, z 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:56:04.026 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:56:04.027 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: Manager daemon x is now available 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:04.028 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:56:04.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:03 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:56:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Standby manager daemon y restarted 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Standby manager daemon y started 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.99:0/799545188' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Standby manager daemon z restarted 2024-09-12T00:56:04.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Standby manager daemon z started 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.? 172.21.15.159:0/1510401801' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Active manager daemon x restarted 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Activating manager daemon x 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: mgrmap e20: x(active, starting, since 0.0120648s), standbys: y, z 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T00:56:04.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T00:56:04.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T00:56:04.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: Manager daemon x is now available 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T00:56:04.121 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:03 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T00:56:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.361+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mon.1 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fa89809b0a0 con 0x7fa8a8100a40 2024-09-12T00:56:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.361+0000 7fa8a57fa640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8900518c0 0x7fa890051ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.361+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa8a8062790 con 0x7fa8900518c0 2024-09-12T00:56:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.363+0000 7fa8a77fe640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8900518c0 0x7fa890051ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.365+0000 7fa8a77fe640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8900518c0 0x7fa890051ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa89c031440 tx=0x7fa89c0315b0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.384+0000 7fa8a57fa640 1 -- 172.21.15.16:0/1780001291 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa8a8062790 con 0x7fa8900518c0 2024-09-12T00:56:04.384 INFO:teuthology.orchestra.run.smithi016.stdout:{} 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8900518c0 msgr2=0x7fa890051ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8900518c0 0x7fa890051ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa89c031440 tx=0x7fa89c0315b0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 msgr2=0x7fa8a806a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 --2- 172.21.15.16:0/1780001291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8a8100a40 0x7fa8a806a410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa89800e9c0 tx=0x7fa89800ee80 comp rx=0 tx=0).stop 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8acafc640 1 -- 172.21.15.16:0/1780001291 reap_dead start 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 shutdown_connections 2024-09-12T00:56:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 >> 172.21.15.16:0/1780001291 conn(0x7fa8a80fc1e0 msgr2=0x7fa8a8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 shutdown_connections 2024-09-12T00:56:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.386+0000 7fa8ae55f640 1 -- 172.21.15.16:0/1780001291 wait complete. 2024-09-12T00:56:04.397 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T00:56:04.411 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T00:56:04.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T00:56:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f0493fff640 1 -- 172.21.15.16:0/973323060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0319b0 con 0x7f049c101c00 2024-09-12T00:56:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 -- 172.21.15.16:0/973323060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 msgr2=0x7f049c102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 --2- 172.21.15.16:0/973323060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c102080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f048c009900 tx=0x7f048c02a410 comp rx=0 tx=0).stop 2024-09-12T00:56:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 -- 172.21.15.16:0/973323060 shutdown_connections 2024-09-12T00:56:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 --2- 172.21.15.16:0/973323060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c1025c0 0x7f049c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 --2- 172.21.15.16:0/973323060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c102080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 --2- 172.21.15.16:0/973323060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c100a10 0x7f049c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 -- 172.21.15.16:0/973323060 >> 172.21.15.16:0/973323060 conn(0x7f049c0fc170 msgr2=0x7f049c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 -- 172.21.15.16:0/973323060 shutdown_connections 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.557+0000 7f04a3354640 1 -- 172.21.15.16:0/973323060 wait complete. 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 Processor -- start 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 -- start start 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c100a10 0x7f049c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1025c0 0x7f049c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10c6e0 con 0x7f049c101c00 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10c560 con 0x7f049c1025c0 2024-09-12T00:56:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a3354640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f049c10c860 con 0x7f049c100a10 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a08ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a08ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40800/0 (socket says 172.21.15.16:40800) 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a08ef640 1 -- 172.21.15.16:0/3951251838 learned_addr learned my addr 172.21.15.16:0/3951251838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a10f0640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c100a10 0x7f049c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.558+0000 7f04a18f1640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1025c0 0x7f049c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 -- 172.21.15.16:0/3951251838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c100a10 msgr2=0x7f049c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c100a10 0x7f049c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 -- 172.21.15.16:0/3951251838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1025c0 msgr2=0x7f049c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1025c0 0x7f049c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 -- 172.21.15.16:0/3951251838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c19fa80 con 0x7f049c101c00 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a08ef640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f048c009230 tx=0x7f048c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c03f030 con 0x7f049c101c00 2024-09-12T00:56:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048c003ce0 con 0x7f049c101c00 2024-09-12T00:56:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c19fd10 con 0x7f049c101c00 2024-09-12T00:56:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c035070 con 0x7f049c101c00 2024-09-12T00:56:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.559+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c1a01b0 con 0x7f049c101c00 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.560+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c1a03e0 con 0x7f049c101c00 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.561+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f048c042030 con 0x7f049c101c00 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.562+0000 7f04927fc640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 0x7f04880a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.562+0000 7f04a10f0640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 0x7f04880a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.562+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f048c0ef660 con 0x7f049c101c00 2024-09-12T00:56:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.563+0000 7f04a10f0640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 0x7f04880a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f049c101a60 tx=0x7f0498005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.569+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f048c0bc130 con 0x7f049c101c00 2024-09-12T00:56:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:04.372+0000 7f1d3f74a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:04.372+0000 7f1d3f74a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:04.372+0000 7f1d3f74a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:04.372+0000 7f1d3f74a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:04.643 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:04 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:04.372+0000 7f1d3f74a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:56:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.750+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f049c1a0b90 con 0x7f049c101c00 2024-09-12T00:56:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.753+0000 7f04927fc640 1 -- 172.21.15.16:0/3951251838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23017 (secure 0 0 0) 0x7f048c0c0430 con 0x7f049c101c00 2024-09-12T00:56:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:04.753 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:56:03.340006+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2952840401":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2290747564":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/90830535":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1572487203":"2024-09-13T00:56:03.339943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 msgr2=0x7f04880a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 0x7f04880a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f049c101a60 tx=0x7f0498005f90 comp rx=0 tx=0).stop 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 msgr2=0x7f049c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f048c009230 tx=0x7f048c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 shutdown_connections 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2990 0x7f04880a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1025c0 0x7f049c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c101c00 0x7f049c1983a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 --2- 172.21.15.16:0/3951251838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c100a10 0x7f049c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 >> 172.21.15.16:0/3951251838 conn(0x7f049c0fc170 msgr2=0x7f049c069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 shutdown_connections 2024-09-12T00:56:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.756+0000 7f04a3354640 1 -- 172.21.15.16:0/3951251838 wait complete. 2024-09-12T00:56:04.771 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T00:49:22.631453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T00:54:00.136499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T00:54:02.731797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T00:56:04.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T00:56:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 -- 172.21.15.16:0/684108308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 msgr2=0x7f4660101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 --2- 172.21.15.16:0/684108308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660101f20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4648009900 tx=0x7f464802a410 comp rx=0 tx=0).stop 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 -- 172.21.15.16:0/684108308 shutdown_connections 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 --2- 172.21.15.16:0/684108308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f4660109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 --2- 172.21.15.16:0/684108308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 0x7f4660103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 --2- 172.21.15.16:0/684108308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660101f20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 -- 172.21.15.16:0/684108308 >> 172.21.15.16:0/684108308 conn(0x7f46600fd2a0 msgr2=0x7f46600ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 -- 172.21.15.16:0/684108308 shutdown_connections 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.882+0000 7f4665a65640 1 -- 172.21.15.16:0/684108308 wait complete. 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.883+0000 7f4665a65640 1 Processor -- start 2024-09-12T00:56:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.883+0000 7f4665a65640 1 -- start start 2024-09-12T00:56:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.883+0000 7f4665a65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.883+0000 7f4665a65640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 0x7f46601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.883+0000 7f4665a65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f466019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f4665a65640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d7f0 con 0x7f4660101b20 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f4665a65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d670 con 0x7f46601036d0 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f4665a65640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d970 con 0x7f4660102d10 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40830/0 (socket says 172.21.15.16:40830) 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 -- 172.21.15.16:0/3358188521 learned_addr learned my addr 172.21.15.16:0/3358188521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465ffff640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f466019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465effd640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 0x7f46601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 -- 172.21.15.16:0/3358188521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 msgr2=0x7f46601983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 0x7f46601983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 -- 172.21.15.16:0/3358188521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 msgr2=0x7f466019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f466019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 -- 172.21.15.16:0/3358188521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466019fac0 con 0x7f4660101b20 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465ffff640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f466019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465f7fe640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4648031440 tx=0x7f4648032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464803f030 con 0x7f4660101b20 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466019fd50 con 0x7f4660101b20 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4648003ce0 con 0x7f4660101b20 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.884+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648035070 con 0x7f4660101b20 2024-09-12T00:56:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.885+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601a01d0 con 0x7f4660101b20 2024-09-12T00:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.886+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4660108f80 con 0x7f4660101b20 2024-09-12T00:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.887+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f4648042030 con 0x7f4660101b20 2024-09-12T00:56:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.888+0000 7f465cff9640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 0x7f464c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.888+0000 7f465effd640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 0x7f464c0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.888+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f46480ef220 con 0x7f4660101b20 2024-09-12T00:56:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.888+0000 7f465effd640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 0x7f464c0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46601993b0 tx=0x7f465000b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:04.892+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46480bbd30 con 0x7f4660101b20 2024-09-12T00:56:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.070+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4660062df0 con 0x7f4660101b20 2024-09-12T00:56:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.070+0000 7f465cff9640 1 -- 172.21.15.16:0/3358188521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f46480c0030 con 0x7f4660101b20 2024-09-12T00:56:05.070 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T00:56:05.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 msgr2=0x7f464c0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 0x7f464c0a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f46601993b0 tx=0x7f465000b390 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 msgr2=0x7f4660197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4648031440 tx=0x7f4648032ce0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 shutdown_connections 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f464c0a2990 0x7f464c0a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46601036d0 0x7f466019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4660102d10 0x7f46601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 --2- 172.21.15.16:0/3358188521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4660101b20 0x7f4660197ea0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 >> 172.21.15.16:0/3358188521 conn(0x7f46600fd2a0 msgr2=0x7f4660106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 shutdown_connections 2024-09-12T00:56:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.073+0000 7f4665a65640 1 -- 172.21.15.16:0/3358188521 wait complete. 2024-09-12T00:56:05.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T00:56:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 -- 172.21.15.16:0/1862118133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 msgr2=0x7f827c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 --2- 172.21.15.16:0/1862118133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c104a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8270009900 tx=0x7f827002a410 comp rx=0 tx=0).stop 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 -- 172.21.15.16:0/1862118133 shutdown_connections 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 --2- 172.21.15.16:0/1862118133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c107880 0x7f827c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 --2- 172.21.15.16:0/1862118133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c104f50 0x7f827c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 --2- 172.21.15.16:0/1862118133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c104a10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 -- 172.21.15.16:0/1862118133 >> 172.21.15.16:0/1862118133 conn(0x7f827c0fc140 msgr2=0x7f827c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 -- 172.21.15.16:0/1862118133 shutdown_connections 2024-09-12T00:56:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.199+0000 7f8282e53640 1 -- 172.21.15.16:0/1862118133 wait complete. 2024-09-12T00:56:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 Processor -- start 2024-09-12T00:56:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 -- start start 2024-09-12T00:56:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 0x7f827c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c107880 0x7f827c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10d7c0 con 0x7f827c102620 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10d640 con 0x7f827c107880 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8282e53640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10d940 con 0x7f827c104f50 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8280bef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8280bef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40846/0 (socket says 172.21.15.16:40846) 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f8280bef640 1 -- 172.21.15.16:0/1109387699 learned_addr learned my addr 172.21.15.16:0/1109387699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f827bfff640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 0x7f827c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.200+0000 7f82813f0640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c107880 0x7f827c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 -- 172.21.15.16:0/1109387699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 msgr2=0x7f827c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 0x7f827c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 -- 172.21.15.16:0/1109387699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c107880 msgr2=0x7f827c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c107880 0x7f827c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 -- 172.21.15.16:0/1109387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c1a1ce0 con 0x7f827c102620 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f827bfff640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 0x7f827c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8280bef640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8270009230 tx=0x7f82700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827003f030 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8270003ce0 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c1a1f70 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270035070 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.201+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c1a2380 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.203+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c109370 con 0x7f827c102620 2024-09-12T00:56:05.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.204+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f8270042030 con 0x7f827c102620 2024-09-12T00:56:05.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.205+0000 7f8279ffb640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 0x7f826c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.205+0000 7f827bfff640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 0x7f826c0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.205+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f82700ef1a0 con 0x7f827c102620 2024-09-12T00:56:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.206+0000 7f827bfff640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 0x7f826c0a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f827c19b5d0 tx=0x7f8268005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.209+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82700bbcb0 con 0x7f827c102620 2024-09-12T00:56:05.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.370+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f827003b460 con 0x7f827c102620 2024-09-12T00:56:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.447+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f827c062e90 con 0x7f827c102620 2024-09-12T00:56:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.447+0000 7f8279ffb640 1 -- 172.21.15.16:0/1109387699 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f8270075f80 con 0x7f827c102620 2024-09-12T00:56:05.447 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 msgr2=0x7f826c0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 0x7f826c0a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f827c19b5d0 tx=0x7f8268005f90 comp rx=0 tx=0).stop 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 msgr2=0x7f827c19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8270009230 tx=0x7f82700042f0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 shutdown_connections 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f826c0a2990 0x7f826c0a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827c107880 0x7f827c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f827c104f50 0x7f827c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 --2- 172.21.15.16:0/1109387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f827c102620 0x7f827c19a0c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 >> 172.21.15.16:0/1109387699 conn(0x7f827c0fc140 msgr2=0x7f827c101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 shutdown_connections 2024-09-12T00:56:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.449+0000 7f8282e53640 1 -- 172.21.15.16:0/1109387699 wait complete. 2024-09-12T00:56:05.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 -- 172.21.15.16:0/630709270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 msgr2=0x7f1238100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 --2- 172.21.15.16:0/630709270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f1238100e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1234009860 tx=0x7f123402a340 comp rx=0 tx=0).stop 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 -- 172.21.15.16:0/630709270 shutdown_connections 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 --2- 172.21.15.16:0/630709270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12381025c0 0x7f1238108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 --2- 172.21.15.16:0/630709270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1238101c00 0x7f1238102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 --2- 172.21.15.16:0/630709270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f1238100e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 -- 172.21.15.16:0/630709270 >> 172.21.15.16:0/630709270 conn(0x7f12380fc190 msgr2=0x7f12380fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 -- 172.21.15.16:0/630709270 shutdown_connections 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.595+0000 7f12404af640 1 -- 172.21.15.16:0/630709270 wait complete. 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 Processor -- start 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 -- start start 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f123819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12381025c0 0x7f12381a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f123810c690 con 0x7f1238101c00 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f123810c510 con 0x7f1238100a10 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f12404af640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f123810c810 con 0x7f12381025c0 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123da4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123da4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123da4a640 1 -- 172.21.15.16:0/793057781 learned_addr learned my addr 172.21.15.16:0/793057781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f123819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123ea4c640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12381025c0 0x7f12381a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 -- 172.21.15.16:0/793057781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12381025c0 msgr2=0x7f12381a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12381025c0 0x7f12381a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 -- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 msgr2=0x7f123819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123e24b640 1 -- 172.21.15.16:0/793057781 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381a1ce0 con 0x7f1238100a10 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.596+0000 7f123da4a640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f123e24b640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f123819a0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1234035040 tx=0x7f1234033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234037040 con 0x7f1238100a10 2024-09-12T00:56:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1234004040 con 0x7f1238100a10 2024-09-12T00:56:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381a1f70 con 0x7f1238100a10 2024-09-12T00:56:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234005650 con 0x7f1238100a10 2024-09-12T00:56:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.597+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a2380 con 0x7f1238100a10 2024-09-12T00:56:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.598+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12381a25b0 con 0x7f1238100a10 2024-09-12T00:56:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.599+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f1234042020 con 0x7f1238100a10 2024-09-12T00:56:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.600+0000 7f122f7fe640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 0x7f12240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.600+0000 7f123da4a640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 0x7f12240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.600+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f12340ef310 con 0x7f1238100a10 2024-09-12T00:56:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.601+0000 7f123da4a640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 0x7f12240a52d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f123819b5d0 tx=0x7f122800b480 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.607+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1234049020 con 0x7f1238100a10 2024-09-12T00:56:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:05 smithi159 ceph-mon[35566]: mgrmap e21: x(active, since 1.01915s), standbys: y, z 2024-09-12T00:56:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:05 smithi159 ceph-mon[35566]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:05 smithi159 ceph-mon[35566]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3951251838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3358188521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:56:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:05 smithi016 ceph-mon[30972]: mgrmap e21: x(active, since 1.01915s), standbys: y, z 2024-09-12T00:56:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:05 smithi016 ceph-mon[30972]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:05 smithi016 ceph-mon[30972]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3951251838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3358188521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:56:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:05 smithi099 ceph-mon[35654]: mgrmap e21: x(active, since 1.01915s), standbys: y, z 2024-09-12T00:56:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:05 smithi099 ceph-mon[35654]: from='client.24901 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:05 smithi099 ceph-mon[35654]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3951251838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3358188521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T00:56:05.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.797+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12381a2780 con 0x7f1238100a10 2024-09-12T00:56:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.798+0000 7f122f7fe640 1 -- 172.21.15.16:0/793057781 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f12340bbea0 con 0x7f1238100a10 2024-09-12T00:56:05.797 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 msgr2=0x7f12240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 0x7f12240a52d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f123819b5d0 tx=0x7f122800b480 comp rx=0 tx=0).stop 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 msgr2=0x7f123819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f123819a0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1234035040 tx=0x7f1234033d90 comp rx=0 tx=0).stop 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 shutdown_connections 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f12240a2e10 0x7f12240a52d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12381025c0 0x7f12381a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1238101c00 0x7f123819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 --2- 172.21.15.16:0/793057781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1238100a10 0x7f123819a0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 >> 172.21.15.16:0/793057781 conn(0x7f12380fc190 msgr2=0x7f123806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 shutdown_connections 2024-09-12T00:56:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:05.801+0000 7f12404af640 1 -- 172.21.15.16:0/793057781 wait complete. 2024-09-12T00:56:05.814 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T00:56:05.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:06.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:06.143 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:06.047+0000 7f1d2094d640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:00:56:06] ENGINE Error in HTTPServer.serve 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: Traceback (most recent call last): 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._connections.run(self.expiration_interval) 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._run(expiration_interval) 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:56:06.144 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: s = self.context.wrap_socket( 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: return self.sslsocket_class._create( 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self.do_handshake() 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: self._sslobj.do_handshake() 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:56:06.145 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:56:06.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T00:56:06.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1109387699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/793057781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: [12/Sep/2024:00:56:05] ENGINE Bus STARTING 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: [12/Sep/2024:00:56:05] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: [12/Sep/2024:00:56:06] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T00:56:06.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: [12/Sep/2024:00:56:06] ENGINE Bus STARTED 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: [12/Sep/2024:00:56:06] ENGINE Error in HTTPServer.serve 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:56:06.511 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.512 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T00:56:06.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T00:56:06.514 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:06 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T00:56:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T00:56:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1109387699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/793057781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: [12/Sep/2024:00:56:05] ENGINE Bus STARTING 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: [12/Sep/2024:00:56:05] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T00:56:06.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: [12/Sep/2024:00:56:06] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: [12/Sep/2024:00:56:06] ENGINE Bus STARTED 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: [12/Sep/2024:00:56:06] ENGINE Error in HTTPServer.serve 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: Traceback (most recent call last): 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: self._connections.run(self.expiration_interval) 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: self._run(expiration_interval) 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:56:06.618 INFO:journalctl@ceph.mon.c.smithi159.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: s = self.context.wrap_socket( 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: return self.sslsocket_class._create( 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: self.do_handshake() 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: self._sslobj.do_handshake() 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T00:56:06.621 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T00:56:06.621 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:06 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T00:56:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 -- 172.21.15.16:0/3098064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b4102d30 msgr2=0x7f30b4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 --2- 172.21.15.16:0/3098064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b4102d30 0x7f30b4103110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009930 tx=0x7f30ac02a420 comp rx=0 tx=0).stop 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 -- 172.21.15.16:0/3098064741 shutdown_connections 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 --2- 172.21.15.16:0/3098064741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41082a0 0x7f30b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 --2- 172.21.15.16:0/3098064741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b41036e0 0x7f30b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 --2- 172.21.15.16:0/3098064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b4102d30 0x7f30b4103110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 -- 172.21.15.16:0/3098064741 >> 172.21.15.16:0/3098064741 conn(0x7f30b40fc7d0 msgr2=0x7f30b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 -- 172.21.15.16:0/3098064741 shutdown_connections 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.593+0000 7f30bab30640 1 -- 172.21.15.16:0/3098064741 wait complete. 2024-09-12T00:56:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 Processor -- start 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 -- start start 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b4102d30 0x7f30b41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41036e0 0x7f30b4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41120a0 con 0x7f30b41082a0 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4111f20 con 0x7f30b41036e0 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30bab30640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4112220 con 0x7f30b4102d30 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40896/0 (socket says 172.21.15.16:40896) 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 -- 172.21.15.16:0/533271624 learned_addr learned my addr 172.21.15.16:0/533271624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b3fff640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41036e0 0x7f30b4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b88cc640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b4102d30 0x7f30b41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 -- 172.21.15.16:0/533271624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b4102d30 msgr2=0x7f30b41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b4102d30 0x7f30b41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 -- 172.21.15.16:0/533271624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41036e0 msgr2=0x7f30b4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41036e0 0x7f30b4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.594+0000 7f30b90cd640 1 -- 172.21.15.16:0/533271624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b419d550 con 0x7f30b41082a0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30b90cd640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30a400eb10 tx=0x7f30a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a400ce20 con 0x7f30b41082a0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4005020 con 0x7f30b41082a0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b419d840 con 0x7f30b41082a0 2024-09-12T00:56:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4010690 con 0x7f30b41082a0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.595+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41a5120 con 0x7f30b41082a0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.596+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b406a850 con 0x7f30b41082a0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.597+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f30a4005540 con 0x7f30b41082a0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.598+0000 7f30b1ffb640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 0x7f30a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.598+0000 7f30b88cc640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 0x7f30a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.598+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f30a40cfbf0 con 0x7f30b41082a0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.598+0000 7f30b88cc640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 0x7f30a00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.602+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30a409c670 con 0x7f30b41082a0 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1109387699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/793057781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: [12/Sep/2024:00:56:05] ENGINE Bus STARTING 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: [12/Sep/2024:00:56:05] ENGINE Serving on http://172.21.15.16:8765 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: [12/Sep/2024:00:56:06] ENGINE Serving on https://172.21.15.16:7150 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: [12/Sep/2024:00:56:06] ENGINE Bus STARTED 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: [12/Sep/2024:00:56:06] ENGINE Error in HTTPServer.serve 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: Traceback (most recent call last): 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._run(expiration_interval) 2024-09-12T00:56:06.775 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: s = self.context.wrap_socket( 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: return self.sslsocket_class._create( 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: self.do_handshake() 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._sslobj.do_handshake() 2024-09-12T00:56:06.776 INFO:journalctl@ceph.mon.b.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T00:56:06.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:06.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T00:56:06.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T00:56:06.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:06 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T00:56:06.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.820+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30b410a2b0 con 0x7f30b41082a0 2024-09-12T00:56:06.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.821+0000 7f30b1ffb640 1 -- 172.21.15.16:0/533271624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f30a409bf70 con 0x7f30b41082a0 2024-09-12T00:56:06.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:06.820 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 msgr2=0x7f30a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 0x7f30a00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000c00 tx=0x7f30ac005a20 comp rx=0 tx=0).stop 2024-09-12T00:56:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 msgr2=0x7f30b419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30a400eb10 tx=0x7f30a400ef10 comp rx=0 tx=0).stop 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 shutdown_connections 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30a00a2d80 0x7f30a00a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b41082a0 0x7f30b419ce70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b41036e0 0x7f30b4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 --2- 172.21.15.16:0/533271624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30b4102d30 0x7f30b41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.823+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 >> 172.21.15.16:0/533271624 conn(0x7f30b40fc7d0 msgr2=0x7f30b4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.824+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 shutdown_connections 2024-09-12T00:56:06.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:06.824+0000 7f30bab30640 1 -- 172.21.15.16:0/533271624 wait complete. 2024-09-12T00:56:07.703 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T00:56:07.703 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T00:56:07.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- 172.21.15.16:0/223280176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c1008a0 msgr2=0x7f289c100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/223280176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c1008a0 0x7f289c100c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2898009900 tx=0x7f289802a410 comp rx=0 tx=0).stop 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- 172.21.15.16:0/223280176 shutdown_connections 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/223280176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c10e450 0x7f289c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/223280176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c101250 0x7f289c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/223280176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c1008a0 0x7f289c100c80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:07.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- 172.21.15.16:0/223280176 >> 172.21.15.16:0/223280176 conn(0x7f289c0fc6a0 msgr2=0x7f289c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- 172.21.15.16:0/223280176 shutdown_connections 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- 172.21.15.16:0/223280176 wait complete. 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 Processor -- start 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.815+0000 7f28a3d9a640 1 -- start start 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c1008a0 0x7f289c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c101250 0x7f289c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1140d0 con 0x7f289c10e450 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f289c113f50 con 0x7f289c101250 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a3d9a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f289c114250 con 0x7f289c1008a0 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40902/0 (socket says 172.21.15.16:40902) 2024-09-12T00:56:07.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 -- 172.21.15.16:0/1129137939 learned_addr learned my addr 172.21.15.16:0/1129137939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a1b36640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c1008a0 0x7f289c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a1335640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c101250 0x7f289c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 -- 172.21.15.16:0/1129137939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c1008a0 msgr2=0x7f289c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c1008a0 0x7f289c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 -- 172.21.15.16:0/1129137939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c101250 msgr2=0x7f289c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c101250 0x7f289c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 -- 172.21.15.16:0/1129137939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c19f630 con 0x7f289c10e450 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f28a2337640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f289400eb40 tx=0x7f289400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.816+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894004b20 con 0x7f289c10e450 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.817+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2894004cc0 con 0x7f289c10e450 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.817+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894010850 con 0x7f289c10e450 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.817+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c19f920 con 0x7f289c10e450 2024-09-12T00:56:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.817+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1a71b0 con 0x7f289c10e450 2024-09-12T00:56:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.818+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c100c80 con 0x7f289c10e450 2024-09-12T00:56:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.819+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f2894005020 con 0x7f289c10e450 2024-09-12T00:56:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.819+0000 7f2892ffd640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 0x7f28880a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.819+0000 7f28a1b36640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 0x7f28880a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.820+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f28940cfd80 con 0x7f289c10e450 2024-09-12T00:56:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.820+0000 7f28a1b36640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 0x7f28880a5360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2898009230 tx=0x7f28980058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:07.824+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f289409c780 con 0x7f289c10e450 2024-09-12T00:56:08.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.001+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f289c1095b0 con 0x7f28880a2ea0 2024-09-12T00:56:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.008+0000 7f2892ffd640 1 -- 172.21.15.16:0/1129137939 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258050 (secure 0 0 0) 0x7f289c1095b0 con 0x7f28880a2ea0 2024-09-12T00:56:08.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:08.012 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 msgr2=0x7f28880a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 0x7f28880a5360 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2898009230 tx=0x7f28980058d0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 msgr2=0x7f289c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f289400eb40 tx=0x7f289400c2d0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 shutdown_connections 2024-09-12T00:56:08.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28880a2ea0 0x7f28880a5360 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f289c10e450 0x7f289c19ef00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289c101250 0x7f289c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.011+0000 7f28a3d9a640 1 --2- 172.21.15.16:0/1129137939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c1008a0 0x7f289c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.012+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 >> 172.21.15.16:0/1129137939 conn(0x7f289c0fc6a0 msgr2=0x7f289c10f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.012+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 shutdown_connections 2024-09-12T00:56:08.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.012+0000 7f28a3d9a640 1 -- 172.21.15.16:0/1129137939 wait complete. 2024-09-12T00:56:08.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/533271624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.023 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:07 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.024 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-12T00:56:07.358288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189621,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1344,"num_read_kb":837,"num_write":553,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8458599,"data_compressed":201479,"data_compressed_allocated":258048,"data_compressed_original":2949120,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987819"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436456+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436456+0000","last_peered":"2024-09-12T00:56:03.436456+0000","last_clean":"2024-09-12T00:56:03.436456+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436456+0000","last_undegraded":"2024-09-12T00:56:03.436456+0000","last_fullsized":"2024-09-12T00:56:03.436456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386221+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386221+0000","last_peered":"2024-09-12T00:56:03.386221+0000","last_clean":"2024-09-12T00:56:03.386221+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386221+0000","last_undegraded":"2024-09-12T00:56:03.386221+0000","last_fullsized":"2024-09-12T00:56:03.386221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386228+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386228+0000","last_peered":"2024-09-12T00:56:03.386228+0000","last_clean":"2024-09-12T00:56:03.386228+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386228+0000","last_undegraded":"2024-09-12T00:56:03.386228+0000","last_fullsized":"2024-09-12T00:56:03.386228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624526+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624526+0000","last_peered":"2024-09-12T00:56:03.624526+0000","last_clean":"2024-09-12T00:56:03.624526+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624526+0000","last_undegraded":"2024-09-12T00:56:03.624526+0000","last_fullsized":"2024-09-12T00:56:03.624526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436872+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436872+0000","last_peered":"2024-09-12T00:56:03.436872+0000","last_clean":"2024-09-12T00:56:03.436872+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436872+0000","last_undegraded":"2024-09-12T00:56:03.436872+0000","last_fullsized":"2024-09-12T00:56:03.436872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437657+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437657+0000","last_peered":"2024-09-12T00:56:03.437657+0000","last_clean":"2024-09-12T00:56:03.437657+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437657+0000","last_undegraded":"2024-09-12T00:56:03.437657+0000","last_fullsized":"2024-09-12T00:56:03.437657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436272+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436272+0000","last_peered":"2024-09-12T00:56:03.436272+0000","last_clean":"2024-09-12T00:56:03.436272+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436272+0000","last_undegraded":"2024-09-12T00:56:03.436272+0000","last_fullsized":"2024-09-12T00:56:03.436272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345432+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345432+0000","last_peered":"2024-09-12T00:56:03.345432+0000","last_clean":"2024-09-12T00:56:03.345432+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345432+0000","last_undegraded":"2024-09-12T00:56:03.345432+0000","last_fullsized":"2024-09-12T00:56:03.345432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386030+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386030+0000","last_peered":"2024-09-12T00:56:03.386030+0000","last_clean":"2024-09-12T00:56:03.386030+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386030+0000","last_undegraded":"2024-09-12T00:56:03.386030+0000","last_fullsized":"2024-09-12T00:56:03.386030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350356+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350356+0000","last_peered":"2024-09-12T00:56:03.350356+0000","last_clean":"2024-09-12T00:56:03.350356+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350356+0000","last_undegraded":"2024-09-12T00:56:03.350356+0000","last_fullsized":"2024-09-12T00:56:03.350356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350400+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350400+0000","last_peered":"2024-09-12T00:56:03.350400+0000","last_clean":"2024-09-12T00:56:03.350400+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350400+0000","last_undegraded":"2024-09-12T00:56:03.350400+0000","last_fullsized":"2024-09-12T00:56:03.350400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445084+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:03.445084+0000","last_peered":"2024-09-12T00:56:03.445084+0000","last_clean":"2024-09-12T00:56:03.445084+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:03.445084+0000","last_undegraded":"2024-09-12T00:56:03.445084+0000","last_fullsized":"2024-09-12T00:56:03.445084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":246,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970707+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:03.970707+0000","last_peered":"2024-09-12T00:56:03.970707+0000","last_clean":"2024-09-12T00:56:03.970707+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:03.970707+0000","last_undegraded":"2024-09-12T00:56:03.970707+0000","last_fullsized":"2024-09-12T00:56:03.970707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'35","reported_seq":303,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.989038+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:03.989038+0000","last_peered":"2024-09-12T00:56:03.989038+0000","last_clean":"2024-09-12T00:56:03.989038+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:03.989038+0000","last_undegraded":"2024-09-12T00:56:03.989038+0000","last_fullsized":"2024-09-12T00:56:03.989038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":173,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624344+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:03.624344+0000","last_peered":"2024-09-12T00:56:03.624344+0000","last_clean":"2024-09-12T00:56:03.624344+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:03.624344+0000","last_undegraded":"2024-09-12T00:56:03.624344+0000","last_fullsized":"2024-09-12T00:56:03.624344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.569030+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:03.569030+0000","last_peered":"2024-09-12T00:56:03.569030+0000","last_clean":"2024-09-12T00:56:03.569030+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:03.569030+0000","last_undegraded":"2024-09-12T00:56:03.569030+0000","last_fullsized":"2024-09-12T00:56:03.569030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.976038+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:03.976038+0000","last_peered":"2024-09-12T00:56:03.976038+0000","last_clean":"2024-09-12T00:56:03.976038+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.976038+0000","last_undegraded":"2024-09-12T00:56:03.976038+0000","last_fullsized":"2024-09-12T00:56:03.976038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627590+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:03.627590+0000","last_peered":"2024-09-12T00:56:03.627590+0000","last_clean":"2024-09-12T00:56:03.627590+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:03.627590+0000","last_undegraded":"2024-09-12T00:56:03.627590+0000","last_fullsized":"2024-09-12T00:56:03.627590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624390+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624390+0000","last_peered":"2024-09-12T00:56:03.624390+0000","last_clean":"2024-09-12T00:56:03.624390+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624390+0000","last_undegraded":"2024-09-12T00:56:03.624390+0000","last_fullsized":"2024-09-12T00:56:03.624390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437353+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437353+0000","last_peered":"2024-09-12T00:56:03.437353+0000","last_clean":"2024-09-12T00:56:03.437353+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437353+0000","last_undegraded":"2024-09-12T00:56:03.437353+0000","last_fullsized":"2024-09-12T00:56:03.437353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627589+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627589+0000","last_peered":"2024-09-12T00:56:03.627589+0000","last_clean":"2024-09-12T00:56:03.627589+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627589+0000","last_undegraded":"2024-09-12T00:56:03.627589+0000","last_fullsized":"2024-09-12T00:56:03.627589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345216+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345216+0000","last_peered":"2024-09-12T00:56:03.345216+0000","last_clean":"2024-09-12T00:56:03.345216+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345216+0000","last_undegraded":"2024-09-12T00:56:03.345216+0000","last_fullsized":"2024-09-12T00:56:03.345216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.975057+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:03.975057+0000","last_peered":"2024-09-12T00:56:03.975057+0000","last_clean":"2024-09-12T00:56:03.975057+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:03.975057+0000","last_undegraded":"2024-09-12T00:56:03.975057+0000","last_fullsized":"2024-09-12T00:56:03.975057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444936+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444936+0000","last_peered":"2024-09-12T00:56:03.444936+0000","last_clean":"2024-09-12T00:56:03.444936+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444936+0000","last_undegraded":"2024-09-12T00:56:03.444936+0000","last_fullsized":"2024-09-12T00:56:03.444936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970683+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:03.970683+0000","last_peered":"2024-09-12T00:56:03.970683+0000","last_clean":"2024-09-12T00:56:03.970683+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:03.970683+0000","last_undegraded":"2024-09-12T00:56:03.970683+0000","last_fullsized":"2024-09-12T00:56:03.970683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447741+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447741+0000","last_peered":"2024-09-12T00:56:03.447741+0000","last_clean":"2024-09-12T00:56:03.447741+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447741+0000","last_undegraded":"2024-09-12T00:56:03.447741+0000","last_fullsized":"2024-09-12T00:56:03.447741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434138+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434138+0000","last_peered":"2024-09-12T00:56:03.434138+0000","last_clean":"2024-09-12T00:56:03.434138+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434138+0000","last_undegraded":"2024-09-12T00:56:03.434138+0000","last_fullsized":"2024-09-12T00:56:03.434138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447811+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447811+0000","last_peered":"2024-09-12T00:56:03.447811+0000","last_clean":"2024-09-12T00:56:03.447811+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447811+0000","last_undegraded":"2024-09-12T00:56:03.447811+0000","last_fullsized":"2024-09-12T00:56:03.447811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628996+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.628996+0000","last_peered":"2024-09-12T00:56:03.628996+0000","last_clean":"2024-09-12T00:56:03.628996+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.628996+0000","last_undegraded":"2024-09-12T00:56:03.628996+0000","last_fullsized":"2024-09-12T00:56:03.628996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345607+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345607+0000","last_peered":"2024-09-12T00:56:03.345607+0000","last_clean":"2024-09-12T00:56:03.345607+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345607+0000","last_undegraded":"2024-09-12T00:56:03.345607+0000","last_fullsized":"2024-09-12T00:56:03.345607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.574425+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:03.574425+0000","last_peered":"2024-09-12T00:56:03.574425+0000","last_clean":"2024-09-12T00:56:03.574425+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:03.574425+0000","last_undegraded":"2024-09-12T00:56:03.574425+0000","last_fullsized":"2024-09-12T00:56:03.574425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437778+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437778+0000","last_peered":"2024-09-12T00:56:03.437778+0000","last_clean":"2024-09-12T00:56:03.437778+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437778+0000","last_undegraded":"2024-09-12T00:56:03.437778+0000","last_fullsized":"2024-09-12T00:56:03.437778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624460+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624460+0000","last_peered":"2024-09-12T00:56:03.624460+0000","last_clean":"2024-09-12T00:56:03.624460+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624460+0000","last_undegraded":"2024-09-12T00:56:03.624460+0000","last_fullsized":"2024-09-12T00:56:03.624460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386273+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386273+0000","last_peered":"2024-09-12T00:56:03.386273+0000","last_clean":"2024-09-12T00:56:03.386273+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386273+0000","last_undegraded":"2024-09-12T00:56:03.386273+0000","last_fullsized":"2024-09-12T00:56:03.386273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346761+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346761+0000","last_peered":"2024-09-12T00:56:03.346761+0000","last_clean":"2024-09-12T00:56:03.346761+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346761+0000","last_undegraded":"2024-09-12T00:56:03.346761+0000","last_fullsized":"2024-09-12T00:56:03.346761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437122+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437122+0000","last_peered":"2024-09-12T00:56:03.437122+0000","last_clean":"2024-09-12T00:56:03.437122+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437122+0000","last_undegraded":"2024-09-12T00:56:03.437122+0000","last_fullsized":"2024-09-12T00:56:03.437122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436279+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436279+0000","last_peered":"2024-09-12T00:56:03.436279+0000","last_clean":"2024-09-12T00:56:03.436279+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436279+0000","last_undegraded":"2024-09-12T00:56:03.436279+0000","last_fullsized":"2024-09-12T00:56:03.436279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350476+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.350476+0000","last_peered":"2024-09-12T00:56:03.350476+0000","last_clean":"2024-09-12T00:56:03.350476+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.350476+0000","last_undegraded":"2024-09-12T00:56:03.350476+0000","last_fullsized":"2024-09-12T00:56:03.350476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437427+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437427+0000","last_peered":"2024-09-12T00:56:03.437427+0000","last_clean":"2024-09-12T00:56:03.437427+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437427+0000","last_undegraded":"2024-09-12T00:56:03.437427+0000","last_fullsized":"2024-09-12T00:56:03.437427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628881+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628881+0000","last_peered":"2024-09-12T00:56:03.628881+0000","last_clean":"2024-09-12T00:56:03.628881+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628881+0000","last_undegraded":"2024-09-12T00:56:03.628881+0000","last_fullsized":"2024-09-12T00:56:03.628881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386061+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386061+0000","last_peered":"2024-09-12T00:56:03.386061+0000","last_clean":"2024-09-12T00:56:03.386061+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386061+0000","last_undegraded":"2024-09-12T00:56:03.386061+0000","last_fullsized":"2024-09-12T00:56:03.386061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970280+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:03.970280+0000","last_peered":"2024-09-12T00:56:03.970280+0000","last_clean":"2024-09-12T00:56:03.970280+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:03.970280+0000","last_undegraded":"2024-09-12T00:56:03.970280+0000","last_fullsized":"2024-09-12T00:56:03.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599019,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1208,"num_read_kb":718,"num_write":370,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1797057,"data_compressed":85571,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":954368,"data_stored":847325,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":1242272,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046544,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":452829,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1176895,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":519151,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":454889,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":520062,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455095,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67161,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199797,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:08.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T00:56:08.029 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:56:08.030 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/533271624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.031 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:07 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-12T00:56:08.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-12T00:56:08.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-12T00:56:08.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/533271624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:08.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.119 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T00:56:08.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.120 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:07 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 -- 172.21.15.16:0/3157359144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102c90 msgr2=0x7f0598103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 --2- 172.21.15.16:0/3157359144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102c90 0x7f0598103070 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0588009900 tx=0x7f058802a480 comp rx=0 tx=0).stop 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 -- 172.21.15.16:0/3157359144 shutdown_connections 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 --2- 172.21.15.16:0/3157359144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f059810c0e0 0x7f059810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 --2- 172.21.15.16:0/3157359144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05981035b0 0x7f059810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 --2- 172.21.15.16:0/3157359144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598102c90 0x7f0598103070 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 -- 172.21.15.16:0/3157359144 >> 172.21.15.16:0/3157359144 conn(0x7f05980fc6a0 msgr2=0x7f05980feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 -- 172.21.15.16:0/3157359144 shutdown_connections 2024-09-12T00:56:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.169+0000 7f059ca82640 1 -- 172.21.15.16:0/3157359144 wait complete. 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 Processor -- start 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 -- start start 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598102c90 0x7f0598198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f059810c0e0 0x7f059819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0598111f10 con 0x7f05981035b0 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0598111d90 con 0x7f0598102c90 2024-09-12T00:56:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059ca82640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0598112090 con 0x7f059810c0e0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40926/0 (socket says 172.21.15.16:40926) 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 -- 172.21.15.16:0/2716852480 learned_addr learned my addr 172.21.15.16:0/2716852480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f059659c640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598102c90 0x7f0598198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0596d9d640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f059810c0e0 0x7f059819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 -- 172.21.15.16:0/2716852480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f059810c0e0 msgr2=0x7f059819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f059810c0e0 0x7f059819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 -- 172.21.15.16:0/2716852480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598102c90 msgr2=0x7f0598198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598102c90 0x7f0598198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 -- 172.21.15.16:0/2716852480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059819d430 con 0x7f05981035b0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.170+0000 7f0595d9b640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f058c00d740 tx=0x7f058c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.171+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c017070 con 0x7f05981035b0 2024-09-12T00:56:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.171+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f058c00de50 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.171+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059819d720 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.171+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c00f620 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.171+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981a4fb0 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.172+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0598103070 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.173+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f058c009c30 con 0x7f05981035b0 2024-09-12T00:56:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.173+0000 7f05877fe640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 0x7f05800a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.173+0000 7f059659c640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 0x7f05800a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.174+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f058c0cf0c0 con 0x7f05981035b0 2024-09-12T00:56:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.174+0000 7f059659c640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 0x7f05800a5360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f0588005b50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.178+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f058c09b510 con 0x7f05981035b0 2024-09-12T00:56:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.354+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05980fe490 con 0x7f05800a2ea0 2024-09-12T00:56:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.360+0000 7f05877fe640 1 -- 172.21.15.16:0/2716852480 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258050 (secure 0 0 0) 0x7f05980fe490 con 0x7f05800a2ea0 2024-09-12T00:56:08.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.363+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 msgr2=0x7f05800a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.363+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 0x7f05800a5360 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f0588005b50 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.363+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 msgr2=0x7f0598198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.363+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f058c00d740 tx=0x7f058c00dc00 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 shutdown_connections 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f05800a2ea0 0x7f05800a5360 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f059810c0e0 0x7f059819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05981035b0 0x7f0598198980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 --2- 172.21.15.16:0/2716852480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0598102c90 0x7f0598198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 >> 172.21.15.16:0/2716852480 conn(0x7f05980fc6a0 msgr2=0x7f05980fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 shutdown_connections 2024-09-12T00:56:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.364+0000 7f059ca82640 1 -- 172.21.15.16:0/2716852480 wait complete. 2024-09-12T00:56:08.374 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-12T00:56:07.358288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189621,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1344,"num_read_kb":837,"num_write":553,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8458599,"data_compressed":201479,"data_compressed_allocated":258048,"data_compressed_original":2949120,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987819"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436456+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436456+0000","last_peered":"2024-09-12T00:56:03.436456+0000","last_clean":"2024-09-12T00:56:03.436456+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436456+0000","last_undegraded":"2024-09-12T00:56:03.436456+0000","last_fullsized":"2024-09-12T00:56:03.436456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386221+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386221+0000","last_peered":"2024-09-12T00:56:03.386221+0000","last_clean":"2024-09-12T00:56:03.386221+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386221+0000","last_undegraded":"2024-09-12T00:56:03.386221+0000","last_fullsized":"2024-09-12T00:56:03.386221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386228+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386228+0000","last_peered":"2024-09-12T00:56:03.386228+0000","last_clean":"2024-09-12T00:56:03.386228+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386228+0000","last_undegraded":"2024-09-12T00:56:03.386228+0000","last_fullsized":"2024-09-12T00:56:03.386228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624526+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624526+0000","last_peered":"2024-09-12T00:56:03.624526+0000","last_clean":"2024-09-12T00:56:03.624526+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624526+0000","last_undegraded":"2024-09-12T00:56:03.624526+0000","last_fullsized":"2024-09-12T00:56:03.624526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436872+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436872+0000","last_peered":"2024-09-12T00:56:03.436872+0000","last_clean":"2024-09-12T00:56:03.436872+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436872+0000","last_undegraded":"2024-09-12T00:56:03.436872+0000","last_fullsized":"2024-09-12T00:56:03.436872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437657+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437657+0000","last_peered":"2024-09-12T00:56:03.437657+0000","last_clean":"2024-09-12T00:56:03.437657+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437657+0000","last_undegraded":"2024-09-12T00:56:03.437657+0000","last_fullsized":"2024-09-12T00:56:03.437657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436272+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436272+0000","last_peered":"2024-09-12T00:56:03.436272+0000","last_clean":"2024-09-12T00:56:03.436272+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436272+0000","last_undegraded":"2024-09-12T00:56:03.436272+0000","last_fullsized":"2024-09-12T00:56:03.436272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345432+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345432+0000","last_peered":"2024-09-12T00:56:03.345432+0000","last_clean":"2024-09-12T00:56:03.345432+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345432+0000","last_undegraded":"2024-09-12T00:56:03.345432+0000","last_fullsized":"2024-09-12T00:56:03.345432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386030+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386030+0000","last_peered":"2024-09-12T00:56:03.386030+0000","last_clean":"2024-09-12T00:56:03.386030+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386030+0000","last_undegraded":"2024-09-12T00:56:03.386030+0000","last_fullsized":"2024-09-12T00:56:03.386030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350356+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350356+0000","last_peered":"2024-09-12T00:56:03.350356+0000","last_clean":"2024-09-12T00:56:03.350356+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350356+0000","last_undegraded":"2024-09-12T00:56:03.350356+0000","last_fullsized":"2024-09-12T00:56:03.350356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350400+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350400+0000","last_peered":"2024-09-12T00:56:03.350400+0000","last_clean":"2024-09-12T00:56:03.350400+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350400+0000","last_undegraded":"2024-09-12T00:56:03.350400+0000","last_fullsized":"2024-09-12T00:56:03.350400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445084+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:03.445084+0000","last_peered":"2024-09-12T00:56:03.445084+0000","last_clean":"2024-09-12T00:56:03.445084+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:03.445084+0000","last_undegraded":"2024-09-12T00:56:03.445084+0000","last_fullsized":"2024-09-12T00:56:03.445084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":246,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970707+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:03.970707+0000","last_peered":"2024-09-12T00:56:03.970707+0000","last_clean":"2024-09-12T00:56:03.970707+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:03.970707+0000","last_undegraded":"2024-09-12T00:56:03.970707+0000","last_fullsized":"2024-09-12T00:56:03.970707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'35","reported_seq":303,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.989038+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:03.989038+0000","last_peered":"2024-09-12T00:56:03.989038+0000","last_clean":"2024-09-12T00:56:03.989038+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:03.989038+0000","last_undegraded":"2024-09-12T00:56:03.989038+0000","last_fullsized":"2024-09-12T00:56:03.989038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":173,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624344+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:03.624344+0000","last_peered":"2024-09-12T00:56:03.624344+0000","last_clean":"2024-09-12T00:56:03.624344+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:03.624344+0000","last_undegraded":"2024-09-12T00:56:03.624344+0000","last_fullsized":"2024-09-12T00:56:03.624344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.569030+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:03.569030+0000","last_peered":"2024-09-12T00:56:03.569030+0000","last_clean":"2024-09-12T00:56:03.569030+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:03.569030+0000","last_undegraded":"2024-09-12T00:56:03.569030+0000","last_fullsized":"2024-09-12T00:56:03.569030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.976038+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:03.976038+0000","last_peered":"2024-09-12T00:56:03.976038+0000","last_clean":"2024-09-12T00:56:03.976038+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.976038+0000","last_undegraded":"2024-09-12T00:56:03.976038+0000","last_fullsized":"2024-09-12T00:56:03.976038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627590+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:03.627590+0000","last_peered":"2024-09-12T00:56:03.627590+0000","last_clean":"2024-09-12T00:56:03.627590+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:03.627590+0000","last_undegraded":"2024-09-12T00:56:03.627590+0000","last_fullsized":"2024-09-12T00:56:03.627590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624390+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624390+0000","last_peered":"2024-09-12T00:56:03.624390+0000","last_clean":"2024-09-12T00:56:03.624390+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624390+0000","last_undegraded":"2024-09-12T00:56:03.624390+0000","last_fullsized":"2024-09-12T00:56:03.624390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437353+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437353+0000","last_peered":"2024-09-12T00:56:03.437353+0000","last_clean":"2024-09-12T00:56:03.437353+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437353+0000","last_undegraded":"2024-09-12T00:56:03.437353+0000","last_fullsized":"2024-09-12T00:56:03.437353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627589+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627589+0000","last_peered":"2024-09-12T00:56:03.627589+0000","last_clean":"2024-09-12T00:56:03.627589+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627589+0000","last_undegraded":"2024-09-12T00:56:03.627589+0000","last_fullsized":"2024-09-12T00:56:03.627589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345216+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345216+0000","last_peered":"2024-09-12T00:56:03.345216+0000","last_clean":"2024-09-12T00:56:03.345216+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345216+0000","last_undegraded":"2024-09-12T00:56:03.345216+0000","last_fullsized":"2024-09-12T00:56:03.345216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.975057+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:03.975057+0000","last_peered":"2024-09-12T00:56:03.975057+0000","last_clean":"2024-09-12T00:56:03.975057+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:03.975057+0000","last_undegraded":"2024-09-12T00:56:03.975057+0000","last_fullsized":"2024-09-12T00:56:03.975057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444936+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444936+0000","last_peered":"2024-09-12T00:56:03.444936+0000","last_clean":"2024-09-12T00:56:03.444936+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444936+0000","last_undegraded":"2024-09-12T00:56:03.444936+0000","last_fullsized":"2024-09-12T00:56:03.444936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970683+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:03.970683+0000","last_peered":"2024-09-12T00:56:03.970683+0000","last_clean":"2024-09-12T00:56:03.970683+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:03.970683+0000","last_undegraded":"2024-09-12T00:56:03.970683+0000","last_fullsized":"2024-09-12T00:56:03.970683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447741+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447741+0000","last_peered":"2024-09-12T00:56:03.447741+0000","last_clean":"2024-09-12T00:56:03.447741+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447741+0000","last_undegraded":"2024-09-12T00:56:03.447741+0000","last_fullsized":"2024-09-12T00:56:03.447741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434138+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434138+0000","last_peered":"2024-09-12T00:56:03.434138+0000","last_clean":"2024-09-12T00:56:03.434138+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434138+0000","last_undegraded":"2024-09-12T00:56:03.434138+0000","last_fullsized":"2024-09-12T00:56:03.434138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447811+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447811+0000","last_peered":"2024-09-12T00:56:03.447811+0000","last_clean":"2024-09-12T00:56:03.447811+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447811+0000","last_undegraded":"2024-09-12T00:56:03.447811+0000","last_fullsized":"2024-09-12T00:56:03.447811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628996+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.628996+0000","last_peered":"2024-09-12T00:56:03.628996+0000","last_clean":"2024-09-12T00:56:03.628996+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.628996+0000","last_undegraded":"2024-09-12T00:56:03.628996+0000","last_fullsized":"2024-09-12T00:56:03.628996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345607+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345607+0000","last_peered":"2024-09-12T00:56:03.345607+0000","last_clean":"2024-09-12T00:56:03.345607+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345607+0000","last_undegraded":"2024-09-12T00:56:03.345607+0000","last_fullsized":"2024-09-12T00:56:03.345607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.574425+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:03.574425+0000","last_peered":"2024-09-12T00:56:03.574425+0000","last_clean":"2024-09-12T00:56:03.574425+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:03.574425+0000","last_undegraded":"2024-09-12T00:56:03.574425+0000","last_fullsized":"2024-09-12T00:56:03.574425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437778+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437778+0000","last_peered":"2024-09-12T00:56:03.437778+0000","last_clean":"2024-09-12T00:56:03.437778+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437778+0000","last_undegraded":"2024-09-12T00:56:03.437778+0000","last_fullsized":"2024-09-12T00:56:03.437778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624460+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624460+0000","last_peered":"2024-09-12T00:56:03.624460+0000","last_clean":"2024-09-12T00:56:03.624460+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624460+0000","last_undegraded":"2024-09-12T00:56:03.624460+0000","last_fullsized":"2024-09-12T00:56:03.624460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386273+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386273+0000","last_peered":"2024-09-12T00:56:03.386273+0000","last_clean":"2024-09-12T00:56:03.386273+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386273+0000","last_undegraded":"2024-09-12T00:56:03.386273+0000","last_fullsized":"2024-09-12T00:56:03.386273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346761+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346761+0000","last_peered":"2024-09-12T00:56:03.346761+0000","last_clean":"2024-09-12T00:56:03.346761+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346761+0000","last_undegraded":"2024-09-12T00:56:03.346761+0000","last_fullsized":"2024-09-12T00:56:03.346761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437122+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437122+0000","last_peered":"2024-09-12T00:56:03.437122+0000","last_clean":"2024-09-12T00:56:03.437122+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437122+0000","last_undegraded":"2024-09-12T00:56:03.437122+0000","last_fullsized":"2024-09-12T00:56:03.437122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436279+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436279+0000","last_peered":"2024-09-12T00:56:03.436279+0000","last_clean":"2024-09-12T00:56:03.436279+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436279+0000","last_undegraded":"2024-09-12T00:56:03.436279+0000","last_fullsized":"2024-09-12T00:56:03.436279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350476+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.350476+0000","last_peered":"2024-09-12T00:56:03.350476+0000","last_clean":"2024-09-12T00:56:03.350476+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.350476+0000","last_undegraded":"2024-09-12T00:56:03.350476+0000","last_fullsized":"2024-09-12T00:56:03.350476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437427+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437427+0000","last_peered":"2024-09-12T00:56:03.437427+0000","last_clean":"2024-09-12T00:56:03.437427+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437427+0000","last_undegraded":"2024-09-12T00:56:03.437427+0000","last_fullsized":"2024-09-12T00:56:03.437427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628881+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628881+0000","last_peered":"2024-09-12T00:56:03.628881+0000","last_clean":"2024-09-12T00:56:03.628881+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628881+0000","last_undegraded":"2024-09-12T00:56:03.628881+0000","last_fullsized":"2024-09-12T00:56:03.628881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386061+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386061+0000","last_peered":"2024-09-12T00:56:03.386061+0000","last_clean":"2024-09-12T00:56:03.386061+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386061+0000","last_undegraded":"2024-09-12T00:56:03.386061+0000","last_fullsized":"2024-09-12T00:56:03.386061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970280+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:03.970280+0000","last_peered":"2024-09-12T00:56:03.970280+0000","last_clean":"2024-09-12T00:56:03.970280+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:03.970280+0000","last_undegraded":"2024-09-12T00:56:03.970280+0000","last_fullsized":"2024-09-12T00:56:03.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599019,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1208,"num_read_kb":718,"num_write":370,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1797057,"data_compressed":85571,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":954368,"data_stored":847325,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":1242272,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046544,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":452829,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1176895,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":519151,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":454889,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":520062,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455095,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67161,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199797,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:08.379 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T00:56:08.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T00:56:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1593452927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c10e450 msgr2=0x7f7d9c110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1593452927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c10e450 0x7f7d9c110840 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009ca0 tx=0x7f7d9402a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1593452927 shutdown_connections 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1593452927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c10e450 0x7f7d9c110840 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1593452927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c101250 0x7f7d9c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1593452927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.518+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1593452927 >> 172.21.15.16:0/1593452927 conn(0x7f7d9c0fc6a0 msgr2=0x7f7d9c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1593452927 shutdown_connections 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1593452927 wait complete. 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 Processor -- start 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- start start 2024-09-12T00:56:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c101250 0x7f7d9c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c10e450 0x7f7d9c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c1140b0 con 0x7f7d9c1008a0 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c113f30 con 0x7f7d9c101250 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.519+0000 7f7da3ef1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c114230 con 0x7f7d9c10e450 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46184/0 (socket says 172.21.15.16:46184) 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 -- 172.21.15.16:0/1663917210 learned_addr learned my addr 172.21.15.16:0/1663917210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da148c640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c101250 0x7f7d9c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da248e640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c10e450 0x7f7d9c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 -- 172.21.15.16:0/1663917210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c10e450 msgr2=0x7f7d9c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c10e450 0x7f7d9c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 -- 172.21.15.16:0/1663917210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c101250 msgr2=0x7f7d9c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c101250 0x7f7d9c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 -- 172.21.15.16:0/1663917210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c19f510 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da1c8d640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c920 tx=0x7f7d9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007c70 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d98007e10 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c19f800 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98018890 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.520+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c1a7090 con 0x7f7d9c1008a0 2024-09-12T00:56:08.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.521+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c100c80 con 0x7f7d9c1008a0 2024-09-12T00:56:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.522+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f7d98018a50 con 0x7f7d9c1008a0 2024-09-12T00:56:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.523+0000 7f7d92ffd640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 0x7f7d880a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.523+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7d980cfa40 con 0x7f7d9c1008a0 2024-09-12T00:56:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.523+0000 7f7da148c640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 0x7f7d880a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.524+0000 7f7da148c640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 0x7f7d880a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c10ecc0 tx=0x7f7d8c005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.527+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d98014030 con 0x7f7d9c1008a0 2024-09-12T00:56:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.702+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d9c10f700 con 0x7f7d880a2ea0 2024-09-12T00:56:08.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.708+0000 7f7d92ffd640 1 -- 172.21.15.16:0/1663917210 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258050 (secure 0 0 0) 0x7f7d9c10f700 con 0x7f7d880a2ea0 2024-09-12T00:56:08.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:08.711 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 msgr2=0x7f7d880a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 0x7f7d880a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c10ecc0 tx=0x7f7d8c005f90 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 msgr2=0x7f7d9c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c920 tx=0x7f7d9800cde0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 shutdown_connections 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d880a2ea0 0x7f7d880a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d9c10e450 0x7f7d9c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d9c101250 0x7f7d9c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 --2- 172.21.15.16:0/1663917210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9c1008a0 0x7f7d9c19a520 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 >> 172.21.15.16:0/1663917210 conn(0x7f7d9c0fc6a0 msgr2=0x7f7d9c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 shutdown_connections 2024-09-12T00:56:08.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:08.711+0000 7f7da3ef1640 1 -- 172.21.15.16:0/1663917210 wait complete. 2024-09-12T00:56:08.721 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-12T00:56:07.358288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189621,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1344,"num_read_kb":837,"num_write":553,"num_write_kb":4400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":936,"ondisk_log_size":936,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377448,"kb_used_data":11516,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564376,"statfs":{"total":1151940427776,"available":1151553921024,"internally_reserved":0,"allocated":11792384,"data_stored":8458599,"data_compressed":201479,"data_compressed_allocated":258048,"data_compressed_original":2949120,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987819"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436456+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436456+0000","last_peered":"2024-09-12T00:56:03.436456+0000","last_clean":"2024-09-12T00:56:03.436456+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436456+0000","last_undegraded":"2024-09-12T00:56:03.436456+0000","last_fullsized":"2024-09-12T00:56:03.436456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386221+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386221+0000","last_peered":"2024-09-12T00:56:03.386221+0000","last_clean":"2024-09-12T00:56:03.386221+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386221+0000","last_undegraded":"2024-09-12T00:56:03.386221+0000","last_fullsized":"2024-09-12T00:56:03.386221+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386228+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386228+0000","last_peered":"2024-09-12T00:56:03.386228+0000","last_clean":"2024-09-12T00:56:03.386228+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386228+0000","last_undegraded":"2024-09-12T00:56:03.386228+0000","last_fullsized":"2024-09-12T00:56:03.386228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624526+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624526+0000","last_peered":"2024-09-12T00:56:03.624526+0000","last_clean":"2024-09-12T00:56:03.624526+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624526+0000","last_undegraded":"2024-09-12T00:56:03.624526+0000","last_fullsized":"2024-09-12T00:56:03.624526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436872+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436872+0000","last_peered":"2024-09-12T00:56:03.436872+0000","last_clean":"2024-09-12T00:56:03.436872+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436872+0000","last_undegraded":"2024-09-12T00:56:03.436872+0000","last_fullsized":"2024-09-12T00:56:03.436872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437657+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437657+0000","last_peered":"2024-09-12T00:56:03.437657+0000","last_clean":"2024-09-12T00:56:03.437657+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437657+0000","last_undegraded":"2024-09-12T00:56:03.437657+0000","last_fullsized":"2024-09-12T00:56:03.437657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436272+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436272+0000","last_peered":"2024-09-12T00:56:03.436272+0000","last_clean":"2024-09-12T00:56:03.436272+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436272+0000","last_undegraded":"2024-09-12T00:56:03.436272+0000","last_fullsized":"2024-09-12T00:56:03.436272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345432+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345432+0000","last_peered":"2024-09-12T00:56:03.345432+0000","last_clean":"2024-09-12T00:56:03.345432+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345432+0000","last_undegraded":"2024-09-12T00:56:03.345432+0000","last_fullsized":"2024-09-12T00:56:03.345432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386030+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386030+0000","last_peered":"2024-09-12T00:56:03.386030+0000","last_clean":"2024-09-12T00:56:03.386030+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386030+0000","last_undegraded":"2024-09-12T00:56:03.386030+0000","last_fullsized":"2024-09-12T00:56:03.386030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350356+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350356+0000","last_peered":"2024-09-12T00:56:03.350356+0000","last_clean":"2024-09-12T00:56:03.350356+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350356+0000","last_undegraded":"2024-09-12T00:56:03.350356+0000","last_fullsized":"2024-09-12T00:56:03.350356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350400+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350400+0000","last_peered":"2024-09-12T00:56:03.350400+0000","last_clean":"2024-09-12T00:56:03.350400+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350400+0000","last_undegraded":"2024-09-12T00:56:03.350400+0000","last_fullsized":"2024-09-12T00:56:03.350400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":139,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445084+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:03.445084+0000","last_peered":"2024-09-12T00:56:03.445084+0000","last_clean":"2024-09-12T00:56:03.445084+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:03.445084+0000","last_undegraded":"2024-09-12T00:56:03.445084+0000","last_fullsized":"2024-09-12T00:56:03.445084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":246,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970707+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:03.970707+0000","last_peered":"2024-09-12T00:56:03.970707+0000","last_clean":"2024-09-12T00:56:03.970707+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:03.970707+0000","last_undegraded":"2024-09-12T00:56:03.970707+0000","last_fullsized":"2024-09-12T00:56:03.970707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'35","reported_seq":303,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.989038+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:03.989038+0000","last_peered":"2024-09-12T00:56:03.989038+0000","last_clean":"2024-09-12T00:56:03.989038+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:03.989038+0000","last_undegraded":"2024-09-12T00:56:03.989038+0000","last_fullsized":"2024-09-12T00:56:03.989038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":173,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624344+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:03.624344+0000","last_peered":"2024-09-12T00:56:03.624344+0000","last_clean":"2024-09-12T00:56:03.624344+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:03.624344+0000","last_undegraded":"2024-09-12T00:56:03.624344+0000","last_fullsized":"2024-09-12T00:56:03.624344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'41","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.569030+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:03.569030+0000","last_peered":"2024-09-12T00:56:03.569030+0000","last_clean":"2024-09-12T00:56:03.569030+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:03.569030+0000","last_undegraded":"2024-09-12T00:56:03.569030+0000","last_fullsized":"2024-09-12T00:56:03.569030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.976038+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:03.976038+0000","last_peered":"2024-09-12T00:56:03.976038+0000","last_clean":"2024-09-12T00:56:03.976038+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.976038+0000","last_undegraded":"2024-09-12T00:56:03.976038+0000","last_fullsized":"2024-09-12T00:56:03.976038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627590+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:03.627590+0000","last_peered":"2024-09-12T00:56:03.627590+0000","last_clean":"2024-09-12T00:56:03.627590+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:03.627590+0000","last_undegraded":"2024-09-12T00:56:03.627590+0000","last_fullsized":"2024-09-12T00:56:03.627590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624390+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624390+0000","last_peered":"2024-09-12T00:56:03.624390+0000","last_clean":"2024-09-12T00:56:03.624390+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624390+0000","last_undegraded":"2024-09-12T00:56:03.624390+0000","last_fullsized":"2024-09-12T00:56:03.624390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437353+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437353+0000","last_peered":"2024-09-12T00:56:03.437353+0000","last_clean":"2024-09-12T00:56:03.437353+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437353+0000","last_undegraded":"2024-09-12T00:56:03.437353+0000","last_fullsized":"2024-09-12T00:56:03.437353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627589+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627589+0000","last_peered":"2024-09-12T00:56:03.627589+0000","last_clean":"2024-09-12T00:56:03.627589+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627589+0000","last_undegraded":"2024-09-12T00:56:03.627589+0000","last_fullsized":"2024-09-12T00:56:03.627589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345216+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345216+0000","last_peered":"2024-09-12T00:56:03.345216+0000","last_clean":"2024-09-12T00:56:03.345216+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345216+0000","last_undegraded":"2024-09-12T00:56:03.345216+0000","last_fullsized":"2024-09-12T00:56:03.345216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.975057+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:03.975057+0000","last_peered":"2024-09-12T00:56:03.975057+0000","last_clean":"2024-09-12T00:56:03.975057+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:03.975057+0000","last_undegraded":"2024-09-12T00:56:03.975057+0000","last_fullsized":"2024-09-12T00:56:03.975057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444936+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444936+0000","last_peered":"2024-09-12T00:56:03.444936+0000","last_clean":"2024-09-12T00:56:03.444936+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444936+0000","last_undegraded":"2024-09-12T00:56:03.444936+0000","last_fullsized":"2024-09-12T00:56:03.444936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'6","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970683+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:03.970683+0000","last_peered":"2024-09-12T00:56:03.970683+0000","last_clean":"2024-09-12T00:56:03.970683+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:03.970683+0000","last_undegraded":"2024-09-12T00:56:03.970683+0000","last_fullsized":"2024-09-12T00:56:03.970683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447741+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447741+0000","last_peered":"2024-09-12T00:56:03.447741+0000","last_clean":"2024-09-12T00:56:03.447741+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447741+0000","last_undegraded":"2024-09-12T00:56:03.447741+0000","last_fullsized":"2024-09-12T00:56:03.447741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434138+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434138+0000","last_peered":"2024-09-12T00:56:03.434138+0000","last_clean":"2024-09-12T00:56:03.434138+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434138+0000","last_undegraded":"2024-09-12T00:56:03.434138+0000","last_fullsized":"2024-09-12T00:56:03.434138+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447811+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447811+0000","last_peered":"2024-09-12T00:56:03.447811+0000","last_clean":"2024-09-12T00:56:03.447811+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447811+0000","last_undegraded":"2024-09-12T00:56:03.447811+0000","last_fullsized":"2024-09-12T00:56:03.447811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628996+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.628996+0000","last_peered":"2024-09-12T00:56:03.628996+0000","last_clean":"2024-09-12T00:56:03.628996+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.628996+0000","last_undegraded":"2024-09-12T00:56:03.628996+0000","last_fullsized":"2024-09-12T00:56:03.628996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345607+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345607+0000","last_peered":"2024-09-12T00:56:03.345607+0000","last_clean":"2024-09-12T00:56:03.345607+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345607+0000","last_undegraded":"2024-09-12T00:56:03.345607+0000","last_fullsized":"2024-09-12T00:56:03.345607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'51","reported_seq":114,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.574425+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:03.574425+0000","last_peered":"2024-09-12T00:56:03.574425+0000","last_clean":"2024-09-12T00:56:03.574425+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:03.574425+0000","last_undegraded":"2024-09-12T00:56:03.574425+0000","last_fullsized":"2024-09-12T00:56:03.574425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437778+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437778+0000","last_peered":"2024-09-12T00:56:03.437778+0000","last_clean":"2024-09-12T00:56:03.437778+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437778+0000","last_undegraded":"2024-09-12T00:56:03.437778+0000","last_fullsized":"2024-09-12T00:56:03.437778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624460+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624460+0000","last_peered":"2024-09-12T00:56:03.624460+0000","last_clean":"2024-09-12T00:56:03.624460+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624460+0000","last_undegraded":"2024-09-12T00:56:03.624460+0000","last_fullsized":"2024-09-12T00:56:03.624460+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386273+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386273+0000","last_peered":"2024-09-12T00:56:03.386273+0000","last_clean":"2024-09-12T00:56:03.386273+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386273+0000","last_undegraded":"2024-09-12T00:56:03.386273+0000","last_fullsized":"2024-09-12T00:56:03.386273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346761+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346761+0000","last_peered":"2024-09-12T00:56:03.346761+0000","last_clean":"2024-09-12T00:56:03.346761+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346761+0000","last_undegraded":"2024-09-12T00:56:03.346761+0000","last_fullsized":"2024-09-12T00:56:03.346761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437122+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437122+0000","last_peered":"2024-09-12T00:56:03.437122+0000","last_clean":"2024-09-12T00:56:03.437122+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437122+0000","last_undegraded":"2024-09-12T00:56:03.437122+0000","last_fullsized":"2024-09-12T00:56:03.437122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436279+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436279+0000","last_peered":"2024-09-12T00:56:03.436279+0000","last_clean":"2024-09-12T00:56:03.436279+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436279+0000","last_undegraded":"2024-09-12T00:56:03.436279+0000","last_fullsized":"2024-09-12T00:56:03.436279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350476+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.350476+0000","last_peered":"2024-09-12T00:56:03.350476+0000","last_clean":"2024-09-12T00:56:03.350476+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.350476+0000","last_undegraded":"2024-09-12T00:56:03.350476+0000","last_fullsized":"2024-09-12T00:56:03.350476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437427+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437427+0000","last_peered":"2024-09-12T00:56:03.437427+0000","last_clean":"2024-09-12T00:56:03.437427+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437427+0000","last_undegraded":"2024-09-12T00:56:03.437427+0000","last_fullsized":"2024-09-12T00:56:03.437427+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628881+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628881+0000","last_peered":"2024-09-12T00:56:03.628881+0000","last_clean":"2024-09-12T00:56:03.628881+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628881+0000","last_undegraded":"2024-09-12T00:56:03.628881+0000","last_fullsized":"2024-09-12T00:56:03.628881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386061+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386061+0000","last_peered":"2024-09-12T00:56:03.386061+0000","last_clean":"2024-09-12T00:56:03.386061+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386061+0000","last_undegraded":"2024-09-12T00:56:03.386061+0000","last_fullsized":"2024-09-12T00:56:03.386061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.970280+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:03.970280+0000","last_peered":"2024-09-12T00:56:03.970280+0000","last_clean":"2024-09-12T00:56:03.970280+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:03.970280+0000","last_undegraded":"2024-09-12T00:56:03.970280+0000","last_fullsized":"2024-09-12T00:56:03.970280+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599019,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1208,"num_read_kb":718,"num_write":370,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1797057,"data_compressed":85571,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514534,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":954368,"data_stored":847325,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":1242272,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046544,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":452829,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1176895,"data_compressed":45402,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":519151,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":443719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":454889,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1003520,"data_stored":520062,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455095,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67161,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134420,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199797,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:08.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:09.016 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:08 smithi099 ceph-mon[35654]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.118 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:08 smithi159 ceph-mon[35566]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:09.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: Updating smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:09.138 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:08 smithi016 ceph-mon[30972]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 -- 172.21.15.16:0/2830939956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 msgr2=0x7f2cf01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/2830939956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf01021c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009900 tx=0x7f2cec02a410 comp rx=0 tx=0).stop 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 -- 172.21.15.16:0/2830939956 shutdown_connections 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/2830939956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0106f10 0x7f2cf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/2830939956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0102790 0x7f2cf01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/2830939956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf01021c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 -- 172.21.15.16:0/2830939956 >> 172.21.15.16:0/2830939956 conn(0x7f2cf00fd8c0 msgr2=0x7f2cf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 -- 172.21.15.16:0/2830939956 shutdown_connections 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.562+0000 7f2cf6c19640 1 -- 172.21.15.16:0/2830939956 wait complete. 2024-09-12T00:56:09.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 Processor -- start 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 -- start start 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0102790 0x7f2cf0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0106f10 0x7f2cf0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111f70 con 0x7f2cf0101de0 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0111df0 con 0x7f2cf0106f10 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf6c19640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf01120f0 con 0x7f2cf0102790 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf49b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf49b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46204/0 (socket says 172.21.15.16:46204) 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf49b5640 1 -- 172.21.15.16:0/502711658 learned_addr learned my addr 172.21.15.16:0/502711658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:09.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2cf51b6640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0106f10 0x7f2cf0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.563+0000 7f2ce7fff640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0102790 0x7f2cf0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 -- 172.21.15.16:0/502711658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0102790 msgr2=0x7f2cf0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0102790 0x7f2cf0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 -- 172.21.15.16:0/502711658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0106f10 msgr2=0x7f2cf0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0106f10 0x7f2cf0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 -- 172.21.15.16:0/502711658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a6990 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf49b5640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2cec006010 tx=0x7f2cec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec03f030 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a6c20 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cec003ce0 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec035070 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.564+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a7150 con 0x7f2cf0101de0 2024-09-12T00:56:09.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.565+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf01021c0 con 0x7f2cf0101de0 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.566+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f2cec042030 con 0x7f2cf0101de0 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.567+0000 7f2ce5ffb640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 0x7f2cd80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.567+0000 7f2ce7fff640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 0x7f2cd80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.567+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2cec0efa90 con 0x7f2cf0101de0 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.568+0000 7f2ce7fff640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 0x7f2cd80a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0075f80 tx=0x7f2ce000b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:09.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.571+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cec0bbfd0 con 0x7f2cf0101de0 2024-09-12T00:56:09.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:09 smithi016 ceph-mon[30972]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:09.786 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:09 smithi016 ceph-mon[30972]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.786+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cf010a0f0 con 0x7f2cf0101de0 2024-09-12T00:56:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.787+0000 7f2ce5ffb640 1 -- 172.21.15.16:0/502711658 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f2cec0c02d0 con 0x7f2cf0101de0 2024-09-12T00:56:09.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:09.788 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 msgr2=0x7f2cd80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 0x7f2cd80a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0075f80 tx=0x7f2ce000b390 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 msgr2=0x7f2cf00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2cec006010 tx=0x7f2cec0042f0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 shutdown_connections 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2cd80a2e10 0x7f2cd80a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cf0106f10 0x7f2cf0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2cf0102790 0x7f2cf0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 --2- 172.21.15.16:0/502711658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0101de0 0x7f2cf00744f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 >> 172.21.15.16:0/502711658 conn(0x7f2cf00fd8c0 msgr2=0x7f2cf00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:09.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 shutdown_connections 2024-09-12T00:56:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:09.790+0000 7f2cf6c19640 1 -- 172.21.15.16:0/502711658 wait complete. 2024-09-12T00:56:10.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:09 smithi099 ceph-mon[35654]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:10.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:09 smithi099 ceph-mon[35654]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:10.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:09 smithi159 ceph-mon[35566]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:10.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:09 smithi159 ceph-mon[35566]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:10.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:56:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:10.972 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:10 smithi016 ceph-mon[30972]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:10.972 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:10 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/502711658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:11.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:10 smithi099 ceph-mon[35654]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:10 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/502711658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:10 smithi159 ceph-mon[35566]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T00:56:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:10 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/502711658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- 172.21.15.16:0/2516899200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820100e20 msgr2=0x7f1820101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 --2- 172.21.15.16:0/2516899200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820100e20 0x7f1820101200 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1810009900 tx=0x7f181002a480 comp rx=0 tx=0).stop 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- 172.21.15.16:0/2516899200 shutdown_connections 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 --2- 172.21.15.16:0/2516899200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820102ef0 0x7f182010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 --2- 172.21.15.16:0/2516899200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 0x7f18201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 --2- 172.21.15.16:0/2516899200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820100e20 0x7f1820101200 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- 172.21.15.16:0/2516899200 >> 172.21.15.16:0/2516899200 conn(0x7f18200fc7d0 msgr2=0x7f18200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- 172.21.15.16:0/2516899200 shutdown_connections 2024-09-12T00:56:11.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- 172.21.15.16:0/2516899200 wait complete. 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 Processor -- start 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.212+0000 7f18267df640 1 -- start start 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 0x7f1820199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 0x7f18201998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1820110f80 con 0x7f1820102ef0 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1820110e00 con 0x7f1820100e20 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f18267df640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1820111100 con 0x7f1820102570 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 -- 172.21.15.16:0/3563070763 learned_addr learned my addr 172.21.15.16:0/3563070763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1817fff640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 0x7f18201998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f181ffff640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 0x7f1820199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 -- 172.21.15.16:0/3563070763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 msgr2=0x7f18201998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 0x7f18201998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 -- 172.21.15.16:0/3563070763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 msgr2=0x7f1820199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 0x7f1820199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 -- 172.21.15.16:0/3563070763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18201a4950 con 0x7f1820102ef0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f181ffff640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 0x7f1820199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.213+0000 7f1824d7c640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f181800cad0 tx=0x7f181800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.214+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800ae20 con 0x7f1820102ef0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.214+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1818005020 con 0x7f1820102ef0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.214+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a4b20 con 0x7f1820102ef0 2024-09-12T00:56:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.214+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181800f6f0 con 0x7f1820102ef0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.214+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a5010 con 0x7f1820102ef0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.215+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1820101200 con 0x7f1820102ef0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.216+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f18180051d0 con 0x7f1820102ef0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.216+0000 7f181dffb640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 0x7f180c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.217+0000 7f181ffff640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 0x7f180c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.217+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f18180cf6c0 con 0x7f1820102ef0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.217+0000 7f181ffff640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 0x7f180c0a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1810005ab0 tx=0x7f1810005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.221+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1818014030 con 0x7f1820102ef0 2024-09-12T00:56:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.402+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f182006a5a0 con 0x7f1820102ef0 2024-09-12T00:56:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.404+0000 7f181dffb640 1 -- 172.21.15.16:0/3563070763 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23017 (secure 0 0 0) 0x7f182006a5a0 con 0x7f1820102ef0 2024-09-12T00:56:11.403 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:11.404 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:56:03.340006+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2952840401":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2290747564":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/90830535":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1572487203":"2024-09-13T00:56:03.339943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:56:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 msgr2=0x7f180c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 0x7f180c0a52d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1810005ab0 tx=0x7f1810005a20 comp rx=0 tx=0).stop 2024-09-12T00:56:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 msgr2=0x7f18201a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f181800cad0 tx=0x7f181800cf90 comp rx=0 tx=0).stop 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 shutdown_connections 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f180c0a2e10 0x7f180c0a52d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1820102ef0 0x7f18201a4410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1820102570 0x7f18201998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 --2- 172.21.15.16:0/3563070763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1820100e20 0x7f1820199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 >> 172.21.15.16:0/3563070763 conn(0x7f18200fc7d0 msgr2=0x7f18200fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 shutdown_connections 2024-09-12T00:56:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:11.406+0000 7f18267df640 1 -- 172.21.15.16:0/3563070763 wait complete. 2024-09-12T00:56:11.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3563070763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:12.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3563070763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:12.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3563070763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:12.128 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T00:56:12.128 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T00:56:12.139 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T00:56:12.139 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T00:56:12.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T00:56:12.140 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T00:56:12.140 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:56:12.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:56:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 -- 172.21.15.16:0/539324939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb810c0d0 msgr2=0x7f0cb810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/539324939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb810c0d0 0x7f0cb810e570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009ca0 tx=0x7f0cb002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 -- 172.21.15.16:0/539324939 shutdown_connections 2024-09-12T00:56:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/539324939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb810c0d0 0x7f0cb810e570 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/539324939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb81035c0 0x7f0cb810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/539324939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 -- 172.21.15.16:0/539324939 >> 172.21.15.16:0/539324939 conn(0x7f0cb80fc6b0 msgr2=0x7f0cb80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 -- 172.21.15.16:0/539324939 shutdown_connections 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 -- 172.21.15.16:0/539324939 wait complete. 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.260+0000 7f0cbd88c640 1 Processor -- start 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 -- start start 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb810c0d0 0x7f0cb819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8111f30 con 0x7f0cb81035c0 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8111db0 con 0x7f0cb8102ca0 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cbd88c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81120b0 con 0x7f0cb810c0d0 2024-09-12T00:56:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46246/0 (socket says 172.21.15.16:46246) 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 -- 172.21.15.16:0/772611706 learned_addr learned my addr 172.21.15.16:0/772611706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb77fe640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb810c0d0 0x7f0cb819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 -- 172.21.15.16:0/772611706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb810c0d0 msgr2=0x7f0cb819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb810c0d0 0x7f0cb819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 -- 172.21.15.16:0/772611706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 msgr2=0x7f0cb8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb67fc640 1 -- 172.21.15.16:0/772611706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb819d330 con 0x7f0cb81035c0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.261+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cb67fc640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b450 tx=0x7f0cac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac015020 con 0x7f0cb81035c0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cac011d70 con 0x7f0cb81035c0 2024-09-12T00:56:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb819d620 con 0x7f0cb81035c0 2024-09-12T00:56:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f340 con 0x7f0cb81035c0 2024-09-12T00:56:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.262+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a4eb0 con 0x7f0cb81035c0 2024-09-12T00:56:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.263+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb8103080 con 0x7f0cb81035c0 2024-09-12T00:56:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.264+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f0cac018020 con 0x7f0cb81035c0 2024-09-12T00:56:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.265+0000 7f0cbc88a640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 0x7f0ca00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.265+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 0x7f0ca00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.265+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0cac0cf170 con 0x7f0cb81035c0 2024-09-12T00:56:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.266+0000 7f0cb6ffd640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 0x7f0ca00a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800baa0 tx=0x7f0ca8005d90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.269+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cac01f020 con 0x7f0cb81035c0 2024-09-12T00:56:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.533+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f0cb81a51a0 con 0x7f0cb81035c0 2024-09-12T00:56:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.537+0000 7f0cbc88a640 1 -- 172.21.15.16:0/772611706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f0cac00d040 con 0x7f0cb81035c0 2024-09-12T00:56:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 msgr2=0x7f0ca00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 0x7f0ca00a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800baa0 tx=0x7f0ca8005d90 comp rx=0 tx=0).stop 2024-09-12T00:56:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 msgr2=0x7f0cb8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00b450 tx=0x7f0cac00b910 comp rx=0 tx=0).stop 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 shutdown_connections 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0ca00a2ec0 0x7f0ca00a5380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb810c0d0 0x7f0cb819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb81035c0 0x7f0cb8198880 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 --2- 172.21.15.16:0/772611706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb8102ca0 0x7f0cb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 >> 172.21.15.16:0/772611706 conn(0x7f0cb80fc6b0 msgr2=0x7f0cb80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 shutdown_connections 2024-09-12T00:56:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.540+0000 7f0cbd88c640 1 -- 172.21.15.16:0/772611706 wait complete. 2024-09-12T00:56:12.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 -- 172.21.15.16:0/2206151434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878101370 msgr2=0x7f787810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 --2- 172.21.15.16:0/2206151434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878101370 0x7f787810d8a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7870009cd0 tx=0x7f787002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 -- 172.21.15.16:0/2206151434 shutdown_connections 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 --2- 172.21.15.16:0/2206151434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878101370 0x7f787810d8a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 --2- 172.21.15.16:0/2206151434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 0x7f7878100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 --2- 172.21.15.16:0/2206151434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78781069e0 0x7f7878106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 -- 172.21.15.16:0/2206151434 >> 172.21.15.16:0/2206151434 conn(0x7f78780fc6f0 msgr2=0x7f78780feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 -- 172.21.15.16:0/2206151434 shutdown_connections 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.664+0000 7f787f331640 1 -- 172.21.15.16:0/2206151434 wait complete. 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 Processor -- start 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 -- start start 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 0x7f7878197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7878101370 0x7f78781977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7878110e20 con 0x7f78781069e0 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7878110ca0 con 0x7f7878101370 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787f331640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7878110fa0 con 0x7f78781009d0 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787d8ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787d8ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46272/0 (socket says 172.21.15.16:46272) 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787d8ce640 1 -- 172.21.15.16:0/3789288046 learned_addr learned my addr 172.21.15.16:0/3789288046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787d0cd640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 0x7f7878197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.665+0000 7f787c8cc640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7878101370 0x7f78781977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 -- 172.21.15.16:0/3789288046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 msgr2=0x7f7878197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 0x7f7878197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 -- 172.21.15.16:0/3789288046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7878101370 msgr2=0x7f78781977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7878101370 0x7f78781977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 -- 172.21.15.16:0/3789288046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787819c280 con 0x7f78781069e0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787d8ce640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7870005cc0 tx=0x7f7870032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870004040 con 0x7f78781069e0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78700046c0 con 0x7f78781069e0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870030e00 con 0x7f78781069e0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787819c510 con 0x7f78781069e0 2024-09-12T00:56:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.666+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787806a300 con 0x7f78781069e0 2024-09-12T00:56:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.667+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7878102110 con 0x7f78781069e0 2024-09-12T00:56:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.668+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f78700305a0 con 0x7f78781069e0 2024-09-12T00:56:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.669+0000 7f786e7fc640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 0x7f78640a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.669+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f787002e030 con 0x7f78781069e0 2024-09-12T00:56:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.669+0000 7f787d0cd640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 0x7f78640a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.670+0000 7f787d0cd640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 0x7f78640a5400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f787400bb60 tx=0x7f787400b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:12.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.672+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f787004b060 con 0x7f78781069e0 2024-09-12T00:56:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.858+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f787806a560 con 0x7f78781069e0 2024-09-12T00:56:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.862+0000 7f786e7fc640 1 -- 172.21.15.16:0/3789288046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f78700bd6c0 con 0x7f78781069e0 2024-09-12T00:56:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.864+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 msgr2=0x7f78640a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.864+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 0x7f78640a5400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f787400bb60 tx=0x7f787400b390 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.864+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 msgr2=0x7f787819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.864+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7870005cc0 tx=0x7f7870032f40 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 shutdown_connections 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f78640a2f40 0x7f78640a5400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781069e0 0x7f787819bb50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7878101370 0x7f78781977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 --2- 172.21.15.16:0/3789288046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78781009d0 0x7f7878197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 >> 172.21.15.16:0/3789288046 conn(0x7f78780fc6f0 msgr2=0x7f78780fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 shutdown_connections 2024-09-12T00:56:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.865+0000 7f787f331640 1 -- 172.21.15.16:0/3789288046 wait complete. 2024-09-12T00:56:12.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-12T00:56:12.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/772611706' entity='client.admin' 2024-09-12T00:56:12.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:12.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:12.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:12.876 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:12 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:12.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 -- 172.21.15.16:0/2223632498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86201009b0 msgr2=0x7f8620100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 --2- 172.21.15.16:0/2223632498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86201009b0 0x7f8620100e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8614009930 tx=0x7f861402a420 comp rx=0 tx=0).stop 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 -- 172.21.15.16:0/2223632498 shutdown_connections 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 --2- 172.21.15.16:0/2223632498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8620101350 0x7f862010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 --2- 172.21.15.16:0/2223632498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86201009b0 0x7f8620100e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 --2- 172.21.15.16:0/2223632498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201069c0 0x7f8620106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 -- 172.21.15.16:0/2223632498 >> 172.21.15.16:0/2223632498 conn(0x7f8620077990 msgr2=0x7f86200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 -- 172.21.15.16:0/2223632498 shutdown_connections 2024-09-12T00:56:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7f86260e6640 1 -- 172.21.15.16:0/2223632498 wait complete. 2024-09-12T00:56:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 Processor -- start 2024-09-12T00:56:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 -- start start 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 0x7f862019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110e50 con 0x7f8620101350 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110cd0 con 0x7f86201069c0 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f86260e6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8620110fd0 con 0x7f86201009b0 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7f861effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:57296/0 (socket says 172.21.15.16:57296) 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46300/0 (socket says 172.21.15.16:46300) 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861f7fe640 1 -- 172.21.15.16:0/3255933257 learned_addr learned my addr 172.21.15.16:0/3255933257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861ffff640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 0x7f862019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 -- 172.21.15.16:0/3255933257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 msgr2=0x7f8620198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 -- 172.21.15.16:0/3255933257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 msgr2=0x7f862019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 0x7f862019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 -- 172.21.15.16:0/3255933257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862019d380 con 0x7f8620101350 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861f7fe640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861ffff640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 0x7f862019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7f861effd640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f861402a940 tx=0x7f8614003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861403f070 con 0x7f8620101350 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8614031c90 con 0x7f8620101350 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862019d610 con 0x7f8620101350 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8614035030 con 0x7f8620101350 2024-09-12T00:56:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86201a4f00 con 0x7f8620101350 2024-09-12T00:56:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.000+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8620100e10 con 0x7f8620101350 2024-09-12T00:56:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.002+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f8614042070 con 0x7f8620101350 2024-09-12T00:56:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7f861cff9640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 0x7f86080a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7f861f7fe640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 0x7f86080a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f86140f0440 con 0x7f8620101350 2024-09-12T00:56:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7f861f7fe640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 0x7f86080a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8610011490 tx=0x7f8610005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.006+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86140bc8c0 con 0x7f8620101350 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/772611706' entity='client.admin' 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:12 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1907639970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102570 msgr2=0x7fd7981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1907639970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102570 0x7fd7981029b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd784009900 tx=0x7fd78402a480 comp rx=0 tx=0).stop 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1907639970 shutdown_connections 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1907639970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798102ef0 0x7fd79810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1907639970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102570 0x7fd7981029b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1907639970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798100e20 0x7fd798101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1907639970 >> 172.21.15.16:0/1907639970 conn(0x7fd7980fc7d0 msgr2=0x7fd7980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1907639970 shutdown_connections 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.996+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1907639970 wait complete. 2024-09-12T00:56:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 Processor -- start 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 -- start start 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798100e20 0x7fd798199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102ef0 0x7fd7981a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd798110fd0 con 0x7fd798102ef0 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd798110e50 con 0x7fd798102570 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.997+0000 7fd79ecb2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd798111150 con 0x7fd798100e20 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd79d24f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102ef0 0x7fd7981a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:52458/0 (socket says 172.21.15.16:52458) 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 -- 172.21.15.16:0/1305498912 learned_addr learned my addr 172.21.15.16:0/1305498912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798100e20 0x7fd798199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 -- 172.21.15.16:0/1305498912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798100e20 msgr2=0x7fd798199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798100e20 0x7fd798199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 -- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102ef0 msgr2=0x7fd7981a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102ef0 0x7fd7981a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 -- 172.21.15.16:0/1305498912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7981a48e0 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd797fff640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd78402aa70 tx=0x7fd784003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd795ffb640 1 -- 172.21.15.16:0/1305498912 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78403f070 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd795ffb640 1 -- 172.21.15.16:0/1305498912 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd784031c30 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.998+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7981a4ab0 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7fd795ffb640 1 -- 172.21.15.16:0/1305498912 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784035030 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:12.999+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7981a4f00 con 0x7fd798102570 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.002+0000 7fd7a108b740 1 Processor -- start 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.002+0000 7fd7a108b740 1 -- start start 2024-09-12T00:56:13.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f5ceaf280 con 0x556f5cf596c0 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556f5ce56b10 con 0x556f5cf58da0 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd7a108b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556f5ceab240 con 0x556f5cf53d60 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:52464/0 (socket says 172.21.15.16:52464) 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79ca4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 -- 172.21.15.16:0/2885322397 learned_addr learned my addr 172.21.15.16:0/2885322397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd797fff640 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd795ffb640 1 -- 172.21.15.16:0/1305498912 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fd784042070 con 0x7fd798102570 2024-09-12T00:56:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 -- 172.21.15.16:0/2885322397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 msgr2=0x556f5cf541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 -- 172.21.15.16:0/2885322397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 msgr2=0x556f5cf536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79d24f640 1 -- 172.21.15.16:0/2885322397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5ce631f0 con 0x556f5cf58da0 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.003+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7fd79d24f640 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd790009f50 tx=0x7fd7900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c8e953ef7b3165 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7fd782ffd640 1 -- 172.21.15.16:0/2885322397 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790004d20 con 0x556f5cf58da0 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7fd782ffd640 1 -- 172.21.15.16:0/2885322397 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd790004050 con 0x556f5cf58da0 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.004+0000 7fd782ffd640 1 -- 172.21.15.16:0/2885322397 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790005240 con 0x556f5cf58da0 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd795ffb640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 0x7fd7880a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 -- 172.21.15.16:0/2885322397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 msgr2=0x556f5cf59180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd790009f50 tx=0x7fd7900118d0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 0x7fd7880a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd795ffb640 1 -- 172.21.15.16:0/1305498912 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd7840efcc0 con 0x7fd798102570 2024-09-12T00:56:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 -- 172.21.15.16:0/2885322397 shutdown_connections 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2885322397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf59180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 -- 172.21.15.16:0/2885322397 >> 172.21.15.16:0/2885322397 conn(0x556f5cf45ba0 msgr2=0x556f5cf45f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.005+0000 7fd7a108b740 1 -- 172.21.15.16:0/2885322397 shutdown_connections 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.006+0000 7fd7a108b740 1 -- 172.21.15.16:0/2885322397 wait complete. 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.006+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 0x7fd7880a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x556f5cf472b0 tx=0x7fd78c005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.006+0000 7fd7a108b740 1 -- start start 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.006+0000 7fd7a108b740 1 Processor -- start 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf53d60 0x556f5cf5cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf596c0 0x556f5cf5d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556f5cf600e0 con 0x556f5cf58da0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556f5ce69f10 con 0x556f5cf53d60 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556f5cf60260 con 0x556f5cf596c0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd7a108b740 10 client.0 _ll_register_callbacks cb 0x556f5ce32a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46326/0 (socket says 172.21.15.16:46326) 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 -- 172.21.15.16:0/4054660963 learned_addr learned my addr 172.21.15.16:0/4054660963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf53d60 0x556f5cf5cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd79d24f640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf596c0 0x556f5cf5d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 -- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf596c0 msgr2=0x556f5cf5d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf596c0 0x556f5cf5d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 -- 172.21.15.16:0/4054660963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf53d60 msgr2=0x556f5cf5cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf53d60 0x556f5cf5cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.007+0000 7fd797fff640 1 -- 172.21.15.16:0/4054660963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5cc206a0 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd797fff640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4573ec8c658f524e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd777fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784049070 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd790004050 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd7840c4810 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7840b82d0 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd7a108b740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f5cf6bda0 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.008+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd790004050 con 0x556f5cf58da0 2024-09-12T00:56:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7840b6ba0 con 0x556f5cf58da0 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 20 client.15372 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cf6d140 0x556f5cf6f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 4 client.15372 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f5cebc7f0 con 0x556f5cf6d140 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.009+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cf6d140 0x556f5cf6f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.010+0000 7fd777fff640 10 client.15372 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.010+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cf6d140 0x556f5cf6f590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcad0ff49934d124 server_cookie=5d6b9cb5124d9364 in_seq=0 out_seq=0 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.010+0000 7fd777fff640 1 -- 172.21.15.16:0/4054660963 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f5cebc7f0 con 0x556f5cf6d140 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.010+0000 7fd777fff640 10 client.15372 handle_command_reply: tid=0 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.010+0000 7fd7a108b740 1 client.15372 shutdown 2024-09-12T00:56:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 msgr2=0x556f5cf5d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd784006010 tx=0x7fd784009260 comp rx=0 tx=0).stop 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 shutdown_connections 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cf6d140 0x556f5cf6f590 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf596c0 0x556f5cf5d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd777fff640 0 client.15372 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf58da0 0x556f5cf5d100 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 --2- 172.21.15.16:0/4054660963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf53d60 0x556f5cf5cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 >> 172.21.15.16:0/4054660963 conn(0x556f5cf45ba0 msgr2=0x556f5cf45270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 shutdown_connections 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 1 -- 172.21.15.16:0/4054660963 wait complete. 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.011+0000 7fd7a108b740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.028+0000 7fd7a108b740 1 Processor -- start 2024-09-12T00:56:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 -- start start 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf58da0 0x556f5cf585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 0x556f5cf45160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981ab4f0 con 0x556f5cf53d60 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981ad030 con 0x556f5cf596c0 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd7a108b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981abbc0 con 0x556f5cf58da0 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46338/0 (socket says 172.21.15.16:46338) 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 -- 172.21.15.16:0/85976804 learned_addr learned my addr 172.21.15.16:0/85976804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd797fff640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf58da0 0x556f5cf585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79d24f640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 0x556f5cf45160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 -- 172.21.15.16:0/85976804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf58da0 msgr2=0x556f5cf585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf58da0 0x556f5cf585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 -- 172.21.15.16:0/85976804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 msgr2=0x556f5cf45160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 0x556f5cf45160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79ca4e640 1 -- 172.21.15.16:0/85976804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5cf5bbb0 con 0x556f5cf53d60 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.029+0000 7fd79d24f640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 0x556f5cf45160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.030+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x556f5cf6fe40 tx=0x7fd78c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e29119adc1bcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.030+0000 7fd7827fc640 1 -- 172.21.15.16:0/85976804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c031070 con 0x556f5cf53d60 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.030+0000 7fd7827fc640 1 -- 172.21.15.16:0/85976804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c02c430 con 0x556f5cf53d60 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.030+0000 7fd7827fc640 1 -- 172.21.15.16:0/85976804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c021650 con 0x556f5cf53d60 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- 172.21.15.16:0/85976804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 msgr2=0x556f5cf536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x556f5cf6fe40 tx=0x7fd78c017000 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- 172.21.15.16:0/85976804 shutdown_connections 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf596c0 0x556f5cf45160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf58da0 0x556f5cf585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 --2- 172.21.15.16:0/85976804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf53d60 0x556f5cf536f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- 172.21.15.16:0/85976804 >> 172.21.15.16:0/85976804 conn(0x556f5cf45ba0 msgr2=0x556f5cf44a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- 172.21.15.16:0/85976804 shutdown_connections 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- 172.21.15.16:0/85976804 wait complete. 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 -- start start 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.031+0000 7fd7a108b740 1 Processor -- start 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981ac960 con 0x556f5cf596c0 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981ac290 con 0x556f5cf58da0 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7981a94d0 con 0x556f5cf53d60 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd79ca4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd7a108b740 10 client.0 _ll_register_callbacks cb 0x7fd7840ed2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd79ca4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:57344/0 (socket says 172.21.15.16:57344) 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd79ca4e640 1 -- 172.21.15.16:0/2526532941 learned_addr learned my addr 172.21.15.16:0/2526532941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd79d24f640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 -- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 msgr2=0x556f5cf656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 -- 172.21.15.16:0/2526532941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 msgr2=0x556f5cf624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd797fff640 1 -- 172.21.15.16:0/2526532941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f5cf62c00 con 0x556f5cf58da0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.032+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd797fff640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf65c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7840bbfc0 tx=0x7fd7840effd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1f6795e71665f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd777fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784049040 con 0x556f5cf58da0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd7840f32a0 con 0x556f5cf58da0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c02c430 con 0x556f5cf58da0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd784005d50 con 0x556f5cf58da0 2024-09-12T00:56:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd7a108b740 10 client.25001 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f5cfd88c0 con 0x556f5cf58da0 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.033+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c02c430 con 0x556f5cf58da0 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7840edb30 con 0x556f5cf58da0 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 10 client.25001 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 10 client.25001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 20 client.25001 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cfd8e80 0x556f5cfdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 4 client.25001 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f5ceb6d00 con 0x556f5cfd8e80 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.034+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cfd8e80 0x556f5cfdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.035+0000 7fd777fff640 10 client.25001 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:13.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.035+0000 7fd79ca4e640 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cfd8e80 0x556f5cfdb2a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdddb331f4cbc61c server_cookie=5b466513e813d801 in_seq=0 out_seq=0 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd777fff640 1 -- 172.21.15.16:0/2526532941 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f5ceb6d00 con 0x556f5cfd8e80 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd777fff640 10 client.25001 handle_command_reply: tid=0 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 client.25001 shutdown 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 msgr2=0x556f5cf65c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf65c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd7840bbfc0 tx=0x7fd7840effd0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 shutdown_connections 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556f5cfd8e80 0x556f5cfdb2a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f5cf596c0 0x556f5cf624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556f5cf58da0 0x556f5cf65c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 --2- 172.21.15.16:0/2526532941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f5cf53d60 0x556f5cf656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 >> 172.21.15.16:0/2526532941 conn(0x556f5cf45ba0 msgr2=0x556f5cf45270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd777fff640 0 client.25001 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 shutdown_connections 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 1 -- 172.21.15.16:0/2526532941 wait complete. 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.036+0000 7fd7a108b740 20 client.25001 trim_cache size 0 max 16384 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "9bb20896-b580-44d9-ad59-a2dd4c9fe6f5", 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:56:13.052 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 msgr2=0x7fd7880a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 0x7fd7880a5350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x556f5cf472b0 tx=0x7fd78c005f90 comp rx=0 tx=0).stop 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 msgr2=0x7fd798199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd78402aa70 tx=0x7fd784003c10 comp rx=0 tx=0).stop 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 shutdown_connections 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7880a2e90 0x7fd7880a5350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd798102ef0 0x7fd7981a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd798102570 0x7fd798199860 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 --2- 172.21.15.16:0/1305498912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd798100e20 0x7fd798199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 >> 172.21.15.16:0/1305498912 conn(0x7fd7980fc7d0 msgr2=0x7fd798106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 shutdown_connections 2024-09-12T00:56:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.037+0000 7fd79ecb2640 1 -- 172.21.15.16:0/1305498912 wait complete. 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/772611706' entity='client.admin' 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:12 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.190+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f86201996e0 con 0x7f8620101350 2024-09-12T00:56:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.193+0000 7f861cff9640 1 -- 172.21.15.16:0/3255933257 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f86140c0bc0 con 0x7f8620101350 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 msgr2=0x7f86080a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 0x7f86080a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8610011490 tx=0x7f8610005f90 comp rx=0 tx=0).stop 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 msgr2=0x7f86201988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f861402a940 tx=0x7f8614003b20 comp rx=0 tx=0).stop 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 shutdown_connections 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f86080a2f40 0x7f86080a5400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86201069c0 0x7f862019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.195+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8620101350 0x7f86201988d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.196+0000 7f86260e6640 1 --2- 172.21.15.16:0/3255933257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86201009b0 0x7f8620198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.196+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 >> 172.21.15.16:0/3255933257 conn(0x7f8620077990 msgr2=0x7f86200699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.196+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 shutdown_connections 2024-09-12T00:56:13.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.196+0000 7f86260e6640 1 -- 172.21.15.16:0/3255933257 wait complete. 2024-09-12T00:56:13.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T00:56:13.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 -- 172.21.15.16:0/690217491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2168101350 msgr2=0x7f216810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 --2- 172.21.15.16:0/690217491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2168101350 0x7f216810d880 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2160009cd0 tx=0x7f216002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 -- 172.21.15.16:0/690217491 shutdown_connections 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 --2- 172.21.15.16:0/690217491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2168101350 0x7f216810d880 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 --2- 172.21.15.16:0/690217491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21681009b0 0x7f2168100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 --2- 172.21.15.16:0/690217491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681069c0 0x7f2168106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 -- 172.21.15.16:0/690217491 >> 172.21.15.16:0/690217491 conn(0x7f2168077990 msgr2=0x7f21680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 -- 172.21.15.16:0/690217491 shutdown_connections 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.335+0000 7f216ef0a640 1 -- 172.21.15.16:0/690217491 wait complete. 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 Processor -- start 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 -- start start 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681009b0 0x7f21680fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2168101350 0x7f21680fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2168110e00 con 0x7f21681069c0 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2168110c80 con 0x7f21681009b0 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216ef0a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2168110f80 con 0x7f2168101350 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46358/0 (socket says 172.21.15.16:46358) 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 -- 172.21.15.16:0/344653526 learned_addr learned my addr 172.21.15.16:0/344653526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216cca6640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681009b0 0x7f21680fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f2167fff640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2168101350 0x7f21680fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 -- 172.21.15.16:0/344653526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2168101350 msgr2=0x7f21680fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2168101350 0x7f21680fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 -- 172.21.15.16:0/344653526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681009b0 msgr2=0x7f21680fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681009b0 0x7f21680fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.336+0000 7f216d4a7640 1 -- 172.21.15.16:0/344653526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21681a6890 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f216d4a7640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2160005cc0 tx=0x7f2160003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160004040 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21681a6b20 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21600046c0 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160046480 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.337+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a7070 con 0x7f21681069c0 2024-09-12T00:56:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.338+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2168100e10 con 0x7f21681069c0 2024-09-12T00:56:13.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.339+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f21600305a0 con 0x7f21681069c0 2024-09-12T00:56:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.340+0000 7f2165ffb640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 0x7f21500a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.340+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f216002e030 con 0x7f21681069c0 2024-09-12T00:56:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.340+0000 7f216cca6640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 0x7f21500a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.341+0000 7f216cca6640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 0x7f21500a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f215800bb60 tx=0x7f215800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.343+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f216004b060 con 0x7f21681069c0 2024-09-12T00:56:13.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.549+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2168109fc0 con 0x7f21500a2ec0 2024-09-12T00:56:13.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.614+0000 7f2165ffb640 1 -- 172.21.15.16:0/344653526 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f215800b7b0 con 0x7f21500a2ec0 2024-09-12T00:56:13.614 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-09-12T00:56:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 msgr2=0x7f21500a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 0x7f21500a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f215800bb60 tx=0x7f215800b390 comp rx=0 tx=0).stop 2024-09-12T00:56:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 msgr2=0x7f21680fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2160005cc0 tx=0x7f2160003970 comp rx=0 tx=0).stop 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 shutdown_connections 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21500a2ec0 0x7f21500a5380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681069c0 0x7f21680fd860 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2168101350 0x7f21680fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.616+0000 7f216ef0a640 1 --2- 172.21.15.16:0/344653526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21681009b0 0x7f21680fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.617+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 >> 172.21.15.16:0/344653526 conn(0x7f2168077990 msgr2=0x7f2168069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.617+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 shutdown_connections 2024-09-12T00:56:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.617+0000 7f216ef0a640 1 -- 172.21.15.16:0/344653526 wait complete. 2024-09-12T00:56:13.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T00:56:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- 172.21.15.16:0/2646346748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8410c0d0 msgr2=0x7fbb8410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/2646346748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8410c0d0 0x7fbb8410e550 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009900 tx=0x7fbb6c02a480 comp rx=0 tx=0).stop 2024-09-12T00:56:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- 172.21.15.16:0/2646346748 shutdown_connections 2024-09-12T00:56:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/2646346748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8410c0d0 0x7fbb8410e550 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/2646346748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb841035a0 0x7fbb8410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/2646346748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 0x7fbb84103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- 172.21.15.16:0/2646346748 >> 172.21.15.16:0/2646346748 conn(0x7fbb84077990 msgr2=0x7fbb840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- 172.21.15.16:0/2646346748 shutdown_connections 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- 172.21.15.16:0/2646346748 wait complete. 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 Processor -- start 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.735+0000 7fbb8aab7640 1 -- start start 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 0x7fbb84198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8410c0d0 0x7fbb8419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111e70 con 0x7fbb841035a0 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111cf0 con 0x7fbb8410c0d0 2024-09-12T00:56:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb8aab7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84111ff0 con 0x7fbb84102c80 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46368/0 (socket says 172.21.15.16:46368) 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 -- 172.21.15.16:0/3974261785 learned_addr learned my addr 172.21.15.16:0/3974261785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb89054640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8410c0d0 0x7fbb8419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb88853640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 0x7fbb84198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 -- 172.21.15.16:0/3974261785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 msgr2=0x7fbb84198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 0x7fbb84198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 -- 172.21.15.16:0/3974261785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8410c0d0 msgr2=0x7fbb8419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8410c0d0 0x7fbb8419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 -- 172.21.15.16:0/3974261785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8419d270 con 0x7fbb841035a0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.736+0000 7fbb83fff640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c940 tx=0x7fbb7400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.737+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74004220 con 0x7fbb841035a0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.737+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb740043c0 con 0x7fbb841035a0 2024-09-12T00:56:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.737+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8419d560 con 0x7fbb841035a0 2024-09-12T00:56:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.737+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb74004a00 con 0x7fbb841035a0 2024-09-12T00:56:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.737+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841a4df0 con 0x7fbb841035a0 2024-09-12T00:56:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.738+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84103060 con 0x7fbb841035a0 2024-09-12T00:56:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.739+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fbb740079b0 con 0x7fbb841035a0 2024-09-12T00:56:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.739+0000 7fbb81ffb640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 0x7fbb700a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.740+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fbb740cf610 con 0x7fbb841035a0 2024-09-12T00:56:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.740+0000 7fbb88853640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 0x7fbb700a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.740+0000 7fbb88853640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 0x7fbb700a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007350 tx=0x7fbb7c0072e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.744+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb74014030 con 0x7fbb841035a0 2024-09-12T00:56:13.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:56:13.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:13.945+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fbb840fd420 con 0x7fbb700a2ec0 2024-09-12T00:56:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.006+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3974261785 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fbb840fd420 con 0x7fbb700a2ec0 2024-09-12T00:56:14.006 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 msgr2=0x7fbb700a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 0x7fbb700a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c007350 tx=0x7fbb7c0072e0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 msgr2=0x7fbb841987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400c940 tx=0x7fbb7400ce00 comp rx=0 tx=0).stop 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 shutdown_connections 2024-09-12T00:56:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb700a2ec0 0x7fbb700a5380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8410c0d0 0x7fbb8419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb841035a0 0x7fbb841987c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 --2- 172.21.15.16:0/3974261785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb84102c80 0x7fbb84198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.008+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 >> 172.21.15.16:0/3974261785 conn(0x7fbb84077990 msgr2=0x7fbb840fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.009+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 shutdown_connections 2024-09-12T00:56:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.009+0000 7fbb8aab7640 1 -- 172.21.15.16:0/3974261785 wait complete. 2024-09-12T00:56:14.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3789288046' entity='client.admin' 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3255933257' entity='client.admin' 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/3202339439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01035c0 msgr2=0x7f2eb010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/3202339439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01035c0 0x7f2eb010baa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea002a410 comp rx=0 tx=0).stop 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/3202339439 shutdown_connections 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/3202339439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb010c0d0 0x7f2eb010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/3202339439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb01035c0 0x7f2eb010baa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/3202339439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/3202339439 >> 172.21.15.16:0/3202339439 conn(0x7f2eb00fc6b0 msgr2=0x7f2eb00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/3202339439 shutdown_connections 2024-09-12T00:56:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.137+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/3202339439 wait complete. 2024-09-12T00:56:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 Processor -- start 2024-09-12T00:56:14.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 -- start start 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb01035c0 0x7f2eb01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0111ed0 con 0x7f2eb010c0d0 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0111d50 con 0x7f2eb01035c0 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb7ffd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0112050 con 0x7f2eb0102ca0 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb659a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb659a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46396/0 (socket says 172.21.15.16:46396) 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.138+0000 7f2eb659a640 1 -- 172.21.15.16:0/2302631417 learned_addr learned my addr 172.21.15.16:0/2302631417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb5d99640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb5598640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb01035c0 0x7f2eb01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 -- 172.21.15.16:0/2302631417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 msgr2=0x7f2eb0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 -- 172.21.15.16:0/2302631417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb01035c0 msgr2=0x7f2eb01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb01035c0 0x7f2eb01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 -- 172.21.15.16:0/2302631417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb019d350 con 0x7f2eb010c0d0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb5d99640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb659a640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800eb10 tx=0x7f2ea800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004b20 con 0x7f2eb010c0d0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea8004cc0 con 0x7f2eb010c0d0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8010760 con 0x7f2eb010c0d0 2024-09-12T00:56:14.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb019d640 con 0x7f2eb010c0d0 2024-09-12T00:56:14.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.139+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a4ed0 con 0x7f2eb010c0d0 2024-09-12T00:56:14.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.140+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb0103080 con 0x7f2eb010c0d0 2024-09-12T00:56:14.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.142+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f2ea8010900 con 0x7f2eb010c0d0 2024-09-12T00:56:14.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.142+0000 7f2ea6ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 0x7f2e9c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.142+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2ea80cff20 con 0x7f2eb010c0d0 2024-09-12T00:56:14.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.143+0000 7f2eb5d99640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 0x7f2e9c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.143+0000 7f2eb5d99640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 0x7f2e9c0a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0058c0 tx=0x7f2eac005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3789288046' entity='client.admin' 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:56:14.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3255933257' entity='client.admin' 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.146+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ea80541b0 con 0x7f2eb010c0d0 2024-09-12T00:56:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3789288046' entity='client.admin' 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3255933257' entity='client.admin' 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:56:14.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:56:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.347+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2eb0062630 con 0x7f2e9c0a2ec0 2024-09-12T00:56:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.354+0000 7f2ea6ffd640 1 -- 172.21.15.16:0/2302631417 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2eb0062630 con 0x7f2e9c0a2ec0 2024-09-12T00:56:14.353 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T00:56:13", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 msgr2=0x7f2e9c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 0x7f2e9c0a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0058c0 tx=0x7f2eac005850 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 msgr2=0x7f2eb019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800eb10 tx=0x7f2ea800c2d0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 shutdown_connections 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e9c0a2ec0 0x7f2e9c0a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2eb010c0d0 0x7f2eb019cc20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2eb01035c0 0x7f2eb01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 --2- 172.21.15.16:0/2302631417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2eb0102ca0 0x7f2eb0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 >> 172.21.15.16:0/2302631417 conn(0x7f2eb00fc6b0 msgr2=0x7f2eb00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 shutdown_connections 2024-09-12T00:56:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.356+0000 7f2eb7ffd640 1 -- 172.21.15.16:0/2302631417 wait complete. 2024-09-12T00:56:14.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 -- 172.21.15.16:0/2172223779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214103600 msgr2=0x7f021410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 --2- 172.21.15.16:0/2172223779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214103600 0x7f021410bae0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0208009900 tx=0x7f020802a480 comp rx=0 tx=0).stop 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 -- 172.21.15.16:0/2172223779 shutdown_connections 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 --2- 172.21.15.16:0/2172223779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 0x7f021410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 --2- 172.21.15.16:0/2172223779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214103600 0x7f021410bae0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 --2- 172.21.15.16:0/2172223779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214102ce0 0x7f02141030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 -- 172.21.15.16:0/2172223779 >> 172.21.15.16:0/2172223779 conn(0x7f0214077990 msgr2=0x7f02140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 -- 172.21.15.16:0/2172223779 shutdown_connections 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.475+0000 7f0218d31640 1 -- 172.21.15.16:0/2172223779 wait complete. 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 Processor -- start 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 -- start start 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214103600 0x7f02141988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 0x7f021419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0214111f00 con 0x7f0214102ce0 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0214111d80 con 0x7f021410c0b0 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0218d31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0214112080 con 0x7f0214103600 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f021259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f021259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46412/0 (socket says 172.21.15.16:46412) 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f021259c640 1 -- 172.21.15.16:0/4031633671 learned_addr learned my addr 172.21.15.16:0/4031633671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0211d9b640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214103600 0x7f02141988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.476+0000 7f0212d9d640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 0x7f021419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 -- 172.21.15.16:0/4031633671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214103600 msgr2=0x7f02141988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214103600 0x7f02141988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 -- 172.21.15.16:0/4031633671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 msgr2=0x7f021419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 0x7f021419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 -- 172.21.15.16:0/4031633671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021419d380 con 0x7f0214102ce0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f021259c640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f020400d770 tx=0x7f020400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204017070 con 0x7f0214102ce0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0204005500 con 0x7f0214102ce0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021419d670 con 0x7f0214102ce0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204004e00 con 0x7f0214102ce0 2024-09-12T00:56:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.477+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02141a4f00 con 0x7f0214102ce0 2024-09-12T00:56:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.478+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02141030c0 con 0x7f0214102ce0 2024-09-12T00:56:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.479+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f0204004fa0 con 0x7f0214102ce0 2024-09-12T00:56:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.480+0000 7f02037fe640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 0x7f01fc0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.480+0000 7f0211d9b640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 0x7f01fc0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.480+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f02040cf030 con 0x7f0214102ce0 2024-09-12T00:56:14.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.481+0000 7f0211d9b640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 0x7f01fc0a5400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0208000c00 tx=0x7f0208005b50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.484+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f020409b3e0 con 0x7f0214102ce0 2024-09-12T00:56:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 -- 172.21.15.16:0/956424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b81082e0 msgr2=0x7f65b810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 --2- 172.21.15.16:0/956424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b81082e0 0x7f65b810ec50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009930 tx=0x7f65b002a420 comp rx=0 tx=0).stop 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 -- 172.21.15.16:0/956424119 shutdown_connections 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 --2- 172.21.15.16:0/956424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b81082e0 0x7f65b810ec50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 --2- 172.21.15.16:0/956424119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b8103760 0x7f65b8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 --2- 172.21.15.16:0/956424119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8102db0 0x7f65b8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 -- 172.21.15.16:0/956424119 >> 172.21.15.16:0/956424119 conn(0x7f65b80fc7d0 msgr2=0x7f65b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 -- 172.21.15.16:0/956424119 shutdown_connections 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.675+0000 7f65bda52640 1 -- 172.21.15.16:0/956424119 wait complete. 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 Processor -- start 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 -- start start 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8103760 0x7f65b8100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b81082e0 0x7f65b8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81120b0 con 0x7f65b8102db0 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8111f30 con 0x7f65b8103760 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65bda52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8112230 con 0x7f65b81082e0 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46426/0 (socket says 172.21.15.16:46426) 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.676+0000 7f65b6ffd640 1 -- 172.21.15.16:0/1465634461 learned_addr learned my addr 172.21.15.16:0/1465634461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b77fe640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b81082e0 0x7f65b8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b67fc640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8103760 0x7f65b8100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 -- 172.21.15.16:0/1465634461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b81082e0 msgr2=0x7f65b8101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b81082e0 0x7f65b8101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 -- 172.21.15.16:0/1465634461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8103760 msgr2=0x7f65b8100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8103760 0x7f65b8100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 -- 172.21.15.16:0/1465634461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b81a6a10 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ebc0 tx=0x7f65a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65bca50640 1 -- 172.21.15.16:0/1465634461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8010630 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65bca50640 1 -- 172.21.15.16:0/1465634461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a8010cb0 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b81a6d00 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65bca50640 1 -- 172.21.15.16:0/1465634461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8018bd0 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.677+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a71e0 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.680+0000 7f65bfe2b740 1 Processor -- start 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.680+0000 7f65bfe2b740 1 -- start start 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.680+0000 7f65bca50640 1 -- 172.21.15.16:0/1465634461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f65a80107d0 con 0x7f65b8102db0 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a79140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5630099d42e0 con 0x563009a79680 2024-09-12T00:56:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56300997bce0 con 0x563009a6e4a0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5630099d02a0 con 0x563009a6edc0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46438/0 (socket says 172.21.15.16:46438) 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b67fc640 1 -- 172.21.15.16:0/2416078445 learned_addr learned my addr 172.21.15.16:0/2416078445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a79140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 -- 172.21.15.16:0/2416078445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 msgr2=0x563009a79140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a79140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 -- 172.21.15.16:0/2416078445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 msgr2=0x563009a7ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b77fe640 1 -- 172.21.15.16:0/2416078445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563009960440 con 0x563009a6e4a0 2024-09-12T00:56:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b67fc640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.681+0000 7f65bca50640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 0x7f65a00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 0x7f65a00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f65b77fe640 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a6e880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98eb8e36bddfa94d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f65bca50640 1 -- 172.21.15.16:0/1465634461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f65a80cfda0 con 0x7f65b8102db0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f659dffb640 1 -- 172.21.15.16:0/2416078445 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0035030 con 0x563009a6e4a0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f659dffb640 1 -- 172.21.15.16:0/2416078445 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0040070 con 0x563009a6e4a0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.682+0000 7f659dffb640 1 -- 172.21.15.16:0/2416078445 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0031840 con 0x563009a6e4a0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- 172.21.15.16:0/2416078445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 msgr2=0x563009a6e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65b6ffd640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 0x7f65a00a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65a80cd400 tx=0x7f65a800cf90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a6e880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b0005dc0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- 172.21.15.16:0/2416078445 shutdown_connections 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/2416078445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a6e880 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- 172.21.15.16:0/2416078445 >> 172.21.15.16:0/2416078445 conn(0x563009a6aa30 msgr2=0x563009a6ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- 172.21.15.16:0/2416078445 shutdown_connections 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- 172.21.15.16:0/2416078445 wait complete. 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 -- start start 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.683+0000 7f65bfe2b740 1 Processor -- start 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a79030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563009a7c250 con 0x563009a79680 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56300998f050 con 0x563009a6e4a0 2024-09-12T00:56:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563009a7c3d0 con 0x563009a6edc0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65bfe2b740 10 client.0 _ll_register_callbacks cb 0x5630099cf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46448/0 (socket says 172.21.15.16:46448) 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 -- 172.21.15.16:0/3850469777 learned_addr learned my addr 172.21.15.16:0/3850469777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a79030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b67fc640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 -- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 msgr2=0x563009a762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 -- 172.21.15.16:0/3850469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 msgr2=0x563009a79030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a79030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.684+0000 7f65b77fe640 1 -- 172.21.15.16:0/3850469777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630097453c0 con 0x563009a79680 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65b77fe640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006010 tx=0x7f65b003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1424a814949b69bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0035070 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f65b00319a0 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b0040070 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65b0031ba0 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563009a87c60 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0040070 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65b0004a60 con 0x563009a79680 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 20 client.15414 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009a88fb0 0x563009a8b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 4 client.15414 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.685+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630099e1850 con 0x563009a88fb0 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.686+0000 7f65b67fc640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009a88fb0 0x563009a8b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.686+0000 7f65b67fc640 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009a88fb0 0x563009a8b3a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8693a98c76a57a4 server_cookie=32483d5b804845d3 in_seq=0 out_seq=0 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.686+0000 7f6596ffd640 10 client.15414 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f6596ffd640 1 -- 172.21.15.16:0/3850469777 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630099e1850 con 0x563009a88fb0 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f6596ffd640 10 client.15414 handle_command_reply: tid=0 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 client.15414 shutdown 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 msgr2=0x563009a76880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f65b0006010 tx=0x7f65b003e040 comp rx=0 tx=0).stop 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 shutdown_connections 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009a88fb0 0x563009a8b3a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a79680 0x563009a76880 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/3850469777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a6e4a0 0x563009a79030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 >> 172.21.15.16:0/3850469777 conn(0x563009a6aa30 msgr2=0x563009a70710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f6596ffd640 0 client.15414 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 shutdown_connections 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 1 -- 172.21.15.16:0/3850469777 wait complete. 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.687+0000 7f65bfe2b740 20 client.15414 trim_cache size 0 max 16384 2024-09-12T00:56:14.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.693+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0214062630 con 0x7f01fc0a2f40 2024-09-12T00:56:14.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.700+0000 7f02037fe640 1 -- 172.21.15.16:0/4031633671 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0214062630 con 0x7f01fc0a2f40 2024-09-12T00:56:14.699 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 msgr2=0x7f01fc0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 0x7f01fc0a5400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0208000c00 tx=0x7f0208005b50 comp rx=0 tx=0).stop 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 msgr2=0x7f0214198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f020400d770 tx=0x7f020400dc30 comp rx=0 tx=0).stop 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 shutdown_connections 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01fc0a2f40 0x7f01fc0a5400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f021410c0b0 0x7f021419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0214103600 0x7f02141988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 --2- 172.21.15.16:0/4031633671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0214102ce0 0x7f0214198390 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 >> 172.21.15.16:0/4031633671 conn(0x7f0214077990 msgr2=0x7f02140fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 shutdown_connections 2024-09-12T00:56:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.702+0000 7f0218d31640 1 -- 172.21.15.16:0/4031633671 wait complete. 2024-09-12T00:56:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 Processor -- start 2024-09-12T00:56:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 -- start start 2024-09-12T00:56:14.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a73aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a69370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81afa10 con 0x563009a6e4a0 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81b1550 con 0x563009a79680 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81b00e0 con 0x563009a6edc0 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46456/0 (socket says 172.21.15.16:46456) 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.706+0000 7f65b77fe640 1 -- 172.21.15.16:0/1363089171 learned_addr learned my addr 172.21.15.16:0/1363089171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b67fc640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a73aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a69370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 -- 172.21.15.16:0/1363089171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 msgr2=0x563009a73aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a73aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 -- 172.21.15.16:0/1363089171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 msgr2=0x563009a69370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a69370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 -- 172.21.15.16:0/1363089171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563009a6e2d0 con 0x563009a6e4a0 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b67fc640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a73aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f65b77fe640 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001eb0 tx=0x7f65b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbd2de5dd2ec546 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f659d7fa640 1 -- 172.21.15.16:0/1363089171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0035040 con 0x563009a6e4a0 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f659d7fa640 1 -- 172.21.15.16:0/1363089171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0040070 con 0x563009a6e4a0 2024-09-12T00:56:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.707+0000 7f659d7fa640 1 -- 172.21.15.16:0/1363089171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0003ac0 con 0x563009a6e4a0 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 -- 172.21.15.16:0/1363089171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 msgr2=0x563009a73560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001eb0 tx=0x7f65b002a420 comp rx=0 tx=0).stop 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 -- 172.21.15.16:0/1363089171 shutdown_connections 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a69370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6edc0 0x563009a73aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/1363089171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6e4a0 0x563009a73560 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 -- 172.21.15.16:0/1363089171 >> 172.21.15.16:0/1363089171 conn(0x563009a6aa30 msgr2=0x563009a753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 -- 172.21.15.16:0/1363089171 shutdown_connections 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.708+0000 7f65bfe2b740 1 -- 172.21.15.16:0/1363089171 wait complete. 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 -- start start 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 Processor -- start 2024-09-12T00:56:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6e4a0 0x563009a8e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a8c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81b0e80 con 0x563009a6edc0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81b07b0 con 0x563009a79680 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81ad9f0 con 0x563009a6e4a0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65bfe2b740 10 client.0 _ll_register_callbacks cb 0x7f65ac009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65b67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46470/0 (socket says 172.21.15.16:46470) 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65b67fc640 1 -- 172.21.15.16:0/4080081299 learned_addr learned my addr 172.21.15.16:0/4080081299 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.709+0000 7f65b77fe640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6e4a0 0x563009a8e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b77fe640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 -- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6e4a0 msgr2=0x563009a8e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6e4a0 0x563009a8e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 -- 172.21.15.16:0/4080081299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 msgr2=0x563009a8c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a8c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 -- 172.21.15.16:0/4080081299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563009a8c930 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65b67fc640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x563009a75b70 tx=0x7f65ac014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b68810c86406988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f6596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0142d0 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f65ac004050 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b0040070 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65ac004250 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65bfe2b740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.710+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563009af3a30 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0040070 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65ac012450 con 0x563009a6edc0 2024-09-12T00:56:14.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 20 client.15420 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009af41b0 0x563009af65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 4 client.15420 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630099dbd60 con 0x563009af41b0 2024-09-12T00:56:14.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.711+0000 7f65b77fe640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009af41b0 0x563009af65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65b77fe640 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009af41b0 0x563009af65d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b4a04bac40b1fa server_cookie=5b93fe9934118968 in_seq=0 out_seq=0 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f6596ffd640 10 client.15420 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f6596ffd640 1 -- 172.21.15.16:0/4080081299 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630099dbd60 con 0x563009af41b0 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f6596ffd640 10 client.15420 handle_command_reply: tid=0 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 client.15420 shutdown 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 msgr2=0x563009a8bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x563009a75b70 tx=0x7f65ac014840 comp rx=0 tx=0).stop 2024-09-12T00:56:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 shutdown_connections 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563009af41b0 0x563009af65d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f6596ffd640 0 client.15420 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563009a79680 0x563009a8c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563009a6edc0 0x563009a8bb60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 --2- 172.21.15.16:0/4080081299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563009a6e4a0 0x563009a8e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 >> 172.21.15.16:0/4080081299 conn(0x563009a6aa30 msgr2=0x563009a69350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 shutdown_connections 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.712+0000 7f65bfe2b740 1 -- 172.21.15.16:0/4080081299 wait complete. 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bfe2b740 20 client.15420 trim_cache size 0 max 16384 2024-09-12T00:56:14.716 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "9bb20896-b580-44d9-ad59-a2dd4c9fe6f5": { 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "9bb20896-b580-44d9-ad59-a2dd4c9fe6f5", 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:14.717 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 msgr2=0x7f65a00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 0x7f65a00a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f65a80cd400 tx=0x7f65a800cf90 comp rx=0 tx=0).stop 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 msgr2=0x7f65b8100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ebc0 tx=0x7f65a800c6a0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 shutdown_connections 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f65a00a2e10 0x7f65a00a52d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65b81082e0 0x7f65b8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65b8103760 0x7f65b8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 --2- 172.21.15.16:0/1465634461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65b8102db0 0x7f65b8100a10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:14.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 >> 172.21.15.16:0/1465634461 conn(0x7f65b80fc7d0 msgr2=0x7f65b8105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:14.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 shutdown_connections 2024-09-12T00:56:14.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:14.713+0000 7f65bda52640 1 -- 172.21.15.16:0/1465634461 wait complete. 2024-09-12T00:56:14.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T00:56:14.749 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T00:56:14.759 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T00:56:14.802 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T00:56:14.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:56:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:14 smithi159 ceph-mon[35566]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-12T00:56:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:14 smithi159 ceph-mon[35566]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:14 smithi159 ceph-mon[35566]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:14 smithi016 ceph-mon[30972]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-12T00:56:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:14 smithi016 ceph-mon[30972]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:14 smithi016 ceph-mon[30972]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.162 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:14 smithi099 ceph-mon[35654]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-12T00:56:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:14 smithi099 ceph-mon[35654]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:14 smithi099 ceph-mon[35654]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:15.641 INFO:tasks.cephfs.filesystem:scrub status for tag:9bb20896-b580-44d9-ad59-a2dd4c9fe6f5 - {'path': '/', 'tag': '9bb20896-b580-44d9-ad59-a2dd4c9fe6f5', 'options': 'recursive,force'} 2024-09-12T00:56:16.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.086+0000 7f7c5e87e640 1 -- 172.21.15.16:0/3605813730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 msgr2=0x7f7c58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:16.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.086+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/3605813730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c58106e60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009930 tx=0x7f7c4802a420 comp rx=0 tx=0).stop 2024-09-12T00:56:16.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 -- 172.21.15.16:0/3605813730 shutdown_connections 2024-09-12T00:56:16.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/3605813730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 0x7f7c5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/3605813730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 0x7f7c58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/3605813730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c58106e60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 -- 172.21.15.16:0/3605813730 >> 172.21.15.16:0/3605813730 conn(0x7f7c580fc7a0 msgr2=0x7f7c580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 -- 172.21.15.16:0/3605813730 shutdown_connections 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 -- 172.21.15.16:0/3605813730 wait complete. 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 Processor -- start 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.087+0000 7f7c5e87e640 1 -- start start 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 0x7f7c58199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 0x7f7c58199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58110ed0 con 0x7f7c58106a80 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58110d50 con 0x7f7c58101420 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5e87e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58111050 con 0x7f7c58100a80 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:16.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46482/0 (socket says 172.21.15.16:46482) 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 -- 172.21.15.16:0/449904833 learned_addr learned my addr 172.21.15.16:0/449904833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c57fff640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 0x7f7c58199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c577fe640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 0x7f7c58199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 -- 172.21.15.16:0/449904833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 msgr2=0x7f7c58199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 0x7f7c58199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 -- 172.21.15.16:0/449904833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 msgr2=0x7f7c58199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 0x7f7c58199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 -- 172.21.15.16:0/449904833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581a4800 con 0x7f7c58106a80 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c5ce1b640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cce0 tx=0x7f7c5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.088+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000f450 con 0x7f7c58106a80 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.089+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c5000fad0 con 0x7f7c58106a80 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.089+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c581a49d0 con 0x7f7c58106a80 2024-09-12T00:56:16.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.089+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c50018c10 con 0x7f7c58106a80 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.089+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581a4ec0 con 0x7f7c58106a80 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.090+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c58100ee0 con 0x7f7c58106a80 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.091+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f7c5000f5f0 con 0x7f7c58106a80 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.091+0000 7f7c557fa640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 0x7f7c400a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.092+0000 7f7c57fff640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 0x7f7c400a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.092+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7c500cfc50 con 0x7f7c58106a80 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.092+0000 7f7c57fff640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 0x7f7c400a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006010 tx=0x7f7c48000bc0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.095+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c500d5020 con 0x7f7c58106a80 2024-09-12T00:56:16.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:15 smithi016 ceph-mon[30972]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:15 smithi016 ceph-mon[30972]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.267+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c580fd530 con 0x7f7c400a2e10 2024-09-12T00:56:16.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.273+0000 7f7c557fa640 1 -- 172.21.15.16:0/449904833 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258097 (secure 0 0 0) 0x7f7c580fd530 con 0x7f7c400a2e10 2024-09-12T00:56:16.276 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 msgr2=0x7f7c400a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 0x7f7c400a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006010 tx=0x7f7c48000bc0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 msgr2=0x7f7c581a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cce0 tx=0x7f7c5000a620 comp rx=0 tx=0).stop 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 shutdown_connections 2024-09-12T00:56:16.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c400a2e10 0x7f7c400a52d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c58106a80 0x7f7c581a42c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c58101420 0x7f7c58199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 --2- 172.21.15.16:0/449904833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c58100a80 0x7f7c58199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 >> 172.21.15.16:0/449904833 conn(0x7f7c580fc7a0 msgr2=0x7f7c580fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 shutdown_connections 2024-09-12T00:56:16.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:16.277+0000 7f7c5e87e640 1 -- 172.21.15.16:0/449904833 wait complete. 2024-09-12T00:56:16.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:15 smithi099 ceph-mon[35654]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.280 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:15 smithi099 ceph-mon[35654]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:15 smithi159 ceph-mon[35566]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:15 smithi159 ceph-mon[35566]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T00:56:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:16 smithi016 ceph-mon[30972]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-12T00:56:16.996 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-12T00:56:15.359971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391090,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1793,"num_read_kb":1327,"num_write":704,"num_write_kb":5324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377580,"kb_used_data":11656,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564244,"statfs":{"total":1151940427776,"available":1151553785856,"internally_reserved":0,"allocated":11935744,"data_stored":9597150,"data_compressed":256411,"data_compressed_allocated":331776,"data_compressed_original":4026368,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":490,"num_write":151,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.989502"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436504+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436504+0000","last_peered":"2024-09-12T00:56:03.436504+0000","last_clean":"2024-09-12T00:56:03.436504+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436504+0000","last_undegraded":"2024-09-12T00:56:03.436504+0000","last_fullsized":"2024-09-12T00:56:03.436504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386269+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386269+0000","last_peered":"2024-09-12T00:56:03.386269+0000","last_clean":"2024-09-12T00:56:03.386269+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386269+0000","last_undegraded":"2024-09-12T00:56:03.386269+0000","last_fullsized":"2024-09-12T00:56:03.386269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386258+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386258+0000","last_peered":"2024-09-12T00:56:03.386258+0000","last_clean":"2024-09-12T00:56:03.386258+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386258+0000","last_undegraded":"2024-09-12T00:56:03.386258+0000","last_fullsized":"2024-09-12T00:56:03.386258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624567+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624567+0000","last_peered":"2024-09-12T00:56:03.624567+0000","last_clean":"2024-09-12T00:56:03.624567+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624567+0000","last_undegraded":"2024-09-12T00:56:03.624567+0000","last_fullsized":"2024-09-12T00:56:03.624567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436902+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436902+0000","last_peered":"2024-09-12T00:56:03.436902+0000","last_clean":"2024-09-12T00:56:03.436902+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436902+0000","last_undegraded":"2024-09-12T00:56:03.436902+0000","last_fullsized":"2024-09-12T00:56:03.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437706+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437706+0000","last_peered":"2024-09-12T00:56:03.437706+0000","last_clean":"2024-09-12T00:56:03.437706+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437706+0000","last_undegraded":"2024-09-12T00:56:03.437706+0000","last_fullsized":"2024-09-12T00:56:03.437706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436327+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436327+0000","last_peered":"2024-09-12T00:56:03.436327+0000","last_clean":"2024-09-12T00:56:03.436327+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436327+0000","last_undegraded":"2024-09-12T00:56:03.436327+0000","last_fullsized":"2024-09-12T00:56:03.436327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345565+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345565+0000","last_peered":"2024-09-12T00:56:03.345565+0000","last_clean":"2024-09-12T00:56:03.345565+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345565+0000","last_undegraded":"2024-09-12T00:56:03.345565+0000","last_fullsized":"2024-09-12T00:56:03.345565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386083+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386083+0000","last_peered":"2024-09-12T00:56:03.386083+0000","last_clean":"2024-09-12T00:56:03.386083+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386083+0000","last_undegraded":"2024-09-12T00:56:03.386083+0000","last_fullsized":"2024-09-12T00:56:03.386083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350417+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350417+0000","last_peered":"2024-09-12T00:56:03.350417+0000","last_clean":"2024-09-12T00:56:03.350417+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350417+0000","last_undegraded":"2024-09-12T00:56:03.350417+0000","last_fullsized":"2024-09-12T00:56:03.350417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350455+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350455+0000","last_peered":"2024-09-12T00:56:03.350455+0000","last_clean":"2024-09-12T00:56:03.350455+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350455+0000","last_undegraded":"2024-09-12T00:56:03.350455+0000","last_fullsized":"2024-09-12T00:56:03.350455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436340+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436340+0000","last_peered":"2024-09-12T00:56:03.436340+0000","last_clean":"2024-09-12T00:56:03.436340+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436340+0000","last_undegraded":"2024-09-12T00:56:03.436340+0000","last_fullsized":"2024-09-12T00:56:03.436340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":266,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.983255+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:13.983255+0000","last_peered":"2024-09-12T00:56:13.983255+0000","last_clean":"2024-09-12T00:56:13.983255+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:13.983255+0000","last_undegraded":"2024-09-12T00:56:13.983255+0000","last_fullsized":"2024-09-12T00:56:13.983255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.004424+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:14.004424+0000","last_peered":"2024-09-12T00:56:14.004424+0000","last_clean":"2024-09-12T00:56:14.004424+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:14.004424+0000","last_undegraded":"2024-09-12T00:56:14.004424+0000","last_fullsized":"2024-09-12T00:56:14.004424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":189,"num_write":40,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.986447+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:12.986447+0000","last_peered":"2024-09-12T00:56:12.986447+0000","last_clean":"2024-09-12T00:56:12.986447+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:12.986447+0000","last_undegraded":"2024-09-12T00:56:12.986447+0000","last_fullsized":"2024-09-12T00:56:12.986447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'67","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.005115+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:14.005115+0000","last_peered":"2024-09-12T00:56:14.005115+0000","last_clean":"2024-09-12T00:56:14.005115+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:14.005115+0000","last_undegraded":"2024-09-12T00:56:14.005115+0000","last_fullsized":"2024-09-12T00:56:14.005115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'9","reported_seq":158,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.987987+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:13.987987+0000","last_peered":"2024-09-12T00:56:13.987987+0000","last_clean":"2024-09-12T00:56:13.987987+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:13.987987+0000","last_undegraded":"2024-09-12T00:56:13.987987+0000","last_fullsized":"2024-09-12T00:56:13.987987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":392,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:05.443212+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:05.443212+0000","last_peered":"2024-09-12T00:56:05.443212+0000","last_clean":"2024-09-12T00:56:05.443212+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:05.443212+0000","last_undegraded":"2024-09-12T00:56:05.443212+0000","last_fullsized":"2024-09-12T00:56:05.443212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624442+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624442+0000","last_peered":"2024-09-12T00:56:03.624442+0000","last_clean":"2024-09-12T00:56:03.624442+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624442+0000","last_undegraded":"2024-09-12T00:56:03.624442+0000","last_fullsized":"2024-09-12T00:56:03.624442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437402+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437402+0000","last_peered":"2024-09-12T00:56:03.437402+0000","last_clean":"2024-09-12T00:56:03.437402+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437402+0000","last_undegraded":"2024-09-12T00:56:03.437402+0000","last_fullsized":"2024-09-12T00:56:03.437402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627654+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627654+0000","last_peered":"2024-09-12T00:56:03.627654+0000","last_clean":"2024-09-12T00:56:03.627654+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627654+0000","last_undegraded":"2024-09-12T00:56:03.627654+0000","last_fullsized":"2024-09-12T00:56:03.627654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345288+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345288+0000","last_peered":"2024-09-12T00:56:03.345288+0000","last_clean":"2024-09-12T00:56:03.345288+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345288+0000","last_undegraded":"2024-09-12T00:56:03.345288+0000","last_fullsized":"2024-09-12T00:56:03.345288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.985939+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:12.985939+0000","last_peered":"2024-09-12T00:56:12.985939+0000","last_clean":"2024-09-12T00:56:12.985939+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:12.985939+0000","last_undegraded":"2024-09-12T00:56:12.985939+0000","last_fullsized":"2024-09-12T00:56:12.985939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444998+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444998+0000","last_peered":"2024-09-12T00:56:03.444998+0000","last_clean":"2024-09-12T00:56:03.444998+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444998+0000","last_undegraded":"2024-09-12T00:56:03.444998+0000","last_fullsized":"2024-09-12T00:56:03.444998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447787+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447787+0000","last_peered":"2024-09-12T00:56:03.447787+0000","last_clean":"2024-09-12T00:56:03.447787+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447787+0000","last_undegraded":"2024-09-12T00:56:03.447787+0000","last_fullsized":"2024-09-12T00:56:03.447787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434188+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434188+0000","last_peered":"2024-09-12T00:56:03.434188+0000","last_clean":"2024-09-12T00:56:03.434188+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434188+0000","last_undegraded":"2024-09-12T00:56:03.434188+0000","last_fullsized":"2024-09-12T00:56:03.434188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447861+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447861+0000","last_peered":"2024-09-12T00:56:03.447861+0000","last_clean":"2024-09-12T00:56:03.447861+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447861+0000","last_undegraded":"2024-09-12T00:56:03.447861+0000","last_fullsized":"2024-09-12T00:56:03.447861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.629022+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.629022+0000","last_peered":"2024-09-12T00:56:03.629022+0000","last_clean":"2024-09-12T00:56:03.629022+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.629022+0000","last_undegraded":"2024-09-12T00:56:03.629022+0000","last_fullsized":"2024-09-12T00:56:03.629022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345653+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345653+0000","last_peered":"2024-09-12T00:56:03.345653+0000","last_clean":"2024-09-12T00:56:03.345653+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345653+0000","last_undegraded":"2024-09-12T00:56:03.345653+0000","last_fullsized":"2024-09-12T00:56:03.345653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'103","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.353712+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:14.353712+0000","last_peered":"2024-09-12T00:56:14.353712+0000","last_clean":"2024-09-12T00:56:14.353712+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:14.353712+0000","last_undegraded":"2024-09-12T00:56:14.353712+0000","last_fullsized":"2024-09-12T00:56:14.353712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437822+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437822+0000","last_peered":"2024-09-12T00:56:03.437822+0000","last_clean":"2024-09-12T00:56:03.437822+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437822+0000","last_undegraded":"2024-09-12T00:56:03.437822+0000","last_fullsized":"2024-09-12T00:56:03.437822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624508+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624508+0000","last_peered":"2024-09-12T00:56:03.624508+0000","last_clean":"2024-09-12T00:56:03.624508+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624508+0000","last_undegraded":"2024-09-12T00:56:03.624508+0000","last_fullsized":"2024-09-12T00:56:03.624508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386330+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386330+0000","last_peered":"2024-09-12T00:56:03.386330+0000","last_clean":"2024-09-12T00:56:03.386330+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386330+0000","last_undegraded":"2024-09-12T00:56:03.386330+0000","last_fullsized":"2024-09-12T00:56:03.386330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346808+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346808+0000","last_peered":"2024-09-12T00:56:03.346808+0000","last_clean":"2024-09-12T00:56:03.346808+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346808+0000","last_undegraded":"2024-09-12T00:56:03.346808+0000","last_fullsized":"2024-09-12T00:56:03.346808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437175+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437175+0000","last_peered":"2024-09-12T00:56:03.437175+0000","last_clean":"2024-09-12T00:56:03.437175+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437175+0000","last_undegraded":"2024-09-12T00:56:03.437175+0000","last_fullsized":"2024-09-12T00:56:03.437175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436354+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436354+0000","last_peered":"2024-09-12T00:56:03.436354+0000","last_clean":"2024-09-12T00:56:03.436354+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436354+0000","last_undegraded":"2024-09-12T00:56:03.436354+0000","last_fullsized":"2024-09-12T00:56:03.436354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.825998+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.825998+0000","last_peered":"2024-09-12T00:56:03.825998+0000","last_clean":"2024-09-12T00:56:03.825998+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.825998+0000","last_undegraded":"2024-09-12T00:56:03.825998+0000","last_fullsized":"2024-09-12T00:56:03.825998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437494+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437494+0000","last_peered":"2024-09-12T00:56:03.437494+0000","last_clean":"2024-09-12T00:56:03.437494+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437494+0000","last_undegraded":"2024-09-12T00:56:03.437494+0000","last_fullsized":"2024-09-12T00:56:03.437494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'13","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628935+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628935+0000","last_peered":"2024-09-12T00:56:03.628935+0000","last_clean":"2024-09-12T00:56:03.628935+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628935+0000","last_undegraded":"2024-09-12T00:56:03.628935+0000","last_fullsized":"2024-09-12T00:56:03.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386112+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386112+0000","last_peered":"2024-09-12T00:56:03.386112+0000","last_clean":"2024-09-12T00:56:03.386112+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386112+0000","last_undegraded":"2024-09-12T00:56:03.386112+0000","last_fullsized":"2024-09-12T00:56:03.386112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":159,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.037214+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:13.037214+0000","last_peered":"2024-09-12T00:56:13.037214+0000","last_clean":"2024-09-12T00:56:13.037214+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:13.037214+0000","last_undegraded":"2024-09-12T00:56:13.037214+0000","last_fullsized":"2024-09-12T00:56:13.037214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800488,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1556,"num_read_kb":997,"num_write":501,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2401464,"data_compressed":113132,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514536,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":847718,"data_compressed":22221,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1442955,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046937,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":453615,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1373527,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":523202,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":1174102,"data_compressed":37423,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":455282,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":524506,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455488,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34489,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":71212,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331052,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400480,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395728,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:17.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:56:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:17.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:16 smithi099 ceph-mon[35654]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-12T00:56:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:16 smithi159 ceph-mon[35566]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 54 op/s 2024-09-12T00:56:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 -- 172.21.15.16:0/3071931862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8101420 msgr2=0x7f3af810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 --2- 172.21.15.16:0/3071931862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8101420 0x7f3af810d950 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009cd0 tx=0x7f3af002a8e0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 -- 172.21.15.16:0/3071931862 shutdown_connections 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 --2- 172.21.15.16:0/3071931862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8101420 0x7f3af810d950 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 --2- 172.21.15.16:0/3071931862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8100a80 0x7f3af8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 --2- 172.21.15.16:0/3071931862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8106a80 0x7f3af8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 -- 172.21.15.16:0/3071931862 >> 172.21.15.16:0/3071931862 conn(0x7f3af80fc7a0 msgr2=0x7f3af80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 -- 172.21.15.16:0/3071931862 shutdown_connections 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.772+0000 7f3afcacc640 1 -- 172.21.15.16:0/3071931862 wait complete. 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 Processor -- start 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 -- start start 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:17.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8101420 0x7f3af81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8106a80 0x7f3af81a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8110e80 con 0x7f3af8100a80 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8110d00 con 0x7f3af8106a80 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3afcacc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3af8111000 con 0x7f3af8101420 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3af659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3af659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46494/0 (socket says 172.21.15.16:46494) 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3af659c640 1 -- 172.21.15.16:0/943942825 learned_addr learned my addr 172.21.15.16:0/943942825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3aedd9b640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8101420 0x7f3af81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:17.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.773+0000 7f3af6d9d640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8106a80 0x7f3af81a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 -- 172.21.15.16:0/943942825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8101420 msgr2=0x7f3af81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8101420 0x7f3af81996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 -- 172.21.15.16:0/943942825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8106a80 msgr2=0x7f3af81a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8106a80 0x7f3af81a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 -- 172.21.15.16:0/943942825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af81a4760 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3af659c640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800d950 tx=0x7f3ae800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8017070 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af81a4930 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae800de80 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800f620 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.774+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af81a4ec0 con 0x7f3af8100a80 2024-09-12T00:56:17.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.775+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af8100ee0 con 0x7f3af8100a80 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.776+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f3ae8009c30 con 0x7f3af8100a80 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.777+0000 7f3aeffff640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 0x7f3ae40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.777+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f3ae80cf200 con 0x7f3af8100a80 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.777+0000 7f3aedd9b640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 0x7f3ae40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.778+0000 7f3aedd9b640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 0x7f3ae40a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3af806b320 tx=0x7f3ae000a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:17.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.780+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ae809bc00 con 0x7f3af8100a80 2024-09-12T00:56:17.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.953+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3af80fd5c0 con 0x7f3ae40a2e10 2024-09-12T00:56:17.953 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:17 smithi016 ceph-mon[30972]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:17.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.960+0000 7f3aeffff640 1 -- 172.21.15.16:0/943942825 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258098 (secure 0 0 0) 0x7f3af80fd5c0 con 0x7f3ae40a2e10 2024-09-12T00:56:17.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 msgr2=0x7f3ae40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 0x7f3ae40a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3af806b320 tx=0x7f3ae000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 msgr2=0x7f3af81991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800d950 tx=0x7f3ae800dd50 comp rx=0 tx=0).stop 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 shutdown_connections 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2e10 0x7f3ae40a52d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3af8106a80 0x7f3af81a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3af8101420 0x7f3af81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 --2- 172.21.15.16:0/943942825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af8100a80 0x7f3af81991a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.962+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 >> 172.21.15.16:0/943942825 conn(0x7f3af80fc7a0 msgr2=0x7f3af80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.963+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 shutdown_connections 2024-09-12T00:56:17.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:17.963+0000 7f3afcacc640 1 -- 172.21.15.16:0/943942825 wait complete. 2024-09-12T00:56:18.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:17 smithi099 ceph-mon[35654]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:17 smithi159 ceph-mon[35566]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:18.633 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-12T00:56:17.360437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391090,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1793,"num_read_kb":1327,"num_write":704,"num_write_kb":5324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377580,"kb_used_data":11656,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564244,"statfs":{"total":1151940427776,"available":1151553785856,"internally_reserved":0,"allocated":11935744,"data_stored":9597150,"data_compressed":256411,"data_compressed_allocated":331776,"data_compressed_original":4026368,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":490,"num_write":151,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002523"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436504+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436504+0000","last_peered":"2024-09-12T00:56:03.436504+0000","last_clean":"2024-09-12T00:56:03.436504+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436504+0000","last_undegraded":"2024-09-12T00:56:03.436504+0000","last_fullsized":"2024-09-12T00:56:03.436504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386269+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386269+0000","last_peered":"2024-09-12T00:56:03.386269+0000","last_clean":"2024-09-12T00:56:03.386269+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386269+0000","last_undegraded":"2024-09-12T00:56:03.386269+0000","last_fullsized":"2024-09-12T00:56:03.386269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386258+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386258+0000","last_peered":"2024-09-12T00:56:03.386258+0000","last_clean":"2024-09-12T00:56:03.386258+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386258+0000","last_undegraded":"2024-09-12T00:56:03.386258+0000","last_fullsized":"2024-09-12T00:56:03.386258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624567+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624567+0000","last_peered":"2024-09-12T00:56:03.624567+0000","last_clean":"2024-09-12T00:56:03.624567+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624567+0000","last_undegraded":"2024-09-12T00:56:03.624567+0000","last_fullsized":"2024-09-12T00:56:03.624567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436902+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436902+0000","last_peered":"2024-09-12T00:56:03.436902+0000","last_clean":"2024-09-12T00:56:03.436902+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436902+0000","last_undegraded":"2024-09-12T00:56:03.436902+0000","last_fullsized":"2024-09-12T00:56:03.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437706+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437706+0000","last_peered":"2024-09-12T00:56:03.437706+0000","last_clean":"2024-09-12T00:56:03.437706+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437706+0000","last_undegraded":"2024-09-12T00:56:03.437706+0000","last_fullsized":"2024-09-12T00:56:03.437706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436327+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436327+0000","last_peered":"2024-09-12T00:56:03.436327+0000","last_clean":"2024-09-12T00:56:03.436327+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436327+0000","last_undegraded":"2024-09-12T00:56:03.436327+0000","last_fullsized":"2024-09-12T00:56:03.436327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345565+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345565+0000","last_peered":"2024-09-12T00:56:03.345565+0000","last_clean":"2024-09-12T00:56:03.345565+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345565+0000","last_undegraded":"2024-09-12T00:56:03.345565+0000","last_fullsized":"2024-09-12T00:56:03.345565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386083+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386083+0000","last_peered":"2024-09-12T00:56:03.386083+0000","last_clean":"2024-09-12T00:56:03.386083+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386083+0000","last_undegraded":"2024-09-12T00:56:03.386083+0000","last_fullsized":"2024-09-12T00:56:03.386083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350417+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350417+0000","last_peered":"2024-09-12T00:56:03.350417+0000","last_clean":"2024-09-12T00:56:03.350417+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350417+0000","last_undegraded":"2024-09-12T00:56:03.350417+0000","last_fullsized":"2024-09-12T00:56:03.350417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350455+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350455+0000","last_peered":"2024-09-12T00:56:03.350455+0000","last_clean":"2024-09-12T00:56:03.350455+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350455+0000","last_undegraded":"2024-09-12T00:56:03.350455+0000","last_fullsized":"2024-09-12T00:56:03.350455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436340+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436340+0000","last_peered":"2024-09-12T00:56:03.436340+0000","last_clean":"2024-09-12T00:56:03.436340+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436340+0000","last_undegraded":"2024-09-12T00:56:03.436340+0000","last_fullsized":"2024-09-12T00:56:03.436340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":266,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.983255+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:13.983255+0000","last_peered":"2024-09-12T00:56:13.983255+0000","last_clean":"2024-09-12T00:56:13.983255+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:13.983255+0000","last_undegraded":"2024-09-12T00:56:13.983255+0000","last_fullsized":"2024-09-12T00:56:13.983255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.004424+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:14.004424+0000","last_peered":"2024-09-12T00:56:14.004424+0000","last_clean":"2024-09-12T00:56:14.004424+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:14.004424+0000","last_undegraded":"2024-09-12T00:56:14.004424+0000","last_fullsized":"2024-09-12T00:56:14.004424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":189,"num_write":40,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.986447+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:12.986447+0000","last_peered":"2024-09-12T00:56:12.986447+0000","last_clean":"2024-09-12T00:56:12.986447+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:12.986447+0000","last_undegraded":"2024-09-12T00:56:12.986447+0000","last_fullsized":"2024-09-12T00:56:12.986447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'67","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.005115+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:14.005115+0000","last_peered":"2024-09-12T00:56:14.005115+0000","last_clean":"2024-09-12T00:56:14.005115+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:14.005115+0000","last_undegraded":"2024-09-12T00:56:14.005115+0000","last_fullsized":"2024-09-12T00:56:14.005115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'9","reported_seq":158,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.987987+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:13.987987+0000","last_peered":"2024-09-12T00:56:13.987987+0000","last_clean":"2024-09-12T00:56:13.987987+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:13.987987+0000","last_undegraded":"2024-09-12T00:56:13.987987+0000","last_fullsized":"2024-09-12T00:56:13.987987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":392,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:05.443212+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:05.443212+0000","last_peered":"2024-09-12T00:56:05.443212+0000","last_clean":"2024-09-12T00:56:05.443212+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:05.443212+0000","last_undegraded":"2024-09-12T00:56:05.443212+0000","last_fullsized":"2024-09-12T00:56:05.443212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624442+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624442+0000","last_peered":"2024-09-12T00:56:03.624442+0000","last_clean":"2024-09-12T00:56:03.624442+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624442+0000","last_undegraded":"2024-09-12T00:56:03.624442+0000","last_fullsized":"2024-09-12T00:56:03.624442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437402+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437402+0000","last_peered":"2024-09-12T00:56:03.437402+0000","last_clean":"2024-09-12T00:56:03.437402+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437402+0000","last_undegraded":"2024-09-12T00:56:03.437402+0000","last_fullsized":"2024-09-12T00:56:03.437402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627654+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627654+0000","last_peered":"2024-09-12T00:56:03.627654+0000","last_clean":"2024-09-12T00:56:03.627654+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627654+0000","last_undegraded":"2024-09-12T00:56:03.627654+0000","last_fullsized":"2024-09-12T00:56:03.627654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345288+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345288+0000","last_peered":"2024-09-12T00:56:03.345288+0000","last_clean":"2024-09-12T00:56:03.345288+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345288+0000","last_undegraded":"2024-09-12T00:56:03.345288+0000","last_fullsized":"2024-09-12T00:56:03.345288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.985939+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:12.985939+0000","last_peered":"2024-09-12T00:56:12.985939+0000","last_clean":"2024-09-12T00:56:12.985939+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:12.985939+0000","last_undegraded":"2024-09-12T00:56:12.985939+0000","last_fullsized":"2024-09-12T00:56:12.985939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444998+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444998+0000","last_peered":"2024-09-12T00:56:03.444998+0000","last_clean":"2024-09-12T00:56:03.444998+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444998+0000","last_undegraded":"2024-09-12T00:56:03.444998+0000","last_fullsized":"2024-09-12T00:56:03.444998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447787+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447787+0000","last_peered":"2024-09-12T00:56:03.447787+0000","last_clean":"2024-09-12T00:56:03.447787+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447787+0000","last_undegraded":"2024-09-12T00:56:03.447787+0000","last_fullsized":"2024-09-12T00:56:03.447787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434188+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434188+0000","last_peered":"2024-09-12T00:56:03.434188+0000","last_clean":"2024-09-12T00:56:03.434188+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434188+0000","last_undegraded":"2024-09-12T00:56:03.434188+0000","last_fullsized":"2024-09-12T00:56:03.434188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447861+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447861+0000","last_peered":"2024-09-12T00:56:03.447861+0000","last_clean":"2024-09-12T00:56:03.447861+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447861+0000","last_undegraded":"2024-09-12T00:56:03.447861+0000","last_fullsized":"2024-09-12T00:56:03.447861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.629022+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.629022+0000","last_peered":"2024-09-12T00:56:03.629022+0000","last_clean":"2024-09-12T00:56:03.629022+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.629022+0000","last_undegraded":"2024-09-12T00:56:03.629022+0000","last_fullsized":"2024-09-12T00:56:03.629022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345653+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345653+0000","last_peered":"2024-09-12T00:56:03.345653+0000","last_clean":"2024-09-12T00:56:03.345653+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345653+0000","last_undegraded":"2024-09-12T00:56:03.345653+0000","last_fullsized":"2024-09-12T00:56:03.345653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'103","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.353712+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:14.353712+0000","last_peered":"2024-09-12T00:56:14.353712+0000","last_clean":"2024-09-12T00:56:14.353712+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:14.353712+0000","last_undegraded":"2024-09-12T00:56:14.353712+0000","last_fullsized":"2024-09-12T00:56:14.353712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437822+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437822+0000","last_peered":"2024-09-12T00:56:03.437822+0000","last_clean":"2024-09-12T00:56:03.437822+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437822+0000","last_undegraded":"2024-09-12T00:56:03.437822+0000","last_fullsized":"2024-09-12T00:56:03.437822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624508+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624508+0000","last_peered":"2024-09-12T00:56:03.624508+0000","last_clean":"2024-09-12T00:56:03.624508+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624508+0000","last_undegraded":"2024-09-12T00:56:03.624508+0000","last_fullsized":"2024-09-12T00:56:03.624508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386330+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386330+0000","last_peered":"2024-09-12T00:56:03.386330+0000","last_clean":"2024-09-12T00:56:03.386330+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386330+0000","last_undegraded":"2024-09-12T00:56:03.386330+0000","last_fullsized":"2024-09-12T00:56:03.386330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346808+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346808+0000","last_peered":"2024-09-12T00:56:03.346808+0000","last_clean":"2024-09-12T00:56:03.346808+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346808+0000","last_undegraded":"2024-09-12T00:56:03.346808+0000","last_fullsized":"2024-09-12T00:56:03.346808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437175+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437175+0000","last_peered":"2024-09-12T00:56:03.437175+0000","last_clean":"2024-09-12T00:56:03.437175+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437175+0000","last_undegraded":"2024-09-12T00:56:03.437175+0000","last_fullsized":"2024-09-12T00:56:03.437175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436354+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436354+0000","last_peered":"2024-09-12T00:56:03.436354+0000","last_clean":"2024-09-12T00:56:03.436354+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436354+0000","last_undegraded":"2024-09-12T00:56:03.436354+0000","last_fullsized":"2024-09-12T00:56:03.436354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.825998+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.825998+0000","last_peered":"2024-09-12T00:56:03.825998+0000","last_clean":"2024-09-12T00:56:03.825998+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.825998+0000","last_undegraded":"2024-09-12T00:56:03.825998+0000","last_fullsized":"2024-09-12T00:56:03.825998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437494+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437494+0000","last_peered":"2024-09-12T00:56:03.437494+0000","last_clean":"2024-09-12T00:56:03.437494+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437494+0000","last_undegraded":"2024-09-12T00:56:03.437494+0000","last_fullsized":"2024-09-12T00:56:03.437494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'13","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628935+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628935+0000","last_peered":"2024-09-12T00:56:03.628935+0000","last_clean":"2024-09-12T00:56:03.628935+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628935+0000","last_undegraded":"2024-09-12T00:56:03.628935+0000","last_fullsized":"2024-09-12T00:56:03.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386112+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386112+0000","last_peered":"2024-09-12T00:56:03.386112+0000","last_clean":"2024-09-12T00:56:03.386112+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386112+0000","last_undegraded":"2024-09-12T00:56:03.386112+0000","last_fullsized":"2024-09-12T00:56:03.386112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":159,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.037214+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:13.037214+0000","last_peered":"2024-09-12T00:56:13.037214+0000","last_clean":"2024-09-12T00:56:13.037214+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:13.037214+0000","last_undegraded":"2024-09-12T00:56:13.037214+0000","last_fullsized":"2024-09-12T00:56:13.037214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800488,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1556,"num_read_kb":997,"num_write":501,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2401464,"data_compressed":113132,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514536,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":847718,"data_compressed":22221,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1442955,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046937,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":453615,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1373527,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":523202,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":1174102,"data_compressed":37423,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":455282,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":524506,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455488,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34489,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":71212,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331052,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400480,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395728,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:18.639 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T00:56:18.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T00:56:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:19.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:18 smithi016 ceph-mon[30972]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:19.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:18 smithi016 ceph-mon[30972]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:18 smithi099 ceph-mon[35654]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:18 smithi099 ceph-mon[35654]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:18 smithi159 ceph-mon[35566]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:18 smithi159 ceph-mon[35566]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 -- 172.21.15.16:0/1828734201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 msgr2=0x7ff948100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 --2- 172.21.15.16:0/1828734201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948100ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff938009930 tx=0x7ff93802a420 comp rx=0 tx=0).stop 2024-09-12T00:56:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 -- 172.21.15.16:0/1828734201 shutdown_connections 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 --2- 172.21.15.16:0/1828734201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948101420 0x7ff94810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 --2- 172.21.15.16:0/1828734201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948100ee0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 --2- 172.21.15.16:0/1828734201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948106a80 0x7ff948106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 -- 172.21.15.16:0/1828734201 >> 172.21.15.16:0/1828734201 conn(0x7ff9480fc7a0 msgr2=0x7ff9480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 -- 172.21.15.16:0/1828734201 shutdown_connections 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.380+0000 7ff9506f1640 1 -- 172.21.15.16:0/1828734201 wait complete. 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 Processor -- start 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 -- start start 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948101420 0x7ff948198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948106a80 0x7ff94819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff948110e40 con 0x7ff948100a80 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff948110cc0 con 0x7ff948101420 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff9506f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff948110fc0 con 0x7ff948106a80 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.381+0000 7ff94e48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35324/0 (socket says 172.21.15.16:35324) 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 -- 172.21.15.16:0/3435175907 learned_addr learned my addr 172.21.15.16:0/3435175907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94dc8c640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948101420 0x7ff948198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94ec8e640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948106a80 0x7ff94819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 -- 172.21.15.16:0/3435175907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948106a80 msgr2=0x7ff94819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948106a80 0x7ff94819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 -- 172.21.15.16:0/3435175907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948101420 msgr2=0x7ff948198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948101420 0x7ff948198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 -- 172.21.15.16:0/3435175907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94819d3c0 con 0x7ff948100a80 2024-09-12T00:56:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff94e48d640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c920 tx=0x7ff94400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944007c70 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94819d6b0 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff944007e10 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944018890 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.382+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481a4f90 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.383+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff948100ee0 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.385+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7ff944018a50 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.385+0000 7ff93f7fe640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 0x7ff9340a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.386+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff9440cf990 con 0x7ff948100a80 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.386+0000 7ff94dc8c640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 0x7ff9340a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.386+0000 7ff94dc8c640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 0x7ff9340a52d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff9380058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:19.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.388+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff944014030 con 0x7ff948100a80 2024-09-12T00:56:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.564+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff948062630 con 0x7ff9340a2e10 2024-09-12T00:56:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.572+0000 7ff93f7fe640 1 -- 172.21.15.16:0/3435175907 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258098 (secure 0 0 0) 0x7ff948062630 con 0x7ff9340a2e10 2024-09-12T00:56:19.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:19.576 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-12T00:56:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 msgr2=0x7ff9340a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 0x7ff9340a52d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff9380058d0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 msgr2=0x7ff948198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c920 tx=0x7ff94400cde0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 shutdown_connections 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9340a2e10 0x7ff9340a52d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff948106a80 0x7ff94819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff948101420 0x7ff948198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 --2- 172.21.15.16:0/3435175907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff948100a80 0x7ff948198410 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 >> 172.21.15.16:0/3435175907 conn(0x7ff9480fc7a0 msgr2=0x7ff94806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 shutdown_connections 2024-09-12T00:56:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:19.575+0000 7ff9506f1640 1 -- 172.21.15.16:0/3435175907 wait complete. 2024-09-12T00:56:20.307 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-12T00:56:19.360753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391090,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1793,"num_read_kb":1327,"num_write":704,"num_write_kb":5324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1239947,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1031,"ondisk_log_size":1031,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377580,"kb_used_data":11656,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564244,"statfs":{"total":1151940427776,"available":1151553785856,"internally_reserved":0,"allocated":11935744,"data_stored":9597150,"data_compressed":256411,"data_compressed_allocated":331776,"data_compressed_original":4026368,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201469,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":490,"num_write":151,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002465"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436504+0000","last_change":"2024-09-12T00:54:03.755049+0000","last_active":"2024-09-12T00:56:03.436504+0000","last_peered":"2024-09-12T00:56:03.436504+0000","last_clean":"2024-09-12T00:56:03.436504+0000","last_became_active":"2024-09-12T00:54:03.754063+0000","last_became_peered":"2024-09-12T00:54:03.754063+0000","last_unstale":"2024-09-12T00:56:03.436504+0000","last_undegraded":"2024-09-12T00:56:03.436504+0000","last_fullsized":"2024-09-12T00:56:03.436504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:33:04.793196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386269+0000","last_change":"2024-09-12T00:54:03.762547+0000","last_active":"2024-09-12T00:56:03.386269+0000","last_peered":"2024-09-12T00:56:03.386269+0000","last_clean":"2024-09-12T00:56:03.386269+0000","last_became_active":"2024-09-12T00:54:03.761939+0000","last_became_peered":"2024-09-12T00:54:03.761939+0000","last_unstale":"2024-09-12T00:56:03.386269+0000","last_undegraded":"2024-09-12T00:56:03.386269+0000","last_fullsized":"2024-09-12T00:56:03.386269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:01.358017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386258+0000","last_change":"2024-09-12T00:54:03.755187+0000","last_active":"2024-09-12T00:56:03.386258+0000","last_peered":"2024-09-12T00:56:03.386258+0000","last_clean":"2024-09-12T00:56:03.386258+0000","last_became_active":"2024-09-12T00:54:03.754896+0000","last_became_peered":"2024-09-12T00:54:03.754896+0000","last_unstale":"2024-09-12T00:56:03.386258+0000","last_undegraded":"2024-09-12T00:56:03.386258+0000","last_fullsized":"2024-09-12T00:56:03.386258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:39.429752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624567+0000","last_change":"2024-09-12T00:54:03.766141+0000","last_active":"2024-09-12T00:56:03.624567+0000","last_peered":"2024-09-12T00:56:03.624567+0000","last_clean":"2024-09-12T00:56:03.624567+0000","last_became_active":"2024-09-12T00:54:03.765839+0000","last_became_peered":"2024-09-12T00:54:03.765839+0000","last_unstale":"2024-09-12T00:56:03.624567+0000","last_undegraded":"2024-09-12T00:56:03.624567+0000","last_fullsized":"2024-09-12T00:56:03.624567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:51.312076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628675+0000","last_change":"2024-09-12T00:54:03.747584+0000","last_active":"2024-09-12T00:56:03.628675+0000","last_peered":"2024-09-12T00:56:03.628675+0000","last_clean":"2024-09-12T00:56:03.628675+0000","last_became_active":"2024-09-12T00:54:03.747173+0000","last_became_peered":"2024-09-12T00:54:03.747173+0000","last_unstale":"2024-09-12T00:56:03.628675+0000","last_undegraded":"2024-09-12T00:56:03.628675+0000","last_fullsized":"2024-09-12T00:56:03.628675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:35.049921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447644+0000","last_change":"2024-09-12T00:54:03.759965+0000","last_active":"2024-09-12T00:56:03.447644+0000","last_peered":"2024-09-12T00:56:03.447644+0000","last_clean":"2024-09-12T00:56:03.447644+0000","last_became_active":"2024-09-12T00:54:03.759647+0000","last_became_peered":"2024-09-12T00:54:03.759647+0000","last_unstale":"2024-09-12T00:56:03.447644+0000","last_undegraded":"2024-09-12T00:56:03.447644+0000","last_fullsized":"2024-09-12T00:56:03.447644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:24.083114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627478+0000","last_change":"2024-09-12T00:54:03.758993+0000","last_active":"2024-09-12T00:56:03.627478+0000","last_peered":"2024-09-12T00:56:03.627478+0000","last_clean":"2024-09-12T00:56:03.627478+0000","last_became_active":"2024-09-12T00:54:03.758573+0000","last_became_peered":"2024-09-12T00:54:03.758573+0000","last_unstale":"2024-09-12T00:56:03.627478+0000","last_undegraded":"2024-09-12T00:56:03.627478+0000","last_fullsized":"2024-09-12T00:56:03.627478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:25.342745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.433965+0000","last_change":"2024-09-12T00:55:09.334005+0000","last_active":"2024-09-12T00:56:03.433965+0000","last_peered":"2024-09-12T00:56:03.433965+0000","last_clean":"2024-09-12T00:56:03.433965+0000","last_became_active":"2024-09-12T00:55:09.333700+0000","last_became_peered":"2024-09-12T00:55:09.333700+0000","last_unstale":"2024-09-12T00:56:03.433965+0000","last_undegraded":"2024-09-12T00:56:03.433965+0000","last_fullsized":"2024-09-12T00:56:03.433965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:22:56.839107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347153+0000","last_change":"2024-09-12T00:54:03.760235+0000","last_active":"2024-09-12T00:56:03.347153+0000","last_peered":"2024-09-12T00:56:03.347153+0000","last_clean":"2024-09-12T00:56:03.347153+0000","last_became_active":"2024-09-12T00:54:03.759939+0000","last_became_peered":"2024-09-12T00:54:03.759939+0000","last_unstale":"2024-09-12T00:56:03.347153+0000","last_undegraded":"2024-09-12T00:56:03.347153+0000","last_fullsized":"2024-09-12T00:56:03.347153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:38:10.322876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447989+0000","last_change":"2024-09-12T00:54:03.761986+0000","last_active":"2024-09-12T00:56:03.447989+0000","last_peered":"2024-09-12T00:56:03.447989+0000","last_clean":"2024-09-12T00:56:03.447989+0000","last_became_active":"2024-09-12T00:54:03.761647+0000","last_became_peered":"2024-09-12T00:54:03.761647+0000","last_unstale":"2024-09-12T00:56:03.447989+0000","last_undegraded":"2024-09-12T00:56:03.447989+0000","last_fullsized":"2024-09-12T00:56:03.447989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:58.275436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627839+0000","last_change":"2024-09-12T00:54:03.766044+0000","last_active":"2024-09-12T00:56:03.627839+0000","last_peered":"2024-09-12T00:56:03.627839+0000","last_clean":"2024-09-12T00:56:03.627839+0000","last_became_active":"2024-09-12T00:54:03.765423+0000","last_became_peered":"2024-09-12T00:54:03.765423+0000","last_unstale":"2024-09-12T00:56:03.627839+0000","last_undegraded":"2024-09-12T00:56:03.627839+0000","last_fullsized":"2024-09-12T00:56:03.627839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:47:06.813366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624593+0000","last_change":"2024-09-12T00:54:03.767531+0000","last_active":"2024-09-12T00:56:03.624593+0000","last_peered":"2024-09-12T00:56:03.624593+0000","last_clean":"2024-09-12T00:56:03.624593+0000","last_became_active":"2024-09-12T00:54:03.767233+0000","last_became_peered":"2024-09-12T00:54:03.767233+0000","last_unstale":"2024-09-12T00:56:03.624593+0000","last_undegraded":"2024-09-12T00:56:03.624593+0000","last_fullsized":"2024-09-12T00:56:03.624593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:20.047366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345479+0000","last_change":"2024-09-12T00:54:03.748779+0000","last_active":"2024-09-12T00:56:03.345479+0000","last_peered":"2024-09-12T00:56:03.345479+0000","last_clean":"2024-09-12T00:56:03.345479+0000","last_became_active":"2024-09-12T00:54:03.748454+0000","last_became_peered":"2024-09-12T00:54:03.748454+0000","last_unstale":"2024-09-12T00:56:03.345479+0000","last_undegraded":"2024-09-12T00:56:03.345479+0000","last_fullsized":"2024-09-12T00:56:03.345479+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:42.836639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385913+0000","last_change":"2024-09-12T00:54:03.752148+0000","last_active":"2024-09-12T00:56:03.385913+0000","last_peered":"2024-09-12T00:56:03.385913+0000","last_clean":"2024-09-12T00:56:03.385913+0000","last_became_active":"2024-09-12T00:54:03.751850+0000","last_became_peered":"2024-09-12T00:54:03.751850+0000","last_unstale":"2024-09-12T00:56:03.385913+0000","last_undegraded":"2024-09-12T00:56:03.385913+0000","last_fullsized":"2024-09-12T00:56:03.385913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:51.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.385752+0000","last_change":"2024-09-12T00:54:03.763613+0000","last_active":"2024-09-12T00:56:03.385752+0000","last_peered":"2024-09-12T00:56:03.385752+0000","last_clean":"2024-09-12T00:56:03.385752+0000","last_became_active":"2024-09-12T00:54:03.763259+0000","last_became_peered":"2024-09-12T00:54:03.763259+0000","last_unstale":"2024-09-12T00:56:03.385752+0000","last_undegraded":"2024-09-12T00:56:03.385752+0000","last_fullsized":"2024-09-12T00:56:03.385752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:20:30.227619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346680+0000","last_change":"2024-09-12T00:54:03.757073+0000","last_active":"2024-09-12T00:56:03.346680+0000","last_peered":"2024-09-12T00:56:03.346680+0000","last_clean":"2024-09-12T00:56:03.346680+0000","last_became_active":"2024-09-12T00:54:03.756264+0000","last_became_peered":"2024-09-12T00:54:03.756264+0000","last_unstale":"2024-09-12T00:56:03.346680+0000","last_undegraded":"2024-09-12T00:56:03.346680+0000","last_fullsized":"2024-09-12T00:56:03.346680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:17:17.719671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436519+0000","last_change":"2024-09-12T00:54:03.755345+0000","last_active":"2024-09-12T00:56:03.436519+0000","last_peered":"2024-09-12T00:56:03.436519+0000","last_clean":"2024-09-12T00:56:03.436519+0000","last_became_active":"2024-09-12T00:54:03.754574+0000","last_became_peered":"2024-09-12T00:54:03.754574+0000","last_unstale":"2024-09-12T00:56:03.436519+0000","last_undegraded":"2024-09-12T00:56:03.436519+0000","last_fullsized":"2024-09-12T00:56:03.436519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:33.788923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434346+0000","last_change":"2024-09-12T00:54:03.753472+0000","last_active":"2024-09-12T00:56:03.434346+0000","last_peered":"2024-09-12T00:56:03.434346+0000","last_clean":"2024-09-12T00:56:03.434346+0000","last_became_active":"2024-09-12T00:54:03.753233+0000","last_became_peered":"2024-09-12T00:54:03.753233+0000","last_unstale":"2024-09-12T00:56:03.434346+0000","last_undegraded":"2024-09-12T00:56:03.434346+0000","last_fullsized":"2024-09-12T00:56:03.434346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:44:06.836772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351111+0000","last_change":"2024-09-12T00:54:03.752408+0000","last_active":"2024-09-12T00:56:03.351111+0000","last_peered":"2024-09-12T00:56:03.351111+0000","last_clean":"2024-09-12T00:56:03.351111+0000","last_became_active":"2024-09-12T00:54:03.752103+0000","last_became_peered":"2024-09-12T00:54:03.752103+0000","last_unstale":"2024-09-12T00:56:03.351111+0000","last_undegraded":"2024-09-12T00:56:03.351111+0000","last_fullsized":"2024-09-12T00:56:03.351111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:35.388591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436161+0000","last_change":"2024-09-12T00:54:03.763509+0000","last_active":"2024-09-12T00:56:03.436161+0000","last_peered":"2024-09-12T00:56:03.436161+0000","last_clean":"2024-09-12T00:56:03.436161+0000","last_became_active":"2024-09-12T00:54:03.763221+0000","last_became_peered":"2024-09-12T00:54:03.763221+0000","last_unstale":"2024-09-12T00:56:03.436161+0000","last_undegraded":"2024-09-12T00:56:03.436161+0000","last_fullsized":"2024-09-12T00:56:03.436161+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:04.131269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345612+0000","last_change":"2024-09-12T00:54:03.759229+0000","last_active":"2024-09-12T00:56:03.345612+0000","last_peered":"2024-09-12T00:56:03.345612+0000","last_clean":"2024-09-12T00:56:03.345612+0000","last_became_active":"2024-09-12T00:54:03.758919+0000","last_became_peered":"2024-09-12T00:54:03.758919+0000","last_unstale":"2024-09-12T00:56:03.345612+0000","last_undegraded":"2024-09-12T00:56:03.345612+0000","last_fullsized":"2024-09-12T00:56:03.345612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:04:24.750334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346801+0000","last_change":"2024-09-12T00:54:03.762932+0000","last_active":"2024-09-12T00:56:03.346801+0000","last_peered":"2024-09-12T00:56:03.346801+0000","last_clean":"2024-09-12T00:56:03.346801+0000","last_became_active":"2024-09-12T00:54:03.762604+0000","last_became_peered":"2024-09-12T00:54:03.762604+0000","last_unstale":"2024-09-12T00:56:03.346801+0000","last_undegraded":"2024-09-12T00:56:03.346801+0000","last_fullsized":"2024-09-12T00:56:03.346801+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:47:10.663446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436902+0000","last_change":"2024-09-12T00:54:03.757002+0000","last_active":"2024-09-12T00:56:03.436902+0000","last_peered":"2024-09-12T00:56:03.436902+0000","last_clean":"2024-09-12T00:56:03.436902+0000","last_became_active":"2024-09-12T00:54:03.756744+0000","last_became_peered":"2024-09-12T00:54:03.756744+0000","last_unstale":"2024-09-12T00:56:03.436902+0000","last_undegraded":"2024-09-12T00:56:03.436902+0000","last_fullsized":"2024-09-12T00:56:03.436902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:12.867891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444896+0000","last_change":"2024-09-12T00:54:03.757828+0000","last_active":"2024-09-12T00:56:03.444896+0000","last_peered":"2024-09-12T00:56:03.444896+0000","last_clean":"2024-09-12T00:56:03.444896+0000","last_became_active":"2024-09-12T00:54:03.757158+0000","last_became_peered":"2024-09-12T00:54:03.757158+0000","last_unstale":"2024-09-12T00:56:03.444896+0000","last_undegraded":"2024-09-12T00:56:03.444896+0000","last_fullsized":"2024-09-12T00:56:03.444896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:42:07.089346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437706+0000","last_change":"2024-09-12T00:54:03.763167+0000","last_active":"2024-09-12T00:56:03.437706+0000","last_peered":"2024-09-12T00:56:03.437706+0000","last_clean":"2024-09-12T00:56:03.437706+0000","last_became_active":"2024-09-12T00:54:03.762850+0000","last_became_peered":"2024-09-12T00:54:03.762850+0000","last_unstale":"2024-09-12T00:56:03.437706+0000","last_undegraded":"2024-09-12T00:56:03.437706+0000","last_fullsized":"2024-09-12T00:56:03.437706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:05:32.550643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.351232+0000","last_change":"2024-09-12T00:54:03.755072+0000","last_active":"2024-09-12T00:56:03.351232+0000","last_peered":"2024-09-12T00:56:03.351232+0000","last_clean":"2024-09-12T00:56:03.351232+0000","last_became_active":"2024-09-12T00:54:03.754759+0000","last_became_peered":"2024-09-12T00:54:03.754759+0000","last_unstale":"2024-09-12T00:56:03.351232+0000","last_undegraded":"2024-09-12T00:56:03.351232+0000","last_fullsized":"2024-09-12T00:56:03.351232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:49:51.433425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436327+0000","last_change":"2024-09-12T00:54:03.750616+0000","last_active":"2024-09-12T00:56:03.436327+0000","last_peered":"2024-09-12T00:56:03.436327+0000","last_clean":"2024-09-12T00:56:03.436327+0000","last_became_active":"2024-09-12T00:54:03.750307+0000","last_became_peered":"2024-09-12T00:54:03.750307+0000","last_unstale":"2024-09-12T00:56:03.436327+0000","last_undegraded":"2024-09-12T00:56:03.436327+0000","last_fullsized":"2024-09-12T00:56:03.436327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:48:27.013846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345565+0000","last_change":"2024-09-12T00:54:03.761797+0000","last_active":"2024-09-12T00:56:03.345565+0000","last_peered":"2024-09-12T00:56:03.345565+0000","last_clean":"2024-09-12T00:56:03.345565+0000","last_became_active":"2024-09-12T00:54:03.761366+0000","last_became_peered":"2024-09-12T00:54:03.761366+0000","last_unstale":"2024-09-12T00:56:03.345565+0000","last_undegraded":"2024-09-12T00:56:03.345565+0000","last_fullsized":"2024-09-12T00:56:03.345565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:32.244090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386083+0000","last_change":"2024-09-12T00:54:03.761848+0000","last_active":"2024-09-12T00:56:03.386083+0000","last_peered":"2024-09-12T00:56:03.386083+0000","last_clean":"2024-09-12T00:56:03.386083+0000","last_became_active":"2024-09-12T00:54:03.761488+0000","last_became_peered":"2024-09-12T00:54:03.761488+0000","last_unstale":"2024-09-12T00:56:03.386083+0000","last_undegraded":"2024-09-12T00:56:03.386083+0000","last_fullsized":"2024-09-12T00:56:03.386083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:20.998616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350417+0000","last_change":"2024-09-12T00:54:03.753882+0000","last_active":"2024-09-12T00:56:03.350417+0000","last_peered":"2024-09-12T00:56:03.350417+0000","last_clean":"2024-09-12T00:56:03.350417+0000","last_became_active":"2024-09-12T00:54:03.753541+0000","last_became_peered":"2024-09-12T00:54:03.753541+0000","last_unstale":"2024-09-12T00:56:03.350417+0000","last_undegraded":"2024-09-12T00:56:03.350417+0000","last_fullsized":"2024-09-12T00:56:03.350417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:30:47.203504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350455+0000","last_change":"2024-09-12T00:54:03.761936+0000","last_active":"2024-09-12T00:56:03.350455+0000","last_peered":"2024-09-12T00:56:03.350455+0000","last_clean":"2024-09-12T00:56:03.350455+0000","last_became_active":"2024-09-12T00:54:03.761542+0000","last_became_peered":"2024-09-12T00:54:03.761542+0000","last_unstale":"2024-09-12T00:56:03.350455+0000","last_undegraded":"2024-09-12T00:56:03.350455+0000","last_fullsized":"2024-09-12T00:56:03.350455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:50:05.872517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436340+0000","last_change":"2024-09-12T00:54:03.757399+0000","last_active":"2024-09-12T00:56:03.436340+0000","last_peered":"2024-09-12T00:56:03.436340+0000","last_clean":"2024-09-12T00:56:03.436340+0000","last_became_active":"2024-09-12T00:54:03.756585+0000","last_became_peered":"2024-09-12T00:54:03.756585+0000","last_unstale":"2024-09-12T00:56:03.436340+0000","last_undegraded":"2024-09-12T00:56:03.436340+0000","last_fullsized":"2024-09-12T00:56:03.436340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:56.470951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628639+0000","last_change":"2024-09-12T00:54:03.752207+0000","last_active":"2024-09-12T00:56:03.628639+0000","last_peered":"2024-09-12T00:56:03.628639+0000","last_clean":"2024-09-12T00:56:03.628639+0000","last_became_active":"2024-09-12T00:54:03.751903+0000","last_became_peered":"2024-09-12T00:54:03.751903+0000","last_unstale":"2024-09-12T00:56:03.628639+0000","last_undegraded":"2024-09-12T00:56:03.628639+0000","last_fullsized":"2024-09-12T00:56:03.628639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:14.264746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":149,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.672782+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.672442+0000","last_became_peered":"2024-09-12T00:54:01.672442+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:48.973541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444971+0000","last_change":"2024-09-12T00:54:03.763268+0000","last_active":"2024-09-12T00:56:03.444971+0000","last_peered":"2024-09-12T00:56:03.444971+0000","last_clean":"2024-09-12T00:56:03.444971+0000","last_became_active":"2024-09-12T00:54:03.762360+0000","last_became_peered":"2024-09-12T00:54:03.762360+0000","last_unstale":"2024-09-12T00:56:03.444971+0000","last_undegraded":"2024-09-12T00:56:03.444971+0000","last_fullsized":"2024-09-12T00:56:03.444971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:59:27.569766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447724+0000","last_change":"2024-09-12T00:54:01.668774+0000","last_active":"2024-09-12T00:56:03.447724+0000","last_peered":"2024-09-12T00:56:03.447724+0000","last_clean":"2024-09-12T00:56:03.447724+0000","last_became_active":"2024-09-12T00:54:01.668432+0000","last_became_peered":"2024-09-12T00:54:01.668432+0000","last_unstale":"2024-09-12T00:56:03.447724+0000","last_undegraded":"2024-09-12T00:56:03.447724+0000","last_fullsized":"2024-09-12T00:56:03.447724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:39.349560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345229+0000","last_change":"2024-09-12T00:54:01.664737+0000","last_active":"2024-09-12T00:56:03.345229+0000","last_peered":"2024-09-12T00:56:03.345229+0000","last_clean":"2024-09-12T00:56:03.345229+0000","last_became_active":"2024-09-12T00:54:01.664359+0000","last_became_peered":"2024-09-12T00:54:01.664359+0000","last_unstale":"2024-09-12T00:56:03.345229+0000","last_undegraded":"2024-09-12T00:56:03.345229+0000","last_fullsized":"2024-09-12T00:56:03.345229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:58.015863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436906+0000","last_change":"2024-09-12T00:54:03.755781+0000","last_active":"2024-09-12T00:56:03.436906+0000","last_peered":"2024-09-12T00:56:03.436906+0000","last_clean":"2024-09-12T00:56:03.436906+0000","last_became_active":"2024-09-12T00:54:03.755140+0000","last_became_peered":"2024-09-12T00:54:03.755140+0000","last_unstale":"2024-09-12T00:56:03.436906+0000","last_undegraded":"2024-09-12T00:56:03.436906+0000","last_fullsized":"2024-09-12T00:56:03.436906+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:51.429312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445125+0000","last_change":"2024-09-12T00:54:01.669990+0000","last_active":"2024-09-12T00:56:03.445125+0000","last_peered":"2024-09-12T00:56:03.445125+0000","last_clean":"2024-09-12T00:56:03.445125+0000","last_became_active":"2024-09-12T00:54:01.669295+0000","last_became_peered":"2024-09-12T00:54:01.669295+0000","last_unstale":"2024-09-12T00:56:03.445125+0000","last_undegraded":"2024-09-12T00:56:03.445125+0000","last_fullsized":"2024-09-12T00:56:03.445125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:22.892655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434015+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.434015+0000","last_peered":"2024-09-12T00:56:03.434015+0000","last_clean":"2024-09-12T00:56:03.434015+0000","last_became_active":"2024-09-12T00:54:03.751712+0000","last_became_peered":"2024-09-12T00:54:03.751712+0000","last_unstale":"2024-09-12T00:56:03.434015+0000","last_undegraded":"2024-09-12T00:56:03.434015+0000","last_fullsized":"2024-09-12T00:56:03.434015+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:10.717506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'6","reported_seq":266,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.983255+0000","last_change":"2024-09-12T00:54:01.666250+0000","last_active":"2024-09-12T00:56:13.983255+0000","last_peered":"2024-09-12T00:56:13.983255+0000","last_clean":"2024-09-12T00:56:13.983255+0000","last_became_active":"2024-09-12T00:54:01.665891+0000","last_became_peered":"2024-09-12T00:54:01.665891+0000","last_unstale":"2024-09-12T00:56:13.983255+0000","last_undegraded":"2024-09-12T00:56:13.983255+0000","last_fullsized":"2024-09-12T00:56:13.983255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:57:59.672660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":236,"num_read_kb":236,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.347164+0000","last_change":"2024-09-12T00:54:03.757797+0000","last_active":"2024-09-12T00:56:03.347164+0000","last_peered":"2024-09-12T00:56:03.347164+0000","last_clean":"2024-09-12T00:56:03.347164+0000","last_became_active":"2024-09-12T00:54:03.755170+0000","last_became_peered":"2024-09-12T00:54:03.755170+0000","last_unstale":"2024-09-12T00:56:03.347164+0000","last_undegraded":"2024-09-12T00:56:03.347164+0000","last_fullsized":"2024-09-12T00:56:03.347164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:35.486731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627853+0000","last_change":"2024-09-12T00:54:01.671634+0000","last_active":"2024-09-12T00:56:03.627853+0000","last_peered":"2024-09-12T00:56:03.627853+0000","last_clean":"2024-09-12T00:56:03.627853+0000","last_became_active":"2024-09-12T00:54:01.671089+0000","last_became_peered":"2024-09-12T00:54:01.671089+0000","last_unstale":"2024-09-12T00:56:03.627853+0000","last_undegraded":"2024-09-12T00:56:03.627853+0000","last_fullsized":"2024-09-12T00:56:03.627853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:52.193667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386202+0000","last_change":"2024-09-12T00:54:03.760757+0000","last_active":"2024-09-12T00:56:03.386202+0000","last_peered":"2024-09-12T00:56:03.386202+0000","last_clean":"2024-09-12T00:56:03.386202+0000","last_became_active":"2024-09-12T00:54:03.760341+0000","last_became_peered":"2024-09-12T00:54:03.760341+0000","last_unstale":"2024-09-12T00:56:03.386202+0000","last_undegraded":"2024-09-12T00:56:03.386202+0000","last_fullsized":"2024-09-12T00:56:03.386202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:56.946503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445128+0000","last_change":"2024-09-12T00:54:01.755331+0000","last_active":"2024-09-12T00:56:03.445128+0000","last_peered":"2024-09-12T00:56:03.445128+0000","last_clean":"2024-09-12T00:56:03.445128+0000","last_became_active":"2024-09-12T00:54:01.754926+0000","last_became_peered":"2024-09-12T00:54:01.754926+0000","last_unstale":"2024-09-12T00:56:03.445128+0000","last_undegraded":"2024-09-12T00:56:03.445128+0000","last_fullsized":"2024-09-12T00:56:03.445128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:37.725559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346867+0000","last_change":"2024-09-12T00:54:03.761750+0000","last_active":"2024-09-12T00:56:03.346867+0000","last_peered":"2024-09-12T00:56:03.346867+0000","last_clean":"2024-09-12T00:56:03.346867+0000","last_became_active":"2024-09-12T00:54:03.761466+0000","last_became_peered":"2024-09-12T00:54:03.761466+0000","last_unstale":"2024-09-12T00:56:03.346867+0000","last_undegraded":"2024-09-12T00:56:03.346867+0000","last_fullsized":"2024-09-12T00:56:03.346867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:07.784903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'38","reported_seq":329,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.004424+0000","last_change":"2024-09-12T00:55:09.911319+0000","last_active":"2024-09-12T00:56:14.004424+0000","last_peered":"2024-09-12T00:56:14.004424+0000","last_clean":"2024-09-12T00:56:14.004424+0000","last_became_active":"2024-09-12T00:55:09.904622+0000","last_became_peered":"2024-09-12T00:55:09.904622+0000","last_unstale":"2024-09-12T00:56:14.004424+0000","last_undegraded":"2024-09-12T00:56:14.004424+0000","last_fullsized":"2024-09-12T00:56:14.004424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:03:00.386809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":189,"num_write":40,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628876+0000","last_change":"2024-09-12T00:54:03.759733+0000","last_active":"2024-09-12T00:56:03.628876+0000","last_peered":"2024-09-12T00:56:03.628876+0000","last_clean":"2024-09-12T00:56:03.628876+0000","last_became_active":"2024-09-12T00:54:03.759290+0000","last_became_peered":"2024-09-12T00:54:03.759290+0000","last_unstale":"2024-09-12T00:56:03.628876+0000","last_undegraded":"2024-09-12T00:56:03.628876+0000","last_fullsized":"2024-09-12T00:56:03.628876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:03:58.940600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.350956+0000","last_change":"2024-09-12T00:54:01.673411+0000","last_active":"2024-09-12T00:56:03.350956+0000","last_peered":"2024-09-12T00:56:03.350956+0000","last_clean":"2024-09-12T00:56:03.350956+0000","last_became_active":"2024-09-12T00:54:01.673077+0000","last_became_peered":"2024-09-12T00:54:01.673077+0000","last_unstale":"2024-09-12T00:56:03.350956+0000","last_undegraded":"2024-09-12T00:56:03.350956+0000","last_fullsized":"2024-09-12T00:56:03.350956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:11.236401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345287+0000","last_change":"2024-09-12T00:54:03.759617+0000","last_active":"2024-09-12T00:56:03.345287+0000","last_peered":"2024-09-12T00:56:03.345287+0000","last_clean":"2024-09-12T00:56:03.345287+0000","last_became_active":"2024-09-12T00:54:03.759025+0000","last_became_peered":"2024-09-12T00:54:03.759025+0000","last_unstale":"2024-09-12T00:56:03.345287+0000","last_undegraded":"2024-09-12T00:56:03.345287+0000","last_fullsized":"2024-09-12T00:56:03.345287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:58.590152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.445060+0000","last_change":"2024-09-12T00:54:01.669702+0000","last_active":"2024-09-12T00:56:03.445060+0000","last_peered":"2024-09-12T00:56:03.445060+0000","last_clean":"2024-09-12T00:56:03.445060+0000","last_became_active":"2024-09-12T00:54:01.669282+0000","last_became_peered":"2024-09-12T00:54:01.669282+0000","last_unstale":"2024-09-12T00:56:03.445060+0000","last_undegraded":"2024-09-12T00:56:03.445060+0000","last_fullsized":"2024-09-12T00:56:03.445060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:38:20.492431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627644+0000","last_change":"2024-09-12T00:54:03.767644+0000","last_active":"2024-09-12T00:56:03.627644+0000","last_peered":"2024-09-12T00:56:03.627644+0000","last_clean":"2024-09-12T00:56:03.627644+0000","last_became_active":"2024-09-12T00:54:03.767348+0000","last_became_peered":"2024-09-12T00:54:03.767348+0000","last_unstale":"2024-09-12T00:56:03.627644+0000","last_undegraded":"2024-09-12T00:56:03.627644+0000","last_fullsized":"2024-09-12T00:56:03.627644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:24.068345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.986447+0000","last_change":"2024-09-12T00:54:01.673865+0000","last_active":"2024-09-12T00:56:12.986447+0000","last_peered":"2024-09-12T00:56:12.986447+0000","last_clean":"2024-09-12T00:56:12.986447+0000","last_became_active":"2024-09-12T00:54:01.673431+0000","last_became_peered":"2024-09-12T00:54:01.673431+0000","last_unstale":"2024-09-12T00:56:12.986447+0000","last_undegraded":"2024-09-12T00:56:12.986447+0000","last_fullsized":"2024-09-12T00:56:12.986447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:23:11.135891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627416+0000","last_change":"2024-09-12T00:54:03.754561+0000","last_active":"2024-09-12T00:56:03.627416+0000","last_peered":"2024-09-12T00:56:03.627416+0000","last_clean":"2024-09-12T00:56:03.627416+0000","last_became_active":"2024-09-12T00:54:03.754277+0000","last_became_peered":"2024-09-12T00:54:03.754277+0000","last_unstale":"2024-09-12T00:56:03.627416+0000","last_undegraded":"2024-09-12T00:56:03.627416+0000","last_fullsized":"2024-09-12T00:56:03.627416+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:22.994692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628588+0000","last_change":"2024-09-12T00:54:01.754352+0000","last_active":"2024-09-12T00:56:03.628588+0000","last_peered":"2024-09-12T00:56:03.628588+0000","last_clean":"2024-09-12T00:56:03.628588+0000","last_became_active":"2024-09-12T00:54:01.754012+0000","last_became_peered":"2024-09-12T00:54:01.754012+0000","last_unstale":"2024-09-12T00:56:03.628588+0000","last_undegraded":"2024-09-12T00:56:03.628588+0000","last_fullsized":"2024-09-12T00:56:03.628588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:41:31.772682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444927+0000","last_change":"2024-09-12T00:55:09.335750+0000","last_active":"2024-09-12T00:56:03.444927+0000","last_peered":"2024-09-12T00:56:03.444927+0000","last_clean":"2024-09-12T00:56:03.444927+0000","last_became_active":"2024-09-12T00:55:09.335365+0000","last_became_peered":"2024-09-12T00:55:09.335365+0000","last_unstale":"2024-09-12T00:56:03.444927+0000","last_undegraded":"2024-09-12T00:56:03.444927+0000","last_fullsized":"2024-09-12T00:56:03.444927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:10:02.982310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628990+0000","last_change":"2024-09-12T00:54:01.668907+0000","last_active":"2024-09-12T00:56:03.628990+0000","last_peered":"2024-09-12T00:56:03.628990+0000","last_clean":"2024-09-12T00:56:03.628990+0000","last_became_active":"2024-09-12T00:54:01.667233+0000","last_became_peered":"2024-09-12T00:54:01.667233+0000","last_unstale":"2024-09-12T00:56:03.628990+0000","last_undegraded":"2024-09-12T00:56:03.628990+0000","last_fullsized":"2024-09-12T00:56:03.628990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:29:48.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436399+0000","last_change":"2024-09-12T00:55:09.337263+0000","last_active":"2024-09-12T00:56:03.436399+0000","last_peered":"2024-09-12T00:56:03.436399+0000","last_clean":"2024-09-12T00:56:03.436399+0000","last_became_active":"2024-09-12T00:55:09.336885+0000","last_became_peered":"2024-09-12T00:55:09.336885+0000","last_unstale":"2024-09-12T00:56:03.436399+0000","last_undegraded":"2024-09-12T00:56:03.436399+0000","last_fullsized":"2024-09-12T00:56:03.436399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:24:47.989207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345523+0000","last_change":"2024-09-12T00:54:01.669025+0000","last_active":"2024-09-12T00:56:03.345523+0000","last_peered":"2024-09-12T00:56:03.345523+0000","last_clean":"2024-09-12T00:56:03.345523+0000","last_became_active":"2024-09-12T00:54:01.668703+0000","last_became_peered":"2024-09-12T00:54:01.668703+0000","last_unstale":"2024-09-12T00:56:03.345523+0000","last_undegraded":"2024-09-12T00:56:03.345523+0000","last_fullsized":"2024-09-12T00:56:03.345523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:49.175018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627872+0000","last_change":"2024-09-12T00:54:03.758074+0000","last_active":"2024-09-12T00:56:03.627872+0000","last_peered":"2024-09-12T00:56:03.627872+0000","last_clean":"2024-09-12T00:56:03.627872+0000","last_became_active":"2024-09-12T00:54:03.755237+0000","last_became_peered":"2024-09-12T00:54:03.755237+0000","last_unstale":"2024-09-12T00:56:03.627872+0000","last_undegraded":"2024-09-12T00:56:03.627872+0000","last_fullsized":"2024-09-12T00:56:03.627872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:03:21.231299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'67","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.005115+0000","last_change":"2024-09-12T00:54:01.666817+0000","last_active":"2024-09-12T00:56:14.005115+0000","last_peered":"2024-09-12T00:56:14.005115+0000","last_clean":"2024-09-12T00:56:14.005115+0000","last_became_active":"2024-09-12T00:54:01.666384+0000","last_became_peered":"2024-09-12T00:54:01.666384+0000","last_unstale":"2024-09-12T00:56:14.005115+0000","last_undegraded":"2024-09-12T00:56:14.005115+0000","last_fullsized":"2024-09-12T00:56:14.005115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.234709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627653+0000","last_change":"2024-09-12T00:54:03.756424+0000","last_active":"2024-09-12T00:56:03.627653+0000","last_peered":"2024-09-12T00:56:03.627653+0000","last_clean":"2024-09-12T00:56:03.627653+0000","last_became_active":"2024-09-12T00:54:03.756115+0000","last_became_peered":"2024-09-12T00:54:03.756115+0000","last_unstale":"2024-09-12T00:56:03.627653+0000","last_undegraded":"2024-09-12T00:56:03.627653+0000","last_fullsized":"2024-09-12T00:56:03.627653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:49:02.656813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'9","reported_seq":158,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.987987+0000","last_change":"2024-09-12T00:54:01.906624+0000","last_active":"2024-09-12T00:56:13.987987+0000","last_peered":"2024-09-12T00:56:13.987987+0000","last_clean":"2024-09-12T00:56:13.987987+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:13.987987+0000","last_undegraded":"2024-09-12T00:56:13.987987+0000","last_fullsized":"2024-09-12T00:56:13.987987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:08.687165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627846+0000","last_change":"2024-09-12T00:54:03.765713+0000","last_active":"2024-09-12T00:56:03.627846+0000","last_peered":"2024-09-12T00:56:03.627846+0000","last_clean":"2024-09-12T00:56:03.627846+0000","last_became_active":"2024-09-12T00:54:03.765028+0000","last_became_peered":"2024-09-12T00:54:03.765028+0000","last_unstale":"2024-09-12T00:56:03.627846+0000","last_undegraded":"2024-09-12T00:56:03.627846+0000","last_fullsized":"2024-09-12T00:56:03.627846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":392,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:05.443212+0000","last_change":"2024-09-12T00:52:51.410614+0000","last_active":"2024-09-12T00:56:05.443212+0000","last_peered":"2024-09-12T00:56:05.443212+0000","last_clean":"2024-09-12T00:56:05.443212+0000","last_became_active":"2024-09-12T00:52:51.410207+0000","last_became_peered":"2024-09-12T00:52:51.410207+0000","last_unstale":"2024-09-12T00:56:05.443212+0000","last_undegraded":"2024-09-12T00:56:05.443212+0000","last_fullsized":"2024-09-12T00:56:05.443212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:49:23.128033+0000","last_clean_scrub_stamp":"2024-09-12T00:49:23.128033+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:55.184464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624442+0000","last_change":"2024-09-12T00:54:01.671018+0000","last_active":"2024-09-12T00:56:03.624442+0000","last_peered":"2024-09-12T00:56:03.624442+0000","last_clean":"2024-09-12T00:56:03.624442+0000","last_became_active":"2024-09-12T00:54:01.670656+0000","last_became_peered":"2024-09-12T00:54:01.670656+0000","last_unstale":"2024-09-12T00:56:03.624442+0000","last_undegraded":"2024-09-12T00:56:03.624442+0000","last_fullsized":"2024-09-12T00:56:03.624442+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:03.499409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437402+0000","last_change":"2024-09-12T00:54:03.751997+0000","last_active":"2024-09-12T00:56:03.437402+0000","last_peered":"2024-09-12T00:56:03.437402+0000","last_clean":"2024-09-12T00:56:03.437402+0000","last_became_active":"2024-09-12T00:54:03.751650+0000","last_became_peered":"2024-09-12T00:54:03.751650+0000","last_unstale":"2024-09-12T00:56:03.437402+0000","last_undegraded":"2024-09-12T00:56:03.437402+0000","last_fullsized":"2024-09-12T00:56:03.437402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:16.182677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627654+0000","last_change":"2024-09-12T00:54:01.674016+0000","last_active":"2024-09-12T00:56:03.627654+0000","last_peered":"2024-09-12T00:56:03.627654+0000","last_clean":"2024-09-12T00:56:03.627654+0000","last_became_active":"2024-09-12T00:54:01.673345+0000","last_became_peered":"2024-09-12T00:54:01.673345+0000","last_unstale":"2024-09-12T00:56:03.627654+0000","last_undegraded":"2024-09-12T00:56:03.627654+0000","last_fullsized":"2024-09-12T00:56:03.627654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:49.412882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345288+0000","last_change":"2024-09-12T00:54:03.762269+0000","last_active":"2024-09-12T00:56:03.345288+0000","last_peered":"2024-09-12T00:56:03.345288+0000","last_clean":"2024-09-12T00:56:03.345288+0000","last_became_active":"2024-09-12T00:54:03.761934+0000","last_became_peered":"2024-09-12T00:54:03.761934+0000","last_unstale":"2024-09-12T00:56:03.345288+0000","last_undegraded":"2024-09-12T00:56:03.345288+0000","last_fullsized":"2024-09-12T00:56:03.345288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:34.036456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.985939+0000","last_change":"2024-09-12T00:54:01.754451+0000","last_active":"2024-09-12T00:56:12.985939+0000","last_peered":"2024-09-12T00:56:12.985939+0000","last_clean":"2024-09-12T00:56:12.985939+0000","last_became_active":"2024-09-12T00:54:01.754081+0000","last_became_peered":"2024-09-12T00:54:01.754081+0000","last_unstale":"2024-09-12T00:56:12.985939+0000","last_undegraded":"2024-09-12T00:56:12.985939+0000","last_fullsized":"2024-09-12T00:56:12.985939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:20.190287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.444998+0000","last_change":"2024-09-12T00:54:03.769880+0000","last_active":"2024-09-12T00:56:03.444998+0000","last_peered":"2024-09-12T00:56:03.444998+0000","last_clean":"2024-09-12T00:56:03.444998+0000","last_became_active":"2024-09-12T00:54:03.769523+0000","last_became_peered":"2024-09-12T00:54:03.769523+0000","last_unstale":"2024-09-12T00:56:03.444998+0000","last_undegraded":"2024-09-12T00:56:03.444998+0000","last_fullsized":"2024-09-12T00:56:03.444998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:01.106919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"85'7","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:12.982309+0000","last_change":"2024-09-12T00:54:01.669829+0000","last_active":"2024-09-12T00:56:12.982309+0000","last_peered":"2024-09-12T00:56:12.982309+0000","last_clean":"2024-09-12T00:56:12.982309+0000","last_became_active":"2024-09-12T00:54:01.669502+0000","last_became_peered":"2024-09-12T00:54:01.669502+0000","last_unstale":"2024-09-12T00:56:12.982309+0000","last_undegraded":"2024-09-12T00:56:12.982309+0000","last_fullsized":"2024-09-12T00:56:12.982309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:11:21.049922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447787+0000","last_change":"2024-09-12T00:55:09.853667+0000","last_active":"2024-09-12T00:56:03.447787+0000","last_peered":"2024-09-12T00:56:03.447787+0000","last_clean":"2024-09-12T00:56:03.447787+0000","last_became_active":"2024-09-12T00:55:09.853312+0000","last_became_peered":"2024-09-12T00:55:09.853312+0000","last_unstale":"2024-09-12T00:56:03.447787+0000","last_undegraded":"2024-09-12T00:56:03.447787+0000","last_fullsized":"2024-09-12T00:56:03.447787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:02.649165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434188+0000","last_change":"2024-09-12T00:54:01.755965+0000","last_active":"2024-09-12T00:56:03.434188+0000","last_peered":"2024-09-12T00:56:03.434188+0000","last_clean":"2024-09-12T00:56:03.434188+0000","last_became_active":"2024-09-12T00:54:01.755575+0000","last_became_peered":"2024-09-12T00:54:01.755575+0000","last_unstale":"2024-09-12T00:56:03.434188+0000","last_undegraded":"2024-09-12T00:56:03.434188+0000","last_fullsized":"2024-09-12T00:56:03.434188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:27.721356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.447861+0000","last_change":"2024-09-12T00:54:03.749382+0000","last_active":"2024-09-12T00:56:03.447861+0000","last_peered":"2024-09-12T00:56:03.447861+0000","last_clean":"2024-09-12T00:56:03.447861+0000","last_became_active":"2024-09-12T00:54:03.749043+0000","last_became_peered":"2024-09-12T00:54:03.749043+0000","last_unstale":"2024-09-12T00:56:03.447861+0000","last_undegraded":"2024-09-12T00:56:03.447861+0000","last_fullsized":"2024-09-12T00:56:03.447861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:46.084736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.629022+0000","last_change":"2024-09-12T00:54:01.755437+0000","last_active":"2024-09-12T00:56:03.629022+0000","last_peered":"2024-09-12T00:56:03.629022+0000","last_clean":"2024-09-12T00:56:03.629022+0000","last_became_active":"2024-09-12T00:54:01.754931+0000","last_became_peered":"2024-09-12T00:54:01.754931+0000","last_unstale":"2024-09-12T00:56:03.629022+0000","last_undegraded":"2024-09-12T00:56:03.629022+0000","last_fullsized":"2024-09-12T00:56:03.629022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:30.062543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.345653+0000","last_change":"2024-09-12T00:54:03.752014+0000","last_active":"2024-09-12T00:56:03.345653+0000","last_peered":"2024-09-12T00:56:03.345653+0000","last_clean":"2024-09-12T00:56:03.345653+0000","last_became_active":"2024-09-12T00:54:03.751437+0000","last_became_peered":"2024-09-12T00:54:03.751437+0000","last_unstale":"2024-09-12T00:56:03.345653+0000","last_undegraded":"2024-09-12T00:56:03.345653+0000","last_fullsized":"2024-09-12T00:56:03.345653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:39.094614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'103","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:14.353712+0000","last_change":"2024-09-12T00:55:09.342679+0000","last_active":"2024-09-12T00:56:14.353712+0000","last_peered":"2024-09-12T00:56:14.353712+0000","last_clean":"2024-09-12T00:56:14.353712+0000","last_became_active":"2024-09-12T00:55:09.336845+0000","last_became_peered":"2024-09-12T00:55:09.336845+0000","last_unstale":"2024-09-12T00:56:14.353712+0000","last_undegraded":"2024-09-12T00:56:14.353712+0000","last_fullsized":"2024-09-12T00:56:14.353712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:58:28.843552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437822+0000","last_change":"2024-09-12T00:54:03.759654+0000","last_active":"2024-09-12T00:56:03.437822+0000","last_peered":"2024-09-12T00:56:03.437822+0000","last_clean":"2024-09-12T00:56:03.437822+0000","last_became_active":"2024-09-12T00:54:03.759301+0000","last_became_peered":"2024-09-12T00:54:03.759301+0000","last_unstale":"2024-09-12T00:56:03.437822+0000","last_undegraded":"2024-09-12T00:56:03.437822+0000","last_fullsized":"2024-09-12T00:56:03.437822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:01.132310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.624508+0000","last_change":"2024-09-12T00:54:01.906627+0000","last_active":"2024-09-12T00:56:03.624508+0000","last_peered":"2024-09-12T00:56:03.624508+0000","last_clean":"2024-09-12T00:56:03.624508+0000","last_became_active":"2024-09-12T00:54:01.906277+0000","last_became_peered":"2024-09-12T00:54:01.906277+0000","last_unstale":"2024-09-12T00:56:03.624508+0000","last_undegraded":"2024-09-12T00:56:03.624508+0000","last_fullsized":"2024-09-12T00:56:03.624508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:28:33.993647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386330+0000","last_change":"2024-09-12T00:54:03.762250+0000","last_active":"2024-09-12T00:56:03.386330+0000","last_peered":"2024-09-12T00:56:03.386330+0000","last_clean":"2024-09-12T00:56:03.386330+0000","last_became_active":"2024-09-12T00:54:03.761543+0000","last_became_peered":"2024-09-12T00:54:03.761543+0000","last_unstale":"2024-09-12T00:56:03.386330+0000","last_undegraded":"2024-09-12T00:56:03.386330+0000","last_fullsized":"2024-09-12T00:56:03.386330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:13:26.705443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346808+0000","last_change":"2024-09-12T00:54:01.754574+0000","last_active":"2024-09-12T00:56:03.346808+0000","last_peered":"2024-09-12T00:56:03.346808+0000","last_clean":"2024-09-12T00:56:03.346808+0000","last_became_active":"2024-09-12T00:54:01.754196+0000","last_became_peered":"2024-09-12T00:54:01.754196+0000","last_unstale":"2024-09-12T00:56:03.346808+0000","last_undegraded":"2024-09-12T00:56:03.346808+0000","last_fullsized":"2024-09-12T00:56:03.346808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:21:49.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628606+0000","last_change":"2024-09-12T00:54:03.750512+0000","last_active":"2024-09-12T00:56:03.628606+0000","last_peered":"2024-09-12T00:56:03.628606+0000","last_clean":"2024-09-12T00:56:03.628606+0000","last_became_active":"2024-09-12T00:54:03.749814+0000","last_became_peered":"2024-09-12T00:54:03.749814+0000","last_unstale":"2024-09-12T00:56:03.628606+0000","last_undegraded":"2024-09-12T00:56:03.628606+0000","last_fullsized":"2024-09-12T00:56:03.628606+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:29.678753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437175+0000","last_change":"2024-09-12T00:54:01.666701+0000","last_active":"2024-09-12T00:56:03.437175+0000","last_peered":"2024-09-12T00:56:03.437175+0000","last_clean":"2024-09-12T00:56:03.437175+0000","last_became_active":"2024-09-12T00:54:01.666248+0000","last_became_peered":"2024-09-12T00:54:01.666248+0000","last_unstale":"2024-09-12T00:56:03.437175+0000","last_undegraded":"2024-09-12T00:56:03.437175+0000","last_fullsized":"2024-09-12T00:56:03.437175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:42.634033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436354+0000","last_change":"2024-09-12T00:54:03.758161+0000","last_active":"2024-09-12T00:56:03.436354+0000","last_peered":"2024-09-12T00:56:03.436354+0000","last_clean":"2024-09-12T00:56:03.436354+0000","last_became_active":"2024-09-12T00:54:03.757027+0000","last_became_peered":"2024-09-12T00:54:03.757027+0000","last_unstale":"2024-09-12T00:56:03.436354+0000","last_undegraded":"2024-09-12T00:56:03.436354+0000","last_fullsized":"2024-09-12T00:56:03.436354+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:03:02.580371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.825998+0000","last_change":"2024-09-12T00:54:01.671978+0000","last_active":"2024-09-12T00:56:03.825998+0000","last_peered":"2024-09-12T00:56:03.825998+0000","last_clean":"2024-09-12T00:56:03.825998+0000","last_became_active":"2024-09-12T00:54:01.671468+0000","last_became_peered":"2024-09-12T00:54:01.671468+0000","last_unstale":"2024-09-12T00:56:03.825998+0000","last_undegraded":"2024-09-12T00:56:03.825998+0000","last_fullsized":"2024-09-12T00:56:03.825998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:00:12.576544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.437494+0000","last_change":"2024-09-12T00:54:03.755249+0000","last_active":"2024-09-12T00:56:03.437494+0000","last_peered":"2024-09-12T00:56:03.437494+0000","last_clean":"2024-09-12T00:56:03.437494+0000","last_became_active":"2024-09-12T00:54:03.754901+0000","last_became_peered":"2024-09-12T00:54:03.754901+0000","last_unstale":"2024-09-12T00:56:03.437494+0000","last_undegraded":"2024-09-12T00:56:03.437494+0000","last_fullsized":"2024-09-12T00:56:03.437494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:13.206945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.436159+0000","last_change":"2024-09-12T00:55:09.343057+0000","last_active":"2024-09-12T00:56:03.436159+0000","last_peered":"2024-09-12T00:56:03.436159+0000","last_clean":"2024-09-12T00:56:03.436159+0000","last_became_active":"2024-09-12T00:55:09.337164+0000","last_became_peered":"2024-09-12T00:55:09.337164+0000","last_unstale":"2024-09-12T00:56:03.436159+0000","last_undegraded":"2024-09-12T00:56:03.436159+0000","last_fullsized":"2024-09-12T00:56:03.436159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:31.305782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.627570+0000","last_change":"2024-09-12T00:54:03.756897+0000","last_active":"2024-09-12T00:56:03.627570+0000","last_peered":"2024-09-12T00:56:03.627570+0000","last_clean":"2024-09-12T00:56:03.627570+0000","last_became_active":"2024-09-12T00:54:03.756179+0000","last_became_peered":"2024-09-12T00:54:03.756179+0000","last_unstale":"2024-09-12T00:56:03.627570+0000","last_undegraded":"2024-09-12T00:56:03.627570+0000","last_fullsized":"2024-09-12T00:56:03.627570+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:03.773436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.434160+0000","last_change":"2024-09-12T00:55:09.903676+0000","last_active":"2024-09-12T00:56:03.434160+0000","last_peered":"2024-09-12T00:56:03.434160+0000","last_clean":"2024-09-12T00:56:03.434160+0000","last_became_active":"2024-09-12T00:55:09.903415+0000","last_became_peered":"2024-09-12T00:55:09.903415+0000","last_unstale":"2024-09-12T00:56:03.434160+0000","last_undegraded":"2024-09-12T00:56:03.434160+0000","last_fullsized":"2024-09-12T00:56:03.434160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:07:05.807722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628818+0000","last_change":"2024-09-12T00:54:01.667413+0000","last_active":"2024-09-12T00:56:03.628818+0000","last_peered":"2024-09-12T00:56:03.628818+0000","last_clean":"2024-09-12T00:56:03.628818+0000","last_became_active":"2024-09-12T00:54:01.666536+0000","last_became_peered":"2024-09-12T00:54:01.666536+0000","last_unstale":"2024-09-12T00:56:03.628818+0000","last_undegraded":"2024-09-12T00:56:03.628818+0000","last_fullsized":"2024-09-12T00:56:03.628818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:45.811674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.346871+0000","last_change":"2024-09-12T00:54:03.764546+0000","last_active":"2024-09-12T00:56:03.346871+0000","last_peered":"2024-09-12T00:56:03.346871+0000","last_clean":"2024-09-12T00:56:03.346871+0000","last_became_active":"2024-09-12T00:54:03.764187+0000","last_became_peered":"2024-09-12T00:54:03.764187+0000","last_unstale":"2024-09-12T00:56:03.346871+0000","last_undegraded":"2024-09-12T00:56:03.346871+0000","last_fullsized":"2024-09-12T00:56:03.346871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:51.837035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"85'13","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.628935+0000","last_change":"2024-09-12T00:54:01.669563+0000","last_active":"2024-09-12T00:56:03.628935+0000","last_peered":"2024-09-12T00:56:03.628935+0000","last_clean":"2024-09-12T00:56:03.628935+0000","last_became_active":"2024-09-12T00:54:01.669173+0000","last_became_peered":"2024-09-12T00:54:01.669173+0000","last_unstale":"2024-09-12T00:56:03.628935+0000","last_undegraded":"2024-09-12T00:56:03.628935+0000","last_fullsized":"2024-09-12T00:56:03.628935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:09:50.694010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.386112+0000","last_change":"2024-09-12T00:54:03.754427+0000","last_active":"2024-09-12T00:56:03.386112+0000","last_peered":"2024-09-12T00:56:03.386112+0000","last_clean":"2024-09-12T00:56:03.386112+0000","last_became_active":"2024-09-12T00:54:03.754129+0000","last_became_peered":"2024-09-12T00:54:03.754129+0000","last_unstale":"2024-09-12T00:56:03.386112+0000","last_undegraded":"2024-09-12T00:56:03.386112+0000","last_fullsized":"2024-09-12T00:56:03.386112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:43.403483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438261+0000","last_change":"2024-09-12T00:54:01.754543+0000","last_active":"2024-09-12T00:56:03.438261+0000","last_peered":"2024-09-12T00:56:03.438261+0000","last_clean":"2024-09-12T00:56:03.438261+0000","last_became_active":"2024-09-12T00:54:01.754188+0000","last_became_peered":"2024-09-12T00:54:01.754188+0000","last_unstale":"2024-09-12T00:56:03.438261+0000","last_undegraded":"2024-09-12T00:56:03.438261+0000","last_fullsized":"2024-09-12T00:56:03.438261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:22.328283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:03.438014+0000","last_change":"2024-09-12T00:54:03.757044+0000","last_active":"2024-09-12T00:56:03.438014+0000","last_peered":"2024-09-12T00:56:03.438014+0000","last_clean":"2024-09-12T00:56:03.438014+0000","last_became_active":"2024-09-12T00:54:03.756713+0000","last_became_peered":"2024-09-12T00:54:03.756713+0000","last_unstale":"2024-09-12T00:56:03.438014+0000","last_undegraded":"2024-09-12T00:56:03.438014+0000","last_fullsized":"2024-09-12T00:56:03.438014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:02.731909+0000","last_clean_scrub_stamp":"2024-09-12T00:54:02.731909+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:40.295669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":159,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T00:56:13.037214+0000","last_change":"2024-09-12T00:54:01.667386+0000","last_active":"2024-09-12T00:56:13.037214+0000","last_peered":"2024-09-12T00:56:13.037214+0000","last_clean":"2024-09-12T00:56:13.037214+0000","last_became_active":"2024-09-12T00:54:01.666957+0000","last_became_peered":"2024-09-12T00:54:01.666957+0000","last_unstale":"2024-09-12T00:56:13.037214+0000","last_undegraded":"2024-09-12T00:56:13.037214+0000","last_fullsized":"2024-09-12T00:56:13.037214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T00:54:00.652904+0000","last_clean_scrub_stamp":"2024-09-12T00:54:00.652904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:09.762353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800488,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1556,"num_read_kb":997,"num_write":501,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":59211,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2401464,"data_compressed":113132,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":381,"ondisk_log_size":381,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108936,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514536,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":847718,"data_compressed":22221,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":1442955,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":978944,"data_stored":846328,"data_compressed":22080,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":453490,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1003520,"data_stored":1046937,"data_compressed":39246,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":937984,"data_stored":453615,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1373527,"data_compressed":54086,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":523202,"data_compressed":12599,"data_compressed_allocated":16384,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1024000,"data_stored":1174102,"data_compressed":37423,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":455282,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":524506,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":455488,"data_compressed":2934,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36312,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":72516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34489,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":71212,"data_compressed":9665,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331052,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400480,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395728,"data_compressed":19287,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T00:56:20.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:21.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:20 smithi016 ceph-mon[30972]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:21.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:20 smithi016 ceph-mon[30972]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 -- 172.21.15.16:0/1569559704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14102790 msgr2=0x7fdf141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/1569559704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14102790 0x7fdf141067e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009900 tx=0x7fdefc02a410 comp rx=0 tx=0).stop 2024-09-12T00:56:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 -- 172.21.15.16:0/1569559704 shutdown_connections 2024-09-12T00:56:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/1569559704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14106f10 0x7fdf1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/1569559704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14102790 0x7fdf141067e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/1569559704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14101de0 0x7fdf141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 -- 172.21.15.16:0/1569559704 >> 172.21.15.16:0/1569559704 conn(0x7fdf140fd8c0 msgr2=0x7fdf140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 -- 172.21.15.16:0/1569559704 shutdown_connections 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.121+0000 7fdf1a63f640 1 -- 172.21.15.16:0/1569559704 wait complete. 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 Processor -- start 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 -- start start 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14101de0 0x7fdf14198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14102790 0x7fdf141989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14111f70 con 0x7fdf14106f10 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf14111df0 con 0x7fdf14101de0 2024-09-12T00:56:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf1a63f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf141120f0 con 0x7fdf14102790 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35348/0 (socket says 172.21.15.16:35348) 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 -- 172.21.15.16:0/669218301 learned_addr learned my addr 172.21.15.16:0/669218301 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf13fff640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14101de0 0x7fdf14198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf0bfff640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14102790 0x7fdf141989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 -- 172.21.15.16:0/669218301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14102790 msgr2=0x7fdf141989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14102790 0x7fdf141989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 -- 172.21.15.16:0/669218301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14101de0 msgr2=0x7fdf14198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14101de0 0x7fdf14198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.122+0000 7fdf18bdc640 1 -- 172.21.15.16:0/669218301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1419d440 con 0x7fdf14106f10 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf18bdc640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00cad0 tx=0x7fdf0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:21.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00ae20 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c005020 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1419d730 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00f6f0 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.123+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf141a5010 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.124+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf141021c0 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.125+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fdf0c0051d0 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.126+0000 7fdf11ffb640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 0x7fdf000a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.126+0000 7fdf13fff640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 0x7fdf000a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.126+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fdf0c0cf700 con 0x7fdf14106f10 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.126+0000 7fdf13fff640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 0x7fdf000a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf040058c0 tx=0x7fdf0400a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.130+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf0c014030 con 0x7fdf14106f10 2024-09-12T00:56:21.168 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:20 smithi159 ceph-mon[35566]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:21.168 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:20 smithi159 ceph-mon[35566]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:21.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:20 smithi099 ceph-mon[35654]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-09-12T00:56:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:20 smithi099 ceph-mon[35654]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T00:56:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.340+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf1402d7c0 con 0x7fdf14106f10 2024-09-12T00:56:21.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.341+0000 7fdf11ffb640 1 -- 172.21.15.16:0/669218301 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fdf0c09c100 con 0x7fdf14106f10 2024-09-12T00:56:21.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:21.341 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:21.341 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.343+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 msgr2=0x7fdf000a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.343+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 0x7fdf000a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf040058c0 tx=0x7fdf0400a2d0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.343+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 msgr2=0x7fdf1419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.343+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00cad0 tx=0x7fdf0c00cf90 comp rx=0 tx=0).stop 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 shutdown_connections 2024-09-12T00:56:21.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdf000a2e10 0x7fdf000a52d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf14106f10 0x7fdf1419cd60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf14102790 0x7fdf141989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 --2- 172.21.15.16:0/669218301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf14101de0 0x7fdf14198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 >> 172.21.15.16:0/669218301 conn(0x7fdf140fd8c0 msgr2=0x7fdf140fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 shutdown_connections 2024-09-12T00:56:21.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:21.344+0000 7fdf1a63f640 1 -- 172.21.15.16:0/669218301 wait complete. 2024-09-12T00:56:21.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:21 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/669218301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:21.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:22.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:21 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/669218301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:22.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:21 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/669218301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 -- 172.21.15.16:0/1252016816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328100a90 msgr2=0x7fd328100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1252016816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328100a90 0x7fd328100ef0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd310009930 tx=0x7fd31002a420 comp rx=0 tx=0).stop 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 -- 172.21.15.16:0/1252016816 shutdown_connections 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1252016816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328101430 0x7fd32810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1252016816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328100a90 0x7fd328100ef0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1252016816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328106a90 0x7fd328106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.712+0000 7fd32eaea640 1 -- 172.21.15.16:0/1252016816 >> 172.21.15.16:0/1252016816 conn(0x7fd3280fc7d0 msgr2=0x7fd3280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- 172.21.15.16:0/1252016816 shutdown_connections 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- 172.21.15.16:0/1252016816 wait complete. 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 Processor -- start 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- start start 2024-09-12T00:56:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328100a90 0x7fd3280fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328106a90 0x7fd3280fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd328110e90 con 0x7fd328101430 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd328110d10 con 0x7fd328100a90 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd32eaea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd328111010 con 0x7fd328106a90 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.713+0000 7fd327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35370/0 (socket says 172.21.15.16:35370) 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 -- 172.21.15.16:0/1126418593 learned_addr learned my addr 172.21.15.16:0/1126418593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd32d087640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328106a90 0x7fd3280fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd32c886640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328100a90 0x7fd3280fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 -- 172.21.15.16:0/1126418593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328106a90 msgr2=0x7fd3280fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328106a90 0x7fd3280fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 -- 172.21.15.16:0/1126418593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328100a90 msgr2=0x7fd3280fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328100a90 0x7fd3280fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 -- 172.21.15.16:0/1126418593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3281a6990 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd327fff640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd310009230 tx=0x7fd310004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31003f030 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd310003ad0 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3281a6c20 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.714+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310035070 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.715+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281a7100 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.715+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd328100ef0 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.717+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fd310042030 con 0x7fd328101430 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.718+0000 7fd325ffb640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 0x7fd3140a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.718+0000 7fd32c886640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 0x7fd3140a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.718+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd3100eff90 con 0x7fd328101430 2024-09-12T00:56:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.718+0000 7fd32c886640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 0x7fd3140a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058c0 tx=0x7fd31800a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.721+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3100bc410 con 0x7fd328101430 2024-09-12T00:56:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.934+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd32810a030 con 0x7fd328101430 2024-09-12T00:56:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.935+0000 7fd325ffb640 1 -- 172.21.15.16:0/1126418593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fd3100c0710 con 0x7fd328101430 2024-09-12T00:56:22.935 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:22.935 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 msgr2=0x7fd3140a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 0x7fd3140a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3180058c0 tx=0x7fd31800a2d0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 msgr2=0x7fd3280fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd310009230 tx=0x7fd310004270 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 shutdown_connections 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3140a2e10 0x7fd3140a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd328106a90 0x7fd3280fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328101430 0x7fd3280fd350 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 --2- 172.21.15.16:0/1126418593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd328100a90 0x7fd3280fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 >> 172.21.15.16:0/1126418593 conn(0x7fd3280fc7d0 msgr2=0x7fd32806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:22.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 shutdown_connections 2024-09-12T00:56:22.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:22.937+0000 7fd32eaea640 1 -- 172.21.15.16:0/1126418593 wait complete. 2024-09-12T00:56:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:22 smithi016 ceph-mon[30972]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-12T00:56:23.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:22 smithi099 ceph-mon[35654]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-12T00:56:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:22 smithi159 ceph-mon[35566]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-12T00:56:23.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T00:56:23.958 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:24.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:23 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1126418593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:24.175 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:23 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1126418593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:24.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:23 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1126418593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 -- 172.21.15.16:0/1819151592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 msgr2=0x7fd084110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/1819151592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd084110870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009ca0 tx=0x7fd07c02a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 -- 172.21.15.16:0/1819151592 shutdown_connections 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/1819151592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd084110870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/1819151592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd08410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.485+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/1819151592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd084100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 -- 172.21.15.16:0/1819151592 >> 172.21.15.16:0/1819151592 conn(0x7fd0840fc7a0 msgr2=0x7fd0840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 -- 172.21.15.16:0/1819151592 shutdown_connections 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 -- 172.21.15.16:0/1819151592 wait complete. 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 Processor -- start 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 -- start start 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd0841a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.486+0000 7fd08b8ef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd0841a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd08419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd084114240 con 0x7fd08410e480 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841140c0 con 0x7fd0841008d0 2024-09-12T00:56:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841143c0 con 0x7fd084101280 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08968b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd0841a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd08419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08968b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd0841a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:51064/0 (socket says 172.21.15.16:51064) 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08968b640 1 -- 172.21.15.16:0/560784647 learned_addr learned my addr 172.21.15.16:0/560784647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd088e8a640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd0841a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 -- 172.21.15.16:0/560784647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 msgr2=0x7fd0841a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd0841a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 -- 172.21.15.16:0/560784647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 msgr2=0x7fd0841a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd0841a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 -- 172.21.15.16:0/560784647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08419ad20 con 0x7fd08410e480 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd088e8a640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd0841a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd089e8c640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd08419a5f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02ade0 tx=0x7fd07c031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003c30 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c003dd0 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004770 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08419afb0 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.487+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a7300 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.488+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084100d30 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.490+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fd07c02f710 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.490+0000 7fd07a7fc640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 0x7fd0700a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.490+0000 7fd08968b640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 0x7fd0700a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.490+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd07c0f0840 con 0x7fd08410e480 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.491+0000 7fd08968b640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 0x7fd0700a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd080009a40 tx=0x7fd0800091c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.494+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd07c0bd470 con 0x7fd08410e480 2024-09-12T00:56:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.671+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd084109640 con 0x7fd08410e480 2024-09-12T00:56:24.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.673+0000 7fd07a7fc640 1 -- 172.21.15.16:0/560784647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23017 (secure 0 0 0) 0x7fd07c0bcd70 con 0x7fd08410e480 2024-09-12T00:56:24.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:24.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T00:56:03.340006+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2952840401":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2290747564":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/90830535":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1572487203":"2024-09-13T00:56:03.339943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 msgr2=0x7fd0700a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 0x7fd0700a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd080009a40 tx=0x7fd0800091c0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 msgr2=0x7fd08419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd08419a5f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd07c02ade0 tx=0x7fd07c031a60 comp rx=0 tx=0).stop 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 shutdown_connections 2024-09-12T00:56:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2e10 0x7fd0700a52d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08410e480 0x7fd08419a5f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084101280 0x7fd0841a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 --2- 172.21.15.16:0/560784647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0841008d0 0x7fd0841a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 >> 172.21.15.16:0/560784647 conn(0x7fd0840fc7a0 msgr2=0x7fd0840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.676+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 shutdown_connections 2024-09-12T00:56:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:24.677+0000 7fd08b8ef640 1 -- 172.21.15.16:0/560784647 wait complete. 2024-09-12T00:56:25.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:24 smithi016 ceph-mon[30972]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-12T00:56:25.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:24 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:56:25.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:24 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/560784647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:25.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:24 smithi099 ceph-mon[35654]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-12T00:56:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:24 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:56:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:24 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/560784647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:24 smithi159 ceph-mon[35566]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-12T00:56:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:24 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:56:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:24 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/560784647' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T00:56:25.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T00:56:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:26.800 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T00:56:26.801 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T00:56:26.801 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T00:56:26.801 DEBUG:tasks.quiescer:Yielding 2024-09-12T00:56:26.802 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T00:56:26.819 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T00:56:26.819 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T00:56:26.819 INFO:tasks.workunit:timeout=3h 2024-09-12T00:56:26.819 INFO:tasks.workunit:cleanup=True 2024-09-12T00:56:26.820 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:26.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T00:56:26.840 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-12 00:54:23.964691987 +0000 2024-09-12T00:56:26.841 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-12 00:54:23.964691987 +0000 2024-09-12T00:56:26.841 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-12 00:56:01.913893155 +0000 2024-09-12T00:56:26.841 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-12 00:54:23.964691987 +0000 2024-09-12T00:56:26.841 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T00:56:26.841 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T00:56:26.919 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T00:56:26.976 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T00:56:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:27.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:26 smithi016 ceph-mon[30972]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-12T00:56:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:26 smithi099 ceph-mon[35654]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-12T00:56:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:26 smithi159 ceph-mon[35566]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-12T00:56:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 -- 172.21.15.16:0/17296183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 msgr2=0x7f44141031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 --2- 172.21.15.16:0/17296183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f44141031b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f440c009930 tx=0x7f440c02a420 comp rx=0 tx=0).stop 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 -- 172.21.15.16:0/17296183 shutdown_connections 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 --2- 172.21.15.16:0/17296183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414108300 0x7f441410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 --2- 172.21.15.16:0/17296183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414103780 0x7f4414107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 --2- 172.21.15.16:0/17296183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f44141031b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 -- 172.21.15.16:0/17296183 >> 172.21.15.16:0/17296183 conn(0x7f44140fc7d0 msgr2=0x7f44140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 -- 172.21.15.16:0/17296183 shutdown_connections 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.604+0000 7f44193b0640 1 -- 172.21.15.16:0/17296183 wait complete. 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 Processor -- start 2024-09-12T00:56:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 -- start start 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414103780 0x7f4414198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414108300 0x7f441419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112090 con 0x7f4414102dd0 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4414111f10 con 0x7f4414108300 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.605+0000 7f44193b0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112210 con 0x7f4414103780 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35406/0 (socket says 172.21.15.16:35406) 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 -- 172.21.15.16:0/1222242154 learned_addr learned my addr 172.21.15.16:0/1222242154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f44137fe640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414108300 0x7f441419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f44127fc640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414103780 0x7f4414198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 -- 172.21.15.16:0/1222242154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414103780 msgr2=0x7f4414198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414103780 0x7f4414198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 -- 172.21.15.16:0/1222242154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414108300 msgr2=0x7f441419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414108300 0x7f441419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 -- 172.21.15.16:0/1222242154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441419d600 con 0x7f4414102dd0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f4412ffd640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f440c009230 tx=0x7f440c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c03f030 con 0x7f4414102dd0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441419d890 con 0x7f4414102dd0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c003ad0 con 0x7f4414102dd0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.606+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c035070 con 0x7f4414102dd0 2024-09-12T00:56:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.607+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a51d0 con 0x7f4414102dd0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.608+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44141031b0 con 0x7f4414102dd0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.610+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f440c042030 con 0x7f4414102dd0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.610+0000 7f440bfff640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 0x7f44000a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.610+0000 7f44127fc640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 0x7f44000a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.610+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f440c0efa60 con 0x7f4414102dd0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.611+0000 7f44127fc640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 0x7f44000a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44140fefa0 tx=0x7f43fc00b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:27.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.615+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f440c0bbee0 con 0x7f4414102dd0 2024-09-12T00:56:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.830+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f441410a290 con 0x7f4414102dd0 2024-09-12T00:56:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.831+0000 7f440bfff640 1 -- 172.21.15.16:0/1222242154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f440c0c01e0 con 0x7f4414102dd0 2024-09-12T00:56:27.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:27.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 msgr2=0x7f44000a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 0x7f44000a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44140fefa0 tx=0x7f43fc00b390 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 msgr2=0x7f4414198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f440c009230 tx=0x7f440c004270 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 shutdown_connections 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f44000a2e10 0x7f44000a52d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4414108300 0x7f441419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4414103780 0x7f4414198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 --2- 172.21.15.16:0/1222242154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4414102dd0 0x7f4414198650 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 >> 172.21.15.16:0/1222242154 conn(0x7f44140fc7d0 msgr2=0x7f4414105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 shutdown_connections 2024-09-12T00:56:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:27.833+0000 7f44193b0640 1 -- 172.21.15.16:0/1222242154 wait complete. 2024-09-12T00:56:28.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:27 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1222242154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:28.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:27 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1222242154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:27 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1222242154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:28.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T00:54:04.465326+0000', 'modified': '2024-09-12T00:56:02.636132+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24505, 'mds_1': 24487, 'mds_2': 24482, 'mds_3': 24481, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14589': {'gid': 14589, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6839/3482588678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3482588678}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3482588678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6841/2826352912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6840', 'nonce': 2826352912}, {'type': 'v1', 'addr': '172.21.15.16:6841', 'nonce': 2826352912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/4113169911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 4113169911}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 4113169911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/3000703552', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 3000703552}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 3000703552}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/310857906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 310857906}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 310857906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/1045510613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 1045510613}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 1045510613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1983976622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1983976622}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1983976622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.99:6839/850851643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 850851643}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 850851643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.159:6837/247659815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 247659815}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 247659815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.159:6839/1957754994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 1957754994}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 1957754994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24505, 'qdb_cluster': [24452, 24481, 24482, 24487, 24505]} 2024-09-12T00:56:28.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T00:56:28.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T00:56:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:29.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:28 smithi016 ceph-mon[30972]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T00:56:29.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:28 smithi099 ceph-mon[35654]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T00:56:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 -- 172.21.15.16:0/4140428263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c98103660 msgr2=0x7f7c9810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/4140428263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c98103660 0x7f7c9810bb50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009900 tx=0x7f7c8c02a480 comp rx=0 tx=0).stop 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 -- 172.21.15.16:0/4140428263 shutdown_connections 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/4140428263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c9810c170 0x7f7c9810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/4140428263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c98103660 0x7f7c9810bb50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/4140428263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98102d40 0x7f7c98103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 -- 172.21.15.16:0/4140428263 >> 172.21.15.16:0/4140428263 conn(0x7f7c980fc7a0 msgr2=0x7f7c980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 -- 172.21.15.16:0/4140428263 shutdown_connections 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.314+0000 7f7c9e55e640 1 -- 172.21.15.16:0/4140428263 wait complete. 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 Processor -- start 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 -- start start 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c98102d40 0x7f7c981009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98103660 0x7f7c98100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98111f70 con 0x7f7c9810c170 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98111df0 con 0x7f7c98103660 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9e55e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981120f0 con 0x7f7c98102d40 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9cafb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9cafb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39534/0 (socket says 172.21.15.16:39534) 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c9cafb640 1 -- 172.21.15.16:0/3007120324 learned_addr learned my addr 172.21.15.16:0/3007120324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c97fff640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c98102d40 0x7f7c981009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.315+0000 7f7c977fe640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98103660 0x7f7c98100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 -- 172.21.15.16:0/3007120324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c98102d40 msgr2=0x7f7c981009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c98102d40 0x7f7c981009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 -- 172.21.15.16:0/3007120324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98103660 msgr2=0x7f7c98100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98103660 0x7f7c98100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 -- 172.21.15.16:0/3007120324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c981a6910 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9cafb640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cce0 tx=0x7f7c9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c957fa640 1 -- 172.21.15.16:0/3007120324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000f450 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c957fa640 1 -- 172.21.15.16:0/3007120324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c9000fad0 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c957fa640 1 -- 172.21.15.16:0/3007120324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90018c10 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981a6c00 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.316+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c981a70c0 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7c957fa640 1 -- 172.21.15.16:0/3007120324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f7c9000f5f0 con 0x7f7c9810c170 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 Processor -- start 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 -- start start 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 0x55c44bd06db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd11bb0 0x55c44bd13fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44bc6d090 con 0x55c44bd072f0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c44bc14a70 con 0x55c44bd11bb0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.319+0000 7f7ca0937740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c44bc69050 con 0x55c44bd069d0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39538/0 (socket says 172.21.15.16:39538) 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 -- 172.21.15.16:0/629683257 learned_addr learned my addr 172.21.15.16:0/629683257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c977fe640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 0x55c44bd06db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c957fa640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c977fe640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd11bb0 0x55c44bd13fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 -- 172.21.15.16:0/629683257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 msgr2=0x55c44bd06db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 0x55c44bd06db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 -- 172.21.15.16:0/629683257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd11bb0 msgr2=0x55c44bd13fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd11bb0 0x55c44bd13fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 -- 172.21.15.16:0/629683257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44bc21130 con 0x55c44bd072f0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c977fe640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 0x55c44bd06db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c957fa640 1 -- 172.21.15.16:0/3007120324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7c900cfbd0 con 0x7f7c9810c170 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c9cafb640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c97fff640 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800bd10 tx=0x7f7c880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b382beb84b665028 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/629683257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88004030 con 0x55c44bd072f0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/629683257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c880041d0 con 0x55c44bd072f0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.320+0000 7f7c7e7fc640 1 -- 172.21.15.16:0/629683257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88011840 con 0x55c44bd072f0 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 -- 172.21.15.16:0/629683257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 msgr2=0x55c44bd11670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7c9cafb640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 0x7f7c800a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cd230 tx=0x7f7c9000ff60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800bd10 tx=0x7f7c880107a0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 -- 172.21.15.16:0/629683257 shutdown_connections 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd11bb0 0x55c44bd13fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd11670 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 --2- 172.21.15.16:0/629683257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd069d0 0x55c44bd06db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 -- 172.21.15.16:0/629683257 >> 172.21.15.16:0/629683257 conn(0x55c44bd02f60 msgr2=0x55c44bd03340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 -- 172.21.15.16:0/629683257 shutdown_connections 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.321+0000 7f7ca0937740 1 -- 172.21.15.16:0/629683257 wait complete. 2024-09-12T00:56:29.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 -- start start 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 Processor -- start 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd11480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd0e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd14780 con 0x55c44bd11bb0 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c44bc27f20 con 0x55c44bd069d0 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd14900 con 0x55c44bd072f0 2024-09-12T00:56:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7ca0937740 10 client.0 _ll_register_callbacks cb 0x55c44bbf1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7c977fe640 1 -- 172.21.15.16:0/1616348628 learned_addr learned my addr 172.21.15.16:0/1616348628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7c97fff640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd0e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.322+0000 7f7c977fe640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd11480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 -- 172.21.15.16:0/1616348628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 msgr2=0x55c44bd0e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd0e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 -- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 msgr2=0x55c44bd11480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd11480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 -- 172.21.15.16:0/1616348628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44b9de2b0 con 0x55c44bd11bb0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c977fe640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c02a9a0 tx=0x7f7c8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad99294f8e9fea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c035070 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7c8c004480 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c880041d0 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c051050 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7ca0937740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c44bd200d0 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c880041d0 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.323+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c8c03d750 con 0x55c44bd11bb0 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 10 client.15474 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 20 client.15474 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd22690 0x55c44bd23b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 4 client.15474 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44bc7a600 con 0x55c44bd22690 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7c97fff640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd22690 0x55c44bd23b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7c97fff640 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd22690 0x55c44bd23b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=524abeef0950d918 server_cookie=f5afb9eb70f30eb9 in_seq=0 out_seq=0 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.324+0000 7f7c777fe640 10 client.15474 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7c777fe640 1 -- 172.21.15.16:0/1616348628 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c44bc7a600 con 0x55c44bd22690 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7c777fe640 10 client.15474 handle_command_reply: tid=0 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 client.15474 shutdown 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 msgr2=0x55c44bd0ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c02a9a0 tx=0x7f7c8c009260 comp rx=0 tx=0).stop 2024-09-12T00:56:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 shutdown_connections 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd22690 0x55c44bd23b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd11bb0 0x55c44bd0ed10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd0e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7c777fe640 0 client.15474 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 --2- 172.21.15.16:0/1616348628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd11480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 >> 172.21.15.16:0/1616348628 conn(0x55c44bd02f60 msgr2=0x55c44bd08c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 shutdown_connections 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 1 -- 172.21.15.16:0/1616348628 wait complete. 2024-09-12T00:56:29.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.325+0000 7f7ca0937740 20 client.15474 trim_cache size 0 max 16384 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.343+0000 7f7ca0937740 1 Processor -- start 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.343+0000 7f7ca0937740 1 -- start start 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd02f60 0x55c44bd0e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd21e40 con 0x55c44bd072f0 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd22510 con 0x55c44bd069d0 2024-09-12T00:56:29.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7ca0937740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd22be0 con 0x55c44bd02f60 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39554/0 (socket says 172.21.15.16:39554) 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 -- 172.21.15.16:0/608568277 learned_addr learned my addr 172.21.15.16:0/608568277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c97fff640 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd02f60 0x55c44bd0e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 -- 172.21.15.16:0/608568277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd02f60 msgr2=0x55c44bd0e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd02f60 0x55c44bd0e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 -- 172.21.15.16:0/608568277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 msgr2=0x55c44bd0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 -- 172.21.15.16:0/608568277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44bd06800 con 0x55c44bd072f0 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c977fe640 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0096c0 tx=0x7f7c8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d536bd74a68d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.344+0000 7f7c7dffb640 1 -- 172.21.15.16:0/608568277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c035070 con 0x55c44bd072f0 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7c7dffb640 1 -- 172.21.15.16:0/608568277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c03d930 con 0x55c44bd072f0 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7c7dffb640 1 -- 172.21.15.16:0/608568277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0046c0 con 0x55c44bd072f0 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 -- 172.21.15.16:0/608568277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 msgr2=0x55c44bd0ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0096c0 tx=0x7f7c8c000bc0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 -- 172.21.15.16:0/608568277 shutdown_connections 2024-09-12T00:56:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd072f0 0x55c44bd0ba50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd069d0 0x55c44bd0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 --2- 172.21.15.16:0/608568277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd02f60 0x55c44bd0e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 -- 172.21.15.16:0/608568277 >> 172.21.15.16:0/608568277 conn(0x55c44bd0ee70 msgr2=0x55c44bd10740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 -- 172.21.15.16:0/608568277 shutdown_connections 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.345+0000 7f7ca0937740 1 -- 172.21.15.16:0/608568277 wait complete. 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 -- start start 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 Processor -- start 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 0x55c44bd27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd23980 con 0x55c44bd069d0 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd232b0 con 0x55c44bd02f60 2024-09-12T00:56:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c44bd24050 con 0x55c44bd072f0 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7ca0937740 10 client.0 _ll_register_callbacks cb 0x7f7c8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39570/0 (socket says 172.21.15.16:39570) 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7c97fff640 1 -- 172.21.15.16:0/2619529179 learned_addr learned my addr 172.21.15.16:0/2619529179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7c977fe640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 0x55c44bd27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.346+0000 7f7c977fe640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 -- 172.21.15.16:0/2619529179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 msgr2=0x55c44bd28670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd28670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 -- 172.21.15.16:0/2619529179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 msgr2=0x55c44bd28130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 -- 172.21.15.16:0/2619529179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44bd8be20 con 0x55c44bd02f60 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c97fff640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c977fe640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 0x55c44bd27bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c02a9a0 tx=0x7f7c8c005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=492d1b433bfe7c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c777fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c035040 con 0x55c44bd02f60 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c8c03d930 con 0x55c44bd02f60 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7c8c03d650 con 0x55c44bd02f60 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c8c06a050 con 0x55c44bd02f60 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7ca0937740 10 client.25082 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.347+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c44bd8d100 con 0x55c44bd02f60 2024-09-12T00:56:29.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c8c03d930 con 0x55c44bd02f60 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c8c003c90 con 0x55c44bd02f60 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 10 client.25082 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 10 client.25082 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 20 client.25082 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd11bb0 0x55c44bd90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 4 client.25082 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "status"}) -- 0x55c44bc74b10 con 0x55c44bd11bb0 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.348+0000 7f7c97fff640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd11bb0 0x55c44bd90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7c777fe640 10 client.25082 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7c97fff640 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd11bb0 0x55c44bd90700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67059b20a6022ad0 server_cookie=7f2d7ba7096e3f9e in_seq=0 out_seq=0 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7c777fe640 1 -- 172.21.15.16:0/2619529179 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c44bc74b10 con 0x55c44bd11bb0 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7c777fe640 10 client.25082 handle_command_reply: tid=0 2024-09-12T00:56:29.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 client.25082 shutdown 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 msgr2=0x55c44bd27bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 0x55c44bd27bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c02a9a0 tx=0x7f7c8c005390 comp rx=0 tx=0).stop 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 shutdown_connections 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c44bd11bb0 0x55c44bd90700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c44bd072f0 0x55c44bd28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7c777fe640 0 client.25082 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44bd069d0 0x55c44bd28130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 --2- 172.21.15.16:0/2619529179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c44bd02f60 0x55c44bd27bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 >> 172.21.15.16:0/2619529179 conn(0x55c44bd0ee70 msgr2=0x55c44bd1cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 shutdown_connections 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 1 -- 172.21.15.16:0/2619529179 wait complete. 2024-09-12T00:56:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.349+0000 7f7ca0937740 20 client.25082 trim_cache size 0 max 16384 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 4, 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24452, 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 133.53643304900001, 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-12T00:56:29.361 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 167.50515027500001 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 msgr2=0x7f7c800a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 0x7f7c800a5350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c900cd230 tx=0x7f7c9000ff60 comp rx=0 tx=0).stop 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 msgr2=0x7f7c98101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cce0 tx=0x7f7c9000a620 comp rx=0 tx=0).stop 2024-09-12T00:56:29.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 shutdown_connections 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c800a2e90 0x7f7c800a5350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c9810c170 0x7f7c98101420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c98103660 0x7f7c98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 --2- 172.21.15.16:0/3007120324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c98102d40 0x7f7c981009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 >> 172.21.15.16:0/3007120324 conn(0x7f7c980fc7a0 msgr2=0x7f7c980fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 shutdown_connections 2024-09-12T00:56:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:29.350+0000 7f7c9e55e640 1 -- 172.21.15.16:0/3007120324 wait complete. 2024-09-12T00:56:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:28 smithi159 ceph-mon[35566]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T00:56:30.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T00:56:30.394 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2699724926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518100a80 msgr2=0x7ff518100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2699724926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518100a80 0x7ff518100ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009930 tx=0x7ff50c02a420 comp rx=0 tx=0).stop 2024-09-12T00:56:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2699724926 shutdown_connections 2024-09-12T00:56:30.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2699724926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518101420 0x7ff51810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2699724926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518100a80 0x7ff518100ee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2699724926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518106a80 0x7ff518106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2699724926 >> 172.21.15.16:0/2699724926 conn(0x7ff5180fc7a0 msgr2=0x7ff5180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2699724926 shutdown_connections 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2699724926 wait complete. 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.921+0000 7ff51d9b3640 1 Processor -- start 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 -- start start 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518100a80 0x7ff5180fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518106a80 0x7ff5180fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff518110e40 con 0x7ff518101420 2024-09-12T00:56:30.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff518110cc0 con 0x7ff518106a80 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff51d9b3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff518110fc0 con 0x7ff518100a80 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39594/0 (socket says 172.21.15.16:39594) 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 -- 172.21.15.16:0/2446094813 learned_addr learned my addr 172.21.15.16:0/2446094813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518100a80 0x7ff5180fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518106a80 0x7ff5180fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 -- 172.21.15.16:0/2446094813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518100a80 msgr2=0x7ff5180fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518100a80 0x7ff5180fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 -- 172.21.15.16:0/2446094813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518106a80 msgr2=0x7ff5180fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518106a80 0x7ff5180fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 -- 172.21.15.16:0/2446094813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5181a6910 con 0x7ff518101420 2024-09-12T00:56:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.922+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009230 tx=0x7ff50c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.923+0000 7ff51c9b1640 1 -- 172.21.15.16:0/2446094813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c03f030 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.923+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5181a6ba0 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.923+0000 7ff51c9b1640 1 -- 172.21.15.16:0/2446094813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c003ad0 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.923+0000 7ff51c9b1640 1 -- 172.21.15.16:0/2446094813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c035070 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.923+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5181a7080 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.925+0000 7ff51c9b1640 1 -- 172.21.15.16:0/2446094813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7ff50c042030 con 0x7ff518101420 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.925+0000 7ff51fd8c740 1 Processor -- start 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 -- start start 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2d9d0 0x565158e2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2e2f0 0x565158e38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565158d94090 con 0x565158e38bb0 2024-09-12T00:56:30.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565158d3ba70 con 0x565158e2e2f0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565158d90050 con 0x565158e2d9d0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39600/0 (socket says 172.21.15.16:39600) 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 -- 172.21.15.16:0/3095548088 learned_addr learned my addr 172.21.15.16:0/3095548088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff516ffd640 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2e2f0 0x565158e38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5177fe640 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2d9d0 0x565158e2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 -- 172.21.15.16:0/3095548088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2d9d0 msgr2=0x565158e2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2d9d0 0x565158e2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 -- 172.21.15.16:0/3095548088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2e2f0 msgr2=0x565158e38670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2e2f0 0x565158e38670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff5167fc640 1 -- 172.21.15.16:0/3095548088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565158d48130 con 0x565158e38bb0 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.926+0000 7ff51c9b1640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 0x7ff5000a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 0x7ff5000a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff5167fc640 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ed2c0 tx=0x7ff50c0f3e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97764138cb916c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff51c9b1640 1 -- 172.21.15.16:0/2446094813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff50c0efbb0 con 0x7ff518101420 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff4fdffb640 1 -- 172.21.15.16:0/3095548088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c049030 con 0x565158e38bb0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff4fdffb640 1 -- 172.21.15.16:0/3095548088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c0bc680 con 0x565158e38bb0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.927+0000 7ff4fdffb640 1 -- 172.21.15.16:0/3095548088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0bbac0 con 0x565158e38bb0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- 172.21.15.16:0/3095548088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 msgr2=0x565158e3afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 0x7ff5000a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff510005fd0 tx=0x7ff510009600 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ed2c0 tx=0x7ff50c0f3e80 comp rx=0 tx=0).stop 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- 172.21.15.16:0/3095548088 shutdown_connections 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e3afa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2e2f0 0x565158e38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/3095548088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2d9d0 0x565158e2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- 172.21.15.16:0/3095548088 >> 172.21.15.16:0/3095548088 conn(0x565158e29f60 msgr2=0x565158e2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- 172.21.15.16:0/3095548088 shutdown_connections 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- 172.21.15.16:0/3095548088 wait complete. 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 -- start start 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.928+0000 7ff51fd8c740 1 Processor -- start 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e35c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565158e3b540 con 0x565158e2e2f0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565158d4ef20 con 0x565158e2d9d0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565158e3b6c0 con 0x565158e38bb0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff51fd8c740 10 client.0 _ll_register_callbacks cb 0x565158d18030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff516ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff516ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:57758/0 (socket says 172.21.15.16:57758) 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39602/0 (socket says 172.21.15.16:39602) 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff516ffd640 1 -- 172.21.15.16:0/2424023574 learned_addr learned my addr 172.21.15.16:0/2424023574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e35c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 -- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 msgr2=0x565158e35c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e35c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 -- 172.21.15.16:0/2424023574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 msgr2=0x565158e38370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 -- 172.21.15.16:0/2424023574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565158b052b0 con 0x565158e2e2f0 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0f3460 tx=0x7ff50c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8576f3fe7495ac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff4f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:30.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c049070 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.929+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff50c0b85f0 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff50c0bc680 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff50c0bb8e0 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565158e3b540 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c0d6070 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff50c0bc1e0 con 0x565158e2e2f0 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 10 client.15492 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 20 client.15492 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158e49050 0x565158e4a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 4 client.15492 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.930+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565158da1600 con 0x565158e49050 2024-09-12T00:56:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.931+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158e49050 0x565158e4a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.931+0000 7ff4f6ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:30.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.931+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158e49050 0x565158e4a4c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8be336c7b4ed1ad server_cookie=56848c5b5624c44b in_seq=0 out_seq=0 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2424023574 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565158da1600 con 0x565158e49050 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff4f6ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 client.15492 shutdown 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 msgr2=0x565158e356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0f3460 tx=0x7ff50c009260 comp rx=0 tx=0).stop 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 shutdown_connections 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e35c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158e49050 0x565158e4a4c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e356c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2424023574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e38370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff4f6ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 >> 172.21.15.16:0/2424023574 conn(0x565158e29f60 msgr2=0x565158e2fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 shutdown_connections 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2424023574 wait complete. 2024-09-12T00:56:30.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.932+0000 7ff51fd8c740 20 client.15492 trim_cache size 0 max 16384 2024-09-12T00:56:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 Processor -- start 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 -- start start 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e30cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 0x565158e32940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565158e48800 con 0x565158e38bb0 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565158e48ed0 con 0x565158e2d9d0 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565158e495a0 con 0x565158e2e2f0 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.950+0000 7ff5177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39616/0 (socket says 172.21.15.16:39616) 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 -- 172.21.15.16:0/2425400942 learned_addr learned my addr 172.21.15.16:0/2425400942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e30cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 0x565158e32940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 -- 172.21.15.16:0/2425400942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 msgr2=0x565158e32940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 0x565158e32940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 -- 172.21.15.16:0/2425400942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 msgr2=0x565158e30cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e30cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 -- 172.21.15.16:0/2425400942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565158e386e0 con 0x565158e38bb0 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 0x565158e32940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff510012880 tx=0x7ff510014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff3c04e37de8717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff4fd7fa640 1 -- 172.21.15.16:0/2425400942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100209b0 con 0x565158e38bb0 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff4fd7fa640 1 -- 172.21.15.16:0/2425400942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510003e50 con 0x565158e38bb0 2024-09-12T00:56:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.951+0000 7ff4fd7fa640 1 -- 172.21.15.16:0/2425400942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510004790 con 0x565158e38bb0 2024-09-12T00:56:30.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2425400942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 msgr2=0x565158e32e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff510012880 tx=0x7ff510014000 comp rx=0 tx=0).stop 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2425400942 shutdown_connections 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e38bb0 0x565158e32e80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e2e2f0 0x565158e32940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2425400942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e30cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2425400942 >> 172.21.15.16:0/2425400942 conn(0x565158e29f60 msgr2=0x565158e305c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2425400942 shutdown_connections 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2425400942 wait complete. 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 -- start start 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.952+0000 7ff51fd8c740 1 Processor -- start 2024-09-12T00:56:30.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565158e4a340 con 0x565158e2e2f0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565158e49c70 con 0x565158e2d9d0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565158e4aa10 con 0x565158e38bb0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff51fd8c740 10 client.0 _ll_register_callbacks cb 0x7ff50800c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39624/0 (socket says 172.21.15.16:39624) 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 -- 172.21.15.16:0/2072818070 learned_addr learned my addr 172.21.15.16:0/2072818070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 -- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 msgr2=0x565158e32430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e32430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 -- 172.21.15.16:0/2072818070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 msgr2=0x565158e319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 -- 172.21.15.16:0/2072818070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565158e51870 con 0x565158e2e2f0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5177fe640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff5167fc640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0f4010 tx=0x7ff50c0f3e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4287d2ab3c11b145 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.953+0000 7ff4f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c049070 con 0x565158e2e2f0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff50c031bd0 con 0x565158e2e2f0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510003e50 con 0x565158e2e2f0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff50c0bbf80 con 0x565158e2e2f0 2024-09-12T00:56:30.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565158e51870 con 0x565158e2e2f0 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510003e50 con 0x565158e2e2f0 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff50c0c4760 con 0x565158e2e2f0 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 10 client.15498 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 20 client.15498 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158eb7440 0x565158eb6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 4 client.15498 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.954+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "status"}) -- 0x565158d9bb10 con 0x565158eb7440 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158eb7440 0x565158eb6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff516ffd640 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158eb7440 0x565158eb6150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=753127de879e2245 server_cookie=7897c42f8544c82c in_seq=0 out_seq=0 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff4f6ffd640 10 client.15498 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2072818070 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x565158d9bb10 con 0x565158eb7440 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff4f6ffd640 10 client.15498 handle_command_reply: tid=0 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 client.15498 shutdown 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 msgr2=0x565158e31ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0f4010 tx=0x7ff50c0f3e80 comp rx=0 tx=0).stop 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 shutdown_connections 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565158e38bb0 0x565158e32430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x565158eb7440 0x565158eb6150 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565158e2e2f0 0x565158e31ef0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.955+0000 7ff51fd8c740 1 --2- 172.21.15.16:0/2072818070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565158e2d9d0 0x565158e319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 >> 172.21.15.16:0/2072818070 conn(0x565158e29f60 msgr2=0x565158e33f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff4f6ffd640 0 client.15498 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 shutdown_connections 2024-09-12T00:56:30.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51fd8c740 1 -- 172.21.15.16:0/2072818070 wait complete. 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51fd8c740 20 client.15498 trim_cache size 0 max 16384 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 3, 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24481, 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:56:30.960 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 136.148124795, 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 174.33960861099999 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 msgr2=0x7ff5000a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 0x7ff5000a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff510005fd0 tx=0x7ff510009600 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 msgr2=0x7ff5180fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009230 tx=0x7ff50c004270 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 shutdown_connections 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff5000a2e10 0x7ff5000a52d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff518106a80 0x7ff5180fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff518101420 0x7ff5180fd270 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 --2- 172.21.15.16:0/2446094813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff518100a80 0x7ff5180fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 >> 172.21.15.16:0/2446094813 conn(0x7ff5180fc7a0 msgr2=0x7ff51806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 shutdown_connections 2024-09-12T00:56:30.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:30.956+0000 7ff51d9b3640 1 -- 172.21.15.16:0/2446094813 wait complete. 2024-09-12T00:56:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:30 smithi016 ceph-mon[30972]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T00:56:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:30 smithi099 ceph-mon[35654]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T00:56:31.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:30 smithi159 ceph-mon[35566]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T00:56:31.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T00:56:31.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.386+0000 7f14f47dc640 1 -- 172.21.15.16:0/2460988948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec103660 msgr2=0x7f14ec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.386+0000 7f14f47dc640 1 --2- 172.21.15.16:0/2460988948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec103660 0x7f14ec10bb50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc02a480 comp rx=0 tx=0).stop 2024-09-12T00:56:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 -- 172.21.15.16:0/2460988948 shutdown_connections 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- 172.21.15.16:0/2460988948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 0x7f14ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- 172.21.15.16:0/2460988948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec103660 0x7f14ec10bb50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- 172.21.15.16:0/2460988948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec102d40 0x7f14ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 -- 172.21.15.16:0/2460988948 >> 172.21.15.16:0/2460988948 conn(0x7f14ec0fc7a0 msgr2=0x7f14ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 -- 172.21.15.16:0/2460988948 shutdown_connections 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 -- 172.21.15.16:0/2460988948 wait complete. 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 Processor -- start 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 -- start start 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec103660 0x7f14ec198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.387+0000 7f14f47dc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 0x7f14ec19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f47dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec111f70 con 0x7f14ec102d40 2024-09-12T00:56:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f47dc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec111df0 con 0x7f14ec10c170 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f47dc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec1120f0 con 0x7f14ec103660 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39650/0 (socket says 172.21.15.16:39650) 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 -- 172.21.15.16:0/989442858 learned_addr learned my addr 172.21.15.16:0/989442858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f1d77640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec103660 0x7f14ec198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2d79640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 0x7f14ec19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 -- 172.21.15.16:0/989442858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec103660 msgr2=0x7f14ec198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec103660 0x7f14ec198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 -- 172.21.15.16:0/989442858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 msgr2=0x7f14ec19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 0x7f14ec19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 -- 172.21.15.16:0/989442858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec19d400 con 0x7f14ec102d40 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f2578640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d770 tx=0x7f14e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14e37fe640 1 -- 172.21.15.16:0/989442858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8017070 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec19d6f0 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14e37fe640 1 -- 172.21.15.16:0/989442858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e8005500 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14e37fe640 1 -- 172.21.15.16:0/989442858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8004e00 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.388+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec1a4fd0 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.391+0000 7f14e37fe640 1 -- 172.21.15.16:0/989442858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f14e8004fa0 con 0x7f14ec102d40 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.391+0000 7f14f6bb5740 1 Processor -- start 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.391+0000 7f14f6bb5740 1 -- start start 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb994da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bb98a5090 con 0x564bb994dfc0 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bb984ca70 con 0x564bb993e8d0 2024-09-12T00:56:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bb98a1050 con 0x564bb993f1f0 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f1d77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:57806/0 (socket says 172.21.15.16:57806) 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f1d77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39654/0 (socket says 172.21.15.16:39654) 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 -- 172.21.15.16:0/313111926 learned_addr learned my addr 172.21.15.16:0/313111926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2578640 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb994da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 -- 172.21.15.16:0/313111926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 msgr2=0x564bb994da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb994da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 -- 172.21.15.16:0/313111926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 msgr2=0x564bb99503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14e37fe640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 0x7f14d80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 -- 172.21.15.16:0/313111926 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb9859130 con 0x564bb993e8d0 2024-09-12T00:56:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f1d77640 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 0x7f14d80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14e37fe640 1 -- 172.21.15.16:0/989442858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f14e80cf100 con 0x7f14ec102d40 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.392+0000 7f14f2d79640 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009f40 tx=0x7f14e4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e1b1347ceee2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14e0ff9640 1 -- 172.21.15.16:0/313111926 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4003d70 con 0x564bb993e8d0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14e0ff9640 1 -- 172.21.15.16:0/313111926 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40119e0 con 0x564bb993e8d0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14e0ff9640 1 -- 172.21.15.16:0/313111926 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e401ea60 con 0x564bb993e8d0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14f6bb5740 1 -- 172.21.15.16:0/313111926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 msgr2=0x564bb993ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14f2d79640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 0x7f14d80a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009fd0 tx=0x7f14e4020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.393+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009f40 tx=0x7f14e4010860 comp rx=0 tx=0).stop 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- 172.21.15.16:0/313111926 shutdown_connections 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb99503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb994da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/313111926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb993ecb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- 172.21.15.16:0/313111926 >> 172.21.15.16:0/313111926 conn(0x564bb993af60 msgr2=0x564bb993b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- 172.21.15.16:0/313111926 shutdown_connections 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- 172.21.15.16:0/313111926 wait complete. 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- start start 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 Processor -- start 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993e8d0 0x564bb994d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993f1f0 0x564bb994ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bb9950bf0 con 0x564bb994dfc0 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bb985ff20 con 0x564bb993f1f0 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bb9950d70 con 0x564bb993e8d0 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.394+0000 7f14f6bb5740 10 client.0 _ll_register_callbacks cb 0x564bb9829030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:32.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39660/0 (socket says 172.21.15.16:39660) 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 -- 172.21.15.16:0/2192405635 learned_addr learned my addr 172.21.15.16:0/2192405635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f1d77640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993f1f0 0x564bb994ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2578640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993e8d0 0x564bb994d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 -- 172.21.15.16:0/2192405635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993e8d0 msgr2=0x564bb994d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993e8d0 0x564bb994d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 -- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993f1f0 msgr2=0x564bb994ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993f1f0 0x564bb994ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 -- 172.21.15.16:0/2192405635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb96162b0 con 0x564bb994dfc0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f2d79640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f14e4010d60 tx=0x7f14e401e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e00478170aca26b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4003d70 con 0x564bb994dfc0 2024-09-12T00:56:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f14e401ed90 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e40119e0 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.395+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14e4011b90 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bb995c600 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e40119e0 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14e4005050 con 0x564bb994dfc0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 10 client.15510 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 20 client.15510 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb995e840 0x564bb995fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 4 client.15510 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb98b2600 con 0x564bb995e840 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.396+0000 7f14f2578640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb995e840 0x564bb995fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.397+0000 7f14d5ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.397+0000 7f14f2578640 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb995e840 0x564bb995fcb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31798d08804ed401 server_cookie=831d24d55047f69a in_seq=0 out_seq=0 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14d5ffb640 1 -- 172.21.15.16:0/2192405635 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bb98b2600 con 0x564bb995e840 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14d5ffb640 10 client.15510 handle_command_reply: tid=0 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 client.15510 shutdown 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 msgr2=0x564bb994b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f14e4010d60 tx=0x7f14e401e420 comp rx=0 tx=0).stop 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 shutdown_connections 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb995e840 0x564bb995fcb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb994dfc0 0x564bb994b1c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993f1f0 0x564bb994ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14d5ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/2192405635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993e8d0 0x564bb994d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 >> 172.21.15.16:0/2192405635 conn(0x564bb993af60 msgr2=0x564bb9940cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 shutdown_connections 2024-09-12T00:56:32.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 1 -- 172.21.15.16:0/2192405635 wait complete. 2024-09-12T00:56:32.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.398+0000 7f14f6bb5740 20 client.15510 trim_cache size 0 max 16384 2024-09-12T00:56:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 Processor -- start 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 -- start start 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb995f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb994dfc0 0x564bb9945a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bb995dff0 con 0x564bb993f1f0 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bb995e6c0 con 0x564bb993e8d0 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f6bb5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bb995ed90 con 0x564bb994dfc0 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f1d77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f1d77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39664/0 (socket says 172.21.15.16:39664) 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f1d77640 1 -- 172.21.15.16:0/1894728224 learned_addr learned my addr 172.21.15.16:0/1894728224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f2578640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb995f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.416+0000 7f14f2d79640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb994dfc0 0x564bb9945a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 -- 172.21.15.16:0/1894728224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb994dfc0 msgr2=0x564bb9945a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb994dfc0 0x564bb9945a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 -- 172.21.15.16:0/1894728224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 msgr2=0x564bb995f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb995f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 -- 172.21.15.16:0/1894728224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb993bbf0 con 0x564bb993f1f0 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f2578640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb995f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ee63d49fffe534 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14d7fff640 1 -- 172.21.15.16:0/1894728224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc035040 con 0x564bb993f1f0 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14d7fff640 1 -- 172.21.15.16:0/1894728224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14dc031810 con 0x564bb993f1f0 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.417+0000 7f14d7fff640 1 -- 172.21.15.16:0/1894728224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0047a0 con 0x564bb993f1f0 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1894728224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 msgr2=0x564bb9940b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc031660 comp rx=0 tx=0).stop 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1894728224 shutdown_connections 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb994dfc0 0x564bb9945a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993f1f0 0x564bb9940b60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1894728224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb993e8d0 0x564bb995f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1894728224 >> 172.21.15.16:0/1894728224 conn(0x564bb993af60 msgr2=0x564bb9940450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1894728224 shutdown_connections 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1894728224 wait complete. 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 -- start start 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 Processor -- start 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb9961790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f6bb5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb994dfc0 0x564bb9961cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.418+0000 7f14f2578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39680/0 (socket says 172.21.15.16:39680) 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 -- 172.21.15.16:0/1034681821 learned_addr learned my addr 172.21.15.16:0/1034681821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bb995fb30 con 0x564bb993e8d0 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bb995f460 con 0x564bb994dfc0 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bb9960200 con 0x564bb993f1f0 2024-09-12T00:56:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 10 client.0 _ll_register_callbacks cb 0x7f14e40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb9961790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2d79640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb994dfc0 0x564bb9961cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 -- 172.21.15.16:0/1034681821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 msgr2=0x564bb9961790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb9961790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 -- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb994dfc0 msgr2=0x564bb9961cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb994dfc0 0x564bb9961cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb994b530 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f2578640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d740 tx=0x7f14e8097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1e83849e212d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8020040 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f14e8009cc0 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc031810 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14e809bb00 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.419+0000 7f14f6bb5740 10 client.15516 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bb9966c80 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14dc031810 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14e8097a70 con 0x564bb993e8d0 2024-09-12T00:56:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 10 client.15516 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 20 client.15516 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb99cd750 0x564bb99cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 4 client.15516 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "status"}) -- 0x564bb98acb10 con 0x564bb99cd750 2024-09-12T00:56:32.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.420+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb99cd750 0x564bb99cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f1d77640 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb99cd750 0x564bb99cc460 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af2971ae5729e8ce server_cookie=e24b31d8e441ec56 in_seq=0 out_seq=0 2024-09-12T00:56:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14d5ffb640 10 client.15516 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14d5ffb640 1 -- 172.21.15.16:0/1034681821 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564bb98acb10 con 0x564bb99cd750 2024-09-12T00:56:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14d5ffb640 10 client.15516 handle_command_reply: tid=0 2024-09-12T00:56:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 client.15516 shutdown 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 msgr2=0x564bb9968460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d740 tx=0x7f14e8097040 comp rx=0 tx=0).stop 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 shutdown_connections 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564bb99cd750 0x564bb99cc460 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bb994dfc0 0x564bb9961cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bb993f1f0 0x564bb9961790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14d5ffb640 0 client.15516 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 --2- 172.21.15.16:0/1034681821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bb993e8d0 0x564bb9968460 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 >> 172.21.15.16:0/1034681821 conn(0x564bb993af60 msgr2=0x564bb993a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 shutdown_connections 2024-09-12T00:56:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 1 -- 172.21.15.16:0/1034681821 wait complete. 2024-09-12T00:56:32.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.421+0000 7f14f6bb5740 20 client.15516 trim_cache size 0 max 16384 2024-09-12T00:56:32.429 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24482, 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:56:32.430 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 138.623501049, 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 158.919295737 2024-09-12T00:56:32.431 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 msgr2=0x7f14d80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 0x7f14d80a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009fd0 tx=0x7f14e4020000 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 msgr2=0x7f14ec198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d770 tx=0x7f14e800dc30 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 shutdown_connections 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14d80a2e10 0x7f14d80a52d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14ec10c170 0x7f14ec19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14ec103660 0x7f14ec198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 --2- 172.21.15.16:0/989442858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ec102d40 0x7f14ec198450 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:32.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 >> 172.21.15.16:0/989442858 conn(0x7f14ec0fc7a0 msgr2=0x7f14ec0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:32.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 shutdown_connections 2024-09-12T00:56:32.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:32.422+0000 7f14f47dc640 1 -- 172.21.15.16:0/989442858 wait complete. 2024-09-12T00:56:33.106 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:32 smithi016 ceph-mon[30972]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-12T00:56:33.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T00:56:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:32 smithi099 ceph-mon[35654]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-12T00:56:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:32 smithi159 ceph-mon[35566]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-12T00:56:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.893+0000 7fa426521640 1 -- 172.21.15.16:0/3806425329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420106f10 msgr2=0x7fa42010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.893+0000 7fa426521640 1 --2- 172.21.15.16:0/3806425329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420106f10 0x7fa42010ea90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa418009ca0 tx=0x7fa41802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 -- 172.21.15.16:0/3806425329 shutdown_connections 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 --2- 172.21.15.16:0/3806425329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420106f10 0x7fa42010ea90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 --2- 172.21.15.16:0/3806425329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420102790 0x7fa4201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 --2- 172.21.15.16:0/3806425329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 0x7fa4201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 -- 172.21.15.16:0/3806425329 >> 172.21.15.16:0/3806425329 conn(0x7fa4200fd8c0 msgr2=0x7fa4200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 -- 172.21.15.16:0/3806425329 shutdown_connections 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 -- 172.21.15.16:0/3806425329 wait complete. 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 Processor -- start 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.894+0000 7fa426521640 1 -- start start 2024-09-12T00:56:33.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 0x7fa4201983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420106f10 0x7fa42019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa420111f70 con 0x7fa420102790 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa420111df0 con 0x7fa420101de0 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa426521640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4201120f0 con 0x7fa420106f10 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39702/0 (socket says 172.21.15.16:39702) 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 -- 172.21.15.16:0/3905675352 learned_addr learned my addr 172.21.15.16:0/3905675352 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa424abe640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420106f10 0x7fa42019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa41ffff640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 0x7fa4201983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 -- 172.21.15.16:0/3905675352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420106f10 msgr2=0x7fa42019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420106f10 0x7fa42019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 -- 172.21.15.16:0/3905675352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 msgr2=0x7fa4201983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 0x7fa4201983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 -- 172.21.15.16:0/3905675352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42019d360 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.895+0000 7fa417fff640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa40800d770 tx=0x7fa40800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.896+0000 7fa41dffb640 1 -- 172.21.15.16:0/3905675352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408017070 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.896+0000 7fa41dffb640 1 -- 172.21.15.16:0/3905675352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa408005500 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.896+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42019d650 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.896+0000 7fa41dffb640 1 -- 172.21.15.16:0/3905675352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408004e00 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.896+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4201a4f30 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.898+0000 7fa41dffb640 1 -- 172.21.15.16:0/3905675352 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fa408004fa0 con 0x7fa420102790 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.898+0000 7fa4288fa740 1 Processor -- start 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.898+0000 7fa4288fa740 1 -- start start 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.898+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5df1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5dfefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4288fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5d58090 con 0x55b8c5df22f0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4288fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5cffa70 con 0x55b8c5df19d0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4288fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5d54050 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39706/0 (socket says 172.21.15.16:39706) 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 -- 172.21.15.16:0/1757956068 learned_addr learned my addr 172.21.15.16:0/1757956068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41dffb640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 0x7fa40c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa424abe640 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5df1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa417fff640 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5dfefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa424abe640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 0x7fa40c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 -- 172.21.15.16:0/1757956068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 msgr2=0x55b8c5dfefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5dfefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 -- 172.21.15.16:0/1757956068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 msgr2=0x55b8c5df1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5df1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 -- 172.21.15.16:0/1757956068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c5d0c130 con 0x55b8c5df22f0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41dffb640 1 -- 172.21.15.16:0/3905675352 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa4080cf150 con 0x7fa420102790 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa41000bcb0 tx=0x7fa41000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1632bf74e2a48f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.899+0000 7fa4167fc640 1 -- 172.21.15.16:0/1757956068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410015070 con 0x55b8c5df22f0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa424abe640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 0x7fa40c0a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa418004af0 tx=0x7fa41802a8c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4167fc640 1 -- 172.21.15.16:0/1757956068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41001a940 con 0x55b8c5df22f0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4167fc640 1 -- 172.21.15.16:0/1757956068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410011a40 con 0x55b8c5df22f0 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 -- 172.21.15.16:0/1757956068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 msgr2=0x55b8c5dfc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa41000bcb0 tx=0x7fa41000f730 comp rx=0 tx=0).stop 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 -- 172.21.15.16:0/1757956068 shutdown_connections 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5dfefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dfc670 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1757956068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5df1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 -- 172.21.15.16:0/1757956068 >> 172.21.15.16:0/1757956068 conn(0x55b8c5dedf60 msgr2=0x55b8c5dee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 -- 172.21.15.16:0/1757956068 shutdown_connections 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.900+0000 7fa4288fa740 1 -- 172.21.15.16:0/1757956068 wait complete. 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 -- start start 2024-09-12T00:56:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 Processor -- start 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5df9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5dff780 con 0x55b8c5df19d0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5d12f20 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5dff900 con 0x55b8c5df22f0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa4288fa740 10 client.0 _ll_register_callbacks cb 0x55b8c5cdc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39718/0 (socket says 172.21.15.16:39718) 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.901+0000 7fa41ffff640 1 -- 172.21.15.16:0/1357685996 learned_addr learned my addr 172.21.15.16:0/1357685996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa417fff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5df9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa424abe640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 -- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 msgr2=0x55b8c5df9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5df9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 -- 172.21.15.16:0/1357685996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 msgr2=0x55b8c5df9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 -- 172.21.15.16:0/1357685996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c5ac92b0 con 0x55b8c5df19d0 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa424abe640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa410011780 tx=0x7fa41000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4269395d0d41ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa3ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410028060 con 0x55b8c5df19d0 2024-09-12T00:56:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa41001a730 con 0x55b8c5df19d0 2024-09-12T00:56:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41001a940 con 0x55b8c5df19d0 2024-09-12T00:56:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa410015070 con 0x55b8c5df19d0 2024-09-12T00:56:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa4288fa740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.902+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8c5e0b070 con 0x55b8c5df19d0 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa41001a940 con 0x55b8c5df19d0 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa410011a90 con 0x55b8c5df19d0 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 10 client.15528 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 20 client.15528 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e0d2d0 0x55b8c5e0c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 4 client.15528 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8c5d65600 con 0x55b8c5e0d2d0 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa417fff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e0d2d0 0x55b8c5e0c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa3ff7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:56:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.903+0000 7fa417fff640 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e0d2d0 0x55b8c5e0c530 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8031ff8724384110 server_cookie=3cd54a38baa3612d in_seq=0 out_seq=0 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1357685996 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8c5d65600 con 0x55b8c5e0d2d0 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa3ff7fe640 10 client.15528 handle_command_reply: tid=0 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 client.15528 shutdown 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 msgr2=0x55b8c5dfc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa410011780 tx=0x7fa41000bfd0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 shutdown_connections 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e0d2d0 0x55b8c5e0c530 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5df9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa3ff7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1357685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df19d0 0x55b8c5dfc420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.904+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 >> 172.21.15.16:0/1357685996 conn(0x55b8c5dedf60 msgr2=0x55b8c5df3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.905+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 shutdown_connections 2024-09-12T00:56:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.905+0000 7fa4288fa740 1 -- 172.21.15.16:0/1357685996 wait complete. 2024-09-12T00:56:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.905+0000 7fa4288fa740 20 client.15528 trim_cache size 0 max 16384 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 Processor -- start 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 -- start start 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e0c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0ca80 con 0x55b8c5df22f0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0d150 con 0x55b8c5df19d0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa4288fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0d820 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39730/0 (socket says 172.21.15.16:39730) 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 -- 172.21.15.16:0/4157644202 learned_addr learned my addr 172.21.15.16:0/4157644202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa41ffff640 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e0c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 -- 172.21.15.16:0/4157644202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 msgr2=0x55b8c5df69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 -- 172.21.15.16:0/4157644202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 msgr2=0x55b8c5e0c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e0c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 -- 172.21.15.16:0/4157644202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c5dedd90 con 0x55b8c5df22f0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.924+0000 7fa417fff640 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4080970d0 tx=0x7fa4080d15a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fc628c6964277c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa415ffb640 1 -- 172.21.15.16:0/4157644202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408020040 con 0x55b8c5df22f0 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa415ffb640 1 -- 172.21.15.16:0/4157644202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40809b3f0 con 0x55b8c5df22f0 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa415ffb640 1 -- 172.21.15.16:0/4157644202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408097a60 con 0x55b8c5df22f0 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 -- 172.21.15.16:0/4157644202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 msgr2=0x55b8c5dee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4080970d0 tx=0x7fa4080d15a0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 -- 172.21.15.16:0/4157644202 shutdown_connections 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5df69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5df22f0 0x55b8c5dee340 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 --2- 172.21.15.16:0/4157644202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e0c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 -- 172.21.15.16:0/4157644202 >> 172.21.15.16:0/4157644202 conn(0x55b8c5dedf60 msgr2=0x55b8c5df4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.925+0000 7fa4288fa740 1 -- 172.21.15.16:0/4157644202 shutdown_connections 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 -- 172.21.15.16:0/4157644202 wait complete. 2024-09-12T00:56:33.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 -- start start 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 Processor -- start 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e16f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5e102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0e5c0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0def0 con 0x55b8c5df19d0 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8c5e0ec90 con 0x55b8c5df22f0 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa4288fa740 10 client.0 _ll_register_callbacks cb 0x7fa41000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa424abe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.926+0000 7fa424abe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39734/0 (socket says 172.21.15.16:39734) 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 -- 172.21.15.16:0/1149514830 learned_addr learned my addr 172.21.15.16:0/1149514830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:33.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa417fff640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5e102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e16f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 -- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 msgr2=0x55b8c5e102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5e102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 -- 172.21.15.16:0/1149514830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 msgr2=0x55b8c5e16f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e16f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 -- 172.21.15.16:0/1149514830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c5e10e00 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa424abe640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa418009ca0 tx=0x7fa418005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aedcca8d892ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa3ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41802f810 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa41802f9b0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa40809b3f0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa41802fbb0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa4288fa740 10 client.15534 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:33.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.927+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8c5e15870 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40809b3f0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4180305c0 con 0x55b8c5dfcbb0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 10 client.15534 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 10 client.15534 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 20 client.15534 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e7c3b0 0x55b8c5e7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 4 client.15534 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "status"}) -- 0x55b8c5d5fb10 con 0x55b8c5e7c3b0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.928+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e7c3b0 0x55b8c5e7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa3ff7fe640 10 client.15534 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa41ffff640 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e7c3b0 0x55b8c5e7b0c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac8eff258de8af88 server_cookie=1c1825eafa629566 in_seq=0 out_seq=0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa3ff7fe640 1 -- 172.21.15.16:0/1149514830 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b8c5d5fb10 con 0x55b8c5e7c3b0 2024-09-12T00:56:33.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa3ff7fe640 10 client.15534 handle_command_reply: tid=0 2024-09-12T00:56:33.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 client.15534 shutdown 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 msgr2=0x55b8c5e10810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa418009ca0 tx=0x7fa418005f30 comp rx=0 tx=0).stop 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 shutdown_connections 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b8c5dfcbb0 0x55b8c5e10810 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8c5df22f0 0x55b8c5e102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b8c5e7c3b0 0x55b8c5e7b0c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa3ff7fe640 0 client.15534 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 --2- 172.21.15.16:0/1149514830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8c5df19d0 0x55b8c5e16f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 >> 172.21.15.16:0/1149514830 conn(0x55b8c5dedf60 msgr2=0x55b8c5e07b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 shutdown_connections 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 1 -- 172.21.15.16:0/1149514830 wait complete. 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.929+0000 7fa4288fa740 20 client.15534 trim_cache size 0 max 16384 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-12T00:56:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24487, 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 141.13799626700001, 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 174.781924057 2024-09-12T00:56:33.936 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 msgr2=0x7fa40c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 0x7fa40c0a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa418004af0 tx=0x7fa41802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 msgr2=0x7fa4201988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa40800d770 tx=0x7fa40800dc30 comp rx=0 tx=0).stop 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 shutdown_connections 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa40c0a2e10 0x7fa40c0a52d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa420106f10 0x7fa42019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa420102790 0x7fa4201988f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 --2- 172.21.15.16:0/3905675352 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa420101de0 0x7fa4201983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 >> 172.21.15.16:0/3905675352 conn(0x7fa4200fd8c0 msgr2=0x7fa4200fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 shutdown_connections 2024-09-12T00:56:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:33.930+0000 7fa426521640 1 -- 172.21.15.16:0/3905675352 wait complete. 2024-09-12T00:56:34.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T00:56:34.875 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:35.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:34 smithi016 ceph-mon[30972]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:35.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:34 smithi099 ceph-mon[35654]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:34 smithi159 ceph-mon[35566]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- 172.21.15.16:0/159685078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 msgr2=0x7fe990103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 --2- 172.21.15.16:0/159685078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe990103110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009930 tx=0x7fe98c02a440 comp rx=0 tx=0).stop 2024-09-12T00:56:35.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- 172.21.15.16:0/159685078 shutdown_connections 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 --2- 172.21.15.16:0/159685078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 0x7fe99010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 --2- 172.21.15.16:0/159685078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 --2- 172.21.15.16:0/159685078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe990103110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- 172.21.15.16:0/159685078 >> 172.21.15.16:0/159685078 conn(0x7fe9900fc7d0 msgr2=0x7fe9900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- 172.21.15.16:0/159685078 shutdown_connections 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- 172.21.15.16:0/159685078 wait complete. 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 Processor -- start 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.352+0000 7fe997f52640 1 -- start start 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 0x7fe99019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901120e0 con 0x7fe990102d30 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe990111f60 con 0x7fe9901082a0 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe997f52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe990112260 con 0x7fe9901036e0 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39760/0 (socket says 172.21.15.16:39760) 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 -- 172.21.15.16:0/2793271427 learned_addr learned my addr 172.21.15.16:0/2793271427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe9964ef640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 0x7fe99019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe9954ed640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 -- 172.21.15.16:0/2793271427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 msgr2=0x7fe990198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 -- 172.21.15.16:0/2793271427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 msgr2=0x7fe99019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 0x7fe99019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 -- 172.21.15.16:0/2793271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019d550 con 0x7fe990102d30 2024-09-12T00:56:35.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe9954ed640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe995cee640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009a50 tx=0x7fe98c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe986ffd640 1 -- 172.21.15.16:0/2793271427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c03f030 con 0x7fe990102d30 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.353+0000 7fe986ffd640 1 -- 172.21.15.16:0/2793271427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c003ce0 con 0x7fe990102d30 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.354+0000 7fe986ffd640 1 -- 172.21.15.16:0/2793271427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c035070 con 0x7fe990102d30 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.354+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99019d7e0 con 0x7fe990102d30 2024-09-12T00:56:35.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.354+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9901a5120 con 0x7fe990102d30 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "434eab66-70a0-11ef-bcea-c7b262605968", 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24505, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 150.62456436900001, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 28, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 170.95949807900001 2024-09-12T00:56:35.405 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.356+0000 7fe986ffd640 1 -- 172.21.15.16:0/2793271427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fe98c042030 con 0x7fe990102d30 2024-09-12T00:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.356+0000 7fe99a32b740 1 Processor -- start 2024-09-12T00:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 -- start start 2024-09-12T00:56:35.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe375cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe3873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbe2dc090 con 0x557dbe376210 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dbe283a70 con 0x557dbe384fc0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe99a32b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dbe2d8050 con 0x557dbe3758f0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39768/0 (socket says 172.21.15.16:39768) 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 -- 172.21.15.16:0/1813463178 learned_addr learned my addr 172.21.15.16:0/1813463178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe986ffd640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 0x7fe97c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe9964ef640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe375cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe9954ed640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe3873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe9964ef640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 0x7fe97c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 -- 172.21.15.16:0/1813463178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 msgr2=0x557dbe375cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe375cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 -- 172.21.15.16:0/1813463178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 msgr2=0x557dbe3873b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.357+0000 7fe995cee640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe3873b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe986ffd640 1 -- 172.21.15.16:0/2793271427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe98c0efc50 con 0x7fe990102d30 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe995cee640 1 -- 172.21.15.16:0/1813463178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe290130 con 0x557dbe376210 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe9954ed640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe3873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe995cee640 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0bbcc0 tx=0x7fe98c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e3dc68eda7c975 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe9964ef640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 0x7fe97c0a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe988007fd0 tx=0x7fe988007520 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe97bfff640 1 -- 172.21.15.16:0/1813463178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c049070 con 0x557dbe376210 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe97bfff640 1 -- 172.21.15.16:0/1813463178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c0bc6c0 con 0x557dbe376210 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.358+0000 7fe97bfff640 1 -- 172.21.15.16:0/1813463178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c03b460 con 0x557dbe376210 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- 172.21.15.16:0/1813463178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 msgr2=0x557dbe384a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0bbcc0 tx=0x7fe98c043000 comp rx=0 tx=0).stop 2024-09-12T00:56:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- 172.21.15.16:0/1813463178 shutdown_connections 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe3873b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe384a80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 --2- 172.21.15.16:0/1813463178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe375cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- 172.21.15.16:0/1813463178 >> 172.21.15.16:0/1813463178 conn(0x557dbe371f60 msgr2=0x557dbe372340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- 172.21.15.16:0/1813463178 shutdown_connections 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- 172.21.15.16:0/1813463178 wait complete. 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 -- start start 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.359+0000 7fe99a32b740 1 Processor -- start 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe384890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe382120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbe2dc090 con 0x557dbe376210 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dbe296f20 con 0x557dbe384fc0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dbe3878f0 con 0x557dbe3758f0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe99a32b740 10 client.0 _ll_register_callbacks cb 0x557dbe260030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39772/0 (socket says 172.21.15.16:39772) 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 -- 172.21.15.16:0/741881879 learned_addr learned my addr 172.21.15.16:0/741881879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9964ef640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe382120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe995cee640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe384890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 -- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 msgr2=0x557dbe384890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe384890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 -- 172.21.15.16:0/741881879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 msgr2=0x557dbe382120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe382120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 -- 172.21.15.16:0/741881879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe04d2b0 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9954ed640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe98000bf40 tx=0x7fe98000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3554e87be15a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.360+0000 7fe9797fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980018020 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe980014660 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98c0bc6c0 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe980023050 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dbe393160 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c0bc6c0 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9800139d0 con 0x557dbe376210 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 10 client.15546 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 20 client.15546 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe395820 0x557dbe396c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 4 client.15546 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.361+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbe2e9600 con 0x557dbe395820 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.362+0000 7fe995cee640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe395820 0x557dbe396c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.362+0000 7fe9797fa640 10 client.15546 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.362+0000 7fe995cee640 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe395820 0x557dbe396c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5631860e41eb8ab4 server_cookie=65de3401a6d97dab in_seq=0 out_seq=0 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe9797fa640 1 -- 172.21.15.16:0/741881879 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dbe2e9600 con 0x557dbe395820 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe9797fa640 10 client.15546 handle_command_reply: tid=0 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 client.15546 shutdown 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 msgr2=0x557dbe381be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe98000bf40 tx=0x7fe98000f5b0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 shutdown_connections 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe395820 0x557dbe396c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe382120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe381be0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 --2- 172.21.15.16:0/741881879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe384890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe9797fa640 0 client.15546 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 >> 172.21.15.16:0/741881879 conn(0x557dbe371f60 msgr2=0x557dbe377bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 shutdown_connections 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 1 -- 172.21.15.16:0/741881879 wait complete. 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.363+0000 7fe99a32b740 20 client.15546 trim_cache size 0 max 16384 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 Processor -- start 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 -- start start 2024-09-12T00:56:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe3758f0 0x557dbe396c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe384fc0 0x557dbe39f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbe394fd0 con 0x557dbe376210 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dbe3956a0 con 0x557dbe3758f0 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.381+0000 7fe99a32b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dbe395d70 con 0x557dbe384fc0 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39774/0 (socket says 172.21.15.16:39774) 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 -- 172.21.15.16:0/3653387258 learned_addr learned my addr 172.21.15.16:0/3653387258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:35.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 -- 172.21.15.16:0/3653387258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe384fc0 msgr2=0x557dbe39f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe384fc0 0x557dbe39f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 -- 172.21.15.16:0/3653387258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe3758f0 msgr2=0x557dbe396c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe995cee640 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe3758f0 0x557dbe396c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe3758f0 0x557dbe396c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 -- 172.21.15.16:0/3653387258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe382c20 con 0x557dbe376210 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe9954ed640 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe980000c00 tx=0x7fe98000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bf8f3487aa99e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe97b7fe640 1 -- 172.21.15.16:0/3653387258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980018030 con 0x557dbe376210 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe97b7fe640 1 -- 172.21.15.16:0/3653387258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9800138a0 con 0x557dbe376210 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.382+0000 7fe97b7fe640 1 -- 172.21.15.16:0/3653387258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980014c50 con 0x557dbe376210 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- 172.21.15.16:0/3653387258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 msgr2=0x557dbe39ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe980000c00 tx=0x7fe98000b3e0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- 172.21.15.16:0/3653387258 shutdown_connections 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe384fc0 0x557dbe39f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39ef70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 --2- 172.21.15.16:0/3653387258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe3758f0 0x557dbe396c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- 172.21.15.16:0/3653387258 >> 172.21.15.16:0/3653387258 conn(0x557dbe371f60 msgr2=0x557dbe380cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- 172.21.15.16:0/3653387258 shutdown_connections 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- 172.21.15.16:0/3653387258 wait complete. 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 -- start start 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.383+0000 7fe99a32b740 1 Processor -- start 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe39f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe37e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557dbe396b10 con 0x557dbe376210 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dbe396440 con 0x557dbe384fc0 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dbe3971e0 con 0x557dbe3758f0 2024-09-12T00:56:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe99a32b740 10 client.0 _ll_register_callbacks cb 0x7fe98000b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39778/0 (socket says 172.21.15.16:39778) 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 -- 172.21.15.16:0/2447006949 learned_addr learned my addr 172.21.15.16:0/2447006949 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe995cee640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe39f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 -- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 msgr2=0x557dbe39f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe39f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9964ef640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe37e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 -- 172.21.15.16:0/2447006949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 msgr2=0x557dbe37e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe37e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 -- 172.21.15.16:0/2447006949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe37ede0 con 0x557dbe376210 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9954ed640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe980013fb0 tx=0x7fe98000bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb100497a4fe9d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.384+0000 7fe9797fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980018040 con 0x557dbe376210 2024-09-12T00:56:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe9800133c0 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9800138a0 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe980013640 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 10 client.15552 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dbe39ead0 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9800138a0 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe980029c80 con 0x557dbe376210 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 10 client.15552 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 10 client.15552 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 20 client.15552 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe404180 0x557dbe402e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 4 client.15552 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.385+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "status"}) -- 0x557dbe2e3b10 con 0x557dbe404180 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe995cee640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe404180 0x557dbe402e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:35.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe9797fa640 10 client.15552 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe995cee640 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe404180 0x557dbe402e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881d3cd8ce98b73e server_cookie=20219ba47c5c8da9 in_seq=0 out_seq=0 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe9797fa640 1 -- 172.21.15.16:0/2447006949 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557dbe2e3b10 con 0x557dbe404180 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe9797fa640 10 client.15552 handle_command_reply: tid=0 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 client.15552 shutdown 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 msgr2=0x557dbe39f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe980013fb0 tx=0x7fe98000bb60 comp rx=0 tx=0).stop 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 shutdown_connections 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557dbe404180 0x557dbe402e90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dbe384fc0 0x557dbe37e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dbe376210 0x557dbe39f710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 --2- 172.21.15.16:0/2447006949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dbe3758f0 0x557dbe39f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe9797fa640 0 client.15552 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.386+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 >> 172.21.15.16:0/2447006949 conn(0x557dbe371f60 msgr2=0x557dbe377340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 shutdown_connections 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe99a32b740 1 -- 172.21.15.16:0/2447006949 wait complete. 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe99a32b740 20 client.15552 trim_cache size 0 max 16384 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 msgr2=0x7fe97c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 0x7fe97c0a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe988007fd0 tx=0x7fe988007520 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 msgr2=0x7fe9901985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009a50 tx=0x7fe98c004270 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 shutdown_connections 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2e10 0x7fe97c0a52d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9901082a0 0x7fe99019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9901036e0 0x7fe990198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 --2- 172.21.15.16:0/2793271427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe990102d30 0x7fe9901985a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 >> 172.21.15.16:0/2793271427 conn(0x7fe9900fc7d0 msgr2=0x7fe990105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 shutdown_connections 2024-09-12T00:56:35.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:35.387+0000 7fe997f52640 1 -- 172.21.15.16:0/2793271427 wait complete. 2024-09-12T00:56:36.157 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T00:56:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:36 smithi099 ceph-mon[35654]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:36 smithi159 ceph-mon[35566]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:36 smithi016 ceph-mon[30972]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:38 smithi099 ceph-mon[35654]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:38 smithi159 ceph-mon[35566]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:38 smithi016 ceph-mon[30972]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-12T00:56:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:40 smithi099 ceph-mon[35654]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:40 smithi159 ceph-mon[35566]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:41.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:40 smithi016 ceph-mon[30972]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-12T00:56:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:42 smithi099 ceph-mon[35654]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T00:56:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:42 smithi159 ceph-mon[35566]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T00:56:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:42 smithi016 ceph-mon[30972]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T00:56:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:44 smithi099 ceph-mon[35654]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:44 smithi159 ceph-mon[35566]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:44 smithi016 ceph-mon[30972]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:45.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:56:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 -- 172.21.15.16:0/1017425113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8102570 msgr2=0x7f4ff81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/1017425113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8102570 0x7f4ff81029b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009900 tx=0x7f4fe402a480 comp rx=0 tx=0).stop 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 -- 172.21.15.16:0/1017425113 shutdown_connections 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/1017425113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102ef0 0x7f4ff810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/1017425113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8102570 0x7f4ff81029b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/1017425113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff8100e20 0x7f4ff8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 -- 172.21.15.16:0/1017425113 >> 172.21.15.16:0/1017425113 conn(0x7f4ff80fc7d0 msgr2=0x7f4ff80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 -- 172.21.15.16:0/1017425113 shutdown_connections 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 -- 172.21.15.16:0/1017425113 wait complete. 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.591+0000 7f4ffeb64640 1 Processor -- start 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 -- start start 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 0x7f4ff80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff8102ef0 0x7f4ff80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8110f80 con 0x7f4ff8100e20 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8110e00 con 0x7f4ff8102570 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffeb64640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8111100 con 0x7f4ff8102ef0 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47060/0 (socket says 172.21.15.16:47060) 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 -- 172.21.15.16:0/2282641781 learned_addr learned my addr 172.21.15.16:0/2282641781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 0x7f4ff80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 -- 172.21.15.16:0/2282641781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff8102ef0 msgr2=0x7f4ff80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff8102ef0 0x7f4ff80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 -- 172.21.15.16:0/2282641781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 msgr2=0x7f4ff80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 0x7f4ff80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 -- 172.21.15.16:0/2282641781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81a6940 con 0x7f4ff8100e20 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 0x7f4ff80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.592+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00d770 tx=0x7f4fec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.593+0000 7f4ff5ffb640 1 -- 172.21.15.16:0/2282641781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec017070 con 0x7f4ff8100e20 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.593+0000 7f4ff5ffb640 1 -- 172.21.15.16:0/2282641781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fec005500 con 0x7f4ff8100e20 2024-09-12T00:56:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.593+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81a6c30 con 0x7f4ff8100e20 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.593+0000 7f4ff5ffb640 1 -- 172.21.15.16:0/2282641781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec004e00 con 0x7f4ff8100e20 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.593+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a7170 con 0x7f4ff8100e20 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.595+0000 7f4ff5ffb640 1 -- 172.21.15.16:0/2282641781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f4fec004fa0 con 0x7f4ff8100e20 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ff5ffb640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 0x7f4fe80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 Processor -- start 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 0x7f4fe80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 -- start start 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ff5ffb640 1 -- 172.21.15.16:0/2282641781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4fec0cf1f0 con 0x7f4ff8100e20 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956a0dc0 0x559c956ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 0x559c956ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c956062e0 con 0x559c956a04a0 2024-09-12T00:56:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c955adce0 con 0x559c956ab680 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f5000f3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c956022a0 con 0x559c956a0dc0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47070/0 (socket says 172.21.15.16:47070) 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.596+0000 7f4ffc900640 1 -- 172.21.15.16:0/3618418907 learned_addr learned my addr 172.21.15.16:0/3618418907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956a0dc0 0x559c956ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffd101640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 0x559c956ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 0x7f4fe80a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000c00 tx=0x7f4fe4004100 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 -- 172.21.15.16:0/3618418907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956a0dc0 msgr2=0x559c956ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956a0dc0 0x559c956ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 -- 172.21.15.16:0/3618418907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 msgr2=0x559c956ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 0x559c956ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 -- 172.21.15.16:0/3618418907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c95592440 con 0x559c956a04a0 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffd101640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 0x559c956ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fec09b190 tx=0x7f4fec004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feb032a7c8e0c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/3618418907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec020070 con 0x559c956a04a0 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/3618418907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fec09f840 con 0x559c956a04a0 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.597+0000 7f4fe2ffd640 1 -- 172.21.15.16:0/3618418907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0cca70 con 0x559c956a04a0 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 -- 172.21.15.16:0/3618418907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 msgr2=0x559c956a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fec09b190 tx=0x7f4fec004780 comp rx=0 tx=0).stop 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 -- 172.21.15.16:0/3618418907 shutdown_connections 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956ab680 0x559c956ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956a0dc0 0x559c956ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3618418907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956a0880 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 -- 172.21.15.16:0/3618418907 >> 172.21.15.16:0/3618418907 conn(0x559c9569ca30 msgr2=0x559c9569ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 -- 172.21.15.16:0/3618418907 shutdown_connections 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.598+0000 7f5000f3d740 1 -- 172.21.15.16:0/3618418907 wait complete. 2024-09-12T00:56:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 -- start start 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 Processor -- start 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a0dc0 0x559c956a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559c956ae250 con 0x559c956a04a0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c955c1050 con 0x559c956a0dc0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.599+0000 7f5000f3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c956ae3d0 con 0x559c956ab680 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47086/0 (socket says 172.21.15.16:47086) 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 -- 172.21.15.16:0/2132603964 learned_addr learned my addr 172.21.15.16:0/2132603964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f5000f3d740 10 client.0 _ll_register_callbacks cb 0x559c95601160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffd101640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a0dc0 0x559c956a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 -- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 msgr2=0x559c956a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 -- 172.21.15.16:0/2132603964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a0dc0 msgr2=0x559c956a81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a0dc0 0x559c956a81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 -- 172.21.15.16:0/2132603964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c953773c0 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4ffc900640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0a3bd0 tx=0x7f4fec0cc730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c35d8c197030d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0b8020 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4fec097b70 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.600+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec09f840 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fec084050 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c956b9ba0 con 0x559c956a04a0 2024-09-12T00:56:46.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fec09f840 con 0x559c956a04a0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec09a8e0 con 0x559c956a04a0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 20 client.15564 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c956bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 4 client.15564 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c95613850 con 0x559c956baef0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.601+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c956bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.602+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c956bd2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e17bb53d7b882cd server_cookie=4c4a1b067941783b in_seq=0 out_seq=0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.602+0000 7f4fd7fff640 10 client.15564 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f4fd7fff640 1 -- 172.21.15.16:0/2132603964 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c95613850 con 0x559c956baef0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f4fd7fff640 10 client.15564 handle_command_reply: tid=0 2024-09-12T00:56:46.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 client.15564 shutdown 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 msgr2=0x559c956aaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0a3bd0 tx=0x7f4fec0cc730 comp rx=0 tx=0).stop 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 shutdown_connections 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c956bd2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f4fd7fff640 0 client.15564 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a0dc0 0x559c956a81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 --2- 172.21.15.16:0/2132603964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a04a0 0x559c956aaf40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.603+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 >> 172.21.15.16:0/2132603964 conn(0x559c9569ca30 msgr2=0x559c956a2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.604+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 shutdown_connections 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.604+0000 7f5000f3d740 1 -- 172.21.15.16:0/2132603964 wait complete. 2024-09-12T00:56:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.604+0000 7f5000f3d740 20 client.15564 trim_cache size 0 max 16384 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 Processor -- start 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 -- start start 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81af9e0 con 0x559c956a0dc0 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b1520 con 0x559c956a04a0 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f5000f3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b00b0 con 0x559c956ab680 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.621+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47098/0 (socket says 172.21.15.16:47098) 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 -- 172.21.15.16:0/3575883526 learned_addr learned my addr 172.21.15.16:0/3575883526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ffd101640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 -- 172.21.15.16:0/3575883526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 msgr2=0x559c956c2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956c2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 -- 172.21.15.16:0/3575883526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 msgr2=0x559c956b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 -- 172.21.15.16:0/3575883526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c956a0bf0 con 0x559c956a0dc0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009900 tx=0x7f4fe403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a21c85e377410c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4fe27fc640 1 -- 172.21.15.16:0/3575883526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4058070 con 0x559c956a0dc0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4fe27fc640 1 -- 172.21.15.16:0/3575883526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4041040 con 0x559c956a0dc0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.622+0000 7f4fe27fc640 1 -- 172.21.15.16:0/3575883526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4035050 con 0x559c956a0dc0 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 -- 172.21.15.16:0/3575883526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 msgr2=0x559c956c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009900 tx=0x7f4fe403d000 comp rx=0 tx=0).stop 2024-09-12T00:56:46.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 -- 172.21.15.16:0/3575883526 shutdown_connections 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956c28e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3575883526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 -- 172.21.15.16:0/3575883526 >> 172.21.15.16:0/3575883526 conn(0x559c9569ca30 msgr2=0x559c956a72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 -- 172.21.15.16:0/3575883526 shutdown_connections 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.623+0000 7f5000f3d740 1 -- 172.21.15.16:0/3575883526 wait complete. 2024-09-12T00:56:46.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.624+0000 7f5000f3d740 1 -- start start 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.624+0000 7f5000f3d740 1 Processor -- start 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b0e50 con 0x559c956a0dc0 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81b0780 con 0x559c956a04a0 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81ad9c0 con 0x559c956ab680 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f5000f3d740 10 client.0 _ll_register_callbacks cb 0x7f4fe4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47106/0 (socket says 172.21.15.16:47106) 2024-09-12T00:56:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ff7fff640 1 -- 172.21.15.16:0/3142194279 learned_addr learned my addr 172.21.15.16:0/3142194279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffd101640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 -- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 msgr2=0x559c956b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 -- 172.21.15.16:0/3142194279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 msgr2=0x559c956b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 -- 172.21.15.16:0/3142194279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c956b13a0 con 0x559c956a04a0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4ffc900640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b3750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x559c956a9630 tx=0x7f4fec0cc8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2e499e9f0e048c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.625+0000 7f4fd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec0b8070 con 0x559c956a04a0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4fec09a8e0 con 0x559c956a04a0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe4041040 con 0x559c956a04a0 2024-09-12T00:56:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4fec020070 con 0x559c956a04a0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 10 client.25154 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c957253d0 con 0x559c956a04a0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4041040 con 0x559c956a04a0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4fec0b4260 con 0x559c956a04a0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 10 client.25154 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 10 client.25154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 20 client.25154 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.626+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c95727850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.627+0000 7f5000f3d740 4 client.25154 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.627+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c9560dd60 con 0x559c956baef0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.627+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c95727850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.627+0000 7f4ff7fff640 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c95727850 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e4ffa9a106a2cd server_cookie=31b32d718e0ced96 in_seq=0 out_seq=0 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.627+0000 7f4fd7fff640 10 client.25154 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f4fd7fff640 1 -- 172.21.15.16:0/3142194279 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c9560dd60 con 0x559c956baef0 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f4fd7fff640 10 client.25154 handle_command_reply: tid=0 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 client.25154 shutdown 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 msgr2=0x559c956b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b3750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x559c956a9630 tx=0x7f4fec0cc8d0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 shutdown_connections 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559c956baef0 0x559c95727850 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c956ab680 0x559c956b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c956a0dc0 0x559c956b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 --2- 172.21.15.16:0/3142194279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c956a04a0 0x559c956b3750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 >> 172.21.15.16:0/3142194279 conn(0x559c9569ca30 msgr2=0x559c9569b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.628+0000 7f4fd7fff640 0 client.25154 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 shutdown_connections 2024-09-12T00:56:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f5000f3d740 1 -- 172.21.15.16:0/3142194279 wait complete. 2024-09-12T00:56:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f5000f3d740 20 client.25154 trim_cache size 0 max 16384 2024-09-12T00:56:46.641 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:46.641 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T00:56:46.641 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T00:56:46.641 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 msgr2=0x7f4fe80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 0x7f4fe80a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000c00 tx=0x7f4fe4004100 comp rx=0 tx=0).stop 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 msgr2=0x7f4ff80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.629+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00d770 tx=0x7f4fec00dc30 comp rx=0 tx=0).stop 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 shutdown_connections 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4fe80a2e90 0x7f4fe80a5350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ff8102ef0 0x7f4ff80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff8102570 0x7f4ff80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 --2- 172.21.15.16:0/2282641781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff8100e20 0x7f4ff80fcc80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 >> 172.21.15.16:0/2282641781 conn(0x7f4ff80fc7d0 msgr2=0x7f4ff8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 shutdown_connections 2024-09-12T00:56:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:46.630+0000 7f4ffeb64640 1 -- 172.21.15.16:0/2282641781 wait complete. 2024-09-12T00:56:47.217 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:46 smithi016 ceph-mon[30972]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:46 smithi099 ceph-mon[35654]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:47.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:56:47.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:46 smithi159 ceph-mon[35566]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:47.611 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 -- 172.21.15.16:0/1918480591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102ef0 msgr2=0x7f6dec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 --2- 172.21.15.16:0/1918480591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102ef0 0x7f6dec10da40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009ca0 tx=0x7f6de402a970 comp rx=0 tx=0).stop 2024-09-12T00:56:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 -- 172.21.15.16:0/1918480591 shutdown_connections 2024-09-12T00:56:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 --2- 172.21.15.16:0/1918480591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102ef0 0x7f6dec10da40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 --2- 172.21.15.16:0/1918480591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec102570 0x7f6dec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 --2- 172.21.15.16:0/1918480591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec100e20 0x7f6dec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 -- 172.21.15.16:0/1918480591 >> 172.21.15.16:0/1918480591 conn(0x7f6dec0fc7d0 msgr2=0x7f6dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 -- 172.21.15.16:0/1918480591 shutdown_connections 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.164+0000 7f6df2a76640 1 -- 172.21.15.16:0/1918480591 wait complete. 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 Processor -- start 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 -- start start 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec100e20 0x7f6dec1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec102ef0 0x7f6dec19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec110fc0 con 0x7f6dec102570 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec110e40 con 0x7f6dec100e20 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df2a76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111140 con 0x7f6dec102ef0 2024-09-12T00:56:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6debfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6debfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47116/0 (socket says 172.21.15.16:47116) 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6debfff640 1 -- 172.21.15.16:0/3813532010 learned_addr learned my addr 172.21.15.16:0/3813532010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.165+0000 7f6df0812640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec100e20 0x7f6dec1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6df1013640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec102ef0 0x7f6dec19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 -- 172.21.15.16:0/3813532010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec102ef0 msgr2=0x7f6dec19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec102ef0 0x7f6dec19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 -- 172.21.15.16:0/3813532010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec100e20 msgr2=0x7f6dec1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec100e20 0x7f6dec1a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 -- 172.21.15.16:0/3813532010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec19ace0 con 0x7f6dec102570 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6debfff640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d950 tx=0x7f6dd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8017070 con 0x7f6dec102570 2024-09-12T00:56:48.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8009c30 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.166+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800f6b0 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.167+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec19afd0 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.167+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1a72c0 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.168+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec101200 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.169+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f6dd80109d0 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.170+0000 7f6de9ffb640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 0x7f6ddc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.170+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6dd80cecd0 con 0x7f6dec102570 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.170+0000 7f6df0812640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 0x7f6ddc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.171+0000 7f6df0812640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 0x7f6ddc0a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6de0006fd0 tx=0x7f6de00090e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.174+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6dd8009de0 con 0x7f6dec102570 2024-09-12T00:56:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.393+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dec1a7490 con 0x7f6dec102570 2024-09-12T00:56:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.394+0000 7f6de9ffb640 1 -- 172.21.15.16:0/3813532010 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f6dd809b670 con 0x7f6dec102570 2024-09-12T00:56:48.392 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:48.392 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 msgr2=0x7f6ddc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 0x7f6ddc0a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6de0006fd0 tx=0x7f6de00090e0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 msgr2=0x7f6dec1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d950 tx=0x7f6dd800de10 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 shutdown_connections 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6ddc0a2e10 0x7f6ddc0a52d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dec102ef0 0x7f6dec19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec102570 0x7f6dec1a0930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 --2- 172.21.15.16:0/3813532010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dec100e20 0x7f6dec1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 >> 172.21.15.16:0/3813532010 conn(0x7f6dec0fc7d0 msgr2=0x7f6dec06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 shutdown_connections 2024-09-12T00:56:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:48.396+0000 7f6df2a76640 1 -- 172.21.15.16:0/3813532010 wait complete. 2024-09-12T00:56:49.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:48 smithi016 ceph-mon[30972]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:49.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:48 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3813532010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:49.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T00:56:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:48 smithi099 ceph-mon[35654]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:48 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3813532010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:48 smithi159 ceph-mon[35566]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:48 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3813532010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 -- 172.21.15.16:0/983717353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102ef0 msgr2=0x7f865c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 --2- 172.21.15.16:0/983717353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102ef0 0x7f865c10da40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8654009ca0 tx=0x7f865402a950 comp rx=0 tx=0).stop 2024-09-12T00:56:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 -- 172.21.15.16:0/983717353 shutdown_connections 2024-09-12T00:56:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 --2- 172.21.15.16:0/983717353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102ef0 0x7f865c10da40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 --2- 172.21.15.16:0/983717353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102570 0x7f865c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 --2- 172.21.15.16:0/983717353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 0x7f865c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 -- 172.21.15.16:0/983717353 >> 172.21.15.16:0/983717353 conn(0x7f865c0fc7d0 msgr2=0x7f865c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 -- 172.21.15.16:0/983717353 shutdown_connections 2024-09-12T00:56:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.906+0000 7f8662a8d640 1 -- 172.21.15.16:0/983717353 wait complete. 2024-09-12T00:56:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 Processor -- start 2024-09-12T00:56:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 -- start start 2024-09-12T00:56:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 0x7f865c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102ef0 0x7f865c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f865c110f80 con 0x7f865c102570 2024-09-12T00:56:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f865c110e00 con 0x7f865c102ef0 2024-09-12T00:56:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8662a8d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f865c111100 con 0x7f865c100e20 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f865bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f865bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46564/0 (socket says 172.21.15.16:46564) 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f865bfff640 1 -- 172.21.15.16:0/1946386877 learned_addr learned my addr 172.21.15.16:0/1946386877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f8660829640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 0x7f865c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.907+0000 7f866102a640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102ef0 0x7f865c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 -- 172.21.15.16:0/1946386877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 msgr2=0x7f865c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 0x7f865c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 -- 172.21.15.16:0/1946386877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102ef0 msgr2=0x7f865c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102ef0 0x7f865c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 -- 172.21.15.16:0/1946386877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c19f600 con 0x7f865c102570 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f865bfff640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f864800c890 tx=0x7f864800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f8659ffb640 1 -- 172.21.15.16:0/1946386877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004220 con 0x7f865c102570 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c19f8f0 con 0x7f865c102570 2024-09-12T00:56:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f8659ffb640 1 -- 172.21.15.16:0/1946386877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86480043c0 con 0x7f865c102570 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f8659ffb640 1 -- 172.21.15.16:0/1946386877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648004a00 con 0x7f865c102570 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.908+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a71d0 con 0x7f865c102570 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.910+0000 7f8659ffb640 1 -- 172.21.15.16:0/1946386877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f86480079b0 con 0x7f865c102570 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 Processor -- start 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 -- start start 2024-09-12T00:56:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8659ffb640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 0x7f864c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56525a0232f0 con 0x56525a0beb40 2024-09-12T00:56:49.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565259fcacf0 con 0x56525a0c8370 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8664e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56525a01f2b0 con 0x56525a0bd5d0 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f865bfff640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 0x7f864c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f866102a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f866102a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46566/0 (socket says 172.21.15.16:46566) 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f866102a640 1 -- 172.21.15.16:0/644449503 learned_addr learned my addr 172.21.15.16:0/644449503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.911+0000 7f8660829640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f866102a640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8659ffb640 1 -- 172.21.15.16:0/1946386877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f86480cf3d0 con 0x7f865c102570 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 -- 172.21.15.16:0/644449503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 msgr2=0x56525a0ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 -- 172.21.15.16:0/644449503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 msgr2=0x56525a0bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 -- 172.21.15.16:0/644449503 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565259faf3a0 con 0x56525a0bd5d0 2024-09-12T00:56:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f866102a640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f865bfff640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 0x7f864c0a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86480cca30 tx=0x7f8648086040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8660829640 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0bda10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f865000ab40 tx=0x7f865000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0ebabb85d022190 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.912+0000 7f8646ffd640 1 -- 172.21.15.16:0/644449503 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650004020 con 0x56525a0bd5d0 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8646ffd640 1 -- 172.21.15.16:0/644449503 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86500041c0 con 0x56525a0bd5d0 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8646ffd640 1 -- 172.21.15.16:0/644449503 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650018680 con 0x56525a0bd5d0 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 -- 172.21.15.16:0/644449503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 msgr2=0x56525a0bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0bda10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f865000ab40 tx=0x7f865000f3c0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 -- 172.21.15.16:0/644449503 shutdown_connections 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0bda10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 --2- 172.21.15.16:0/644449503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 -- 172.21.15.16:0/644449503 >> 172.21.15.16:0/644449503 conn(0x56525a086310 msgr2=0x56525a0b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 -- 172.21.15.16:0/644449503 shutdown_connections 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.913+0000 7f8664e66740 1 -- 172.21.15.16:0/644449503 wait complete. 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 -- start start 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 Processor -- start 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56525a0cb070 con 0x56525a0beb40 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565259fde060 con 0x56525a0bd5d0 2024-09-12T00:56:49.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56525a0cb1f0 con 0x56525a0c8370 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8664e66740 10 client.0 _ll_register_callbacks cb 0x565259d318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46582/0 (socket says 172.21.15.16:46582) 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f8660829640 1 -- 172.21.15.16:0/1186046914 learned_addr learned my addr 172.21.15.16:0/1186046914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f866102a640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.914+0000 7f866102a640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 -- 172.21.15.16:0/1186046914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 msgr2=0x56525a0c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 -- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 msgr2=0x56525a0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 -- 172.21.15.16:0/1186046914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565259d943c0 con 0x56525a0beb40 2024-09-12T00:56:49.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8660829640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f865000a740 tx=0x7f8650003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e69697bfb8ac99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f863bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650027020 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86500041c0 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f8650003fc0 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8650032050 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8664e66740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56525a0d6a80 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.915+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86500041c0 con 0x56525a0beb40 2024-09-12T00:56:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8650004920 con 0x56525a0beb40 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 10 client.15588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 20 client.15588 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a0d7e90 0x56525a0da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 4 client.15588 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56525a030860 con 0x56525a0d7e90 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f866102a640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a0d7e90 0x56525a0da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f866102a640 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a0d7e90 0x56525a0da280 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=233af0d958495129 server_cookie=6342a69acc86e638 in_seq=0 out_seq=0 2024-09-12T00:56:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.916+0000 7f863bfff640 10 client.15588 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f863bfff640 1 -- 172.21.15.16:0/1186046914 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56525a030860 con 0x56525a0d7e90 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f863bfff640 10 client.15588 handle_command_reply: tid=0 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 client.15588 shutdown 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 msgr2=0x56525a0c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f865000a740 tx=0x7f8650003be0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 shutdown_connections 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a0d7e90 0x56525a0da280 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0c4f90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 --2- 172.21.15.16:0/1186046914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f863bfff640 0 client.15588 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.917+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 >> 172.21.15.16:0/1186046914 conn(0x56525a086310 msgr2=0x56525a0bf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.918+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 shutdown_connections 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.918+0000 7f8664e66740 1 -- 172.21.15.16:0/1186046914 wait complete. 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.918+0000 7f8664e66740 20 client.15588 trim_cache size 0 max 16384 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.931+0000 7f8664e66740 1 Processor -- start 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.931+0000 7f8664e66740 1 -- start start 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.931+0000 7f8664e66740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.931+0000 7f8664e66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8664e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8664e66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1afa30 con 0x56525a0beb40 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8664e66740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b1570 con 0x56525a0bd5d0 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8664e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b0100 con 0x56525a0c8370 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46592/0 (socket says 172.21.15.16:46592) 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 -- 172.21.15.16:0/601813865 learned_addr learned my addr 172.21.15.16:0/601813865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f866102a640 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f866102a640 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 -- 172.21.15.16:0/601813865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 msgr2=0x56525a0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 -- 172.21.15.16:0/601813865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 msgr2=0x56525a0d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 -- 172.21.15.16:0/601813865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525a0b9510 con 0x56525a0beb40 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f8660829640 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f865000f400 tx=0x7f865000f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5fbb759c9af569 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f86467fc640 1 -- 172.21.15.16:0/601813865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650027030 con 0x56525a0beb40 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f86467fc640 1 -- 172.21.15.16:0/601813865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865000a9c0 con 0x56525a0beb40 2024-09-12T00:56:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.932+0000 7f86467fc640 1 -- 172.21.15.16:0/601813865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650023890 con 0x56525a0beb40 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 -- 172.21.15.16:0/601813865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 msgr2=0x56525a0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f865000f400 tx=0x7f865000f9f0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 -- 172.21.15.16:0/601813865 shutdown_connections 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0c8370 0x56525a0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0b4ad0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 --2- 172.21.15.16:0/601813865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0bd5d0 0x56525a0d4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 -- 172.21.15.16:0/601813865 >> 172.21.15.16:0/601813865 conn(0x56525a086310 msgr2=0x56525a0c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 -- 172.21.15.16:0/601813865 shutdown_connections 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.933+0000 7f8664e66740 1 -- 172.21.15.16:0/601813865 wait complete. 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 -- start start 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 Processor -- start 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b0ea0 con 0x56525a0beb40 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b07d0 con 0x56525a0c8370 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1ada10 con 0x56525a0bd5d0 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8664e66740 10 client.0 _ll_register_callbacks cb 0x7f8654009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8660829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46606/0 (socket says 172.21.15.16:46606) 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f8660829640 1 -- 172.21.15.16:0/1688542169 learned_addr learned my addr 172.21.15.16:0/1688542169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f866102a640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.934+0000 7f866102a640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 -- 172.21.15.16:0/1688542169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 msgr2=0x56525a0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 -- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 msgr2=0x56525a0c17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0c17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 -- 172.21.15.16:0/1688542169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525a0c2020 con 0x56525a0beb40 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f866102a640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8660829640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8650010010 tx=0x7f865000a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39086db5cc949669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f863bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650027040 con 0x56525a0beb40 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f8650004870 con 0x56525a0beb40 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865000a9c0 con 0x56525a0beb40 2024-09-12T00:56:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86500041b0 con 0x56525a0beb40 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8664e66740 10 client.15594 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.935+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56525a142430 con 0x56525a0beb40 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f865000a9c0 con 0x56525a0beb40 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8650004fb0 con 0x56525a0beb40 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 10 client.15594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 20 client.15594 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a142b70 0x56525a144f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 4 client.15594 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:56:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56525a02ad70 con 0x56525a142b70 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.936+0000 7f866102a640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a142b70 0x56525a144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f863bfff640 10 client.15594 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f866102a640 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a142b70 0x56525a144f90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f2eac0865867598 server_cookie=f0ac235827dc0871 in_seq=0 out_seq=0 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f863bfff640 1 -- 172.21.15.16:0/1688542169 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56525a02ad70 con 0x56525a142b70 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f863bfff640 10 client.15594 handle_command_reply: tid=0 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 client.15594 shutdown 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 msgr2=0x56525a0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8650010010 tx=0x7f865000a820 comp rx=0 tx=0).stop 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 shutdown_connections 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56525a142b70 0x56525a144f90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56525a0c8370 0x56525a0c17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56525a0beb40 0x56525a0d0ba0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 --2- 172.21.15.16:0/1688542169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56525a0bd5d0 0x56525a0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f863bfff640 0 client.15594 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 >> 172.21.15.16:0/1688542169 conn(0x56525a086310 msgr2=0x56525a0b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 shutdown_connections 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 1 -- 172.21.15.16:0/1688542169 wait complete. 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.937+0000 7f8664e66740 20 client.15594 trim_cache size 0 max 16384 2024-09-12T00:56:49.962 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:56:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 msgr2=0x7f864c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 0x7f864c0a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f86480cca30 tx=0x7f8648086040 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 msgr2=0x7f865c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f864800c890 tx=0x7f864800cd50 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 shutdown_connections 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f864c0a2e10 0x7f864c0a52d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c102ef0 0x7f865c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865c102570 0x7f865c19ab90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 --2- 172.21.15.16:0/1946386877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f865c100e20 0x7f865c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 >> 172.21.15.16:0/1946386877 conn(0x7f865c0fc7d0 msgr2=0x7f865c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 shutdown_connections 2024-09-12T00:56:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:49.938+0000 7f8662a8d640 1 -- 172.21.15.16:0/1946386877 wait complete. 2024-09-12T00:56:51.158 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T00:56:51.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:56:51.183 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:50 smithi016 ceph-mon[30972]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:51.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:50 smithi099 ceph-mon[35654]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:50 smithi159 ceph-mon[35566]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:53.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:52 smithi099 ceph-mon[35654]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:53.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:52 smithi159 ceph-mon[35566]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:53.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:52 smithi016 ceph-mon[30972]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:54.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T00:56:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:54.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 -- 172.21.15.16:0/925083309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4102790 msgr2=0x7fa9f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:54.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/925083309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4102790 0x7fa9f41067e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02a410 comp rx=0 tx=0).stop 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 -- 172.21.15.16:0/925083309 shutdown_connections 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/925083309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4106f10 0x7fa9f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/925083309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4102790 0x7fa9f41067e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/925083309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 -- 172.21.15.16:0/925083309 >> 172.21.15.16:0/925083309 conn(0x7fa9f40fd8c0 msgr2=0x7fa9f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 -- 172.21.15.16:0/925083309 shutdown_connections 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.759+0000 7fa9f88d7640 1 -- 172.21.15.16:0/925083309 wait complete. 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 Processor -- start 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 -- start start 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4102790 0x7fa9f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:54.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4111f90 con 0x7fa9f4106f10 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4111e10 con 0x7fa9f4101de0 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.760+0000 7fa9f88d7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4112110 con 0x7fa9f4102790 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46636/0 (socket says 172.21.15.16:46636) 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 -- 172.21.15.16:0/377351478 learned_addr learned my addr 172.21.15.16:0/377351478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f1d9b640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4102790 0x7fa9f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f259c640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 -- 172.21.15.16:0/377351478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4102790 msgr2=0x7fa9f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4102790 0x7fa9f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 -- 172.21.15.16:0/377351478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 msgr2=0x7fa9f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 -- 172.21.15.16:0/377351478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41a6930 con 0x7fa9f4106f10 2024-09-12T00:56:54.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f259c640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f2d9d640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009fd0 tx=0x7fa9e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4011830 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4016070 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400f2e0 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.761+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a6c20 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.762+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a7130 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.763+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f41021c0 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.765+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fa9e40119d0 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.766+0000 7fa9eb7fe640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 0x7fa9e00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.766+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa9e40cf2a0 con 0x7fa9f4106f10 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.766+0000 7fa9f259c640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 0x7fa9e00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.766+0000 7fa9f259c640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 0x7fa9e00a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec005fd0 tx=0x7fa9ec009520 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:54.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:54.769+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9e409bdb0 con 0x7fa9f4106f10 2024-09-12T00:56:54.835 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.006+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9f402d7c0 con 0x7fa9f4106f10 2024-09-12T00:56:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.006+0000 7fa9eb7fe640 1 -- 172.21.15.16:0/377351478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa9e409b6b0 con 0x7fa9f4106f10 2024-09-12T00:56:55.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:56:55.006 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:56:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:56:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 msgr2=0x7fa9e00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 0x7fa9e00a5350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec005fd0 tx=0x7fa9ec009520 comp rx=0 tx=0).stop 2024-09-12T00:56:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 msgr2=0x7fa9f4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009fd0 tx=0x7fa9e400b8e0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 shutdown_connections 2024-09-12T00:56:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9e00a2e90 0x7fa9e00a5350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4106f10 0x7fa9f4074f70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f4102790 0x7fa9f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 --2- 172.21.15.16:0/377351478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9f4101de0 0x7fa9f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 >> 172.21.15.16:0/377351478 conn(0x7fa9f40fd8c0 msgr2=0x7fa9f40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 shutdown_connections 2024-09-12T00:56:55.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.009+0000 7fa9f88d7640 1 -- 172.21.15.16:0/377351478 wait complete. 2024-09-12T00:56:55.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:54 smithi099 ceph-mon[35654]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:54 smithi159 ceph-mon[35566]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:54 smithi016 ceph-mon[30972]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 -- 172.21.15.16:0/1043434655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 msgr2=0x7f4e881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/1043434655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e881021c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e7802a410 comp rx=0 tx=0).stop 2024-09-12T00:56:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 -- 172.21.15.16:0/1043434655 shutdown_connections 2024-09-12T00:56:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/1043434655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 0x7f4e8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/1043434655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 0x7f4e881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/1043434655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e881021c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 -- 172.21.15.16:0/1043434655 >> 172.21.15.16:0/1043434655 conn(0x7f4e880fd8c0 msgr2=0x7f4e880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 -- 172.21.15.16:0/1043434655 shutdown_connections 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.532+0000 7f4e8d42e640 1 -- 172.21.15.16:0/1043434655 wait complete. 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 Processor -- start 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 -- start start 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 0x7f4e88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 0x7f4e88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88111f90 con 0x7f4e88101de0 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88111e10 con 0x7f4e88106f10 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e8d42e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e88112110 con 0x7f4e88102790 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46646/0 (socket says 172.21.15.16:46646) 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.533+0000 7f4e86ffd640 1 -- 172.21.15.16:0/219609900 learned_addr learned my addr 172.21.15.16:0/219609900 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 0x7f4e88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e877fe640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 0x7f4e88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 -- 172.21.15.16:0/219609900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 msgr2=0x7f4e88074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 0x7f4e88074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 -- 172.21.15.16:0/219609900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 msgr2=0x7f4e88074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 0x7f4e88074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 -- 172.21.15.16:0/219609900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e881aaea0 con 0x7f4e88101de0 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4e78006010 tx=0x7f4e780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e84ff9640 1 -- 172.21.15.16:0/219609900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7803f030 con 0x7f4e88101de0 2024-09-12T00:56:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e881ab070 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e84ff9640 1 -- 172.21.15.16:0/219609900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78003ce0 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e84ff9640 1 -- 172.21.15.16:0/219609900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78035070 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.534+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e881ab600 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.537+0000 7f4e84ff9640 1 -- 172.21.15.16:0/219609900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f4e78042030 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.537+0000 7f4e8f807740 1 Processor -- start 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 -- start start 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e84ff9640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 0x7f4e740a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e877fe640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 0x7f4e740a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e84ff9640 1 -- 172.21.15.16:0/219609900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4e780efc20 con 0x7f4e88101de0 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b9ecab2f0 con 0x562b9ed50370 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b9ec52cf0 con 0x562b9ed46b40 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e8f807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b9eca72b0 con 0x562b9ed455d0 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed52760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46660/0 (socket says 172.21.15.16:46660) 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.538+0000 7f4e877fe640 1 -- 172.21.15.16:0/1031983673 learned_addr learned my addr 172.21.15.16:0/1031983673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e877fe640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 0x7f4e740a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005fd0 tx=0x7f4e80010740 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1031983673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 msgr2=0x562b9ed45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1031983673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 msgr2=0x562b9ed52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1031983673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9ec373a0 con 0x562b9ed46b40 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed45090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ed1c0 tx=0x7f4e780bbd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f436537a67cb34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e7d7fa640 1 -- 172.21.15.16:0/1031983673 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78049040 con 0x562b9ed46b40 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e7d7fa640 1 -- 172.21.15.16:0/1031983673 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7803b580 con 0x562b9ed46b40 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.539+0000 7f4e7d7fa640 1 -- 172.21.15.16:0/1031983673 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780c4c00 con 0x562b9ed46b40 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 -- 172.21.15.16:0/1031983673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 msgr2=0x562b9ed45090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed45090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ed1c0 tx=0x7f4e780bbd60 comp rx=0 tx=0).stop 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 -- 172.21.15.16:0/1031983673 shutdown_connections 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1031983673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed45090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 -- 172.21.15.16:0/1031983673 >> 172.21.15.16:0/1031983673 conn(0x562b9ed0e310 msgr2=0x562b9ed41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 -- 172.21.15.16:0/1031983673 shutdown_connections 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.540+0000 7f4e8f807740 1 -- 172.21.15.16:0/1031983673 wait complete. 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 -- start start 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 Processor -- start 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed4fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b9ed52e30 con 0x562b9ed50370 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b9ec66060 con 0x562b9ed46b40 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b9ed52fb0 con 0x562b9ed455d0 2024-09-12T00:56:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e8f807740 10 client.0 _ll_register_callbacks cb 0x562b9e9b98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46676/0 (socket says 172.21.15.16:46676) 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e877fe640 1 -- 172.21.15.16:0/2557655252 learned_addr learned my addr 172.21.15.16:0/2557655252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed4fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.541+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 -- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 msgr2=0x562b9ed4fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed4fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 -- 172.21.15.16:0/2557655252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 msgr2=0x562b9ed4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 -- 172.21.15.16:0/2557655252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9ea1c3c0 con 0x562b9ed50370 2024-09-12T00:56:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed4fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e877fe640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4e80013910 tx=0x7f4e80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92a4722c8a2964e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8001fdc0 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4e80027430 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b9ecb8860 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e80027630 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e8f807740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.542+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b9ed5e8b0 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b9ecb8860 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4e8000f040 con 0x562b9ed50370 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 10 client.15612 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 20 client.15612 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9ed62220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 4 client.15612 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9ecb2d70 con 0x562b9ed5fdc0 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.543+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9ed62220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.544+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9ed62220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4b4e5826dbf797c server_cookie=8b53c556a434a8f6 in_seq=0 out_seq=0 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.544+0000 7f4e6e7fc640 10 client.15612 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.544+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/2557655252 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b9ecb2d70 con 0x562b9ed5fdc0 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.544+0000 7f4e6e7fc640 10 client.15612 handle_command_reply: tid=0 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.544+0000 7f4e8f807740 1 client.15612 shutdown 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 msgr2=0x562b9ed4d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4e80013910 tx=0x7f4e80004ab0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 shutdown_connections 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9ed62220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed4d7d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e6e7fc640 0 client.15612 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2557655252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed455d0 0x562b9ed4fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 >> 172.21.15.16:0/2557655252 conn(0x562b9ed0e310 msgr2=0x562b9ed479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 shutdown_connections 2024-09-12T00:56:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 1 -- 172.21.15.16:0/2557655252 wait complete. 2024-09-12T00:56:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.545+0000 7f4e8f807740 20 client.15612 trim_cache size 0 max 16384 2024-09-12T00:56:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 Processor -- start 2024-09-12T00:56:55.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 -- start start 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 0x562b9ed67af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b3eb0 con 0x562b9ed455d0 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b59f0 con 0x562b9ed46b40 2024-09-12T00:56:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e8f807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b4580 con 0x562b9ed50370 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46688/0 (socket says 172.21.15.16:46688) 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e86ffd640 1 -- 172.21.15.16:0/2566727605 learned_addr learned my addr 172.21.15.16:0/2566727605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e877fe640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 0x562b9ed67af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.559+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 -- 172.21.15.16:0/2566727605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 msgr2=0x562b9ed67af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 0x562b9ed67af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 -- 172.21.15.16:0/2566727605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 msgr2=0x562b9ed4c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 -- 172.21.15.16:0/2566727605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9ed414f0 con 0x562b9ed455d0 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e877fe640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 0x562b9ed67af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c46d0 tx=0x7f4e78005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1707cfc32793724a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e7cff9640 1 -- 172.21.15.16:0/2566727605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78049040 con 0x562b9ed455d0 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e7cff9640 1 -- 172.21.15.16:0/2566727605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e780c4c00 con 0x562b9ed455d0 2024-09-12T00:56:55.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.560+0000 7f4e7cff9640 1 -- 172.21.15.16:0/2566727605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e780bc430 con 0x562b9ed455d0 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- 172.21.15.16:0/2566727605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 msgr2=0x562b9ed5bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4e780c46d0 tx=0x7f4e78005c90 comp rx=0 tx=0).stop 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- 172.21.15.16:0/2566727605 shutdown_connections 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed50370 0x562b9ed67af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed46b40 0x562b9ed4c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 --2- 172.21.15.16:0/2566727605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed455d0 0x562b9ed5bf40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- 172.21.15.16:0/2566727605 >> 172.21.15.16:0/2566727605 conn(0x562b9ed0e310 msgr2=0x562b9ed4b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- 172.21.15.16:0/2566727605 shutdown_connections 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- 172.21.15.16:0/2566727605 wait complete. 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 -- start start 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.561+0000 7f4e8f807740 1 Processor -- start 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed46b40 0x562b9ed588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b5320 con 0x562b9ed50370 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b4c50 con 0x562b9ed455d0 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b1e90 con 0x562b9ed46b40 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e8f807740 10 client.0 _ll_register_callbacks cb 0x7f4e80009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:44796/0 (socket says 172.21.15.16:44796) 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1494852664 learned_addr learned my addr 172.21.15.16:0/1494852664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46692/0 (socket says 172.21.15.16:46692) 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed46b40 0x562b9ed588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed46b40 msgr2=0x562b9ed588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed46b40 0x562b9ed588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1494852664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 msgr2=0x562b9ed53710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e86ffd640 1 -- 172.21.15.16:0/1494852664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9ed53f40 con 0x562b9ed455d0 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.562+0000 7f4e877fe640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e86ffd640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e7803d040 tx=0x7f4e7802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8aa2a4674d61d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78049070 con 0x562b9ed455d0 2024-09-12T00:56:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4e780b8740 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e780c4c00 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e780bbf90 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 10 client.25199 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b9edca640 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e780c4c00 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4e780dabd0 con 0x562b9ed455d0 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 10 client.25199 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 10 client.25199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 20 client.25199 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.563+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9edcca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.564+0000 7f4e8f807740 4 client.25199 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.564+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b9ec2f170 con 0x562b9ed5fdc0 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.564+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9edcca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.564+0000 7f4e7ffff640 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9edcca40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed903b5e7a164aa2 server_cookie=5f3783346e3132f1 in_seq=0 out_seq=0 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.564+0000 7f4e6e7fc640 10 client.25199 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e6e7fc640 1 -- 172.21.15.16:0/1494852664 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b9ec2f170 con 0x562b9ed5fdc0 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e6e7fc640 10 client.25199 handle_command_reply: tid=0 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 client.25199 shutdown 2024-09-12T00:56:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 msgr2=0x562b9ed58380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e7803d040 tx=0x7f4e7802a410 comp rx=0 tx=0).stop 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 shutdown_connections 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562b9ed5fdc0 0x562b9edcca40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b9ed50370 0x562b9ed53710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9ed46b40 0x562b9ed588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e6e7fc640 0 client.25199 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 --2- 172.21.15.16:0/1494852664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b9ed455d0 0x562b9ed58380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 >> 172.21.15.16:0/1494852664 conn(0x562b9ed0e310 msgr2=0x562b9ed4c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 shutdown_connections 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 1 -- 172.21.15.16:0/1494852664 wait complete. 2024-09-12T00:56:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.565+0000 7f4e8f807740 20 client.25199 trim_cache size 0 max 16384 2024-09-12T00:56:55.573 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 msgr2=0x7f4e740a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 0x7f4e740a5350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005fd0 tx=0x7f4e80010740 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 msgr2=0x7f4e880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4e78006010 tx=0x7f4e780042f0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 shutdown_connections 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4e740a2e90 0x7f4e740a5350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e88106f10 0x7f4e88074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e88102790 0x7f4e88074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 --2- 172.21.15.16:0/219609900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e88101de0 0x7f4e880744f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 >> 172.21.15.16:0/219609900 conn(0x7f4e880fd8c0 msgr2=0x7f4e880fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 shutdown_connections 2024-09-12T00:56:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:55.566+0000 7f4e8d42e640 1 -- 172.21.15.16:0/219609900 wait complete. 2024-09-12T00:56:56.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:55 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/377351478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:56.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:55 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/377351478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:56.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:55 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/377351478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:56:56.381 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T00:56:56.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T00:56:56.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:56:56.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:57.131 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:56 smithi016 ceph-mon[30972]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:56 smithi099 ceph-mon[35654]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:57.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:56 smithi159 ceph-mon[35566]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T00:56:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.528+0000 7fe48867e640 1 -- 172.21.15.16:0/1998316348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480102ef0 msgr2=0x7fe48010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.528+0000 7fe48867e640 1 --2- 172.21.15.16:0/1998316348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480102ef0 0x7fe48010da40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe478009900 tx=0x7fe47802a480 comp rx=0 tx=0).stop 2024-09-12T00:56:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 -- 172.21.15.16:0/1998316348 shutdown_connections 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 --2- 172.21.15.16:0/1998316348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480102ef0 0x7fe48010da40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 --2- 172.21.15.16:0/1998316348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 0x7fe4801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 --2- 172.21.15.16:0/1998316348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480100e20 0x7fe480101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 -- 172.21.15.16:0/1998316348 >> 172.21.15.16:0/1998316348 conn(0x7fe4800fc7d0 msgr2=0x7fe4800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 -- 172.21.15.16:0/1998316348 shutdown_connections 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.529+0000 7fe48867e640 1 -- 172.21.15.16:0/1998316348 wait complete. 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.530+0000 7fe48867e640 1 Processor -- start 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.530+0000 7fe48867e640 1 -- start start 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 0x7fe4801967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 0x7fe48019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010fe80 con 0x7fe480100e20 2024-09-12T00:56:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010fd00 con 0x7fe480102ef0 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48867e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe480110000 con 0x7fe480102570 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46722/0 (socket says 172.21.15.16:46722) 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 -- 172.21.15.16:0/825762882 learned_addr learned my addr 172.21.15.16:0/825762882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe485c19640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 0x7fe4801967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe486c1b640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 0x7fe48019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 -- 172.21.15.16:0/825762882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 msgr2=0x7fe4801967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 0x7fe4801967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 -- 172.21.15.16:0/825762882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 msgr2=0x7fe48019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 0x7fe48019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe48641a640 1 -- 172.21.15.16:0/825762882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48019b210 con 0x7fe480100e20 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.531+0000 7fe486c1b640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 0x7fe48019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:57.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe48641a640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe470009fa0 tx=0x7fe47000c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe47f7fe640 1 -- 172.21.15.16:0/825762882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470020070 con 0x7fe480100e20 2024-09-12T00:56:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe47f7fe640 1 -- 172.21.15.16:0/825762882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe470013c50 con 0x7fe480100e20 2024-09-12T00:56:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe47f7fe640 1 -- 172.21.15.16:0/825762882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47001bd70 con 0x7fe480100e20 2024-09-12T00:56:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48019b500 con 0x7fe480100e20 2024-09-12T00:56:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.532+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801a2de0 con 0x7fe480100e20 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe47f7fe640 1 -- 172.21.15.16:0/825762882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fe470023070 con 0x7fe480100e20 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 Processor -- start 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 -- start start 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2fa0 0x5585a12b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5585a12082a0 con 0x5585a12a2680 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585a11afb10 con 0x5585a12b6150 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.536+0000 7fe48aa57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a1204260 con 0x5585a12a2fa0 2024-09-12T00:56:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46734/0 (socket says 172.21.15.16:46734) 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 -- 172.21.15.16:0/4145382000 learned_addr learned my addr 172.21.15.16:0/4145382000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe485c19640 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 -- 172.21.15.16:0/4145382000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2fa0 msgr2=0x5585a12b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2fa0 0x5585a12b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 -- 172.21.15.16:0/4145382000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 msgr2=0x5585a12b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe486c1b640 1 -- 172.21.15.16:0/4145382000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a0f79b90 con 0x5585a12a2680 2024-09-12T00:56:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.537+0000 7fe485c19640 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe47f7fe640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 0x7fe4680a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe486c1b640 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe4780314e0 tx=0x7fe4780041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f1d3dc175336f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe48641a640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 0x7fe4680a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe47f7fe640 1 -- 172.21.15.16:0/825762882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe4700d35a0 con 0x7fe480100e20 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe47cff9640 1 -- 172.21.15.16:0/4145382000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478035030 con 0x5585a12a2680 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe47cff9640 1 -- 172.21.15.16:0/4145382000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478005520 con 0x5585a12a2680 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.538+0000 7fe47cff9640 1 -- 172.21.15.16:0/4145382000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478003850 con 0x5585a12a2680 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 -- 172.21.15.16:0/4145382000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 msgr2=0x5585a12a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48641a640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 0x7fe4680a53d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4700d0cb0 tx=0x7fe47009bc30 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe4780314e0 tx=0x7fe4780041a0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 -- 172.21.15.16:0/4145382000 shutdown_connections 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2fa0 0x5585a12b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 --2- 172.21.15.16:0/4145382000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12a2a60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.539+0000 7fe48aa57740 1 -- 172.21.15.16:0/4145382000 >> 172.21.15.16:0/4145382000 conn(0x5585a129ecf0 msgr2=0x5585a129f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.540+0000 7fe48aa57740 1 -- 172.21.15.16:0/4145382000 shutdown_connections 2024-09-12T00:56:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.540+0000 7fe48aa57740 1 -- 172.21.15.16:0/4145382000 wait complete. 2024-09-12T00:56:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.540+0000 7fe48aa57740 1 -- start start 2024-09-12T00:56:57.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.540+0000 7fe48aa57740 1 Processor -- start 2024-09-12T00:56:57.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5585a12b8c60 con 0x5585a12a2fa0 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585a11906e0 con 0x5585a12b6150 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a12b8de0 con 0x5585a12a2680 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe48aa57740 10 client.0 _ll_register_callbacks cb 0x5585a11b4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46736/0 (socket says 172.21.15.16:46736) 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 -- 172.21.15.16:0/901583840 learned_addr learned my addr 172.21.15.16:0/901583840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe486c1b640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe486c1b640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 -- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 msgr2=0x5585a12b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 -- 172.21.15.16:0/901583840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 msgr2=0x5585a12b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 -- 172.21.15.16:0/901583840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a12c23a0 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe486c1b640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe485c19640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe474010840 tx=0x7fe474010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079a2f2c18e3574 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.541+0000 7fe465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47400cd20 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe474004020 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe478005520 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe474023050 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585a12c4d60 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478005520 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe47401a070 con 0x5585a12a2fa0 2024-09-12T00:56:57.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 10 client.15630 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 20 client.15630 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a12c6300 0x5585a12c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 4 client.15630 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.542+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585a12159d0 con 0x5585a12c6300 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.543+0000 7fe486c1b640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a12c6300 0x5585a12c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.543+0000 7fe465ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:57.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.543+0000 7fe486c1b640 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a12c6300 0x5585a12c86f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a67cb8f9f2d9f7e server_cookie=55d1a76a17470144 in_seq=0 out_seq=0 2024-09-12T00:56:57.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.544+0000 7fe465ffb640 1 -- 172.21.15.16:0/901583840 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585a12159d0 con 0x5585a12c6300 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.544+0000 7fe465ffb640 10 client.15630 handle_command_reply: tid=0 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.544+0000 7fe48aa57740 1 client.15630 shutdown 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 msgr2=0x5585a12b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe474010840 tx=0x7fe474010d00 comp rx=0 tx=0).stop 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 shutdown_connections 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a12c6300 0x5585a12c86f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2fa0 0x5585a12b36b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 --2- 172.21.15.16:0/901583840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 >> 172.21.15.16:0/901583840 conn(0x5585a129ecf0 msgr2=0x5585a12a4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.545+0000 7fe465ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:57.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.546+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 shutdown_connections 2024-09-12T00:56:57.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.546+0000 7fe48aa57740 1 -- 172.21.15.16:0/901583840 wait complete. 2024-09-12T00:56:57.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.546+0000 7fe48aa57740 20 client.15630 trim_cache size 0 max 16384 2024-09-12T00:56:57.577 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:56:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 Processor -- start 2024-09-12T00:56:57.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 -- start start 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 0x5585a12a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12a4db0 0x5585a12cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c00e6d0 con 0x5585a12b6150 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c010210 con 0x5585a12a4db0 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe48aa57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c00eda0 con 0x5585a12a2680 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.581+0000 7fe485c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 0x5585a12a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe485c19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 0x5585a12a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46742/0 (socket says 172.21.15.16:46742) 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:37468/0 (socket says 172.21.15.16:37468) 2024-09-12T00:56:57.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 -- 172.21.15.16:0/709873861 learned_addr learned my addr 172.21.15.16:0/709873861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12a4db0 0x5585a12cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 -- 172.21.15.16:0/709873861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12a4db0 msgr2=0x5585a12cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12a4db0 0x5585a12cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 -- 172.21.15.16:0/709873861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 msgr2=0x5585a12a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 0x5585a12a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 -- 172.21.15.16:0/709873861 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a12c8d20 con 0x5585a12a2680 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe486c1b640 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe478005c40 tx=0x7fe478004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53988c1eb9cdba4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.582+0000 7fe467fff640 1 -- 172.21.15.16:0/709873861 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478035040 con 0x5585a12a2680 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe467fff640 1 -- 172.21.15.16:0/709873861 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47804d040 con 0x5585a12a2680 2024-09-12T00:56:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe467fff640 1 -- 172.21.15.16:0/709873861 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478003cc0 con 0x5585a12a2680 2024-09-12T00:56:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 -- 172.21.15.16:0/709873861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 msgr2=0x5585a12b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe478005c40 tx=0x7fe478004cf0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 -- 172.21.15.16:0/709873861 shutdown_connections 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12a4db0 0x5585a12cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12b6150 0x5585a12a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 --2- 172.21.15.16:0/709873861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a2680 0x5585a12b3a40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.583+0000 7fe48aa57740 1 -- 172.21.15.16:0/709873861 >> 172.21.15.16:0/709873861 conn(0x5585a129ecf0 msgr2=0x5585a12a4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.584+0000 7fe48aa57740 1 -- 172.21.15.16:0/709873861 shutdown_connections 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.584+0000 7fe48aa57740 1 -- 172.21.15.16:0/709873861 wait complete. 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.584+0000 7fe48aa57740 1 -- start start 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.584+0000 7fe48aa57740 1 Processor -- start 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a4db0 0x5585a12baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12baf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c00fb40 con 0x5585a12a2680 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c00f470 con 0x5585a12b6150 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c00c6b0 con 0x5585a12a4db0 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe48aa57740 10 client.0 _ll_register_callbacks cb 0x7fe4740103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe486c1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46748/0 (socket says 172.21.15.16:46748) 2024-09-12T00:56:57.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.585+0000 7fe486c1b640 1 -- 172.21.15.16:0/2034797867 learned_addr learned my addr 172.21.15.16:0/2034797867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 -- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a4db0 msgr2=0x5585a12baa20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a4db0 0x5585a12baa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 -- 172.21.15.16:0/2034797867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 msgr2=0x5585a12baf60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:56:57.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12baf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 -- 172.21.15.16:0/2034797867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a12bb600 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe486c1b640 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe478004a80 tx=0x7fe478004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d956c7723b2f174 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478035040 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47804d040 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.586+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe478003d50 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe478060050 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe48aa57740 10 client.15636 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585a1331da0 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe478004740 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe478031580 con 0x5585a12a2680 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe48aa57740 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:57.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.587+0000 7fe48aa57740 10 client.15636 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe48aa57740 20 client.15636 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a1331fd0 0x5585a13343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe48aa57740 4 client.15636 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x7fe47804d040 con 0x5585a1331fd0 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe485c19640 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a1331fd0 0x5585a13343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.588+0000 7fe485c19640 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a1331fd0 0x5585a13343c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982ae0f06d827ea6 server_cookie=a3fddbda4ee5d71 in_seq=0 out_seq=0 2024-09-12T00:56:57.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.589+0000 7fe465ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.603+0000 7fe465ffb640 1 -- 172.21.15.16:0/2034797867 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fe47804d040 con 0x5585a1331fd0 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.603+0000 7fe465ffb640 10 client.15636 handle_command_reply: tid=0 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 client.15636 shutdown 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 msgr2=0x5585a12be6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe478004a80 tx=0x7fe478004180 comp rx=0 tx=0).stop 2024-09-12T00:56:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 shutdown_connections 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5585a1331fd0 0x5585a13343c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585a12b6150 0x5585a12baf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe465ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a12a4db0 0x5585a12baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 --2- 172.21.15.16:0/2034797867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5585a12a2680 0x5585a12be6e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.604+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 >> 172.21.15.16:0/2034797867 conn(0x5585a129ecf0 msgr2=0x5585a129d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.605+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 shutdown_connections 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.605+0000 7fe48aa57740 1 -- 172.21.15.16:0/2034797867 wait complete. 2024-09-12T00:56:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.605+0000 7fe48aa57740 20 client.15636 trim_cache size 0 max 16384 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "4a8b0dcf": { 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:56:57.606 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:56:57.607 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:56:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 msgr2=0x7fe4680a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 0x7fe4680a53d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4700d0cb0 tx=0x7fe47009bc30 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 msgr2=0x7fe480196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe470009fa0 tx=0x7fe47000c640 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 shutdown_connections 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4680a2f10 0x7fe4680a53d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe480102ef0 0x7fe48019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe480102570 0x7fe4801967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 --2- 172.21.15.16:0/825762882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe480100e20 0x7fe480196260 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 >> 172.21.15.16:0/825762882 conn(0x7fe4800fc7d0 msgr2=0x7fe480106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 shutdown_connections 2024-09-12T00:56:57.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:57.606+0000 7fe48867e640 1 -- 172.21.15.16:0/825762882 wait complete. 2024-09-12T00:56:58.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4a8b0dcf', quiesce took 7.5 seconds. Will release after: 17.0 2024-09-12T00:56:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 -- 172.21.15.16:0/2012361542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 msgr2=0x7f2f4410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 --2- 172.21.15.16:0/2012361542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4410da40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009ca0 tx=0x7f2f3c02a950 comp rx=0 tx=0).stop 2024-09-12T00:56:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 -- 172.21.15.16:0/2012361542 shutdown_connections 2024-09-12T00:56:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 --2- 172.21.15.16:0/2012361542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4410da40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 --2- 172.21.15.16:0/2012361542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 --2- 172.21.15.16:0/2012361542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.859+0000 7f2f4c165640 1 -- 172.21.15.16:0/2012361542 >> 172.21.15.16:0/2012361542 conn(0x7f2f440fc7d0 msgr2=0x7f2f440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.860+0000 7f2f4c165640 1 -- 172.21.15.16:0/2012361542 shutdown_connections 2024-09-12T00:56:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.860+0000 7f2f4c165640 1 -- 172.21.15.16:0/2012361542 wait complete. 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.860+0000 7f2f4c165640 1 Processor -- start 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.860+0000 7f2f4c165640 1 -- start start 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f441983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f44198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44110fd0 con 0x7f2f44102ef0 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f49700640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f44198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f441983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f49700640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f44198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52260/0 (socket says 172.21.15.16:52260) 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f49700640 1 -- 172.21.15.16:0/1679701496 learned_addr learned my addr 172.21.15.16:0/1679701496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4a702640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44110e50 con 0x7f2f44100e20 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.861+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44111150 con 0x7f2f44102570 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 -- 172.21.15.16:0/1679701496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 msgr2=0x7f2f44198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f44198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 -- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 msgr2=0x7f2f4419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 -- 172.21.15.16:0/1679701496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419d370 con 0x7f2f44100e20 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f49f01640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f441983c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c970 tx=0x7f2f4000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f3affd640 1 -- 172.21.15.16:0/1679701496 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40007bb0 con 0x7f2f44100e20 2024-09-12T00:56:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f3affd640 1 -- 172.21.15.16:0/1679701496 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f40007d50 con 0x7f2f44100e20 2024-09-12T00:56:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4419d660 con 0x7f2f44100e20 2024-09-12T00:56:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f3affd640 1 -- 172.21.15.16:0/1679701496 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40018820 con 0x7f2f44100e20 2024-09-12T00:56:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.862+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a4f40 con 0x7f2f44100e20 2024-09-12T00:56:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.865+0000 7f2f3affd640 1 -- 172.21.15.16:0/1679701496 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f2f400189c0 con 0x7f2f44100e20 2024-09-12T00:56:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.866+0000 7f2f4e53e740 1 Processor -- start 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 -- start start 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f3affd640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 0x7f2f300a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4a702640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 0x7f2f300a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda00a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46460/0 (socket says 172.21.15.16:46460) 2024-09-12T00:56:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f49f01640 1 -- 172.21.15.16:0/108113755 learned_addr learned my addr 172.21.15.16:0/108113755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560bed9662f0 con 0x560beda01b40 2024-09-12T00:56:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560bed90dcf0 con 0x560beda005d0 2024-09-12T00:56:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560bed9622b0 con 0x560beda0b370 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f49700640 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda00a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.867+0000 7f2f4a702640 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 -- 172.21.15.16:0/108113755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 msgr2=0x560beda0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda0d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 -- 172.21.15.16:0/108113755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 msgr2=0x560beda00a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda00a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 -- 172.21.15.16:0/108113755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bed8f23a0 con 0x560beda01b40 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f4a702640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 0x7f2f300a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006010 tx=0x7f2f3c02f4c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f3affd640 1 -- 172.21.15.16:0/1679701496 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f400cf690 con 0x7f2f44100e20 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f49f01640 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2f4009b630 tx=0x7f2f4006dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2747382e4c5672e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f2ffff640 1 -- 172.21.15.16:0/108113755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40018cb0 con 0x560beda01b40 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f2ffff640 1 -- 172.21.15.16:0/108113755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4006d490 con 0x560beda01b40 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.868+0000 7f2f2ffff640 1 -- 172.21.15.16:0/108113755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4009b930 con 0x560beda01b40 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 msgr2=0x560beda00090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2f4009b630 tx=0x7f2f4006dba0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 shutdown_connections 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda00a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/108113755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda00090 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.869+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 >> 172.21.15.16:0/108113755 conn(0x560bed9c9310 msgr2=0x560bed9fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.870+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 shutdown_connections 2024-09-12T00:56:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.870+0000 7f2f4e53e740 1 -- 172.21.15.16:0/108113755 wait complete. 2024-09-12T00:56:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.870+0000 7f2f4e53e740 1 -- start start 2024-09-12T00:56:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.870+0000 7f2f4e53e740 1 Processor -- start 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda0adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda01b40 0x560beda080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560beda0e070 con 0x560beda0b370 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560bed921060 con 0x560beda005d0 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560beda0e1f0 con 0x560beda01b40 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4e53e740 10 client.0 _ll_register_callbacks cb 0x560bed6748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46468/0 (socket says 172.21.15.16:46468) 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 -- 172.21.15.16:0/2414709347 learned_addr learned my addr 172.21.15.16:0/2414709347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f49700640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda01b40 0x560beda080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f49f01640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda0adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 -- 172.21.15.16:0/2414709347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda01b40 msgr2=0x560beda080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda01b40 0x560beda080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 -- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 msgr2=0x560beda0adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda0adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f4a702640 1 -- 172.21.15.16:0/2414709347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bed6d73c0 con 0x560beda0b370 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.871+0000 7f2f49f01640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda0adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f4a702640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009ca0 tx=0x7f2f3c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac6d86f9e3f61b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c034040 con 0x560beda0b370 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f3c004450 con 0x560beda0b370 2024-09-12T00:56:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4006d490 con 0x560beda0b370 2024-09-12T00:56:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c004650 con 0x560beda0b370 2024-09-12T00:56:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f4e53e740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560beda19ba0 con 0x560beda0b370 2024-09-12T00:56:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.872+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4006d490 con 0x560beda0b370 2024-09-12T00:56:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f3c061020 con 0x560beda0b370 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 10 client.15648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 20 client.15648 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda25cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 4 client.15648 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bed973860 con 0x560beda23810 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f49f01640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda25cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f2d7fa640 10 client.15648 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.873+0000 7f2f49f01640 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda25cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c64fad6c18c93de server_cookie=a6e2dc6c5461b397 in_seq=0 out_seq=0 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/2414709347 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bed973860 con 0x560beda23810 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f2d7fa640 10 client.15648 handle_command_reply: tid=0 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 client.15648 shutdown 2024-09-12T00:56:58.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 msgr2=0x560beda087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009ca0 tx=0x7f2f3c02a950 comp rx=0 tx=0).stop 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 shutdown_connections 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda25cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f2d7fa640 0 client.15648 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda0b370 0x560beda087e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda01b40 0x560beda080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/2414709347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda0adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.874+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 >> 172.21.15.16:0/2414709347 conn(0x560bed9c9310 msgr2=0x560beda02990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.875+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 shutdown_connections 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.875+0000 7f2f4e53e740 1 -- 172.21.15.16:0/2414709347 wait complete. 2024-09-12T00:56:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.875+0000 7f2f4e53e740 20 client.15648 trim_cache size 0 max 16384 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.887+0000 7f2f4e53e740 1 Processor -- start 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 -- start start 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda03180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44071f50 con 0x560beda01b40 2024-09-12T00:56:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44073a90 con 0x560beda005d0 2024-09-12T00:56:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44072620 con 0x560beda0b370 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda03180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda03180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:47970/0 (socket says 172.21.15.16:47970) 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f49f01640 1 -- 172.21.15.16:0/3513460731 learned_addr learned my addr 172.21.15.16:0/3513460731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f49700640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 -- 172.21.15.16:0/3513460731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 msgr2=0x560beda03180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda03180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 -- 172.21.15.16:0/3513460731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 msgr2=0x560beda036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.888+0000 7f2f4a702640 1 -- 172.21.15.16:0/3513460731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beda03c00 con 0x560beda0b370 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.889+0000 7f2f4a702640 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda2b580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0636a0 tx=0x7f2f3c02b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d2cf59070cdbd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.889+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3513460731 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c070070 con 0x560beda0b370 2024-09-12T00:56:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.889+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3513460731 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c063890 con 0x560beda0b370 2024-09-12T00:56:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.889+0000 7f2f2f7fe640 1 -- 172.21.15.16:0/3513460731 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c034040 con 0x560beda0b370 2024-09-12T00:56:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 -- 172.21.15.16:0/3513460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 msgr2=0x560beda2b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda2b580 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0636a0 tx=0x7f2f3c02b100 comp rx=0 tx=0).stop 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 -- 172.21.15.16:0/3513460731 shutdown_connections 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda0b370 0x560beda2b580 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/3513460731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda005d0 0x560beda03180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 -- 172.21.15.16:0/3513460731 >> 172.21.15.16:0/3513460731 conn(0x560bed9c9310 msgr2=0x560beda0a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 -- 172.21.15.16:0/3513460731 shutdown_connections 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.890+0000 7f2f4e53e740 1 -- 172.21.15.16:0/3513460731 wait complete. 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 -- start start 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 Processor -- start 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda0b370 0x560beda10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f440733c0 con 0x560beda01b40 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44072cf0 con 0x560beda0b370 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4406ff30 con 0x560beda005d0 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.891+0000 7f2f4e53e740 10 client.0 _ll_register_callbacks cb 0x7f2f3c02f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52292/0 (socket says 172.21.15.16:52292) 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 -- 172.21.15.16:0/859070780 learned_addr learned my addr 172.21.15.16:0/859070780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49700640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f4a702640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda0b370 0x560beda10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 -- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda0b370 msgr2=0x560beda10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda0b370 0x560beda10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 -- 172.21.15.16:0/859070780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 msgr2=0x560beda139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 -- 172.21.15.16:0/859070780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560beda13fb0 con 0x560beda005d0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f49f01640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f400a4070 tx=0x7f2f40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f90124b1902b0d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4009b470 con 0x560beda005d0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f4000fac0 con 0x560beda005d0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.892+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f3c063890 con 0x560beda005d0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f4009fce0 con 0x560beda005d0 2024-09-12T00:56:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 10 client.25246 fetch_fsmap learned FSMap version 28 2024-09-12T00:56:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560beda8ea70 con 0x560beda005d0 2024-09-12T00:56:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f400afe50 con 0x560beda005d0 2024-09-12T00:56:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f400ba170 con 0x560beda005d0 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 10 client.25246 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 10 client.25246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 20 client.25246 populate_metadata read hostname 'smithi016' 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:56:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 4 client.25246 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.893+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2f3c063890 con 0x560beda23810 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f49700640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f2d7fa640 10 client.25246 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f49700640 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda90cb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12901afa1b760ef5 server_cookie=b491b9f244499e02 in_seq=0 out_seq=0 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f2d7fa640 1 -- 172.21.15.16:0/859070780 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2f3c063890 con 0x560beda23810 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f2d7fa640 10 client.25246 handle_command_reply: tid=0 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f4e53e740 1 client.25246 shutdown 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 msgr2=0x560beda13480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.894+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f400a4070 tx=0x7f2f40000bc0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 shutdown_connections 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560beda23810 0x560beda90cb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560beda0b370 0x560beda10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560beda01b40 0x560beda139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 --2- 172.21.15.16:0/859070780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560beda005d0 0x560beda13480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f2d7fa640 0 client.25246 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 >> 172.21.15.16:0/859070780 conn(0x560bed9c9310 msgr2=0x560bed9fb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 shutdown_connections 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 1 -- 172.21.15.16:0/859070780 wait complete. 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4e53e740 20 client.25246 trim_cache size 0 max 16384 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 msgr2=0x7f2f300a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 0x7f2f300a5350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006010 tx=0x7f2f3c02f4c0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 msgr2=0x7f2f441983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f441983c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c970 tx=0x7f2f4000ce30 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 shutdown_connections 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f300a2e90 0x7f2f300a5350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44102570 0x7f2f44198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 --2- 172.21.15.16:0/1679701496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f44100e20 0x7f2f441983c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 >> 172.21.15.16:0/1679701496 conn(0x7f2f440fc7d0 msgr2=0x7f2f4406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 shutdown_connections 2024-09-12T00:56:58.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:56:58.895+0000 7f2f4c165640 1 -- 172.21.15.16:0/1679701496 wait complete. 2024-09-12T00:56:59.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:56:58 smithi016 ceph-mon[30972]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:59.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:56:58 smithi099 ceph-mon[35654]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:56:58 smithi159 ceph-mon[35566]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:56:59.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:56:59.908 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:00.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:57:00.017+0000 7f1d1a841640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:57:00.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:57:00.017+0000 7f1d1a841640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:57:00.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:57:00.017+0000 7f1d1a841640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:57:00.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:57:00.017+0000 7f1d1a841640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:57:00.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x[31197]: 2024-09-12T00:57:00.017+0000 7f1d1a841640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T00:57:00.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 -- 172.21.15.16:0/685560654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102570 msgr2=0x7fe8141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 --2- 172.21.15.16:0/685560654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102570 0x7fe8141029b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe804009900 tx=0x7fe80402a480 comp rx=0 tx=0).stop 2024-09-12T00:57:00.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 -- 172.21.15.16:0/685560654 shutdown_connections 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 --2- 172.21.15.16:0/685560654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814102ef0 0x7fe81410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 --2- 172.21.15.16:0/685560654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102570 0x7fe8141029b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 --2- 172.21.15.16:0/685560654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814100e20 0x7fe814101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 -- 172.21.15.16:0/685560654 >> 172.21.15.16:0/685560654 conn(0x7fe8140fc7d0 msgr2=0x7fe8140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 -- 172.21.15.16:0/685560654 shutdown_connections 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.394+0000 7fe81c747640 1 -- 172.21.15.16:0/685560654 wait complete. 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.395+0000 7fe81c747640 1 Processor -- start 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.395+0000 7fe81c747640 1 -- start start 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.395+0000 7fe81c747640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 0x7fe8141a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81c747640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 0x7fe8141a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81c747640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81c747640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe814110fd0 con 0x7fe814102ef0 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81c747640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe814110e50 con 0x7fe814102570 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81c747640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe814111150 con 0x7fe814100e20 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 0x7fe8141a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46538/0 (socket says 172.21.15.16:46538) 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 -- 172.21.15.16:0/2349996461 learned_addr learned my addr 172.21.15.16:0/2349996461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 0x7fe8141a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 -- 172.21.15.16:0/2349996461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 msgr2=0x7fe8141a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 0x7fe8141a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 -- 172.21.15.16:0/2349996461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 msgr2=0x7fe8141a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 0x7fe8141a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81ace4640 1 -- 172.21.15.16:0/2349996461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81419f530 con 0x7fe814102ef0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 0x7fe8141a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.396+0000 7fe819ce2640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 0x7fe8141a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe81ace4640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00eaa0 tx=0x7fe80c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe80b7fe640 1 -- 172.21.15.16:0/2349996461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c004d20 con 0x7fe814102ef0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe80b7fe640 1 -- 172.21.15.16:0/2349996461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80c005020 con 0x7fe814102ef0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe80b7fe640 1 -- 172.21.15.16:0/2349996461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0116d0 con 0x7fe814102ef0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81419f820 con 0x7fe814102ef0 2024-09-12T00:57:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.397+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8141ab730 con 0x7fe814102ef0 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.400+0000 7fe80b7fe640 1 -- 172.21.15.16:0/2349996461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fe80c0051d0 con 0x7fe814102ef0 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.400+0000 7fe80b7fe640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 0x7fe8000a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 Processor -- start 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe80b7fe640 1 -- 172.21.15.16:0/2349996461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe80c0cf790 con 0x7fe814102ef0 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 0x7fe8000a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 -- start start 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a752090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55869a6b82f0 con 0x55869a75d370 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 0x7fe8000a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe81000f3c0 tx=0x7fe81000a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52330/0 (socket says 172.21.15.16:52330) 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe819ce2640 1 -- 172.21.15.16:0/1069336220 learned_addr learned my addr 172.21.15.16:0/1069336220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.401+0000 7fe81eb20740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55869a65fcf0 con 0x55869a753b40 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55869a6b42b0 con 0x55869a7525d0 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a752090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 -- 172.21.15.16:0/1069336220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 msgr2=0x55869a752a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 -- 172.21.15.16:0/1069336220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 msgr2=0x55869a752090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a752090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 -- 172.21.15.16:0/1069336220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a6443a0 con 0x55869a75d370 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe819ce2640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe81ace4640 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75f760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0ccd90 tx=0x7fe80c09c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8738e41bb32e58b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.402+0000 7fe808ff9640 1 -- 172.21.15.16:0/1069336220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c01f040 con 0x55869a75d370 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.403+0000 7fe808ff9640 1 -- 172.21.15.16:0/1069336220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80c0a4700 con 0x55869a75d370 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.403+0000 7fe808ff9640 1 -- 172.21.15.16:0/1069336220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0cd070 con 0x55869a75d370 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 msgr2=0x55869a75f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75f760 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0ccd90 tx=0x7fe80c09c250 comp rx=0 tx=0).stop 2024-09-12T00:57:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 shutdown_connections 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75f760 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a752a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 --2- 172.21.15.16:0/1069336220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a752090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 >> 172.21.15.16:0/1069336220 conn(0x55869a71b310 msgr2=0x55869a74ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 shutdown_connections 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- 172.21.15.16:0/1069336220 wait complete. 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.404+0000 7fe81eb20740 1 -- start start 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 Processor -- start 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a75cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55869a760010 con 0x55869a75d370 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55869a673060 con 0x55869a753b40 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55869a760190 con 0x55869a7525d0 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81eb20740 10 client.0 _ll_register_callbacks cb 0x55869a3c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:48034/0 (socket says 172.21.15.16:48034) 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe819ce2640 1 -- 172.21.15.16:0/3203757235 learned_addr learned my addr 172.21.15.16:0/3203757235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a75cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.405+0000 7fe81ace4640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 -- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 msgr2=0x55869a75cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a75cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 -- 172.21.15.16:0/3203757235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 msgr2=0x55869a75a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 -- 172.21.15.16:0/3203757235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869a4293c0 con 0x55869a753b40 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a75cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe81ace4640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe804033000 tx=0x7fe8040053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba74982f8f99679 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804035040 con 0x55869a753b40 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe80c0a4700 con 0x55869a753b40 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe804004b70 con 0x55869a753b40 2024-09-12T00:57:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe804005610 con 0x55869a753b40 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe81eb20740 10 client.25241 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.406+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55869a76bb30 con 0x55869a753b40 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe80c0a4700 con 0x55869a753b40 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe804044020 con 0x55869a753b40 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 10 client.25241 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 10 client.25241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 20 client.25241 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a76cf90 0x55869a76f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 4 client.25241 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55869a6c5860 con 0x55869a76cf90 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.407+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a76cf90 0x55869a76f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.408+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a76cf90 0x55869a76f3b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bff1d80e3d7ab1c5 server_cookie=8a854b82402db335 in_seq=0 out_seq=0 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.408+0000 7fe7fdffb640 10 client.25241 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.408+0000 7fe7fdffb640 1 -- 172.21.15.16:0/3203757235 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55869a6c5860 con 0x55869a76cf90 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.408+0000 7fe7fdffb640 10 client.25241 handle_command_reply: tid=0 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.408+0000 7fe81eb20740 1 client.25241 shutdown 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 msgr2=0x55869a75a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe804033000 tx=0x7fe8040053b0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 shutdown_connections 2024-09-12T00:57:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a75a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a76cf90 0x55869a76f3b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a75a040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3203757235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a75cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe7fdffb640 0 client.25241 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 >> 172.21.15.16:0/3203757235 conn(0x55869a71b310 msgr2=0x55869a754990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 shutdown_connections 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 1 -- 172.21.15.16:0/3203757235 wait complete. 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.409+0000 7fe81eb20740 20 client.25241 trim_cache size 0 max 16384 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 Processor -- start 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 -- start start 2024-09-12T00:57:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a7525d0 0x55869a7692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a75d370 0x55869a7751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b3cd0 con 0x55869a7525d0 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b5810 con 0x55869a753b40 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81eb20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b43a0 con 0x55869a75d370 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe819ce2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:48036/0 (socket says 172.21.15.16:48036) 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe819ce2640 1 -- 172.21.15.16:0/3119786829 learned_addr learned my addr 172.21.15.16:0/3119786829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.424+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a7525d0 0x55869a7692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe81ace4640 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a75d370 0x55869a7751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 -- 172.21.15.16:0/3119786829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a75d370 msgr2=0x55869a7751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a75d370 0x55869a7751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 -- 172.21.15.16:0/3119786829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a7525d0 msgr2=0x55869a7692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a7525d0 0x55869a7692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 -- 172.21.15.16:0/3119786829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55869a74f960 con 0x55869a753b40 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe819ce2640 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8040096c0 tx=0x7fe80402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ceda192165f1b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe7fffff640 1 -- 172.21.15.16:0/3119786829 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804053070 con 0x55869a753b40 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe7fffff640 1 -- 172.21.15.16:0/3119786829 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe804031ad0 con 0x55869a753b40 2024-09-12T00:57:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.425+0000 7fe7fffff640 1 -- 172.21.15.16:0/3119786829 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804035030 con 0x55869a753b40 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 -- 172.21.15.16:0/3119786829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 msgr2=0x55869a774c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe8040096c0 tx=0x7fe80402a480 comp rx=0 tx=0).stop 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 -- 172.21.15.16:0/3119786829 shutdown_connections 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a75d370 0x55869a7751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a774c60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 --2- 172.21.15.16:0/3119786829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a7525d0 0x55869a7692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 -- 172.21.15.16:0/3119786829 >> 172.21.15.16:0/3119786829 conn(0x55869a71b310 msgr2=0x55869a758910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 -- 172.21.15.16:0/3119786829 shutdown_connections 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.426+0000 7fe81eb20740 1 -- 172.21.15.16:0/3119786829 wait complete. 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 -- start start 2024-09-12T00:57:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 Processor -- start 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a761ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a7d6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b5140 con 0x55869a75d370 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b4a70 con 0x55869a753b40 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8141b1cb0 con 0x55869a7525d0 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81a4e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.427+0000 7fe81eb20740 10 client.0 _ll_register_callbacks cb 0x7fe804005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52346/0 (socket says 172.21.15.16:52346) 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 -- 172.21.15.16:0/180269976 learned_addr learned my addr 172.21.15.16:0/180269976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe819ce2640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a761ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81ace4640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a7d6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 -- 172.21.15.16:0/180269976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 msgr2=0x55869a761ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a761ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 -- 172.21.15.16:0/180269976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 msgr2=0x55869a7d6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a7d6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 -- 172.21.15.16:0/180269976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869a755120 con 0x55869a7525d0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe81a4e3640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe81000aab0 tx=0x7fe810004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd5e43b339fe2f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.428+0000 7fe7fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810016070 con 0x55869a7525d0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe804031ad0 con 0x55869a7525d0 2024-09-12T00:57:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe81001cd00 con 0x55869a7525d0 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe810028410 con 0x55869a7525d0 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe81eb20740 10 client.25270 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55869a7d80f0 con 0x55869a7525d0 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe804031ad0 con 0x55869a7525d0 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.429+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe810032840 con 0x55869a7525d0 2024-09-12T00:57:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 10 client.25270 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 10 client.25270 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 20 client.25270 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a7d87f0 0x55869a7dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 4 client.25270 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55869a6bfd70 con 0x55869a7d87f0 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.430+0000 7fe819ce2640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a7d87f0 0x55869a7dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe7fdffb640 10 client.25270 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe819ce2640 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a7d87f0 0x55869a7dac10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c85cc1db620b995 server_cookie=eed3f466ff5921e3 in_seq=0 out_seq=0 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe7fdffb640 1 -- 172.21.15.16:0/180269976 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55869a6bfd70 con 0x55869a7d87f0 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe7fdffb640 10 client.25270 handle_command_reply: tid=0 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 client.25270 shutdown 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 msgr2=0x55869a761580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe81000aab0 tx=0x7fe810004730 comp rx=0 tx=0).stop 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 shutdown_connections 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55869a75d370 0x55869a7d6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55869a7d87f0 0x55869a7dac10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55869a753b40 0x55869a761ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 --2- 172.21.15.16:0/180269976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55869a7525d0 0x55869a761580 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 >> 172.21.15.16:0/180269976 conn(0x55869a71b310 msgr2=0x55869a75b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe7fdffb640 0 client.25270 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 shutdown_connections 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.431+0000 7fe81eb20740 1 -- 172.21.15.16:0/180269976 wait complete. 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81eb20740 20 client.25270 trim_cache size 0 max 16384 2024-09-12T00:57:00.450 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 msgr2=0x7fe8000a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 0x7fe8000a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe81000f3c0 tx=0x7fe81000a2d0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 msgr2=0x7fe81419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00eaa0 tx=0x7fe80c00eea0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 shutdown_connections 2024-09-12T00:57:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe8000a2e90 0x7fe8000a5350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe814102ef0 0x7fe81419ec70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe814102570 0x7fe8141a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 --2- 172.21.15.16:0/2349996461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe814100e20 0x7fe8141a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 >> 172.21.15.16:0/2349996461 conn(0x7fe8140fc7d0 msgr2=0x7fe814069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 shutdown_connections 2024-09-12T00:57:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:00.432+0000 7fe81c747640 1 -- 172.21.15.16:0/2349996461 wait complete. 2024-09-12T00:57:01.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:00 smithi016 ceph-mon[30972]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T00:57:01.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:57:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:00 smithi099 ceph-mon[35654]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T00:57:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:00 smithi159 ceph-mon[35566]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T00:57:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.048+0000 7f84cf275640 1 -- 172.21.15.16:0/16749770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c810c2f0 msgr2=0x7f84c810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.048+0000 7f84cf275640 1 --2- 172.21.15.16:0/16749770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c810c2f0 0x7f84c810e790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009ca0 tx=0x7f84c002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 -- 172.21.15.16:0/16749770 shutdown_connections 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 --2- 172.21.15.16:0/16749770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c810c2f0 0x7f84c810e790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 --2- 172.21.15.16:0/16749770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c81037a0 0x7f84c810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 --2- 172.21.15.16:0/16749770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c8102df0 0x7f84c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 -- 172.21.15.16:0/16749770 >> 172.21.15.16:0/16749770 conn(0x7f84c80fc7d0 msgr2=0x7f84c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 -- 172.21.15.16:0/16749770 shutdown_connections 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 -- 172.21.15.16:0/16749770 wait complete. 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 Processor -- start 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.049+0000 7f84cf275640 1 -- start start 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c81037a0 0x7f84c8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c810c2f0 0x7f84c819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81120f0 con 0x7f84c8102df0 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8111f70 con 0x7f84c810c2f0 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cf275640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c8112270 con 0x7f84c81037a0 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46600/0 (socket says 172.21.15.16:46600) 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 -- 172.21.15.16:0/3391984990 learned_addr learned my addr 172.21.15.16:0/3391984990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cc810640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c81037a0 0x7f84c8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 -- 172.21.15.16:0/3391984990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c81037a0 msgr2=0x7f84c8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c81037a0 0x7f84c8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd011640 1 -- 172.21.15.16:0/3391984990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c810c2f0 msgr2=0x7f84c819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.050+0000 7f84cd812640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c810c2f0 0x7f84c819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84cd011640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c810c2f0 0x7f84c819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84cd011640 1 -- 172.21.15.16:0/3391984990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c819d540 con 0x7f84c8102df0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84cd011640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9f0 tx=0x7f84c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84be7fc640 1 -- 172.21.15.16:0/3391984990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cd80 con 0x7f84c8102df0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84be7fc640 1 -- 172.21.15.16:0/3391984990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c4010910 con 0x7f84c8102df0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84be7fc640 1 -- 172.21.15.16:0/3391984990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018990 con 0x7f84c8102df0 2024-09-12T00:57:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c819d830 con 0x7f84c8102df0 2024-09-12T00:57:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.051+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c819dc60 con 0x7f84c8102df0 2024-09-12T00:57:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.054+0000 7f84be7fc640 1 -- 172.21.15.16:0/3391984990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f84c4010ac0 con 0x7f84c8102df0 2024-09-12T00:57:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.054+0000 7f84d164e740 1 Processor -- start 2024-09-12T00:57:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 -- start start 2024-09-12T00:57:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f0924332f0 con 0x55f0924cd5d0 2024-09-12T00:57:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f0923dacf0 con 0x55f0924d8370 2024-09-12T00:57:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84d164e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f09242f2b0 con 0x55f0924ceb40 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cd011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cc810640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cd812640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cd812640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52370/0 (socket says 172.21.15.16:52370) 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cd812640 1 -- 172.21.15.16:0/3976884598 learned_addr learned my addr 172.21.15.16:0/3976884598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84cc810640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924da760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:48072/0 (socket says 172.21.15.16:48072) 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.055+0000 7f84be7fc640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 0x7f84b40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 0x7f84b40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84be7fc640 1 -- 172.21.15.16:0/3391984990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84c40cfaa0 con 0x7f84c8102df0 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 -- 172.21.15.16:0/3976884598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 msgr2=0x55f0924da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 -- 172.21.15.16:0/3976884598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 msgr2=0x55f0924cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 -- 172.21.15.16:0/3976884598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0923bf3a0 con 0x55f0924ceb40 2024-09-12T00:57:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.056+0000 7f84cd812640 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005cc0 tx=0x7f84c0031f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59480911ab783762 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.057+0000 7f84b37fe640 1 -- 172.21.15.16:0/3976884598 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0045050 con 0x55f0924ceb40 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.057+0000 7f84cd812640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 0x7f84b40a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84c004a000 tx=0x7f84c004c000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.057+0000 7f84b37fe640 1 -- 172.21.15.16:0/3976884598 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c004f020 con 0x55f0924ceb40 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.057+0000 7f84b37fe640 1 -- 172.21.15.16:0/3976884598 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0004240 con 0x55f0924ceb40 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 -- 172.21.15.16:0/3976884598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 msgr2=0x55f0924cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84c0005cc0 tx=0x7f84c0031f20 comp rx=0 tx=0).stop 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 -- 172.21.15.16:0/3976884598 shutdown_connections 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 --2- 172.21.15.16:0/3976884598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924cd090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 -- 172.21.15.16:0/3976884598 >> 172.21.15.16:0/3976884598 conn(0x55f092496310 msgr2=0x55f0924c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 -- 172.21.15.16:0/3976884598 shutdown_connections 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.058+0000 7f84d164e740 1 -- 172.21.15.16:0/3976884598 wait complete. 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.059+0000 7f84d164e740 1 -- start start 2024-09-12T00:57:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.059+0000 7f84d164e740 1 Processor -- start 2024-09-12T00:57:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.059+0000 7f84d164e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 0x55f0924d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.059+0000 7f84d164e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 0x55f0924d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.059+0000 7f84d164e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd011640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 0x55f0924d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd011640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 0x55f0924d7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:52386/0 (socket says 172.21.15.16:52386) 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46610/0 (socket says 172.21.15.16:46610) 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cc810640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 0x55f0924d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd011640 1 -- 172.21.15.16:0/1583638682 learned_addr learned my addr 172.21.15.16:0/1583638682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f0924db010 con 0x55f0924d8370 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f0923ee060 con 0x55f0924ceb40 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f0924db190 con 0x55f0924cd5d0 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 -- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 msgr2=0x55f0924d7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 0x55f0924d7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 -- 172.21.15.16:0/1583638682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 msgr2=0x55f0924d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 0x55f0924d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.060+0000 7f84cd812640 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0921a43c0 con 0x55f0924d8370 2024-09-12T00:57:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84d164e740 10 client.0 _ll_register_callbacks cb 0x55f0921418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84cc810640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 0x55f0924d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c004f020 con 0x55f0924d8370 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84cd812640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f84c002ade0 tx=0x7f84c003cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5206a9949eb55d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84b0ff9640 10 client.15690 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1583638682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0063060 con 0x55f0924d8370 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.061+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1583638682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84c0004000 con 0x55f0924d8370 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.062+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1583638682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84c0004200 con 0x55f0924d8370 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.062+0000 7f84d164e740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.062+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f0924e66e0 con 0x55f0924d8370 2024-09-12T00:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1583638682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84c00043a0 con 0x55f0924d8370 2024-09-12T00:57:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 20 client.15690 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f0924e7d70 0x55f0924ea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 4 client.15690 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f092440860 con 0x55f0924e7d70 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.063+0000 7f84cd011640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f0924e7d70 0x55f0924ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.064+0000 7f84b0ff9640 10 client.15690 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.064+0000 7f84cd011640 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f0924e7d70 0x55f0924ea230 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f56499496c0950 server_cookie=64fc8f0dee3733e1 in_seq=0 out_seq=0 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.064+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1583638682 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f092440860 con 0x55f0924e7d70 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.064+0000 7f84b0ff9640 10 client.15690 handle_command_reply: tid=0 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 client.15690 shutdown 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 msgr2=0x55f0924d55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f84c002ade0 tx=0x7f84c003cba0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 shutdown_connections 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f0924e7d70 0x55f0924ea230 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924d8370 0x55f0924d55d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924ceb40 0x55f0924d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 --2- 172.21.15.16:0/1583638682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924cd5d0 0x55f0924d7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 >> 172.21.15.16:0/1583638682 conn(0x55f092496310 msgr2=0x55f0924cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84b0ff9640 0 client.15690 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.065+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 shutdown_connections 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.066+0000 7f84d164e740 1 -- 172.21.15.16:0/1583638682 wait complete. 2024-09-12T00:57:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.066+0000 7f84d164e740 20 client.15690 trim_cache size 0 max 16384 2024-09-12T00:57:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 Processor -- start 2024-09-12T00:57:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 -- start start 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 0x55f0924d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924d8370 0x55f0924efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81ab570 con 0x55f0924ceb40 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81ad0b0 con 0x55f0924cd5d0 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84d164e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81abc40 con 0x55f0924d8370 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84cc810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84cc810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46616/0 (socket says 172.21.15.16:46616) 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84cc810640 1 -- 172.21.15.16:0/90016336 learned_addr learned my addr 172.21.15.16:0/90016336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84cd011640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 0x55f0924d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.079+0000 7f84cd812640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924d8370 0x55f0924efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 -- 172.21.15.16:0/90016336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924d8370 msgr2=0x55f0924efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924d8370 0x55f0924efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 -- 172.21.15.16:0/90016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 msgr2=0x55f0924d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 -- 172.21.15.16:0/90016336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0924c8310 con 0x55f0924cd5d0 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cc810640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84cd011640 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 0x55f0924d0120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84c4098210 tx=0x7f84c40d1ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0902b972ffd9089 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84b2ffd640 1 -- 172.21.15.16:0/90016336 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40a0160 con 0x55f0924cd5d0 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84b2ffd640 1 -- 172.21.15.16:0/90016336 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c40a4a20 con 0x55f0924cd5d0 2024-09-12T00:57:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.080+0000 7f84b2ffd640 1 -- 172.21.15.16:0/90016336 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4018b30 con 0x55f0924cd5d0 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- 172.21.15.16:0/90016336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 msgr2=0x55f0924d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 0x55f0924d0120 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84c4098210 tx=0x7f84c40d1ef0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- 172.21.15.16:0/90016336 shutdown_connections 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924d8370 0x55f0924efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924ceb40 0x55f0924d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 --2- 172.21.15.16:0/90016336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924cd5d0 0x55f0924d0120 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- 172.21.15.16:0/90016336 >> 172.21.15.16:0/90016336 conn(0x55f092496310 msgr2=0x55f0924d7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- 172.21.15.16:0/90016336 shutdown_connections 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- 172.21.15.16:0/90016336 wait complete. 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.081+0000 7f84d164e740 1 -- start start 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 Processor -- start 2024-09-12T00:57:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81ac9e0 con 0x55f0924cd5d0 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84cd011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84cd011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46622/0 (socket says 172.21.15.16:46622) 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84cd011640 1 -- 172.21.15.16:0/1740516279 learned_addr learned my addr 172.21.15.16:0/1740516279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81ac310 con 0x55f0924d8370 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a95a0 con 0x55f0924ceb40 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.082+0000 7f84d164e740 10 client.0 _ll_register_callbacks cb 0x7f84c40d4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cc810640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd812640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 -- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 msgr2=0x55f0924dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 -- 172.21.15.16:0/1740516279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 msgr2=0x55f0924dcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924dcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0924c9510 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84cd011640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84c40a44f0 tx=0x7f84c4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28e31ce209a3803 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40b9060 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84c4098420 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.083+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c40a4a20 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84c409c4b0 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 10 client.15696 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f092552b70 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c40a0160 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84c4086040 con 0x55f0924cd5d0 2024-09-12T00:57:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 20 client.15696 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f092553240 0x55f092555630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 4 client.15696 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f84c40a4a20 con 0x55f092553240 2024-09-12T00:57:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.084+0000 7f84cc810640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f092553240 0x55f092555630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84cc810640 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f092553240 0x55f092555630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd51ddec3e5d9d1 server_cookie=a8a085ace2d0e52b in_seq=0 out_seq=0 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84b0ff9640 10 client.15696 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84b0ff9640 1 -- 172.21.15.16:0/1740516279 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f84c40a4a20 con 0x55f092553240 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84b0ff9640 10 client.15696 handle_command_reply: tid=0 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84d164e740 1 client.15696 shutdown 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 msgr2=0x55f0924dc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84c40a44f0 tx=0x7f84c4018420 comp rx=0 tx=0).stop 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 shutdown_connections 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.085+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f092553240 0x55f092555630 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f0924d8370 0x55f0924dcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0924ceb40 0x55f0924dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 --2- 172.21.15.16:0/1740516279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0924cd5d0 0x55f0924dc0a0 secure :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84c40a44f0 tx=0x7f84c4018420 comp rx=0 tx=0).stop 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 >> 172.21.15.16:0/1740516279 conn(0x55f092496310 msgr2=0x55f0924c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84b0ff9640 0 client.15696 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 shutdown_connections 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 1 -- 172.21.15.16:0/1740516279 wait complete. 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.086+0000 7f84d164e740 20 client.15696 trim_cache size 0 max 16384 2024-09-12T00:57:02.089 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 msgr2=0x7f84b40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 0x7f84b40a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84c004a000 tx=0x7f84c004c000 comp rx=0 tx=0).stop 2024-09-12T00:57:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 msgr2=0x7f84c8198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9f0 tx=0x7f84c400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 shutdown_connections 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84b40a2ea0 0x7f84b40a5360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c810c2f0 0x7f84c819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c81037a0 0x7f84c8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 --2- 172.21.15.16:0/3391984990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c8102df0 0x7f84c8198590 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 >> 172.21.15.16:0/3391984990 conn(0x7f84c80fc7d0 msgr2=0x7f84c80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 shutdown_connections 2024-09-12T00:57:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:02.087+0000 7f84cf275640 1 -- 172.21.15.16:0/3391984990 wait complete. 2024-09-12T00:57:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:02 smithi099 ceph-mon[35654]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:02 smithi099 ceph-mon[35654]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-09-12T00:57:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:02 smithi159 ceph-mon[35566]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:02 smithi159 ceph-mon[35566]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-09-12T00:57:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:02 smithi016 ceph-mon[30972]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:02 smithi016 ceph-mon[30972]: mgrmap e24: x(active, since 58s), standbys: y, z 2024-09-12T00:57:03.901 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:57:03.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:57:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 -- 172.21.15.16:0/4087166315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860102790 msgr2=0x7f98601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 --2- 172.21.15.16:0/4087166315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860102790 0x7f98601067e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9848009900 tx=0x7f984802a410 comp rx=0 tx=0).stop 2024-09-12T00:57:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 -- 172.21.15.16:0/4087166315 shutdown_connections 2024-09-12T00:57:04.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 --2- 172.21.15.16:0/4087166315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9860106f10 0x7f986010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 --2- 172.21.15.16:0/4087166315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860102790 0x7f98601067e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 --2- 172.21.15.16:0/4087166315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860101de0 0x7f98601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 -- 172.21.15.16:0/4087166315 >> 172.21.15.16:0/4087166315 conn(0x7f98600fd8c0 msgr2=0x7f98600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 -- 172.21.15.16:0/4087166315 shutdown_connections 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.700+0000 7f9864f38640 1 -- 172.21.15.16:0/4087166315 wait complete. 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 Processor -- start 2024-09-12T00:57:04.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 -- start start 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 0x7f986010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986010e460 0x7f986010e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9860111f70 con 0x7f9860101de0 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9860111df0 con 0x7f986010e460 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f9864f38640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98601120f0 con 0x7f9860106f10 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.701+0000 7f985e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46648/0 (socket says 172.21.15.16:46648) 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 -- 172.21.15.16:0/4243818178 learned_addr learned my addr 172.21.15.16:0/4243818178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985ed9d640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986010e460 0x7f986010e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f9857fff640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 0x7f986010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 -- 172.21.15.16:0/4243818178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 msgr2=0x7f986010df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 0x7f986010df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 -- 172.21.15.16:0/4243818178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986010e460 msgr2=0x7f986010e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986010e460 0x7f986010e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 -- 172.21.15.16:0/4243818178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986019ada0 con 0x7f9860101de0 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f9857fff640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 0x7f986010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f985e59c640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f985000d770 tx=0x7f985000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.702+0000 7f98577fe640 1 -- 172.21.15.16:0/4243818178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850017070 con 0x7f9860101de0 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.703+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986019b090 con 0x7f9860101de0 2024-09-12T00:57:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.703+0000 7f98577fe640 1 -- 172.21.15.16:0/4243818178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850005500 con 0x7f9860101de0 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.703+0000 7f98577fe640 1 -- 172.21.15.16:0/4243818178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9850020060 con 0x7f9860101de0 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.703+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986019b580 con 0x7f9860101de0 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 Processor -- start 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 -- start start 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876dde6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876ddf1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55876dd4c280 con 0x55876ddf1d40 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55876dcf3b10 con 0x55876dde6b60 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.706+0000 7f9867311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55876dd48240 con 0x55876dde7480 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46654/0 (socket says 172.21.15.16:46654) 2024-09-12T00:57:04.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f9857fff640 1 -- 172.21.15.16:0/1341713403 learned_addr learned my addr 172.21.15.16:0/1341713403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f98577fe640 1 -- 172.21.15.16:0/4243818178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f9850004da0 con 0x7f9860101de0 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876dde6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.707+0000 7f9857fff640 1 -- 172.21.15.16:0/1341713403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 msgr2=0x55876ddf1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f985e59c640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876ddf1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f98577fe640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 0x7f984c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f9857fff640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876ddf1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f9857fff640 1 -- 172.21.15.16:0/1341713403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 msgr2=0x55876dde6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f9857fff640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876dde6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f9857fff640 1 -- 172.21.15.16:0/1341713403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876dd001f0 con 0x55876ddf1d40 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876dde6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f98577fe640 1 -- 172.21.15.16:0/4243818178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f98500cf060 con 0x7f9860101de0 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f985ed9d640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 0x7f984c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.708+0000 7f9857fff640 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9860102cf0 tx=0x7f9848032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91abcc5f057526b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f9854ff9640 1 -- 172.21.15.16:0/1341713403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984803f070 con 0x55876ddf1d40 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f9854ff9640 1 -- 172.21.15.16:0/1341713403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98480317b0 con 0x55876ddf1d40 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f9854ff9640 1 -- 172.21.15.16:0/1341713403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848035070 con 0x55876ddf1d40 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f9867311740 1 -- 172.21.15.16:0/1341713403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 msgr2=0x55876ddf4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f9867311740 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9860102cf0 tx=0x7f9848032cb0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.709+0000 7f985ed9d640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 0x7f984c0a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f9858004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 -- 172.21.15.16:0/1341713403 shutdown_connections 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876ddf4130 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876ddf1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 --2- 172.21.15.16:0/1341713403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876dde6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 -- 172.21.15.16:0/1341713403 >> 172.21.15.16:0/1341713403 conn(0x55876dde30f0 msgr2=0x55876dde34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 -- 172.21.15.16:0/1341713403 shutdown_connections 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 -- 172.21.15.16:0/1341713403 wait complete. 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 -- start start 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 Processor -- start 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddeec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.710+0000 7f9867311740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9867311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876ddf1d40 0x55876ddef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9867311740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55876ddf4910 con 0x55876dde7480 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9867311740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55876dd06f10 con 0x55876dde6b60 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9867311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55876ddf4a90 con 0x55876ddf1d40 2024-09-12T00:57:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9867311740 10 client.0 _ll_register_callbacks cb 0x55876dccfa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46662/0 (socket says 172.21.15.16:46662) 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 -- 172.21.15.16:0/1312448043 learned_addr learned my addr 172.21.15.16:0/1312448043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f985e59c640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddeec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876ddf1d40 0x55876ddef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 -- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876ddf1d40 msgr2=0x55876ddef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876ddf1d40 0x55876ddef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 -- 172.21.15.16:0/1312448043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 msgr2=0x55876ddeec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddeec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 -- 172.21.15.16:0/1312448043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876dabd6a0 con 0x55876dde7480 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f985e59c640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddeec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9857fff640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f98480052f0 tx=0x7f9848031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f3d635371bce0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.711+0000 7f9845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984803f070 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98480317b0 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f98480315b0 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9848035040 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 10 client.15708 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55876de004f0 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98480317b0 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f98480046c0 con 0x55876dde7480 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 20 client.15708 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de01a40 0x55876de03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 4 client.15708 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.712+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55876dd597f0 con 0x55876de01a40 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.713+0000 7f985e59c640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de01a40 0x55876de03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.713+0000 7f9845ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.713+0000 7f985e59c640 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de01a40 0x55876de03ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a229403bfbf3ece server_cookie=3ad59c81d0d6833a in_seq=0 out_seq=0 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9845ffb640 1 -- 172.21.15.16:0/1312448043 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55876dd597f0 con 0x55876de01a40 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9845ffb640 10 client.15708 handle_command_reply: tid=0 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 client.15708 shutdown 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 msgr2=0x55876ddef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f98480052f0 tx=0x7f9848031400 comp rx=0 tx=0).stop 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 shutdown_connections 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de01a40 0x55876de03ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876ddf1d40 0x55876ddef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddef140 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 --2- 172.21.15.16:0/1312448043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddeec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 >> 172.21.15.16:0/1312448043 conn(0x55876dde30f0 msgr2=0x55876dde8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9845ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 shutdown_connections 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 1 -- 172.21.15.16:0/1312448043 wait complete. 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.714+0000 7f9867311740 20 client.15708 trim_cache size 0 max 16384 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 Processor -- start 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 -- start start 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddf0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876de097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98601bca70 con 0x55876ddf1d40 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98601be5b0 con 0x55876dde6b60 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9867311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98601bd140 con 0x55876dde7480 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f985ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f985ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46678/0 (socket says 172.21.15.16:46678) 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f985ed9d640 1 -- 172.21.15.16:0/1754024518 learned_addr learned my addr 172.21.15.16:0/1754024518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:04.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f9857fff640 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876de097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.732+0000 7f985e59c640 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddf0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 -- 172.21.15.16:0/1754024518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 msgr2=0x55876de097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876de097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 -- 172.21.15.16:0/1754024518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 msgr2=0x55876ddf0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddf0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 -- 172.21.15.16:0/1754024518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876dde2f20 con 0x55876ddf1d40 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f985ed9d640 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f985800e790 tx=0x7f9858015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7805a121933f835 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f9847fff640 1 -- 172.21.15.16:0/1754024518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985801d0d0 con 0x55876ddf1d40 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f9847fff640 1 -- 172.21.15.16:0/1754024518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985801d270 con 0x55876ddf1d40 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.733+0000 7f9847fff640 1 -- 172.21.15.16:0/1754024518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985801b9f0 con 0x55876ddf1d40 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 -- 172.21.15.16:0/1754024518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 msgr2=0x55876de09d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f985800e790 tx=0x7f9858015000 comp rx=0 tx=0).stop 2024-09-12T00:57:04.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 -- 172.21.15.16:0/1754024518 shutdown_connections 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876ddf1d40 0x55876de09d20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde7480 0x55876de097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 --2- 172.21.15.16:0/1754024518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876dde6b60 0x55876ddf0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 -- 172.21.15.16:0/1754024518 >> 172.21.15.16:0/1754024518 conn(0x55876dde30f0 msgr2=0x55876ddf0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 -- 172.21.15.16:0/1754024518 shutdown_connections 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.734+0000 7f9867311740 1 -- 172.21.15.16:0/1754024518 wait complete. 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 -- start start 2024-09-12T00:57:04.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 Processor -- start 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde6b60 0x55876ddebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46682/0 (socket says 172.21.15.16:46682) 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9857fff640 1 -- 172.21.15.16:0/3280809398 learned_addr learned my addr 172.21.15.16:0/3280809398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876ddf1d40 0x55876ddec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98601bdee0 con 0x55876dde7480 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98601bd810 con 0x55876ddf1d40 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.735+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98601baa30 con 0x55876dde6b60 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9867311740 10 client.0 _ll_register_callbacks cb 0x7f9848005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f985e59c640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde6b60 0x55876ddebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f985ed9d640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876ddf1d40 0x55876ddec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 -- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde6b60 msgr2=0x55876ddebde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde6b60 0x55876ddebde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 -- 172.21.15.16:0/3280809398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876ddf1d40 msgr2=0x55876ddec860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876ddf1d40 0x55876ddec860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876de6b9b0 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9857fff640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9848004b10 tx=0x7f9848032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e22727336884b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9845ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.736+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984803f070 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985801d270 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f9848004240 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9848004440 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9867311740 10 client.15714 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55876de6cc90 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.737+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985801d270 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9848003610 con 0x55876dde7480 2024-09-12T00:57:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 20 client.15714 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de6d350 0x55876de6f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 4 client.15714 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.738+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55876dd53d00 con 0x55876de6d350 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.739+0000 7f985e59c640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de6d350 0x55876de6f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.739+0000 7f985e59c640 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de6d350 0x55876de6f770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ddbac484aaaafcd server_cookie=525351436eac630e in_seq=0 out_seq=0 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.739+0000 7f9845ffb640 10 client.15714 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.740+0000 7f9845ffb640 1 -- 172.21.15.16:0/3280809398 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55876dd53d00 con 0x55876de6d350 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.740+0000 7f9845ffb640 10 client.15714 handle_command_reply: tid=0 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 client.15714 shutdown 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 msgr2=0x55876ddec320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9848004b10 tx=0x7f9848032ee0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 shutdown_connections 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55876de6d350 0x55876de6f770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55876ddf1d40 0x55876ddec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55876dde7480 0x55876ddec320 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 --2- 172.21.15.16:0/3280809398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55876dde6b60 0x55876ddebde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 >> 172.21.15.16:0/3280809398 conn(0x55876dde30f0 msgr2=0x55876dde8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.741+0000 7f9845ffb640 0 client.15714 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.742+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 shutdown_connections 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.742+0000 7f9867311740 1 -- 172.21.15.16:0/3280809398 wait complete. 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.742+0000 7f9867311740 20 client.15714 trim_cache size 0 max 16384 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:57:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "db78ab52-e1a8-4c6e-b32b-524ace6d795b", 2024-09-12T00:57:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:57:04.756 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.742+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 msgr2=0x7f984c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.742+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 0x7f984c0a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f9858004a40 comp rx=0 tx=0).stop 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 msgr2=0x7f986019f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f985000d770 tx=0x7f985000dc30 comp rx=0 tx=0).stop 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 shutdown_connections 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f984c0a2ea0 0x7f984c0a5360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986010e460 0x7f986010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9860106f10 0x7f986010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 --2- 172.21.15.16:0/4243818178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860101de0 0x7f986019f2e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 >> 172.21.15.16:0/4243818178 conn(0x7f98600fd8c0 msgr2=0x7f98600744c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 shutdown_connections 2024-09-12T00:57:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:04.743+0000 7f9864f38640 1 -- 172.21.15.16:0/4243818178 wait complete. 2024-09-12T00:57:05.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:04 smithi099 ceph-mon[35654]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:05.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:04 smithi016 ceph-mon[30972]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:04 smithi159 ceph-mon[35566]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T00:57:05.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:57:05.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:05 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:05 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:05 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:05 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:05 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:57:06.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:05 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:05 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:57:06.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:05 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:05 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:57:06.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:05 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:05 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:57:06.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:05 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.403+0000 7f581a508640 1 -- 172.21.15.16:0/511777134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58141009b0 msgr2=0x7f5814100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.403+0000 7f581a508640 1 --2- 172.21.15.16:0/511777134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58141009b0 0x7f5814100d90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5804009900 tx=0x7f580402a410 comp rx=0 tx=0).stop 2024-09-12T00:57:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 -- 172.21.15.16:0/511777134 shutdown_connections 2024-09-12T00:57:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 --2- 172.21.15.16:0/511777134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581410e4d0 0x7f58141108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 --2- 172.21.15.16:0/511777134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5814101360 0x7f581410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 --2- 172.21.15.16:0/511777134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58141009b0 0x7f5814100d90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 -- 172.21.15.16:0/511777134 >> 172.21.15.16:0/511777134 conn(0x7f58140fc7d0 msgr2=0x7f58140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 -- 172.21.15.16:0/511777134 shutdown_connections 2024-09-12T00:57:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.404+0000 7f581a508640 1 -- 172.21.15.16:0/511777134 wait complete. 2024-09-12T00:57:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.405+0000 7f581a508640 1 Processor -- start 2024-09-12T00:57:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.405+0000 7f581a508640 1 -- start start 2024-09-12T00:57:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 0x7f58141a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5814101360 0x7f58141a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5814114230 con 0x7f581410e4d0 2024-09-12T00:57:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58141140b0 con 0x7f58141009b0 2024-09-12T00:57:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f581a508640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58141143b0 con 0x7f5814101360 2024-09-12T00:57:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f5818aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f5818aa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46698/0 (socket says 172.21.15.16:46698) 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f5818aa5640 1 -- 172.21.15.16:0/2064386193 learned_addr learned my addr 172.21.15.16:0/2064386193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f5813fff640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 0x7f58141a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.406+0000 7f58137fe640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5814101360 0x7f58141a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 -- 172.21.15.16:0/2064386193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5814101360 msgr2=0x7f58141a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5814101360 0x7f58141a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 -- 172.21.15.16:0/2064386193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 msgr2=0x7f58141a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 0x7f58141a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 -- 172.21.15.16:0/2064386193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581419acf0 con 0x7f581410e4d0 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5813fff640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 0x7f58141a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f5818aa5640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f580c00eaa0 tx=0x7f580c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f58117fa640 1 -- 172.21.15.16:0/2064386193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c00cdb0 con 0x7f581410e4d0 2024-09-12T00:57:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f58117fa640 1 -- 172.21.15.16:0/2064386193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c005020 con 0x7f581410e4d0 2024-09-12T00:57:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581419afe0 con 0x7f581410e4d0 2024-09-12T00:57:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f58117fa640 1 -- 172.21.15.16:0/2064386193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0116b0 con 0x7f581410e4d0 2024-09-12T00:57:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.407+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a72d0 con 0x7f581410e4d0 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.410+0000 7f58117fa640 1 -- 172.21.15.16:0/2064386193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f580c005560 con 0x7f581410e4d0 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 Processor -- start 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 -- start start 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce4397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce44c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce39f2e0 con 0x55a3ce44ce90 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce346ce0 con 0x55a3ce439ce0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f581c8e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce39b2a0 con 0x55a3ce4393c0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.411+0000 7f58117fa640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46700/0 (socket says 172.21.15.16:46700) 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 -- 172.21.15.16:0/1770256672 learned_addr learned my addr 172.21.15.16:0/1770256672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58117fa640 1 -- 172.21.15.16:0/2064386193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f580c0cf8e0 con 0x7f581410e4d0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f5818aa5640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce4397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 -- 172.21.15.16:0/1770256672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 msgr2=0x55a3ce44c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce44c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 -- 172.21.15.16:0/1770256672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 msgr2=0x55a3ce44f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 -- 172.21.15.16:0/1770256672 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ce32b440 con 0x55a3ce4393c0 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f5818aa5640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cce80 tx=0x7f580c000f40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.412+0000 7f58137fe640 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce4397a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f580800cef0 tx=0x7f58080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e679fc510ce58f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.413+0000 7f57fa7fc640 1 -- 172.21.15.16:0/1770256672 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58080142d0 con 0x55a3ce4393c0 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.413+0000 7f57fa7fc640 1 -- 172.21.15.16:0/1770256672 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808005500 con 0x55a3ce4393c0 2024-09-12T00:57:06.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.413+0000 7f57fa7fc640 1 -- 172.21.15.16:0/1770256672 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808011730 con 0x55a3ce4393c0 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- 172.21.15.16:0/1770256672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 msgr2=0x55a3ce4397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce4397a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f580800cef0 tx=0x7f58080046c0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- 172.21.15.16:0/1770256672 shutdown_connections 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce44c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1770256672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce4397a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- 172.21.15.16:0/1770256672 >> 172.21.15.16:0/1770256672 conn(0x55a3ce435a30 msgr2=0x55a3ce435e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- 172.21.15.16:0/1770256672 shutdown_connections 2024-09-12T00:57:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- 172.21.15.16:0/1770256672 wait complete. 2024-09-12T00:57:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 -- start start 2024-09-12T00:57:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.414+0000 7f581c8e1740 1 Processor -- start 2024-09-12T00:57:06.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce44c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce449b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce44f9a0 con 0x55a3ce44ce90 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce35a050 con 0x55a3ce439ce0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3ce44fb20 con 0x55a3ce4393c0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f58137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f58137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46702/0 (socket says 172.21.15.16:46702) 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f58137fe640 1 -- 172.21.15.16:0/3753052566 learned_addr learned my addr 172.21.15.16:0/3753052566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f581c8e1740 10 client.0 _ll_register_callbacks cb 0x55a3ce39a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f58137fe640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce44c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.415+0000 7f5813fff640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce449b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 -- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 msgr2=0x55a3ce44c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce44c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 -- 172.21.15.16:0/3753052566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 msgr2=0x55a3ce449b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce449b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 -- 172.21.15.16:0/3753052566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ce1103c0 con 0x55a3ce44ce90 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f58137fe640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f580800c450 tx=0x7f5808004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c169d20b0f2b399 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f57f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808004b00 con 0x55a3ce44ce90 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f5808004030 con 0x55a3ce44ce90 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.416+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5808005500 con 0x55a3ce44ce90 2024-09-12T00:57:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5808004230 con 0x55a3ce44ce90 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 10 client.15726 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3ce45b450 con 0x55a3ce44ce90 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808005500 con 0x55a3ce44ce90 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f58080037b0 con 0x55a3ce44ce90 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 20 client.15726 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce45c7a0 0x55a3ce45eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 4 client.15726 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3ce3ac850 con 0x55a3ce45c7a0 2024-09-12T00:57:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.417+0000 7f5813fff640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce45c7a0 0x55a3ce45eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.418+0000 7f57f37fe640 10 client.15726 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.418+0000 7f5813fff640 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce45c7a0 0x55a3ce45eb90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ccf6a3517757e3 server_cookie=d91b0e379a1dbd5 in_seq=0 out_seq=0 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.419+0000 7f57f37fe640 1 -- 172.21.15.16:0/3753052566 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3ce3ac850 con 0x55a3ce45c7a0 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.419+0000 7f57f37fe640 10 client.15726 handle_command_reply: tid=0 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.419+0000 7f581c8e1740 1 client.15726 shutdown 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 msgr2=0x55a3ce44a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f580800c450 tx=0x7f5808004d00 comp rx=0 tx=0).stop 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 shutdown_connections 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce45c7a0 0x55a3ce45eb90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f57f37fe640 0 client.15726 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce44ce90 0x55a3ce44a100 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce439ce0 0x55a3ce449b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 --2- 172.21.15.16:0/3753052566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce44c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 >> 172.21.15.16:0/3753052566 conn(0x55a3ce435a30 msgr2=0x55a3ce434b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 shutdown_connections 2024-09-12T00:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 1 -- 172.21.15.16:0/3753052566 wait complete. 2024-09-12T00:57:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.420+0000 7f581c8e1740 20 client.15726 trim_cache size 0 max 16384 2024-09-12T00:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.454+0000 7f581c8e1740 1 Processor -- start 2024-09-12T00:57:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.454+0000 7f581c8e1740 1 -- start start 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.454+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce458ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.454+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 0x55a3ce44ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f581c8e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ab6a0 con 0x55a3ce439ce0 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f581c8e1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ad1e0 con 0x55a3ce445110 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f581c8e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58141abd70 con 0x55a3ce4393c0 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46704/0 (socket says 172.21.15.16:46704) 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 -- 172.21.15.16:0/1795544506 learned_addr learned my addr 172.21.15.16:0/1795544506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f58137fe640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce458ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f58137fe640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 0x55a3ce44ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 -- 172.21.15.16:0/1795544506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 msgr2=0x55a3ce458ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce458ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 -- 172.21.15.16:0/1795544506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 msgr2=0x55a3ce44ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 0x55a3ce44ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 -- 172.21.15.16:0/1795544506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3ce4366c0 con 0x55a3ce439ce0 2024-09-12T00:57:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f58137fe640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 0x55a3ce44ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.455+0000 7f5813fff640 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f580402a930 tx=0x7f58040058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6399aeb86f108dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.456+0000 7f57f9ffb640 1 -- 172.21.15.16:0/1795544506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804035040 con 0x55a3ce439ce0 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.456+0000 7f57f9ffb640 1 -- 172.21.15.16:0/1795544506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804048070 con 0x55a3ce439ce0 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.456+0000 7f57f9ffb640 1 -- 172.21.15.16:0/1795544506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004a40 con 0x55a3ce439ce0 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- 172.21.15.16:0/1795544506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 msgr2=0x55a3ce4483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f580402a930 tx=0x7f58040058b0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- 172.21.15.16:0/1795544506 shutdown_connections 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce445110 0x55a3ce44ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce439ce0 0x55a3ce4483e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1795544506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce458ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- 172.21.15.16:0/1795544506 >> 172.21.15.16:0/1795544506 conn(0x55a3ce435a30 msgr2=0x55a3ce44b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- 172.21.15.16:0/1795544506 shutdown_connections 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- 172.21.15.16:0/1795544506 wait complete. 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 -- start start 2024-09-12T00:57:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.457+0000 7f581c8e1740 1 Processor -- start 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce454ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 0x55a3ce455420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58141acb10 con 0x55a3ce445110 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58141ac440 con 0x55a3ce451280 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58141a9680 con 0x55a3ce4393c0 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f581c8e1740 10 client.0 _ll_register_callbacks cb 0x7f5808014690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:48196/0 (socket says 172.21.15.16:48196) 2024-09-12T00:57:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 -- 172.21.15.16:0/1417324665 learned_addr learned my addr 172.21.15.16:0/1417324665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce454ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f5813fff640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 0x55a3ce455420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 -- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 msgr2=0x55a3ce454ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce454ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 -- 172.21.15.16:0/1417324665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 msgr2=0x55a3ce455420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.458+0000 7f58137fe640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 0x55a3ce455420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f58137fe640 1 -- 172.21.15.16:0/1417324665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ce451df0 con 0x55a3ce451280 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f5813fff640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 0x55a3ce455420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f58137fe640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58080113e0 tx=0x7f5808004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e56b0116129a365 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f57f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808036040 con 0x55a3ce451280 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f5808004030 con 0x55a3ce451280 2024-09-12T00:57:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5804048070 con 0x55a3ce451280 2024-09-12T00:57:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f58080035f0 con 0x55a3ce451280 2024-09-12T00:57:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f581c8e1740 10 client.25289 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.459+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3ce4c7ff0 con 0x55a3ce451280 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5808037040 con 0x55a3ce451280 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5808020c80 con 0x55a3ce451280 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 10 client.25289 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 20 client.25289 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce4c86b0 0x55a3ce4caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 4 client.25289 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5804048070 con 0x55a3ce4c86b0 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.460+0000 7f5813fff640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce4c86b0 0x55a3ce4caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f57f37fe640 10 client.25289 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f5813fff640 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce4c86b0 0x55a3ce4caaa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dcbfbcebed27986 server_cookie=e094de8598ea5b97 in_seq=0 out_seq=0 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f57f37fe640 1 -- 172.21.15.16:0/1417324665 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5804048070 con 0x55a3ce4c86b0 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f57f37fe640 10 client.25289 handle_command_reply: tid=0 2024-09-12T00:57:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 client.25289 shutdown 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 msgr2=0x55a3ce451700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58080113e0 tx=0x7f5808004d00 comp rx=0 tx=0).stop 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 shutdown_connections 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a3ce4c86b0 0x55a3ce4caaa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a3ce451280 0x55a3ce451700 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3ce445110 0x55a3ce455420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 --2- 172.21.15.16:0/1417324665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3ce4393c0 0x55a3ce454ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 >> 172.21.15.16:0/1417324665 conn(0x55a3ce435a30 msgr2=0x55a3ce44bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f57f37fe640 0 client.25289 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 shutdown_connections 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.461+0000 7f581c8e1740 1 -- 172.21.15.16:0/1417324665 wait complete. 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "db78ab52-e1a8-4c6e-b32b-524ace6d795b": { 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "db78ab52-e1a8-4c6e-b32b-524ace6d795b", 2024-09-12T00:57:06.464 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.462+0000 7f581c8e1740 20 client.25289 trim_cache size 0 max 16384 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 msgr2=0x7f57fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f580c0cce80 tx=0x7f580c000f40 comp rx=0 tx=0).stop 2024-09-12T00:57:06.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 msgr2=0x7f581419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f580c00eaa0 tx=0x7f580c00ef60 comp rx=0 tx=0).stop 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 shutdown_connections 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f581410e4d0 0x7f581419a5c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5814101360 0x7f58141a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 --2- 172.21.15.16:0/2064386193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58141009b0 0x7f58141a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.464+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 >> 172.21.15.16:0/2064386193 conn(0x7f58140fc7d0 msgr2=0x7f58140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.465+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 shutdown_connections 2024-09-12T00:57:06.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:06.465+0000 7f581a508640 1 -- 172.21.15.16:0/2064386193 wait complete. 2024-09-12T00:57:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:06 smithi016 ceph-mon[30972]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T00:57:07.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:06 smithi099 ceph-mon[35654]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T00:57:07.335 INFO:tasks.cephfs.filesystem:scrub status for tag:db78ab52-e1a8-4c6e-b32b-524ace6d795b - {'path': '/', 'tag': 'db78ab52-e1a8-4c6e-b32b-524ace6d795b', 'options': 'recursive,force'} 2024-09-12T00:57:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:06 smithi159 ceph-mon[35566]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T00:57:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:08 smithi099 ceph-mon[35654]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T00:57:09.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:08 smithi159 ceph-mon[35566]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T00:57:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:08 smithi016 ceph-mon[30972]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T00:57:10.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:09 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:57:10.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:09 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:57:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:09 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:57:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:10 smithi099 ceph-mon[35654]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-12T00:57:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:10 smithi159 ceph-mon[35566]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-12T00:57:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:10 smithi016 ceph-mon[30972]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-12T00:57:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:12 smithi099 ceph-mon[35654]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:12 smithi159 ceph-mon[35566]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:12 smithi016 ceph-mon[30972]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:57:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:14.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:13 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:57:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:14.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:13 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:57:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:13 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:14 smithi099 ceph-mon[35654]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:14 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:57:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:14 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:57:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:14 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:14 smithi159 ceph-mon[35566]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:14 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:57:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:14 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:57:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:14 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:14 smithi016 ceph-mon[30972]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-12T00:57:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:14 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:57:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:14 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:57:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:14 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:57:15.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:57:15.968 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:16.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/524818229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28103710 msgr2=0x7f8f2810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:16.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/524818229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28103710 0x7f8f2810bc00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009980 tx=0x7f8f1002e8b0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/524818229 shutdown_connections 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/524818229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f2810c190 0x7f8f2810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/524818229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28103710 0x7f8f2810bc00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/524818229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f28102df0 0x7f8f281031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/524818229 >> 172.21.15.16:0/524818229 conn(0x7f8f280fc7d0 msgr2=0x7f8f280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/524818229 shutdown_connections 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.524+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/524818229 wait complete. 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.525+0000 7f8f2c9d4640 1 Processor -- start 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.525+0000 7f8f2c9d4640 1 -- start start 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 0x7f8f28196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f2810c190 0x7f8f2819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:16.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fd70 con 0x7f8f28102df0 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fbf0 con 0x7f8f28103710 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2c9d4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810fef0 con 0x7f8f2810c190 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57760/0 (socket says 172.21.15.16:57760) 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.526+0000 7f8f2659c640 1 -- 172.21.15.16:0/3090008252 learned_addr learned my addr 172.21.15.16:0/3090008252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f1dd9b640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 0x7f8f28196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f26d9d640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f2810c190 0x7f8f2819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 -- 172.21.15.16:0/3090008252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f2810c190 msgr2=0x7f8f2819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f2810c190 0x7f8f2819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 -- 172.21.15.16:0/3090008252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 msgr2=0x7f8f28196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 0x7f8f28196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 -- 172.21.15.16:0/3090008252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2819b1e0 con 0x7f8f28102df0 2024-09-12T00:57:16.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f1dd9b640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 0x7f8f28196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f2659c640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d950 tx=0x7f8f1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.527+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18017070 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.528+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2819b4d0 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.528+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f18009c30 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.528+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281a2db0 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.528+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800f6b0 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.530+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f08005290 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.532+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f8f1800f850 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.533+0000 7f8f1ffff640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 0x7f8f140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.533+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f8f180cf5b0 con 0x7f8f28102df0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.533+0000 7f8f1dd9b640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 0x7f8f140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:16.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.534+0000 7f8f1dd9b640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 0x7f8f140a53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f2810d6a0 tx=0x7f8f10005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:16.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.542+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f1809bfc0 con 0x7f8f28102df0 2024-09-12T00:57:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.976+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f08005850 con 0x7f8f28102df0 2024-09-12T00:57:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.977+0000 7f8f1ffff640 1 -- 172.21.15.16:0/3090008252 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f8f18009de0 con 0x7f8f28102df0 2024-09-12T00:57:16.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:57:16.977 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:57:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.980+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 msgr2=0x7f8f140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.980+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 0x7f8f140a53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8f2810d6a0 tx=0x7f8f10005a20 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.980+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 msgr2=0x7f8f28196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.980+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800d950 tx=0x7f8f1800de10 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 shutdown_connections 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8f140a2f20 0x7f8f140a53e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8f2810c190 0x7f8f2819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f28103710 0x7f8f28196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 --2- 172.21.15.16:0/3090008252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f28102df0 0x7f8f28196230 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 >> 172.21.15.16:0/3090008252 conn(0x7f8f280fc7d0 msgr2=0x7f8f280fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 shutdown_connections 2024-09-12T00:57:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:16.981+0000 7f8f2c9d4640 1 -- 172.21.15.16:0/3090008252 wait complete. 2024-09-12T00:57:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:16 smithi099 ceph-mon[35654]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-12T00:57:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:16 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3090008252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:16 smithi159 ceph-mon[35566]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-12T00:57:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:16 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3090008252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:16 smithi016 ceph-mon[30972]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-12T00:57:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:16 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3090008252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:17.890 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 4a8b0dcf --release --await' 2024-09-12T00:57:17.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 4a8b0dcf --release --await 2024-09-12T00:57:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 -- 172.21.15.16:0/3339593709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c102d20 msgr2=0x7f7e9c103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/3339593709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c102d20 0x7f7e9c103100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009900 tx=0x7f7e9802a480 comp rx=0 tx=0).stop 2024-09-12T00:57:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 -- 172.21.15.16:0/3339593709 shutdown_connections 2024-09-12T00:57:18.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/3339593709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/3339593709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c103640 0x7f7e9c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/3339593709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c102d20 0x7f7e9c103100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 -- 172.21.15.16:0/3339593709 >> 172.21.15.16:0/3339593709 conn(0x7f7e9c0fc780 msgr2=0x7f7e9c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 -- 172.21.15.16:0/3339593709 shutdown_connections 2024-09-12T00:57:18.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.742+0000 7f7ea46a5640 1 -- 172.21.15.16:0/3339593709 wait complete. 2024-09-12T00:57:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.743+0000 7f7ea46a5640 1 Processor -- start 2024-09-12T00:57:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.743+0000 7f7ea46a5640 1 -- start start 2024-09-12T00:57:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c102d20 0x7f7e9c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 0x7f7e9c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c111f50 con 0x7f7e9c103640 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c111dd0 con 0x7f7e9c102d20 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea46a5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1120d0 con 0x7f7e9c10c150 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:51856/0 (socket says 172.21.15.16:51856) 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 -- 172.21.15.16:0/1489535699 learned_addr learned my addr 172.21.15.16:0/1489535699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2441640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c102d20 0x7f7e9c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 0x7f7e9c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 -- 172.21.15.16:0/1489535699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c102d20 msgr2=0x7f7e9c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c102d20 0x7f7e9c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.744+0000 7f7ea2c42640 1 -- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 msgr2=0x7f7e9c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea2c42640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 0x7f7e9c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea2c42640 1 -- 172.21.15.16:0/1489535699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c19d3f0 con 0x7f7e9c10c150 2024-09-12T00:57:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 0x7f7e9c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea2c42640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400eaa0 tx=0x7f7e9400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7e937fe640 1 -- 172.21.15.16:0/1489535699 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400cdb0 con 0x7f7e9c10c150 2024-09-12T00:57:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7e937fe640 1 -- 172.21.15.16:0/1489535699 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94005020 con 0x7f7e9c10c150 2024-09-12T00:57:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c19d6e0 con 0x7f7e9c10c150 2024-09-12T00:57:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.745+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1008b0 con 0x7f7e9c10c150 2024-09-12T00:57:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.746+0000 7f7e937fe640 1 -- 172.21.15.16:0/1489535699 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940116b0 con 0x7f7e9c10c150 2024-09-12T00:57:18.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.748+0000 7f7e937fe640 1 -- 172.21.15.16:0/1489535699 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f7e94004a40 con 0x7f7e9c10c150 2024-09-12T00:57:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 Processor -- start 2024-09-12T00:57:18.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 -- start start 2024-09-12T00:57:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b372440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b372980 0x562c9b37cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c9b2d7620 con 0x562c9b37d240 2024-09-12T00:57:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562c9b27ee50 con 0x562c9b372060 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.749+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c9b2d35e0 con 0x562c9b372980 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7e937fe640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 0x7f7e880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45386/0 (socket says 172.21.15.16:45386) 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 -- 172.21.15.16:0/3415229045 learned_addr learned my addr 172.21.15.16:0/3415229045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea2441640 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b372980 0x562c9b37cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea2c42640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 0x7f7e880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 -- 172.21.15.16:0/3415229045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b372980 msgr2=0x562c9b37cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b372980 0x562c9b37cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 -- 172.21.15.16:0/3415229045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 msgr2=0x562c9b372440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b372440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7ea1c40640 1 -- 172.21.15.16:0/3415229045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9b26fd40 con 0x562c9b37d240 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.750+0000 7f7e937fe640 1 -- 172.21.15.16:0/1489535699 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7e940cf6a0 con 0x7f7e9c10c150 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.751+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0104a0 tx=0x7f7e8c010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e67191a4ba79b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.751+0000 7f7e90ff9640 1 -- 172.21.15.16:0/3415229045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c004220 con 0x562c9b37d240 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.751+0000 7f7e90ff9640 1 -- 172.21.15.16:0/3415229045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c0043c0 con 0x562c9b37d240 2024-09-12T00:57:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.751+0000 7f7e90ff9640 1 -- 172.21.15.16:0/3415229045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c011af0 con 0x562c9b37d240 2024-09-12T00:57:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3415229045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 msgr2=0x562c9b37f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea2c42640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 0x7f7e880a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e940ccc40 tx=0x7f7e940d5240 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0104a0 tx=0x7f7e8c010960 comp rx=0 tx=0).stop 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3415229045 shutdown_connections 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b37d240 0x562c9b37f630 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b372980 0x562c9b37cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3415229045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b372440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.752+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3415229045 >> 172.21.15.16:0/3415229045 conn(0x562c9b36e5f0 msgr2=0x562c9b36e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.753+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3415229045 shutdown_connections 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.753+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3415229045 wait complete. 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.753+0000 7f7ea6a7e740 1 -- start start 2024-09-12T00:57:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.753+0000 7f7ea6a7e740 1 Processor -- start 2024-09-12T00:57:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b37abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45396/0 (socket says 172.21.15.16:45396) 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea1c40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea1c40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:56016/0 (socket says 172.21.15.16:56016) 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea2441640 1 -- 172.21.15.16:0/727108165 learned_addr learned my addr 172.21.15.16:0/727108165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c9b37fe10 con 0x562c9b372980 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562c9b25ff10 con 0x562c9b372060 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.754+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c9b37ff90 con 0x562c9b37d240 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 -- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 msgr2=0x562c9b37abd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b37abd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 -- 172.21.15.16:0/727108165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 msgr2=0x562c9b37a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9afe6580 con 0x562c9b372980 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea6a7e740 10 client.0 _ll_register_callbacks cb 0x562c9b283db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c0043c0 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7ea2441640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e9802a9a0 tx=0x7f7e98031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c4e89b8ff41a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.755+0000 7f7e85ffb640 10 client.15750 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.756+0000 7f7e85ffb640 1 -- 172.21.15.16:0/727108165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98035040 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.756+0000 7f7e85ffb640 1 -- 172.21.15.16:0/727108165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7e98004240 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.756+0000 7f7e85ffb640 1 -- 172.21.15.16:0/727108165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e980044a0 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.756+0000 7f7ea6a7e740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.756+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=28}) -- 0x562c9b0b0960 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7e85ffb640 1 -- 172.21.15.16:0/727108165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7e980035f0 con 0x562c9b372980 2024-09-12T00:57:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 10 client.15750 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:57:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 20 client.15750 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b38f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 4 client.15750 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c9b2e4b90 con 0x562c9b38cdf0 2024-09-12T00:57:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.757+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b38f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.758+0000 7f7e85ffb640 10 client.15750 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.758+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b38f270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93359bc2fc5df17 server_cookie=72efb0c360bee127 in_seq=0 out_seq=0 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.758+0000 7f7e85ffb640 1 -- 172.21.15.16:0/727108165 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c9b2e4b90 con 0x562c9b38cdf0 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.758+0000 7f7e85ffb640 10 client.15750 handle_command_reply: tid=0 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.758+0000 7f7ea6a7e740 1 client.15750 shutdown 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 msgr2=0x562c9b37a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e9802a9a0 tx=0x7f7e98031850 comp rx=0 tx=0).stop 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 shutdown_connections 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b38f270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b37abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372980 0x562c9b37a690 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7e85ffb640 0 client.15750 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/727108165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372060 0x562c9b37a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 >> 172.21.15.16:0/727108165 conn(0x562c9b36e5f0 msgr2=0x562c9b3742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 shutdown_connections 2024-09-12T00:57:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.759+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/727108165 wait complete. 2024-09-12T00:57:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.760+0000 7f7ea6a7e740 20 client.15750 trim_cache size 0 max 16384 2024-09-12T00:57:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.793+0000 7f7ea6a7e740 1 Processor -- start 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.793+0000 7f7ea6a7e740 1 -- start start 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b394af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b3773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000e640 con 0x562c9b372060 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80010180 con 0x562c9b372980 2024-09-12T00:57:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000ed10 con 0x562c9b37d240 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:56018/0 (socket says 172.21.15.16:56018) 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 -- 172.21.15.16:0/3762031538 learned_addr learned my addr 172.21.15.16:0/3762031538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 -- 172.21.15.16:0/3762031538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 msgr2=0x562c9b3773b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b3773b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 -- 172.21.15.16:0/3762031538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 msgr2=0x562c9b394af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b394af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.794+0000 7f7ea2441640 1 -- 172.21.15.16:0/3762031538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9b36f760 con 0x562c9b372980 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.795+0000 7f7ea2441640 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009230 tx=0x7f7e9803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43b80424bd884e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.795+0000 7f7e87fff640 1 -- 172.21.15.16:0/3762031538 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98035030 con 0x562c9b372980 2024-09-12T00:57:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.795+0000 7f7e87fff640 1 -- 172.21.15.16:0/3762031538 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c0043c0 con 0x562c9b372980 2024-09-12T00:57:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.795+0000 7f7e87fff640 1 -- 172.21.15.16:0/3762031538 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980046c0 con 0x562c9b372980 2024-09-12T00:57:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3762031538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 msgr2=0x562c9b395030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009230 tx=0x7f7e9803e000 comp rx=0 tx=0).stop 2024-09-12T00:57:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3762031538 shutdown_connections 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b3773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b395030 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/3762031538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b394af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.796+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3762031538 >> 172.21.15.16:0/3762031538 conn(0x562c9b36e5f0 msgr2=0x562c9b378a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.797+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3762031538 shutdown_connections 2024-09-12T00:57:18.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.797+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/3762031538 wait complete. 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.797+0000 7f7ea6a7e740 1 -- start start 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.797+0000 7f7ea6a7e740 1 Processor -- start 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b381290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b381d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000fab0 con 0x562c9b372060 2024-09-12T00:57:18.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000f3e0 con 0x562c9b372980 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8000c620 con 0x562c9b37d240 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.798+0000 7f7ea6a7e740 10 client.0 _ll_register_callbacks cb 0x7f7e980095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:56028/0 (socket says 172.21.15.16:56028) 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 -- 172.21.15.16:0/1056947739 learned_addr learned my addr 172.21.15.16:0/1056947739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 -- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 msgr2=0x562c9b381d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b381d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 -- 172.21.15.16:0/1056947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 msgr2=0x562c9b381290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b381290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 -- 172.21.15.16:0/1056947739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c9b3f6d60 con 0x562c9b372980 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea2441640 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e98005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bde7c752800db2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98035070 con 0x562c9b372980 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7e98003aa0 con 0x562c9b372980 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c0043c0 con 0x562c9b372980 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e98067050 con 0x562c9b372980 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea6a7e740 10 client.25307 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.799+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562c9b3f8080 con 0x562c9b372980 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c0043c0 con 0x562c9b372980 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7e98004c70 con 0x562c9b372980 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 10 client.25307 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 10 client.25307 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 20 client.25307 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b3fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 4 client.25307 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4a8b0dcf", "release": true, "await": true}] 2024-09-12T00:57:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.800+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4a8b0dcf", "release": true, "await": true}) -- 0x562c9b2df0a0 con 0x562c9b38cdf0 2024-09-12T00:57:18.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.801+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b3fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.802+0000 7f7e85ffb640 10 client.25307 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.802+0000 7f7ea1c40640 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b3fa830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ed62f6864315df server_cookie=a03748f99a557627 in_seq=0 out_seq=0 2024-09-12T00:57:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.804+0000 7f7e85ffb640 1 -- 172.21.15.16:0/1056947739 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562c9b2df0a0 con 0x562c9b38cdf0 2024-09-12T00:57:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.804+0000 7f7e85ffb640 10 client.25307 handle_command_reply: tid=0 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.804+0000 7f7ea6a7e740 1 client.25307 shutdown 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 msgr2=0x562c9b3817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e98005c00 comp rx=0 tx=0).stop 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 shutdown_connections 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562c9b38cdf0 0x562c9b3fa830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c9b37d240 0x562c9b381d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c9b372980 0x562c9b3817d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 --2- 172.21.15.16:0/1056947739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c9b372060 0x562c9b381290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 >> 172.21.15.16:0/1056947739 conn(0x562c9b36e5f0 msgr2=0x562c9b373ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7e85ffb640 0 client.25307 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 shutdown_connections 2024-09-12T00:57:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 1 -- 172.21.15.16:0/1056947739 wait complete. 2024-09-12T00:57:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.805+0000 7f7ea6a7e740 20 client.25307 trim_cache size 0 max 16384 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "4a8b0dcf": { 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:57:18.807 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.5, 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:18.808 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 msgr2=0x7f7e880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 0x7f7e880a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e940ccc40 tx=0x7f7e940d5240 comp rx=0 tx=0).stop 2024-09-12T00:57:18.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 msgr2=0x7f7e9c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400eaa0 tx=0x7f7e9400ef60 comp rx=0 tx=0).stop 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 shutdown_connections 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7e880a2f20 0x7f7e880a53e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c10c150 0x7f7e9c19cd10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e9c103640 0x7f7e9c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 --2- 172.21.15.16:0/1489535699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e9c102d20 0x7f7e9c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.807+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 >> 172.21.15.16:0/1489535699 conn(0x7f7e9c0fc780 msgr2=0x7f7e9c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.808+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 shutdown_connections 2024-09-12T00:57:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:18.808+0000 7f7ea46a5640 1 -- 172.21.15.16:0/1489535699 wait complete. 2024-09-12T00:57:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:18 smithi099 ceph-mon[35654]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T00:57:19.335 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:18 smithi016 ceph-mon[30972]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T00:57:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:18 smithi159 ceph-mon[35566]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-12T00:57:19.688 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4a8b0dcf', total seconds elapsed: 28.5 2024-09-12T00:57:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:20 smithi099 ceph-mon[35654]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-12T00:57:21.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:20 smithi159 ceph-mon[35566]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-12T00:57:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:20 smithi016 ceph-mon[30972]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 13 op/s 2024-09-12T00:57:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:22 smithi099 ceph-mon[35654]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:22 smithi159 ceph-mon[35566]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:22 smithi016 ceph-mon[30972]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:24 smithi099 ceph-mon[35654]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:24 smithi159 ceph-mon[35566]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:24 smithi016 ceph-mon[30972]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:27.098 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 93% (12120/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T00:57:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:27 smithi099 ceph-mon[35654]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:27 smithi159 ceph-mon[35566]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:27 smithi016 ceph-mon[30972]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.514 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-12T00:57:27.515 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T00:57:27.520 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T00:57:27.547 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T00:57:27.549 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T00:57:27.549 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T00:57:28.276 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T00:57:28.358 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T00:57:28.400 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T00:57:28.401 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T00:57:28.401 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T00:57:28.441 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T00:57:28.445 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T00:57:28.446 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T00:57:28.503 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-09-12T00:57:28.505 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-09-12T00:57:28.506 DEBUG:teuthology.orchestra.run.smithi016:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T00:57:28.567 INFO:tasks.workunit.client.0.smithi016.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-09-12T00:57:28.567 INFO:tasks.workunit.client.0.smithi016.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-09-12T00:57:28.567 INFO:tasks.workunit.client.0.smithi016.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-09-12T00:57:29.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:29 smithi099 ceph-mon[35654]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:57:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:29 smithi159 ceph-mon[35566]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:57:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:29 smithi016 ceph-mon[30972]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T00:57:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:31 smithi099 ceph-mon[35654]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:31.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:31 smithi159 ceph-mon[35566]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:31.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:31 smithi016 ceph-mon[30972]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-12T00:57:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:33 smithi099 ceph-mon[35654]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:33 smithi159 ceph-mon[35566]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:33.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:33 smithi016 ceph-mon[30972]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:35.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:35 smithi099 ceph-mon[35654]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:35 smithi159 ceph-mon[35566]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:35.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:35 smithi016 ceph-mon[30972]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-12T00:57:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:37 smithi099 ceph-mon[35654]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-09-12T00:57:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:37 smithi099 ceph-mon[35654]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T00:57:37.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:57:37.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:37 smithi016 ceph-mon[30972]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-09-12T00:57:37.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:37 smithi016 ceph-mon[30972]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T00:57:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:37 smithi159 ceph-mon[35566]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-09-12T00:57:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:37 smithi159 ceph-mon[35566]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T00:57:37.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 -- 172.21.15.16:0/4034578561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 msgr2=0x7fa698100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 --2- 172.21.15.16:0/4034578561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa698100ee0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa688009930 tx=0x7fa68802a420 comp rx=0 tx=0).stop 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 -- 172.21.15.16:0/4034578561 shutdown_connections 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 --2- 172.21.15.16:0/4034578561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 0x7fa69810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 --2- 172.21.15.16:0/4034578561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa698100ee0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 --2- 172.21.15.16:0/4034578561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 0x7fa698106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 -- 172.21.15.16:0/4034578561 >> 172.21.15.16:0/4034578561 conn(0x7fa6980fc7a0 msgr2=0x7fa6980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.077+0000 7fa69f475640 1 -- 172.21.15.16:0/4034578561 shutdown_connections 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 -- 172.21.15.16:0/4034578561 wait complete. 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 Processor -- start 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 -- start start 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 0x7fa698198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 0x7fa69819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa698110e80 con 0x7fa698100a80 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa698110d00 con 0x7fa698101420 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69f475640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa698111000 con 0x7fa698106a80 2024-09-12T00:57:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.078+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55256/0 (socket says 172.21.15.16:55256) 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 -- 172.21.15.16:0/512738687 learned_addr learned my addr 172.21.15.16:0/512738687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69ca10640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 0x7fa698198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69da12640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 0x7fa69819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 -- 172.21.15.16:0/512738687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 msgr2=0x7fa69819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 0x7fa69819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 -- 172.21.15.16:0/512738687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 msgr2=0x7fa698198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 0x7fa698198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 -- 172.21.15.16:0/512738687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69819d470 con 0x7fa698100a80 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69d211640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa68e7fc640 1 -- 172.21.15.16:0/512738687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694017070 con 0x7fa698100a80 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69819d760 con 0x7fa698100a80 2024-09-12T00:57:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa68e7fc640 1 -- 172.21.15.16:0/512738687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa694005500 con 0x7fa698100a80 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa68e7fc640 1 -- 172.21.15.16:0/512738687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694004e00 con 0x7fa698100a80 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.079+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a5040 con 0x7fa698100a80 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.082+0000 7fa68e7fc640 1 -- 172.21.15.16:0/512738687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa694004fa0 con 0x7fa698100a80 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 Processor -- start 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 -- start start 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e85e3e2e0 con 0x557e85ed8340 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e85de5ce0 con 0x557e85ed8c60 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa6a184e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e85e3a2a0 con 0x557e85ee7a90 2024-09-12T00:57:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55266/0 (socket says 172.21.15.16:55266) 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 -- 172.21.15.16:0/2116726464 learned_addr learned my addr 172.21.15.16:0/2116726464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69d211640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69ca10640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 -- 172.21.15.16:0/2116726464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 msgr2=0x557e85ee9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 -- 172.21.15.16:0/2116726464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 msgr2=0x557e85ee7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa68e7fc640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69da12640 1 -- 172.21.15.16:0/2116726464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e85dca440 con 0x557e85ed8340 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69ca10640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.083+0000 7fa69d211640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa68e7fc640 1 -- 172.21.15.16:0/512738687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa6940cee40 con 0x7fa698100a80 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa69da12640 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa690008dc0 tx=0x7fa69000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9642c2b998f39bcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa69da12640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa6837fe640 1 -- 172.21.15.16:0/2116726464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690020070 con 0x557e85ed8340 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa6837fe640 1 -- 172.21.15.16:0/2116726464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69000f3c0 con 0x557e85ed8340 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.084+0000 7fa6837fe640 1 -- 172.21.15.16:0/2116726464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690014070 con 0x557e85ed8340 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- 172.21.15.16:0/2116726464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 msgr2=0x557e85ed8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa69da12640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 0x7fa6840a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa69001d000 tx=0x7fa690020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa690008dc0 tx=0x7fa69000e720 comp rx=0 tx=0).stop 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- 172.21.15.16:0/2116726464 shutdown_connections 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2116726464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ed8720 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- 172.21.15.16:0/2116726464 >> 172.21.15.16:0/2116726464 conn(0x557e85ed4a30 msgr2=0x557e85ed4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- 172.21.15.16:0/2116726464 shutdown_connections 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- 172.21.15.16:0/2116726464 wait complete. 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 -- start start 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.085+0000 7fa6a184e740 1 Processor -- start 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e85eea660 con 0x557e85ed8340 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e85df9050 con 0x557e85ed8c60 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e85eea7e0 con 0x557e85ee7a90 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa6a184e740 10 client.0 _ll_register_callbacks cb 0x557e85e39160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55268/0 (socket says 172.21.15.16:55268) 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 -- 172.21.15.16:0/3307631939 learned_addr learned my addr 172.21.15.16:0/3307631939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69ca10640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69da12640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 -- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 msgr2=0x557e85ee4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 -- 172.21.15.16:0/3307631939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 msgr2=0x557e85ee46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 -- 172.21.15.16:0/3307631939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e85baf3c0 con 0x557e85ed8340 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69da12640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa69d211640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa69409b090 tx=0x7fa694097730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed189c0ca0fb4673 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.086+0000 7fa680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694020070 con 0x557e85ed8340 2024-09-12T00:57:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa6940cc840 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69000f3c0 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6940046c0 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e85ef60f0 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69000f3c0 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6940d0070 con 0x557e85ed8340 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 20 client.15762 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85ef75b0 0x557e85ef99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 4 client.15762 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.087+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e85e4b850 con 0x557e85ef75b0 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.088+0000 7fa69ca10640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85ef75b0 0x557e85ef99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.088+0000 7fa680ff9640 10 client.15762 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.088+0000 7fa69ca10640 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85ef75b0 0x557e85ef99a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5dcea349de8535 server_cookie=2a2d0f03bfb5d012 in_seq=0 out_seq=0 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa680ff9640 1 -- 172.21.15.16:0/3307631939 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e85e4b850 con 0x557e85ef75b0 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa680ff9640 10 client.15762 handle_command_reply: tid=0 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 client.15762 shutdown 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 msgr2=0x557e85ee7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa69409b090 tx=0x7fa694097730 comp rx=0 tx=0).stop 2024-09-12T00:57:38.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 shutdown_connections 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85ef75b0 0x557e85ef99a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ee7a90 0x557e85ee4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ed8c60 0x557e85ee46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa680ff9640 0 client.15762 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 --2- 172.21.15.16:0/3307631939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ee7410 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 >> 172.21.15.16:0/3307631939 conn(0x557e85ed4a30 msgr2=0x557e85eda790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 shutdown_connections 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 1 -- 172.21.15.16:0/3307631939 wait complete. 2024-09-12T00:57:38.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.089+0000 7fa6a184e740 20 client.15762 trim_cache size 0 max 16384 2024-09-12T00:57:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.107+0000 7fa6a184e740 1 Processor -- start 2024-09-12T00:57:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 -- start start 2024-09-12T00:57:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85ee2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ab6b0 con 0x557e85ed8340 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ad1f0 con 0x557e85ee7a90 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa6a184e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981abd80 con 0x557e85ed8c60 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55270/0 (socket says 172.21.15.16:55270) 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 -- 172.21.15.16:0/1511531121 learned_addr learned my addr 172.21.15.16:0/1511531121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69ca10640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85ee2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69da12640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 -- 172.21.15.16:0/1511531121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 msgr2=0x557e85ee2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85ee2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 -- 172.21.15.16:0/1511531121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 msgr2=0x557e85eff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 -- 172.21.15.16:0/1511531121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e85ee3670 con 0x557e85ed8340 2024-09-12T00:57:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69ca10640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85ee2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.108+0000 7fa69d211640 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa6940977f0 tx=0x7fa6940998a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fbe9fe8b919a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa682ffd640 1 -- 172.21.15.16:0/1511531121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694020040 con 0x557e85ed8340 2024-09-12T00:57:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa682ffd640 1 -- 172.21.15.16:0/1511531121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69409ba90 con 0x557e85ed8340 2024-09-12T00:57:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa682ffd640 1 -- 172.21.15.16:0/1511531121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940cc840 con 0x557e85ed8340 2024-09-12T00:57:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 -- 172.21.15.16:0/1511531121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 msgr2=0x557e85ef3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa6940977f0 tx=0x7fa6940998a0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 -- 172.21.15.16:0/1511531121 shutdown_connections 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85ee2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 --2- 172.21.15.16:0/1511531121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85ef3900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.109+0000 7fa6a184e740 1 -- 172.21.15.16:0/1511531121 >> 172.21.15.16:0/1511531121 conn(0x557e85ed4a30 msgr2=0x557e85edfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- 172.21.15.16:0/1511531121 shutdown_connections 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- 172.21.15.16:0/1511531121 wait complete. 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- start start 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 Processor -- start 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85eebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981acb20 con 0x557e85ed8340 2024-09-12T00:57:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ac450 con 0x557e85ee7a90 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.110+0000 7fa6a184e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a9690 con 0x557e85ed8c60 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa6a184e740 10 client.0 _ll_register_callbacks cb 0x7fa6940056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55282/0 (socket says 172.21.15.16:55282) 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 -- 172.21.15.16:0/2962166387 learned_addr learned my addr 172.21.15.16:0/2962166387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69da12640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69ca10640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85eebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 -- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 msgr2=0x557e85eebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85eebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 -- 172.21.15.16:0/2962166387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 msgr2=0x557e85eec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 -- 172.21.15.16:0/2962166387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e85ed8a90 con 0x557e85ed8340 2024-09-12T00:57:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa69d211640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa694097d60 tx=0x7fa6940cf140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ce302848401021 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940b8020 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa69409a9e0 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.111+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69409ba90 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa694020050 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e85ed8a90 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69409ba90 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6940d1030 con 0x557e85ed8340 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 20 client.15768 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85f625f0 0x557e85f649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 4 client.15768 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e85e45d60 con 0x557e85f625f0 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.112+0000 7fa69ca10640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85f625f0 0x557e85f649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa69ca10640 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85f625f0 0x557e85f649e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1f3a1fa5c6f5d7 server_cookie=125e917333c438bc in_seq=0 out_seq=0 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa680ff9640 10 client.15768 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa680ff9640 1 -- 172.21.15.16:0/2962166387 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e85e45d60 con 0x557e85f625f0 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa680ff9640 10 client.15768 handle_command_reply: tid=0 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 client.15768 shutdown 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 msgr2=0x557e85eeb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa694097d60 tx=0x7fa6940cf140 comp rx=0 tx=0).stop 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 shutdown_connections 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e85f625f0 0x557e85f649e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e85ee7a90 0x557e85eec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa680ff9640 0 client.15768 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e85ed8c60 0x557e85eebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 --2- 172.21.15.16:0/2962166387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e85ed8340 0x557e85eeb7c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 >> 172.21.15.16:0/2962166387 conn(0x557e85ed4a30 msgr2=0x557e85ed3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 shutdown_connections 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 1 -- 172.21.15.16:0/2962166387 wait complete. 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.113+0000 7fa6a184e740 20 client.15768 trim_cache size 0 max 16384 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T00:57:38.117 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T00:57:38.118 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 msgr2=0x7fa6840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 0x7fa6840a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa69001d000 tx=0x7fa690020000 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 msgr2=0x7fa6981984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 shutdown_connections 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa698106a80 0x7fa69819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa698101420 0x7fa698198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 --2- 172.21.15.16:0/512738687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa698100a80 0x7fa6981984c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 >> 172.21.15.16:0/512738687 conn(0x7fa6980fc7a0 msgr2=0x7fa6981048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 shutdown_connections 2024-09-12T00:57:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:38.114+0000 7fa69f475640 1 -- 172.21.15.16:0/512738687 wait complete. 2024-09-12T00:57:38.934 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:57:38.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:57:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:39.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:39 smithi099 ceph-mon[35654]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-12T00:57:39.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:39 smithi016 ceph-mon[30972]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-12T00:57:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:39 smithi159 ceph-mon[35566]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/780064875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 msgr2=0x7fa2dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/780064875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1067e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009900 tx=0x7fa2d002a410 comp rx=0 tx=0).stop 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/780064875 shutdown_connections 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/780064875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/780064875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1067e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/780064875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 0x7fa2dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/780064875 >> 172.21.15.16:0/780064875 conn(0x7fa2dc0fd8c0 msgr2=0x7fa2dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/780064875 shutdown_connections 2024-09-12T00:57:39.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.718+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/780064875 wait complete. 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 Processor -- start 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 -- start start 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 0x7fa2dc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111f70 con 0x7fa2dc102790 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111df0 con 0x7fa2dc101de0 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e3bb2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1120f0 con 0x7fa2dc106f10 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e114d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e114d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e114d640 1 -- 172.21.15.16:0/3117712461 learned_addr learned my addr 172.21.15.16:0/3117712461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:39.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e194e640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 0x7fa2dc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.719+0000 7fa2e214f640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 -- 172.21.15.16:0/3117712461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 msgr2=0x7fa2dc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 -- 172.21.15.16:0/3117712461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 msgr2=0x7fa2dc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 0x7fa2dc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 -- 172.21.15.16:0/3117712461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc19d440 con 0x7fa2dc102790 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e214f640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e114d640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d003f030 con 0x7fa2dc102790 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc19d6d0 con 0x7fa2dc102790 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d0003ce0 con 0x7fa2dc102790 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0035070 con 0x7fa2dc102790 2024-09-12T00:57:39.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.720+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1a5010 con 0x7fa2dc102790 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.721+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc1021c0 con 0x7fa2dc102790 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.722+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2d0042030 con 0x7fa2dc102790 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.723+0000 7fa2daffd640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.723+0000 7fa2e194e640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.723+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa2d00efda0 con 0x7fa2dc102790 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.723+0000 7fa2e194e640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fd0 tx=0x7fa2d4009680 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:39.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.726+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2d00bca50 con 0x7fa2dc102790 2024-09-12T00:57:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.941+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2dc10a0f0 con 0x7fa2dc102790 2024-09-12T00:57:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.942+0000 7fa2daffd640 1 -- 172.21.15.16:0/3117712461 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa2d00bc350 con 0x7fa2dc102790 2024-09-12T00:57:39.941 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:57:39.941 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:57:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 msgr2=0x7fa2c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fd0 tx=0x7fa2d4009680 comp rx=0 tx=0).stop 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 msgr2=0x7fa2dc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0006010 tx=0x7fa2d00042f0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 shutdown_connections 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2c40a2cf0 0x7fa2c40a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2dc106f10 0x7fa2dc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102790 0x7fa2dc1989d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 --2- 172.21.15.16:0/3117712461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2dc101de0 0x7fa2dc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 >> 172.21.15.16:0/3117712461 conn(0x7fa2dc0fd8c0 msgr2=0x7fa2dc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 shutdown_connections 2024-09-12T00:57:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:39.944+0000 7fa2e3bb2640 1 -- 172.21.15.16:0/3117712461 wait complete. 2024-09-12T00:57:40.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3117712461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:40.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:40 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3117712461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:40 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3117712461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:57:40.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T00:57:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:41 smithi159 ceph-mon[35566]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-12T00:57:41.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:41 smithi016 ceph-mon[30972]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-12T00:57:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 -- 172.21.15.16:0/4184937330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094102d30 msgr2=0x7fb094103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 --2- 172.21.15.16:0/4184937330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094102d30 0x7fb094103110 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb084009cd0 tx=0x7fb08402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 -- 172.21.15.16:0/4184937330 shutdown_connections 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 --2- 172.21.15.16:0/4184937330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941082a0 0x7fb09410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 --2- 172.21.15.16:0/4184937330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0941036e0 0x7fb094107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 --2- 172.21.15.16:0/4184937330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094102d30 0x7fb094103110 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 -- 172.21.15.16:0/4184937330 >> 172.21.15.16:0/4184937330 conn(0x7fb0940fc7d0 msgr2=0x7fb0940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 -- 172.21.15.16:0/4184937330 shutdown_connections 2024-09-12T00:57:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.453+0000 7fb099219640 1 -- 172.21.15.16:0/4184937330 wait complete. 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 Processor -- start 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 -- start start 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 0x7fb0941985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 0x7fb094198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941120a0 con 0x7fb0941082a0 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111f20 con 0x7fb0941036e0 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb099219640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb094112220 con 0x7fb094102d30 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb09359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb09359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58110/0 (socket says 172.21.15.16:58110) 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb09359e640 1 -- 172.21.15.16:0/2768135533 learned_addr learned my addr 172.21.15.16:0/2768135533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb092d9d640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 0x7fb0941985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.454+0000 7fb09259c640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 0x7fb094198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 -- 172.21.15.16:0/2768135533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 msgr2=0x7fb0941985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 0x7fb0941985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 -- 172.21.15.16:0/2768135533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 msgr2=0x7fb094198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 0x7fb094198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 -- 172.21.15.16:0/2768135533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09419d550 con 0x7fb0941082a0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb092d9d640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 0x7fb0941985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09259c640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 0x7fb094198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb09359e640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d750 tx=0x7fb08c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb08bfff640 1 -- 172.21.15.16:0/2768135533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c017070 con 0x7fb0941082a0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09419d840 con 0x7fb0941082a0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb08bfff640 1 -- 172.21.15.16:0/2768135533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c005500 con 0x7fb0941082a0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb08bfff640 1 -- 172.21.15.16:0/2768135533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c004d90 con 0x7fb0941082a0 2024-09-12T00:57:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.455+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a5120 con 0x7fb0941082a0 2024-09-12T00:57:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.458+0000 7fb08bfff640 1 -- 172.21.15.16:0/2768135533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb08c004f30 con 0x7fb0941082a0 2024-09-12T00:57:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.458+0000 7fb09b5f2740 1 Processor -- start 2024-09-12T00:57:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.458+0000 7fb09b5f2740 1 -- start start 2024-09-12T00:57:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac4f370 0x56105ac51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56105abaa2f0 con 0x56105ac45b40 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56105ab51cf0 con 0x56105ac445d0 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56105aba62b0 con 0x56105ac4f370 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58112/0 (socket says 172.21.15.16:58112) 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 -- 172.21.15.16:0/2256774385 learned_addr learned my addr 172.21.15.16:0/2256774385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb092d9d640 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac4f370 0x56105ac51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 -- 172.21.15.16:0/2256774385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac4f370 msgr2=0x56105ac51760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac4f370 0x56105ac51760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 -- 172.21.15.16:0/2256774385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 msgr2=0x56105ac44a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac44a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 -- 172.21.15.16:0/2256774385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105ab363a0 con 0x56105ac45b40 2024-09-12T00:57:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb08bfff640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 0x7fb0800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.459+0000 7fb09259c640 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005d90 tx=0x7fb07c00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1147aa5aab1edc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.460+0000 7fb092d9d640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 0x7fb0800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.460+0000 7fb08bfff640 1 -- 172.21.15.16:0/2768135533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb08c0cee40 con 0x7fb0941082a0 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.460+0000 7fb0897fa640 1 -- 172.21.15.16:0/2256774385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c018030 con 0x56105ac45b40 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.460+0000 7fb0897fa640 1 -- 172.21.15.16:0/2256774385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07c005500 con 0x56105ac45b40 2024-09-12T00:57:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.460+0000 7fb0897fa640 1 -- 172.21.15.16:0/2256774385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c010840 con 0x56105ac45b40 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- 172.21.15.16:0/2256774385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 msgr2=0x56105ac44090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb092d9d640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 0x7fb0800a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56105ac41ac0 tx=0x7fb084009360 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005d90 tx=0x7fb07c00c600 comp rx=0 tx=0).stop 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- 172.21.15.16:0/2256774385 shutdown_connections 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac4f370 0x56105ac51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac44a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/2256774385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac44090 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- 172.21.15.16:0/2256774385 >> 172.21.15.16:0/2256774385 conn(0x56105ac0d310 msgr2=0x56105ac40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- 172.21.15.16:0/2256774385 shutdown_connections 2024-09-12T00:57:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- 172.21.15.16:0/2256774385 wait complete. 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 -- start start 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.461+0000 7fb09b5f2740 1 Processor -- start 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac4c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56105ac520c0 con 0x56105ac445d0 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56105ab65060 con 0x56105ac4f370 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56105ac52240 con 0x56105ac45b40 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09b5f2740 10 client.0 _ll_register_callbacks cb 0x56105a8b88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58122/0 (socket says 172.21.15.16:58122) 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 -- 172.21.15.16:0/3840156724 learned_addr learned my addr 172.21.15.16:0/3840156724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09259c640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 -- 172.21.15.16:0/3840156724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 msgr2=0x56105ac4bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac4bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 -- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 msgr2=0x56105ac4c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb09359e640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac4c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac4c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 -- 172.21.15.16:0/3840156724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105a91b3c0 con 0x56105ac445d0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb092d9d640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb084009cd0 tx=0x7fb08402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d238e5aa61fc0ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb07a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.462+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084056060 con 0x56105ac445d0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07c005500 con 0x56105ac445d0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb084041030 con 0x56105ac445d0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb084003fd0 con 0x56105ac445d0 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105ac5db70 con 0x56105ac445d0 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07c005500 con 0x56105ac445d0 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb084030070 con 0x56105ac445d0 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 10 client.15786 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 20 client.15786 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105ac5efd0 0x56105ac61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 4 client.15786 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.463+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105abb7860 con 0x56105ac5efd0 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.464+0000 7fb09259c640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105ac5efd0 0x56105ac61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.464+0000 7fb07a7fc640 10 client.15786 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:57:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.464+0000 7fb09259c640 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105ac5efd0 0x56105ac61430 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e32d2bc1487623f3 server_cookie=ff5981d71d54a6db in_seq=0 out_seq=0 2024-09-12T00:57:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb07a7fc640 1 -- 172.21.15.16:0/3840156724 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56105abb7860 con 0x56105ac5efd0 2024-09-12T00:57:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb07a7fc640 10 client.15786 handle_command_reply: tid=0 2024-09-12T00:57:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 client.15786 shutdown 2024-09-12T00:57:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 msgr2=0x56105ac4ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb084009cd0 tx=0x7fb08402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 shutdown_connections 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105ac5efd0 0x56105ac61430 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac4c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac4bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb07a7fc640 0 client.15786 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3840156724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac445d0 0x56105ac4ed30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 >> 172.21.15.16:0/3840156724 conn(0x56105ac0d310 msgr2=0x56105ac46930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 shutdown_connections 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3840156724 wait complete. 2024-09-12T00:57:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.465+0000 7fb09b5f2740 20 client.15786 trim_cache size 0 max 16384 2024-09-12T00:57:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.478+0000 7fb09b5f2740 1 Processor -- start 2024-09-12T00:57:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 -- start start 2024-09-12T00:57:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac445d0 0x56105ac47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac66d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ad8a0 con 0x56105ac45b40 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941af3e0 con 0x56105ac4f370 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941adf70 con 0x56105ac445d0 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58130/0 (socket says 172.21.15.16:58130) 2024-09-12T00:57:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 -- 172.21.15.16:0/3330683138 learned_addr learned my addr 172.21.15.16:0/3330683138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb092d9d640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac445d0 0x56105ac47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 -- 172.21.15.16:0/3330683138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac445d0 msgr2=0x56105ac47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09359e640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac66d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac445d0 0x56105ac47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 -- 172.21.15.16:0/3330683138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 msgr2=0x56105ac66d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac66d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09259c640 1 -- 172.21.15.16:0/3330683138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105ac44d50 con 0x56105ac45b40 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.479+0000 7fb09359e640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09259c640 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0124e0 tx=0x7fb07c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e465ce761e30ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb088ff9640 1 -- 172.21.15.16:0/3330683138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c018040 con 0x56105ac45b40 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb088ff9640 1 -- 172.21.15.16:0/3330683138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07c019040 con 0x56105ac45b40 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb088ff9640 1 -- 172.21.15.16:0/3330683138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c010640 con 0x56105ac45b40 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3330683138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 msgr2=0x56105ac47660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0124e0 tx=0x7fb07c00c730 comp rx=0 tx=0).stop 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3330683138 shutdown_connections 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac4f370 0x56105ac66d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac45b40 0x56105ac47660 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/3330683138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac445d0 0x56105ac47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.480+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3330683138 >> 172.21.15.16:0/3330683138 conn(0x56105ac0d310 msgr2=0x56105ac3fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3330683138 shutdown_connections 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- 172.21.15.16:0/3330683138 wait complete. 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- start start 2024-09-12T00:57:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 Processor -- start 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941aed10 con 0x56105ac4f370 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ae640 con 0x56105ac445d0 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ab880 con 0x56105ac45b40 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.481+0000 7fb09b5f2740 10 client.0 _ll_register_callbacks cb 0x7fb084031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58138/0 (socket says 172.21.15.16:58138) 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 -- 172.21.15.16:0/657979914 learned_addr learned my addr 172.21.15.16:0/657979914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09259c640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb092d9d640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 -- 172.21.15.16:0/657979914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 msgr2=0x56105ac544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 -- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 msgr2=0x56105ac578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 -- 172.21.15.16:0/657979914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105ac54f20 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb092d9d640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09359e640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0977f0 tx=0x7fb08c0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec168979c6fce34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb07a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c020040 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb08c0cc840 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb07c019040 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb08c09f740 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09b5f2740 10 client.15792 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.482+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56105acca8e0 con 0x56105ac4f370 2024-09-12T00:57:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07c019040 con 0x56105ac4f370 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb08c0b41b0 con 0x56105ac4f370 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 10 client.15792 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 20 client.15792 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105accb000 0x56105accd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 4 client.15792 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56105abb1d70 con 0x56105accb000 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.483+0000 7fb092d9d640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105accb000 0x56105accd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb092d9d640 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105accb000 0x56105accd410 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8bd9713d31cf4da0 server_cookie=cc0c1c7eae1859db in_seq=0 out_seq=0 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb07a7fc640 10 client.15792 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb07a7fc640 1 -- 172.21.15.16:0/657979914 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56105abb1d70 con 0x56105accb000 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb07a7fc640 10 client.15792 handle_command_reply: tid=0 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 client.15792 shutdown 2024-09-12T00:57:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 msgr2=0x56105ac549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0977f0 tx=0x7fb08c0056b0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 shutdown_connections 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56105accb000 0x56105accd410 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56105ac4f370 0x56105ac549e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56105ac45b40 0x56105ac544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb07a7fc640 0 client.15792 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 --2- 172.21.15.16:0/657979914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56105ac445d0 0x56105ac578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 >> 172.21.15.16:0/657979914 conn(0x56105ac0d310 msgr2=0x56105ac4d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 shutdown_connections 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 1 -- 172.21.15.16:0/657979914 wait complete. 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.484+0000 7fb09b5f2740 20 client.15792 trim_cache size 0 max 16384 2024-09-12T00:57:41.487 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 msgr2=0x7fb0800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 0x7fb0800a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x56105ac41ac0 tx=0x7fb084009360 comp rx=0 tx=0).stop 2024-09-12T00:57:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 msgr2=0x7fb09419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00d750 tx=0x7fb08c00dc10 comp rx=0 tx=0).stop 2024-09-12T00:57:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 shutdown_connections 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0800a2cf0 0x7fb0800a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0941082a0 0x7fb09419ce70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0941036e0 0x7fb094198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 --2- 172.21.15.16:0/2768135533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb094102d30 0x7fb0941985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 >> 172.21.15.16:0/2768135533 conn(0x7fb0940fc7d0 msgr2=0x7fb094105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 shutdown_connections 2024-09-12T00:57:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:41.485+0000 7fb099219640 1 -- 172.21.15.16:0/2768135533 wait complete. 2024-09-12T00:57:41.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:41 smithi099 ceph-mon[35654]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-12T00:57:42.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T00:57:42.430 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:42.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 -- 172.21.15.16:0/2714157777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 msgr2=0x7f923410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2714157777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923410e610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f922c009ca0 tx=0x7f922c02a950 comp rx=0 tx=0).stop 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 -- 172.21.15.16:0/2714157777 shutdown_connections 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2714157777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923410e610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2714157777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 0x7f923410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2714157777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f9234103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 -- 172.21.15.16:0/2714157777 >> 172.21.15.16:0/2714157777 conn(0x7f92340fc7a0 msgr2=0x7f92340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 -- 172.21.15.16:0/2714157777 shutdown_connections 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.929+0000 7f923a2fd640 1 -- 172.21.15.16:0/2714157777 wait complete. 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 Processor -- start 2024-09-12T00:57:42.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 -- start start 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 0x7f9234198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9234111f30 con 0x7f9234102d40 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9234111db0 con 0x7f923410c170 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.930+0000 7f923a2fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92341120b0 con 0x7f9234103660 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58162/0 (socket says 172.21.15.16:58162) 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 -- 172.21.15.16:0/2703378215 learned_addr learned my addr 172.21.15.16:0/2703378215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f923889a640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f92337fe640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 0x7f9234198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 -- 172.21.15.16:0/2703378215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 msgr2=0x7f9234198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 0x7f9234198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 -- 172.21.15.16:0/2703378215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 msgr2=0x7f923419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 -- 172.21.15.16:0/2703378215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923419d3a0 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f9233fff640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f922400e9f0 tx=0x7f922400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f92317fa640 1 -- 172.21.15.16:0/2703378215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922400cd80 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923419d690 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f92317fa640 1 -- 172.21.15.16:0/2703378215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9224010910 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f92317fa640 1 -- 172.21.15.16:0/2703378215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240188f0 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.931+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92341a4f70 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.934+0000 7f92317fa640 1 -- 172.21.15.16:0/2703378215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9224010ac0 con 0x7f9234102d40 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.934+0000 7f923c6d6740 1 Processor -- start 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.934+0000 7f923c6d6740 1 -- start start 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79ca4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f92317fa640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 0x7f921c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79cb3780 0x55df79cb5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df79c0a2f0 con 0x55df79ca5b40 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df79bb1cf0 con 0x55df79cb3780 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923c6d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55df79c062b0 con 0x55df79ca45d0 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58168/0 (socket says 172.21.15.16:58168) 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 -- 172.21.15.16:0/2044172453 learned_addr learned my addr 172.21.15.16:0/2044172453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f9233fff640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 0x7f921c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f92337fe640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79ca4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79cb3780 0x55df79cb5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f92317fa640 1 -- 172.21.15.16:0/2703378215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f92240cf830 con 0x7f9234102d40 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 -- 172.21.15.16:0/2044172453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 msgr2=0x55df79ca4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79ca4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 -- 172.21.15.16:0/2044172453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79cb3780 msgr2=0x55df79cb5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79cb3780 0x55df79cb5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 -- 172.21.15.16:0/2044172453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df79b963a0 con 0x55df79ca5b40 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f92337fe640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79ca4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f9233fff640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 0x7f921c0a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9224098230 tx=0x7f9224009700 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.935+0000 7f923889a640 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55df79ca5070 tx=0x7f922c004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f065984f7a6b9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f921a7fc640 1 -- 172.21.15.16:0/2044172453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c043070 con 0x55df79ca5b40 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f921a7fc640 1 -- 172.21.15.16:0/2044172453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c041070 con 0x55df79ca5b40 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f921a7fc640 1 -- 172.21.15.16:0/2044172453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c034040 con 0x55df79ca5b40 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 -- 172.21.15.16:0/2044172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 msgr2=0x55df79ca4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55df79ca5070 tx=0x7f922c004a60 comp rx=0 tx=0).stop 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 -- 172.21.15.16:0/2044172453 shutdown_connections 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79cb3780 0x55df79cb5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79ca4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2044172453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79ca4090 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 -- 172.21.15.16:0/2044172453 >> 172.21.15.16:0/2044172453 conn(0x55df79c6d310 msgr2=0x55df79ca0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 -- 172.21.15.16:0/2044172453 shutdown_connections 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.936+0000 7f923c6d6740 1 -- 172.21.15.16:0/2044172453 wait complete. 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 -- start start 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 Processor -- start 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cb3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79cb3780 0x55df79cb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df79cb6420 con 0x55df79ca5b40 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df79bc5060 con 0x55df79ca45d0 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55df79cb65a0 con 0x55df79cb3780 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f923c6d6740 10 client.0 _ll_register_callbacks cb 0x55df799188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f92337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f92337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58178/0 (socket says 172.21.15.16:58178) 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.937+0000 7f92337fe640 1 -- 172.21.15.16:0/1512817307 learned_addr learned my addr 172.21.15.16:0/1512817307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923889a640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79cb3780 0x55df79cb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923889a640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cb3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 -- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79cb3780 msgr2=0x55df79cb09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79cb3780 0x55df79cb09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 -- 172.21.15.16:0/1512817307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 msgr2=0x55df79cb3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cb3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 -- 172.21.15.16:0/1512817307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7997b3c0 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923889a640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cb3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92337fe640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9228012700 tx=0x7f9228012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b14cec179d22c75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92137fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228004050 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f922c041070 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f92280041f0 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92280043f0 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923c6d6740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.938+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df79cc1f50 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c041070 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9228003bf0 con 0x55df79ca5b40 2024-09-12T00:57:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 10 client.15804 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 20 client.15804 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79cc3460 0x55df79cc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 4 client.15804 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df79c17860 con 0x55df79cc3460 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923889a640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79cc3460 0x55df79cc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f923889a640 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79cc3460 0x55df79cc5880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec3085d41a85efb0 server_cookie=117f44422eb7d4ad in_seq=0 out_seq=0 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.939+0000 7f92137fe640 10 client.15804 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f92137fe640 1 -- 172.21.15.16:0/1512817307 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df79c17860 con 0x55df79cc3460 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f92137fe640 10 client.15804 handle_command_reply: tid=0 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 client.15804 shutdown 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 msgr2=0x55df79cb0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9228012700 tx=0x7f9228012bc0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 shutdown_connections 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79cb3780 0x55df79cb09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79cc3460 0x55df79cc5880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79ca5b40 0x55df79cb0410 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 --2- 172.21.15.16:0/1512817307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cb3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f92137fe640 0 client.15804 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 >> 172.21.15.16:0/1512817307 conn(0x55df79c6d310 msgr2=0x55df79ca6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 shutdown_connections 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.940+0000 7f923c6d6740 1 -- 172.21.15.16:0/1512817307 wait complete. 2024-09-12T00:57:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.941+0000 7f923c6d6740 20 client.15804 trim_cache size 0 max 16384 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 Processor -- start 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 -- start start 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cbf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 0x55df79caf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92341ab5e0 con 0x55df79cb3780 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92341ad120 con 0x55df79ca45d0 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923c6d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92341abcb0 con 0x55df79ca5b40 2024-09-12T00:57:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58186/0 (socket says 172.21.15.16:58186) 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 -- 172.21.15.16:0/2934104669 learned_addr learned my addr 172.21.15.16:0/2934104669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cbf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f92337fe640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 0x55df79caf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 -- 172.21.15.16:0/2934104669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 msgr2=0x55df79caf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 0x55df79caf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 -- 172.21.15.16:0/2934104669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 msgr2=0x55df79cbf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cbf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 -- 172.21.15.16:0/2934104669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df79ca1960 con 0x55df79cb3780 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f92337fe640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 0x55df79caf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.954+0000 7f923889a640 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f922c004140 tx=0x7f922c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b40c2c864707d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f9219ffb640 1 -- 172.21.15.16:0/2934104669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c05b070 con 0x55df79cb3780 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f9219ffb640 1 -- 172.21.15.16:0/2934104669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c03f070 con 0x55df79cb3780 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f9219ffb640 1 -- 172.21.15.16:0/2934104669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c034040 con 0x55df79cb3780 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 -- 172.21.15.16:0/2934104669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 msgr2=0x55df79ccb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f922c004140 tx=0x7f922c02a950 comp rx=0 tx=0).stop 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 -- 172.21.15.16:0/2934104669 shutdown_connections 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79ccb130 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca5b40 0x55df79caf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 --2- 172.21.15.16:0/2934104669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca45d0 0x55df79cbf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 -- 172.21.15.16:0/2934104669 >> 172.21.15.16:0/2934104669 conn(0x55df79c6d310 msgr2=0x55df79caece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 -- 172.21.15.16:0/2934104669 shutdown_connections 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.955+0000 7f923c6d6740 1 -- 172.21.15.16:0/2934104669 wait complete. 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 -- start start 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 Processor -- start 2024-09-12T00:57:42.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79cbbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca5b40 0x55df79cb7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92341aca50 con 0x55df79cb3780 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92341ac380 con 0x55df79ca5b40 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92341a95c0 con 0x55df79ca45d0 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923c6d6740 10 client.0 _ll_register_callbacks cb 0x7f9228009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58198/0 (socket says 172.21.15.16:58198) 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923889a640 1 -- 172.21.15.16:0/692640507 learned_addr learned my addr 172.21.15.16:0/692640507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f92337fe640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca5b40 0x55df79cb7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.956+0000 7f923889a640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79cbbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 -- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 msgr2=0x55df79cbbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79cbbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 -- 172.21.15.16:0/692640507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca5b40 msgr2=0x55df79cb7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca5b40 0x55df79cb7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 -- 172.21.15.16:0/692640507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df79ca7120 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923889a640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f922c059040 tx=0x7f922c04b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b439d5d9b130534 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f92137fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c034070 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f922c03f070 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f922c004b40 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f922c05b040 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923c6d6740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df79d2de40 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.957+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c03f070 con 0x55df79cb3780 2024-09-12T00:57:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f922c003be0 con 0x55df79cb3780 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 10 client.15810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 20 client.15810 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79d2e070 0x55df79d30460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 4 client.15810 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df79c11d70 con 0x55df79d2e070 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f92337fe640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79d2e070 0x55df79d30460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f92137fe640 10 client.15810 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.958+0000 7f92337fe640 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79d2e070 0x55df79d30460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6eda719ad348e1c2 server_cookie=1f7f9923e640125e in_seq=0 out_seq=0 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f92137fe640 1 -- 172.21.15.16:0/692640507 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df79c11d70 con 0x55df79d2e070 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f92137fe640 10 client.15810 handle_command_reply: tid=0 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 client.15810 shutdown 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 msgr2=0x55df79cb7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f922c059040 tx=0x7f922c04b040 comp rx=0 tx=0).stop 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 shutdown_connections 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55df79d2e070 0x55df79d30460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df79cb3780 0x55df79cb7780 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df79ca5b40 0x55df79cb7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f92137fe640 0 client.15810 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 --2- 172.21.15.16:0/692640507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df79ca45d0 0x55df79cbbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 >> 172.21.15.16:0/692640507 conn(0x55df79c6d310 msgr2=0x55df79caf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 shutdown_connections 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 1 -- 172.21.15.16:0/692640507 wait complete. 2024-09-12T00:57:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.959+0000 7f923c6d6740 20 client.15810 trim_cache size 0 max 16384 2024-09-12T00:57:42.965 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 msgr2=0x7f921c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 0x7f921c0a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9224098230 tx=0x7f9224009700 comp rx=0 tx=0).stop 2024-09-12T00:57:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 msgr2=0x7f92341983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f922400e9f0 tx=0x7f922400eeb0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 shutdown_connections 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f921c0a2cf0 0x7f921c0a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f923410c170 0x7f923419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9234103660 0x7f9234198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 --2- 172.21.15.16:0/2703378215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9234102d40 0x7f92341983f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 >> 172.21.15.16:0/2703378215 conn(0x7f92340fc7a0 msgr2=0x7f92340fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 shutdown_connections 2024-09-12T00:57:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:42.960+0000 7f923a2fd640 1 -- 172.21.15.16:0/2703378215 wait complete. 2024-09-12T00:57:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:43 smithi016 ceph-mon[30972]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:43 smithi159 ceph-mon[35566]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:43 smithi099 ceph-mon[35654]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:43.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:57:43.933 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 -- 172.21.15.16:0/4261365522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 msgr2=0x7f472c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 --2- 172.21.15.16:0/4261365522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c1021c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f471c009900 tx=0x7f471c02a410 comp rx=0 tx=0).stop 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 -- 172.21.15.16:0/4261365522 shutdown_connections 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 --2- 172.21.15.16:0/4261365522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c106f10 0x7f472c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 --2- 172.21.15.16:0/4261365522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c102790 0x7f472c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 --2- 172.21.15.16:0/4261365522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c1021c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 -- 172.21.15.16:0/4261365522 >> 172.21.15.16:0/4261365522 conn(0x7f472c0fd8c0 msgr2=0x7f472c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 -- 172.21.15.16:0/4261365522 shutdown_connections 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.439+0000 7f4731d25640 1 -- 172.21.15.16:0/4261365522 wait complete. 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 Processor -- start 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 -- start start 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c102790 0x7f472c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c106f10 0x7f472c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f472c111f70 con 0x7f472c101de0 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f472c111df0 con 0x7f472c102790 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.440+0000 7f4731d25640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1120f0 con 0x7f472c106f10 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58204/0 (socket says 172.21.15.16:58204) 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 -- 172.21.15.16:0/1948918994 learned_addr learned my addr 172.21.15.16:0/1948918994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472bfff640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c106f10 0x7f472c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472affd640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c102790 0x7f472c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 -- 172.21.15.16:0/1948918994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c106f10 msgr2=0x7f472c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c106f10 0x7f472c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 -- 172.21.15.16:0/1948918994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c102790 msgr2=0x7f472c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c102790 0x7f472c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 -- 172.21.15.16:0/1948918994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19d400 con 0x7f472c101de0 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f471c006010 tx=0x7f471c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f4728ff9640 1 -- 172.21.15.16:0/1948918994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c03f030 con 0x7f472c101de0 2024-09-12T00:57:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19d690 con 0x7f472c101de0 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f4728ff9640 1 -- 172.21.15.16:0/1948918994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471c003ce0 con 0x7f472c101de0 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f4728ff9640 1 -- 172.21.15.16:0/1948918994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c035070 con 0x7f472c101de0 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.441+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a4fd0 con 0x7f472c101de0 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 Processor -- start 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f4728ff9640 1 -- 172.21.15.16:0/1948918994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f471c042030 con 0x7f472c101de0 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 -- start start 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555707333b40 0x555707332090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707332a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5557072982f0 con 0x55570733d370 2024-09-12T00:57:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55570723fcf0 con 0x555707333b40 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f47340fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5557072942b0 con 0x5557073325d0 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58220/0 (socket says 172.21.15.16:58220) 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f472affd640 1 -- 172.21.15.16:0/1259892701 learned_addr learned my addr 172.21.15.16:0/1259892701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707332a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.444+0000 7f472bfff640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555707333b40 0x555707332090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 -- 172.21.15.16:0/1259892701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 msgr2=0x555707332a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707332a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 -- 172.21.15.16:0/1259892701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555707333b40 msgr2=0x555707332090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555707333b40 0x555707332090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 -- 172.21.15.16:0/1259892701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557072243a0 con 0x55570733d370 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707332a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f4728ff9640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 0x7f47140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472affd640 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4720009c90 tx=0x7f4720011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd13f0e55c0be1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f472bfff640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 0x7f47140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f4728ff9640 1 -- 172.21.15.16:0/1948918994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f471c0ef4a0 con 0x7f472c101de0 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f4711ffb640 1 -- 172.21.15.16:0/1259892701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472001e070 con 0x55570733d370 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f4711ffb640 1 -- 172.21.15.16:0/1259892701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4720019b10 con 0x55570733d370 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.445+0000 7f4711ffb640 1 -- 172.21.15.16:0/1259892701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472000ea70 con 0x55570733d370 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- 172.21.15.16:0/1259892701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 msgr2=0x55570733f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f472bfff640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 0x7f47140a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4724005fd0 tx=0x7f4724009600 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4720009c90 tx=0x7f4720011810 comp rx=0 tx=0).stop 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- 172.21.15.16:0/1259892701 shutdown_connections 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55570733d370 0x55570733f760 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707332a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 --2- 172.21.15.16:0/1259892701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555707333b40 0x555707332090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- 172.21.15.16:0/1259892701 >> 172.21.15.16:0/1259892701 conn(0x5557072fb310 msgr2=0x55570732ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- 172.21.15.16:0/1259892701 shutdown_connections 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- 172.21.15.16:0/1259892701 wait complete. 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.446+0000 7f47340fe740 1 -- start start 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 Processor -- start 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x55570733cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x55570733a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555707340070 con 0x555707333b40 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555707253060 con 0x55570733d370 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5557073401f0 con 0x5557073325d0 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f47340fe740 10 client.0 _ll_register_callbacks cb 0x555706fa68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:44.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58230/0 (socket says 172.21.15.16:58230) 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 -- 172.21.15.16:0/1581930370 learned_addr learned my addr 172.21.15.16:0/1581930370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472bfff640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x55570733a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x55570733cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 -- 172.21.15.16:0/1581930370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 msgr2=0x55570733cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x55570733cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 -- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 msgr2=0x55570733a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x55570733a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472affd640 1 -- 172.21.15.16:0/1581930370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557070093c0 con 0x555707333b40 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.447+0000 7f472bfff640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x55570733a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f472affd640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4720005890 tx=0x7f4720009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a8a3b6f68b541 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472001e050 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4720005450 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4720019b10 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4720033050 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55570734bad0 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4720019b10 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4720004a50 con 0x555707333b40 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 20 client.15822 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55570734d140 0x55570734f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 4 client.15822 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:44.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.448+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557072a5860 con 0x55570734d140 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.449+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55570734d140 0x55570734f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.449+0000 7f472b7fe640 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55570734d140 0x55570734f530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=613b58a87987448a server_cookie=9051b21d05c4b2f6 in_seq=0 out_seq=0 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.449+0000 7f470affd640 10 client.15822 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f470affd640 1 -- 172.21.15.16:0/1581930370 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557072a5860 con 0x55570734d140 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f470affd640 10 client.15822 handle_command_reply: tid=0 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 client.15822 shutdown 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 msgr2=0x555707339fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4720005890 tx=0x7f4720009f80 comp rx=0 tx=0).stop 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 shutdown_connections 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55570734d140 0x55570734f530 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x55570733a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x555707339fe0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f470affd640 0 client.15822 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 --2- 172.21.15.16:0/1581930370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x55570733cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 >> 172.21.15.16:0/1581930370 conn(0x5557072fb310 msgr2=0x555707334960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 shutdown_connections 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 1 -- 172.21.15.16:0/1581930370 wait complete. 2024-09-12T00:57:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.450+0000 7f47340fe740 20 client.15822 trim_cache size 0 max 16384 2024-09-12T00:57:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 Processor -- start 2024-09-12T00:57:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 -- start start 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555707333b40 0x555707335120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707335660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1ad860 con 0x5557073325d0 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1af3a0 con 0x55570733d370 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f47340fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1adf30 con 0x555707333b40 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58232/0 (socket says 172.21.15.16:58232) 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.464+0000 7f472b7fe640 1 -- 172.21.15.16:0/2916078585 learned_addr learned my addr 172.21.15.16:0/2916078585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472affd640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555707333b40 0x555707335120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472bfff640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707335660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 -- 172.21.15.16:0/2916078585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555707333b40 msgr2=0x555707335120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555707333b40 0x555707335120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 -- 172.21.15.16:0/2916078585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 msgr2=0x555707335660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707335660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 -- 172.21.15.16:0/2916078585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555707338f90 con 0x5557073325d0 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472bfff640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707335660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f472b7fe640 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27a6922bb32d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f47117fa640 1 -- 172.21.15.16:0/2916078585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c049040 con 0x5557073325d0 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f47117fa640 1 -- 172.21.15.16:0/2916078585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471c0bbb60 con 0x5557073325d0 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.465+0000 7f47117fa640 1 -- 172.21.15.16:0/2916078585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0b9ef0 con 0x5557073325d0 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- 172.21.15.16:0/2916078585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 msgr2=0x5557073398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f471c000c00 tx=0x7f471c03b5d0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- 172.21.15.16:0/2916078585 shutdown_connections 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707335660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555707333b40 0x555707335120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 --2- 172.21.15.16:0/2916078585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557073325d0 0x5557073398a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- 172.21.15.16:0/2916078585 >> 172.21.15.16:0/2916078585 conn(0x5557072fb310 msgr2=0x55570733c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- 172.21.15.16:0/2916078585 shutdown_connections 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- 172.21.15.16:0/2916078585 wait complete. 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 -- start start 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.466+0000 7f47340fe740 1 Processor -- start 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707350470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707350ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1aecd0 con 0x555707333b40 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1ae600 con 0x55570733d370 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1ab840 con 0x5557073325d0 2024-09-12T00:57:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f47340fe740 10 client.0 _ll_register_callbacks cb 0x7f472001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58240/0 (socket says 172.21.15.16:58240) 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 -- 172.21.15.16:0/4180716599 learned_addr learned my addr 172.21.15.16:0/4180716599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472b7fe640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707350470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472bfff640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707350ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 -- 172.21.15.16:0/4180716599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 msgr2=0x555707350470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707350470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 -- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 msgr2=0x555707350ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707350ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 -- 172.21.15.16:0/4180716599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570733d0a0 con 0x555707333b40 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.467+0000 7f472affd640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4720006010 tx=0x7f472000e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cce8608824d4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472001e040 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4720005250 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471c0bbb60 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4720005470 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 10 client.15828 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557073bc0b0 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471c0bbb60 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47200037b0 con 0x555707333b40 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 10 client.15828 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 20 client.15828 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5557073bc7c0 0x5557073bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 4 client.15828 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.468+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55570729fd70 con 0x5557073bc7c0 2024-09-12T00:57:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f472b7fe640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5557073bc7c0 0x5557073bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f472b7fe640 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5557073bc7c0 0x5557073bebe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b935fa2c28c272f server_cookie=ab35b0cf7702074a in_seq=0 out_seq=0 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f470affd640 10 client.15828 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f470affd640 1 -- 172.21.15.16:0/4180716599 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55570729fd70 con 0x5557073bc7c0 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f470affd640 10 client.15828 handle_command_reply: tid=0 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 client.15828 shutdown 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 msgr2=0x5557073509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4720006010 tx=0x7f472000e4c0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 shutdown_connections 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5557073bc7c0 0x5557073bebe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55570733d370 0x555707350ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555707333b40 0x5557073509b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 --2- 172.21.15.16:0/4180716599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557073325d0 0x555707350470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 >> 172.21.15.16:0/4180716599 conn(0x5557072fb310 msgr2=0x55570732d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.469+0000 7f470affd640 0 client.15828 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:57:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 shutdown_connections 2024-09-12T00:57:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f47340fe740 1 -- 172.21.15.16:0/4180716599 wait complete. 2024-09-12T00:57:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f47340fe740 20 client.15828 trim_cache size 0 max 16384 2024-09-12T00:57:44.484 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 msgr2=0x7f47140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 0x7f47140a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4724005fd0 tx=0x7f4724009600 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 msgr2=0x7f472c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f471c006010 tx=0x7f471c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 shutdown_connections 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47140a2cf0 0x7f47140a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f472c106f10 0x7f472c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f472c102790 0x7f472c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 --2- 172.21.15.16:0/1948918994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f472c101de0 0x7f472c198450 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 >> 172.21.15.16:0/1948918994 conn(0x7f472c0fd8c0 msgr2=0x7f472c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 shutdown_connections 2024-09-12T00:57:44.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:44.470+0000 7f4731d25640 1 -- 172.21.15.16:0/1948918994 wait complete. 2024-09-12T00:57:45.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:45 smithi016 ceph-mon[30972]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:45.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:57:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:45 smithi159 ceph-mon[35566]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:45.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:45 smithi099 ceph-mon[35654]: pgmap v53: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T00:57:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 -- 172.21.15.16:0/4176124964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8106f10 msgr2=0x7f84a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 --2- 172.21.15.16:0/4176124964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8106f10 0x7f84a810ea90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009ca0 tx=0x7f84a002a8c0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 -- 172.21.15.16:0/4176124964 shutdown_connections 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 --2- 172.21.15.16:0/4176124964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8106f10 0x7f84a810ea90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 --2- 172.21.15.16:0/4176124964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8102790 0x7f84a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 --2- 172.21.15.16:0/4176124964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8101de0 0x7f84a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 -- 172.21.15.16:0/4176124964 >> 172.21.15.16:0/4176124964 conn(0x7f84a80fd8c0 msgr2=0x7f84a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 -- 172.21.15.16:0/4176124964 shutdown_connections 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.962+0000 7f84b0689640 1 -- 172.21.15.16:0/4176124964 wait complete. 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 Processor -- start 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 -- start start 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8101de0 0x7f84a81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 0x7f84a819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8111f70 con 0x7f84a8102790 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a8111df0 con 0x7f84a8101de0 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84b0689640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81120f0 con 0x7f84a8106f10 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84adc24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.963+0000 7f84adc24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58258/0 (socket says 172.21.15.16:58258) 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 -- 172.21.15.16:0/1943685216 learned_addr learned my addr 172.21.15.16:0/1943685216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84aec26640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 0x7f84a819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84ae425640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8101de0 0x7f84a81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 -- 172.21.15.16:0/1943685216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 msgr2=0x7f84a819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 0x7f84a819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 -- 172.21.15.16:0/1943685216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8101de0 msgr2=0x7f84a81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8101de0 0x7f84a81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 -- 172.21.15.16:0/1943685216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a819d380 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84aec26640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 0x7f84a819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84adc24640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f849800d950 tx=0x7f849800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f849f7fe640 1 -- 172.21.15.16:0/1943685216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f849f7fe640 1 -- 172.21.15.16:0/1943685216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8498009c30 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a819d670 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f849f7fe640 1 -- 172.21.15.16:0/1943685216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849800f6b0 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.964+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a81a4f50 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.966+0000 7f849f7fe640 1 -- 172.21.15.16:0/1943685216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84980109d0 con 0x7f84a8102790 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 Processor -- start 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 -- start start 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8b9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8b012f0 con 0x55c7b8b9cb40 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8aa8cf0 con 0x55c7b8ba6370 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84b2a62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8afd2b0 con 0x55c7b8b9b5d0 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58260/0 (socket says 172.21.15.16:58260) 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f84aec26640 1 -- 172.21.15.16:0/48533856 learned_addr learned my addr 172.21.15.16:0/48533856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.967+0000 7f849f7fe640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 0x7f84940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84ae425640 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8b9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84adc24640 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 0x7f84940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f849f7fe640 1 -- 172.21.15.16:0/1943685216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84980cf060 con 0x7f84a8102790 2024-09-12T00:57:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 -- 172.21.15.16:0/48533856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9b5d0 msgr2=0x55c7b8b9ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8b9ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 -- 172.21.15.16:0/48533856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8ba6370 msgr2=0x55c7b8ba8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 -- 172.21.15.16:0/48533856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b8a8d3a0 con 0x55c7b8b9cb40 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f84a0035000 tx=0x7f84a0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff89fb3bf9762b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f84aec26640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 0x7f84940a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009430 tx=0x7f84a0049000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f849cff9640 1 -- 172.21.15.16:0/48533856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a004f070 con 0x55c7b8b9cb40 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f849cff9640 1 -- 172.21.15.16:0/48533856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a0036070 con 0x55c7b8b9cb40 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.968+0000 7f849cff9640 1 -- 172.21.15.16:0/48533856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a002f610 con 0x55c7b8b9cb40 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 -- 172.21.15.16:0/48533856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 msgr2=0x55c7b8b9b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f84a0035000 tx=0x7f84a0003970 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 -- 172.21.15.16:0/48533856 shutdown_connections 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8b9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 --2- 172.21.15.16:0/48533856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8b9b090 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 -- 172.21.15.16:0/48533856 >> 172.21.15.16:0/48533856 conn(0x55c7b8b64310 msgr2=0x55c7b8b97c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 -- 172.21.15.16:0/48533856 shutdown_connections 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.969+0000 7f84b2a62740 1 -- 172.21.15.16:0/48533856 wait complete. 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 -- start start 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 Processor -- start 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8ba8e90 con 0x55c7b8ba6370 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8abc060 con 0x55c7b8b9b5d0 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8ba9010 con 0x55c7b8b9cb40 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84b2a62740 10 client.0 _ll_register_callbacks cb 0x55c7b880f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58276/0 (socket says 172.21.15.16:58276) 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 -- 172.21.15.16:0/262467040 learned_addr learned my addr 172.21.15.16:0/262467040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84adc24640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84ae425640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 -- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 msgr2=0x55c7b8ba3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 -- 172.21.15.16:0/262467040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 msgr2=0x55c7b8ba5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84aec26640 1 -- 172.21.15.16:0/262467040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b88723c0 con 0x55c7b8ba6370 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.970+0000 7f84adc24640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84aec26640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009bf0 tx=0x7f84a0031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ef059a13307c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:45.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0057020 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84a00318e0 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a0036070 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84a0062050 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7b8ba8e90 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a0036070 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84a0030780 con 0x55c7b8ba6370 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 20 client.15840 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8bb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 4 client.15840 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.971+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7b8b0e860 con 0x55c7b8bb5b00 2024-09-12T00:57:45.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.972+0000 7f84ae425640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8bb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.972+0000 7f8491ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.972+0000 7f84ae425640 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8bb7ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54ff28b22396373c server_cookie=77e842d1035606a7 in_seq=0 out_seq=0 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f8491ffb640 1 -- 172.21.15.16:0/262467040 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7b8b0e860 con 0x55c7b8bb5b00 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f8491ffb640 10 client.15840 handle_command_reply: tid=0 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 client.15840 shutdown 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 msgr2=0x55c7b8ba3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009bf0 tx=0x7f84a0031c70 comp rx=0 tx=0).stop 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 shutdown_connections 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8ba3610 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8bb7ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f8491ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 --2- 172.21.15.16:0/262467040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 >> 172.21.15.16:0/262467040 conn(0x55c7b8b64310 msgr2=0x55c7b8b9d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 shutdown_connections 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 1 -- 172.21.15.16:0/262467040 wait complete. 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.973+0000 7f84b2a62740 20 client.15840 trim_cache size 0 max 16384 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 Processor -- start 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 -- start start 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8bb2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8ba6370 0x55c7b8bbd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81ad7b0 con 0x55c7b8b9cb40 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81af2f0 con 0x55c7b8b9b5d0 2024-09-12T00:57:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.986+0000 7f84b2a62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81ade80 con 0x55c7b8ba6370 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58288/0 (socket says 172.21.15.16:58288) 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 -- 172.21.15.16:0/1125987478 learned_addr learned my addr 172.21.15.16:0/1125987478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84ae425640 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8bb2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 -- 172.21.15.16:0/1125987478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8ba6370 msgr2=0x55c7b8bbd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8ba6370 0x55c7b8bbd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 -- 172.21.15.16:0/1125987478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 msgr2=0x55c7b8bb2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8bb2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 -- 172.21.15.16:0/1125987478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b8b9ec20 con 0x55c7b8b9cb40 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84ae425640 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8bb2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f84adc24640 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f84980cc890 tx=0x7f849806d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bcbba61096e93b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f8493fff640 1 -- 172.21.15.16:0/1125987478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498020040 con 0x55c7b8b9cb40 2024-09-12T00:57:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f8493fff640 1 -- 172.21.15.16:0/1125987478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849809bcb0 con 0x55c7b8b9cb40 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.987+0000 7f8493fff640 1 -- 172.21.15.16:0/1125987478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980cca60 con 0x55c7b8b9cb40 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- 172.21.15.16:0/1125987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 msgr2=0x55c7b8ba18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f84980cc890 tx=0x7f849806d570 comp rx=0 tx=0).stop 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- 172.21.15.16:0/1125987478 shutdown_connections 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8ba6370 0x55c7b8bbd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8b9cb40 0x55c7b8ba18f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1125987478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8bb2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- 172.21.15.16:0/1125987478 >> 172.21.15.16:0/1125987478 conn(0x55c7b8b64310 msgr2=0x55c7b8ba50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- 172.21.15.16:0/1125987478 shutdown_connections 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- 172.21.15.16:0/1125987478 wait complete. 2024-09-12T00:57:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 -- start start 2024-09-12T00:57:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.988+0000 7f84b2a62740 1 Processor -- start 2024-09-12T00:57:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8baa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81aec20 con 0x55c7b8ba6370 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81ae550 con 0x55c7b8b9b5d0 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a81ab790 con 0x55c7b8b9cb40 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84b2a62740 10 client.0 _ll_register_callbacks cb 0x7f84a004d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58298/0 (socket says 172.21.15.16:58298) 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 -- 172.21.15.16:0/1171902584 learned_addr learned my addr 172.21.15.16:0/1171902584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84ae425640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84adc24640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8baa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 -- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 msgr2=0x55c7b8baa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8baa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 -- 172.21.15.16:0/1171902584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 msgr2=0x55c7b8ba9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 -- 172.21.15.16:0/1171902584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b8ba60e0 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f84aec26640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f84a0030d90 tx=0x7f84a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb28f265ade362b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.989+0000 7f8491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0057040 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84a00318e0 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f849809bcb0 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84a004f050 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7b8c20770 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849809bcb0 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84a002f940 con 0x55c7b8ba6370 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 10 client.15846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:57:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 20 client.15846 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8c22e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 4 client.15846 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.990+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7b8b08d70 con 0x55c7b8bb5b00 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84ae425640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8c22e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84ae425640 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8c22e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66d572c1736de229 server_cookie=a091815cdf446c21 in_seq=0 out_seq=0 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f8491ffb640 10 client.15846 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f8491ffb640 1 -- 172.21.15.16:0/1171902584 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7b8b08d70 con 0x55c7b8bb5b00 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f8491ffb640 10 client.15846 handle_command_reply: tid=0 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 client.15846 shutdown 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 msgr2=0x55c7b8baaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f84a0030d90 tx=0x7f84a0004ab0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 shutdown_connections 2024-09-12T00:57:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7b8ba6370 0x55c7b8baaa00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7b8b9cb40 0x55c7b8baa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55c7b8bb5b00 0x55c7b8c22e40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 --2- 172.21.15.16:0/1171902584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7b8b9b5d0 0x55c7b8ba9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 >> 172.21.15.16:0/1171902584 conn(0x55c7b8b64310 msgr2=0x55c7b8b9d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.991+0000 7f8491ffb640 0 client.15846 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 shutdown_connections 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b2a62740 1 -- 172.21.15.16:0/1171902584 wait complete. 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b2a62740 20 client.15846 trim_cache size 0 max 16384 2024-09-12T00:57:46.005 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 msgr2=0x7f84940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 0x7f84940a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009430 tx=0x7f84a0049000 comp rx=0 tx=0).stop 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 msgr2=0x7f84a8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f849800d950 tx=0x7f849800de10 comp rx=0 tx=0).stop 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 shutdown_connections 2024-09-12T00:57:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84940a2cf0 0x7f84940a51b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a8106f10 0x7f84a819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a8102790 0x7f84a8198910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 --2- 172.21.15.16:0/1943685216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a8101de0 0x7f84a81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 >> 172.21.15.16:0/1943685216 conn(0x7f84a80fd8c0 msgr2=0x7f84a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 shutdown_connections 2024-09-12T00:57:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:45.992+0000 7f84b0689640 1 -- 172.21.15.16:0/1943685216 wait complete. 2024-09-12T00:57:46.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:57:47.042 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:47.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:47 smithi016 ceph-mon[30972]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-12T00:57:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:47 smithi159 ceph-mon[35566]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-12T00:57:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:47 smithi099 ceph-mon[35654]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-12T00:57:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 -- 172.21.15.16:0/734588710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4100e20 msgr2=0x7f49f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/734588710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4100e20 0x7f49f4101200 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009900 tx=0x7f49e402a480 comp rx=0 tx=0).stop 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 -- 172.21.15.16:0/734588710 shutdown_connections 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/734588710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102ef0 0x7f49f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/734588710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4102570 0x7f49f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/734588710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4100e20 0x7f49f4101200 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 -- 172.21.15.16:0/734588710 >> 172.21.15.16:0/734588710 conn(0x7f49f40fc7d0 msgr2=0x7f49f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 -- 172.21.15.16:0/734588710 shutdown_connections 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.542+0000 7f49fa2c2640 1 -- 172.21.15.16:0/734588710 wait complete. 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 Processor -- start 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 -- start start 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4100e20 0x7f49f4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102570 0x7f49f4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4110fd0 con 0x7f49f4102ef0 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4110e50 con 0x7f49f4102570 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49fa2c2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4111150 con 0x7f49f4100e20 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49f885f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49f885f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58318/0 (socket says 172.21.15.16:58318) 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49f885f640 1 -- 172.21.15.16:0/2886927997 learned_addr learned my addr 172.21.15.16:0/2886927997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4100e20 0x7f49f4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.543+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102570 0x7f49f4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 -- 172.21.15.16:0/2886927997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4100e20 msgr2=0x7f49f4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4100e20 0x7f49f4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 -- 172.21.15.16:0/2886927997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102570 msgr2=0x7f49f4103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102570 0x7f49f4103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 -- 172.21.15.16:0/2886927997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f41a6a00 con 0x7f49f4102ef0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f885f640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00cad0 tx=0x7f49ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f17fa640 1 -- 172.21.15.16:0/2886927997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00ae20 con 0x7f49f4102ef0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f17fa640 1 -- 172.21.15.16:0/2886927997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ec005020 con 0x7f49f4102ef0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f41a6cf0 con 0x7f49f4102ef0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49f17fa640 1 -- 172.21.15.16:0/2886927997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00f6f0 con 0x7f49f4102ef0 2024-09-12T00:57:47.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.544+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a71e0 con 0x7f49f4102ef0 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f17fa640 1 -- 172.21.15.16:0/2886927997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49ec0051d0 con 0x7f49f4102ef0 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 Processor -- start 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 -- start start 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x556099388090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x556099395760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5560992ee2f0 con 0x5560993885d0 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556099295cf0 con 0x556099393370 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49fc69b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560992ea2b0 con 0x556099389b40 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58330/0 (socket says 172.21.15.16:58330) 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f3fff640 1 -- 172.21.15.16:0/2836119065 learned_addr learned my addr 172.21.15.16:0/2836119065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:47.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x556099388090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x556099395760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.547+0000 7f49f17fa640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 0x7f49dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f885f640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 0x7f49dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 -- 172.21.15.16:0/2836119065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 msgr2=0x556099388090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x556099388090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 -- 172.21.15.16:0/2836119065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 msgr2=0x556099395760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x556099395760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 -- 172.21.15.16:0/2836119065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609927a3a0 con 0x5560993885d0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f17fa640 1 -- 172.21.15.16:0/2886927997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f49ec0cf520 con 0x7f49f4102ef0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009230 tx=0x7f49e4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926b7b5dd33524e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49da7fc640 1 -- 172.21.15.16:0/2836119065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4035030 con 0x5560993885d0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49f885f640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 0x7f49dc0a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49ec097e70 tx=0x7f49ec093040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49da7fc640 1 -- 172.21.15.16:0/2836119065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4005520 con 0x5560993885d0 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.548+0000 7f49da7fc640 1 -- 172.21.15.16:0/2836119065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40035f0 con 0x5560993885d0 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- 172.21.15.16:0/2836119065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 msgr2=0x556099388a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009230 tx=0x7f49e4004200 comp rx=0 tx=0).stop 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- 172.21.15.16:0/2836119065 shutdown_connections 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x556099395760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560993885d0 0x556099388a10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2836119065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x556099388090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- 172.21.15.16:0/2836119065 >> 172.21.15.16:0/2836119065 conn(0x556099351310 msgr2=0x556099384c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- 172.21.15.16:0/2836119065 shutdown_connections 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- 172.21.15.16:0/2836119065 wait complete. 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.549+0000 7f49fc69b740 1 -- start start 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 Processor -- start 2024-09-12T00:57:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x556099392d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609938ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556099396010 con 0x556099393370 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560992a9060 con 0x5560993885d0 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556099396190 con 0x556099389b40 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49fc69b740 10 client.0 _ll_register_callbacks cb 0x556098ffc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58334/0 (socket says 172.21.15.16:58334) 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 -- 172.21.15.16:0/3347329055 learned_addr learned my addr 172.21.15.16:0/3347329055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609938ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x556099392d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 -- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 msgr2=0x55609938ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609938ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 -- 172.21.15.16:0/3347329055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 msgr2=0x556099392d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x556099392d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.550+0000 7f49f37fe640 1 -- 172.21.15.16:0/3347329055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609905f3c0 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49e8014f40 tx=0x7f49e800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f26ba4e528a74fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80142d0 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f49e8005020 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e4005520 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f49e8005220 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49fc69b740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560993a1b20 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4005520 con 0x556099393370 2024-09-12T00:57:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49e800f750 con 0x556099393370 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.551+0000 7f49fc69b740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49fc69b740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49fc69b740 20 client.15858 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560993a3120 0x5560993a5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49fc69b740 4 client.15858 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560992fb860 con 0x5560993a3120 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560993a3120 0x5560993a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560993a3120 0x5560993a5510 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90988ed75d33f0a8 server_cookie=3354adff2b8480a9 in_seq=0 out_seq=0 2024-09-12T00:57:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.552+0000 7f49d37fe640 10 client.15858 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:47.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49d37fe640 1 -- 172.21.15.16:0/3347329055 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560992fb860 con 0x5560993a3120 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49d37fe640 10 client.15858 handle_command_reply: tid=0 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 client.15858 shutdown 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 msgr2=0x556099390570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49e8014f40 tx=0x7f49e800ec00 comp rx=0 tx=0).stop 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 shutdown_connections 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560993a3120 0x5560993a5510 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x556099390570 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609938ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3347329055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x556099392d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 >> 172.21.15.16:0/3347329055 conn(0x556099351310 msgr2=0x55609938a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.553+0000 7f49d37fe640 0 client.15858 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.554+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 shutdown_connections 2024-09-12T00:57:47.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.554+0000 7f49fc69b740 1 -- 172.21.15.16:0/3347329055 wait complete. 2024-09-12T00:57:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.554+0000 7f49fc69b740 20 client.15858 trim_cache size 0 max 16384 2024-09-12T00:57:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 Processor -- start 2024-09-12T00:57:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 -- start start 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 0x556099384e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x55609938b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41afa60 con 0x556099389b40 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41b15a0 con 0x556099393370 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49fc69b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41b0130 con 0x5560993885d0 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58344/0 (socket says 172.21.15.16:58344) 2024-09-12T00:57:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.567+0000 7f49f3fff640 1 -- 172.21.15.16:0/3259013861 learned_addr learned my addr 172.21.15.16:0/3259013861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 -- 172.21.15.16:0/3259013861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 msgr2=0x556099384e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 0x556099384e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 0x556099384e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 -- 172.21.15.16:0/3259013861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 msgr2=0x55609938b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x55609938b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 -- 172.21.15.16:0/3259013861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560993a5660 con 0x556099389b40 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f37fe640 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 0x556099384e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49f3fff640 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004b70 tx=0x7f49e4004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da136a3ef90070a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49d9ffb640 1 -- 172.21.15.16:0/3259013861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4035040 con 0x556099389b40 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49d9ffb640 1 -- 172.21.15.16:0/3259013861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4054070 con 0x556099389b40 2024-09-12T00:57:47.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.568+0000 7f49d9ffb640 1 -- 172.21.15.16:0/3259013861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40317a0 con 0x556099389b40 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- 172.21.15.16:0/3259013861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 msgr2=0x55609939f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49e4004b70 tx=0x7f49e4004b00 comp rx=0 tx=0).stop 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- 172.21.15.16:0/3259013861 shutdown_connections 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556099393370 0x55609938b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099389b40 0x55609939f310 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 --2- 172.21.15.16:0/3259013861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560993885d0 0x556099384e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- 172.21.15.16:0/3259013861 >> 172.21.15.16:0/3259013861 conn(0x556099351310 msgr2=0x556099384700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- 172.21.15.16:0/3259013861 shutdown_connections 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- 172.21.15.16:0/3259013861 wait complete. 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 -- start start 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.569+0000 7f49fc69b740 1 Processor -- start 2024-09-12T00:57:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x55609939b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609939bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41b0ed0 con 0x556099393370 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41b0800 con 0x5560993885d0 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ada40 con 0x556099389b40 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49fc69b740 10 client.0 _ll_register_callbacks cb 0x7f49e800e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58358/0 (socket says 172.21.15.16:58358) 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 -- 172.21.15.16:0/2554962180 learned_addr learned my addr 172.21.15.16:0/2554962180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x55609939b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609939bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 -- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 msgr2=0x55609939bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609939bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 -- 172.21.15.16:0/2554962180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 msgr2=0x55609939b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x55609939b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 -- 172.21.15.16:0/2554962180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560993971d0 con 0x556099393370 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609939bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49f37fe640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e800e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74a34e83489b208 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.570+0000 7f49d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e802e070 con 0x556099393370 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f49e80140d0 con 0x556099393370 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e4054070 con 0x556099393370 2024-09-12T00:57:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f49e803e050 con 0x556099393370 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55609940d6e0 con 0x556099393370 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e4054070 con 0x556099393370 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49e8004e40 con 0x556099393370 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 20 client.15864 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55609940dde0 0x556099410200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 4 client.15864 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:57:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.571+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560992f5d70 con 0x55609940dde0 2024-09-12T00:57:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55609940dde0 0x556099410200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49f3fff640 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55609940dde0 0x556099410200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6841cf4cd01bc4cc server_cookie=2bb4871a03549c3 in_seq=0 out_seq=0 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49d37fe640 10 client.15864 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49d37fe640 1 -- 172.21.15.16:0/2554962180 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560992f5d70 con 0x55609940dde0 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49d37fe640 10 client.15864 handle_command_reply: tid=0 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.572+0000 7f49fc69b740 1 client.15864 shutdown 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 msgr2=0x5560993969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e800e620 comp rx=0 tx=0).stop 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 shutdown_connections 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55609940dde0 0x556099410200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556099393370 0x5560993969a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556099389b40 0x55609939bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49d37fe640 0 client.15864 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 --2- 172.21.15.16:0/2554962180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560993885d0 0x55609939b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 >> 172.21.15.16:0/2554962180 conn(0x556099351310 msgr2=0x5560993834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 shutdown_connections 2024-09-12T00:57:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 1 -- 172.21.15.16:0/2554962180 wait complete. 2024-09-12T00:57:47.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fc69b740 20 client.15864 trim_cache size 0 max 16384 2024-09-12T00:57:47.577 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 msgr2=0x7f49dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 0x7f49dc0a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49ec097e70 tx=0x7f49ec093040 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 msgr2=0x7f49f41041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.573+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00cad0 tx=0x7f49ec00cf90 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 shutdown_connections 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49dc0a2cf0 0x7f49dc0a51b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4102ef0 0x7f49f41041d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49f4102570 0x7f49f4103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 --2- 172.21.15.16:0/2886927997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f4100e20 0x7f49f4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 >> 172.21.15.16:0/2886927997 conn(0x7f49f40fc7d0 msgr2=0x7f49f406a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 shutdown_connections 2024-09-12T00:57:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:47.574+0000 7f49fa2c2640 1 -- 172.21.15.16:0/2886927997 wait complete. 2024-09-12T00:57:49.290 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:57:49.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:57:49.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:49 smithi016 ceph-mon[30972]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T00:57:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:49 smithi159 ceph-mon[35566]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T00:57:49.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:49 smithi099 ceph-mon[35654]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-12T00:57:49.553 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:50.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 -- 172.21.15.16:0/973012216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 msgr2=0x7feba8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 --2- 172.21.15.16:0/973012216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8103130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7feb98009900 tx=0x7feb9802a4a0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 -- 172.21.15.16:0/973012216 shutdown_connections 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 --2- 172.21.15.16:0/973012216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 0x7feba810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 --2- 172.21.15.16:0/973012216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 0x7feba810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 --2- 172.21.15.16:0/973012216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8103130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 -- 172.21.15.16:0/973012216 >> 172.21.15.16:0/973012216 conn(0x7feba80fc7d0 msgr2=0x7feba80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 -- 172.21.15.16:0/973012216 shutdown_connections 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.050+0000 7febadbf9640 1 -- 172.21.15.16:0/973012216 wait complete. 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 Processor -- start 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 -- start start 2024-09-12T00:57:50.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 0x7feba8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 0x7feba819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feba8111fe0 con 0x7feba8102d50 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feba8111e60 con 0x7feba810c180 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7febadbf9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feba8112160 con 0x7feba8103670 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60362/0 (socket says 172.21.15.16:60362) 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7feba77fe640 1 -- 172.21.15.16:0/2608715769 learned_addr learned my addr 172.21.15.16:0/2608715769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7feb9ffff640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 0x7feba8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.051+0000 7feba7fff640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 0x7feba819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 -- 172.21.15.16:0/2608715769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 msgr2=0x7feba8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 0x7feba8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 -- 172.21.15.16:0/2608715769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 msgr2=0x7feba819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 0x7feba819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 -- 172.21.15.16:0/2608715769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba819d510 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba77fe640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba57fa640 1 -- 172.21.15.16:0/2608715769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9803f030 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba819d7a0 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba57fa640 1 -- 172.21.15.16:0/2608715769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb98003ce0 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7feba57fa640 1 -- 172.21.15.16:0/2608715769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98035070 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.052+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81a50e0 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.054+0000 7feba57fa640 1 -- 172.21.15.16:0/2608715769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feb98042030 con 0x7feba8102d50 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 Processor -- start 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 -- start start 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba57fa640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 0x7feb940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a72b40280 con 0x556a72bdab60 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a72ae7b10 con 0x556a72bdb480 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7febaffd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a72b3c240 con 0x556a72be5d40 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba7fff640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 0x7feb940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60368/0 (socket says 172.21.15.16:60368) 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 -- 172.21.15.16:0/3567378195 learned_addr learned my addr 172.21.15.16:0/3567378195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba57fa640 1 -- 172.21.15.16:0/2608715769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb980efcf0 con 0x7feba8102d50 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feb9ffff640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba7fff640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 -- 172.21.15.16:0/3567378195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 msgr2=0x556a72be8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 -- 172.21.15.16:0/3567378195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 msgr2=0x556a72be5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feba77fe640 1 -- 172.21.15.16:0/3567378195 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a72af41f0 con 0x556a72bdab60 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.055+0000 7feb9ffff640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.056+0000 7feba7fff640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 0x7feb940a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feba0005fd0 tx=0x7feba0014bf0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.056+0000 7feba77fe640 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb9802a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1155b717dfa35bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.056+0000 7feb9dffb640 1 -- 172.21.15.16:0/3567378195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98049040 con 0x556a72bdab60 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.056+0000 7feb9dffb640 1 -- 172.21.15.16:0/3567378195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb980bc290 con 0x556a72bdab60 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.056+0000 7feb9dffb640 1 -- 172.21.15.16:0/3567378195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980c2900 con 0x556a72bdab60 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- 172.21.15.16:0/3567378195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 msgr2=0x556a72bdaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb9802a4a0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- 172.21.15.16:0/3567378195 shutdown_connections 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- 172.21.15.16:0/3567378195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bdaf40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- 172.21.15.16:0/3567378195 >> 172.21.15.16:0/3567378195 conn(0x556a72bd70f0 msgr2=0x556a72bd74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- 172.21.15.16:0/3567378195 shutdown_connections 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- 172.21.15.16:0/3567378195 wait complete. 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- start start 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 Processor -- start 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556a72be86d0 con 0x556a72bdab60 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556a72afaf10 con 0x556a72bdb480 2024-09-12T00:57:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.057+0000 7febaffd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a72be8850 con 0x556a72be5d40 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7febaffd2740 10 client.0 _ll_register_callbacks cb 0x556a72ac3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60376/0 (socket says 172.21.15.16:60376) 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 -- 172.21.15.16:0/290498314 learned_addr learned my addr 172.21.15.16:0/290498314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feb9ffff640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba7fff640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 -- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 msgr2=0x556a72be36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 -- 172.21.15.16:0/290498314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 msgr2=0x556a72be3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 -- 172.21.15.16:0/290498314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a728b16a0 con 0x556a72bdab60 2024-09-12T00:57:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feba77fe640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb98004650 tx=0x7feb980057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa97cf1cfee0718f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980d4070 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb980c49c0 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb980bc290 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb98049030 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7febaffd2740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.058+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a72be86d0 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9809e020 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb980b8900 con 0x556a72bdab60 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 20 client.15876 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72bf5530 0x556a72bf7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 4 client.15876 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a72b4d7f0 con 0x556a72bf5530 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.059+0000 7feb9ffff640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72bf5530 0x556a72bf7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.060+0000 7feb86ffd640 10 client.15876 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.060+0000 7feb9ffff640 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72bf5530 0x556a72bf7980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c80ce656717a53 server_cookie=f0fb6261cc065fa7 in_seq=0 out_seq=0 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.060+0000 7feb86ffd640 1 -- 172.21.15.16:0/290498314 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a72b4d7f0 con 0x556a72bf5530 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.060+0000 7feb86ffd640 10 client.15876 handle_command_reply: tid=0 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.060+0000 7febaffd2740 1 client.15876 shutdown 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 msgr2=0x556a72be2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb98004650 tx=0x7feb980057d0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 shutdown_connections 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72bf5530 0x556a72bf7980 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72be36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72be3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 --2- 172.21.15.16:0/290498314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72be2c50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7feb86ffd640 0 client.15876 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 >> 172.21.15.16:0/290498314 conn(0x556a72bd70f0 msgr2=0x556a72bdce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 shutdown_connections 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 1 -- 172.21.15.16:0/290498314 wait complete. 2024-09-12T00:57:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.061+0000 7febaffd2740 20 client.15876 trim_cache size 0 max 16384 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 Processor -- start 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 -- start start 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72bdab60 0x556a72bfd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 0x556a72bdfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feba81ad890 con 0x556a72bdb480 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feba81af3d0 con 0x556a72be5d40 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7febaffd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feba81adf60 con 0x556a72bdab60 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60386/0 (socket says 172.21.15.16:60386) 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 -- 172.21.15.16:0/2940028210 learned_addr learned my addr 172.21.15.16:0/2940028210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feba77fe640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72bdab60 0x556a72bfd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feba7fff640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 0x556a72bdfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 -- 172.21.15.16:0/2940028210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72bdab60 msgr2=0x556a72bfd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72bdab60 0x556a72bfd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 -- 172.21.15.16:0/2940028210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 msgr2=0x556a72bdfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 0x556a72bdfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feb9ffff640 1 -- 172.21.15.16:0/2940028210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a72be3c20 con 0x556a72bdb480 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.079+0000 7feba7fff640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 0x556a72bdfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:50.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.080+0000 7feb9ffff640 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feb90009ad0 tx=0x7feb90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed635db377afb606 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.080+0000 7feb9d7fa640 1 -- 172.21.15.16:0/2940028210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9001d310 con 0x556a72bdb480 2024-09-12T00:57:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.080+0000 7feb9d7fa640 1 -- 172.21.15.16:0/2940028210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9001d990 con 0x556a72bdb480 2024-09-12T00:57:50.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.080+0000 7feb9d7fa640 1 -- 172.21.15.16:0/2940028210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb900159a0 con 0x556a72bdb480 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- 172.21.15.16:0/2940028210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 msgr2=0x556a72bfd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feb90009ad0 tx=0x7feb90008040 comp rx=0 tx=0).stop 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- 172.21.15.16:0/2940028210 shutdown_connections 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72be5d40 0x556a72bdfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdb480 0x556a72bfd740 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- 172.21.15.16:0/2940028210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72bdab60 0x556a72bfd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- 172.21.15.16:0/2940028210 >> 172.21.15.16:0/2940028210 conn(0x556a72bd70f0 msgr2=0x556a72be4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- 172.21.15.16:0/2940028210 shutdown_connections 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- 172.21.15.16:0/2940028210 wait complete. 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 -- start start 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 Processor -- start 2024-09-12T00:57:50.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.081+0000 7febaffd2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72bea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7febaffd2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72bea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7febaffd2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feba81aed00 con 0x556a72bdab60 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7febaffd2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feba81ae630 con 0x556a72bdb480 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7febaffd2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feba81ab870 con 0x556a72be5d40 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7febaffd2740 10 client.0 _ll_register_callbacks cb 0x7feb98003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60392/0 (socket says 172.21.15.16:60392) 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 -- 172.21.15.16:0/790974420 learned_addr learned my addr 172.21.15.16:0/790974420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba7fff640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72bea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feb9ffff640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72bea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 -- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 msgr2=0x556a72bea8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72bea8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 -- 172.21.15.16:0/790974420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 msgr2=0x556a72bea360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72bea360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 -- 172.21.15.16:0/790974420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a72beaf90 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feb9ffff640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72bea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feba77fe640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feb980b86f0 tx=0x7feb980092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf75ef7011d9b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.082+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980d4070 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb9809f080 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9001d990 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb980bbbf0 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a72beaf90 con 0x556a72bdab60 2024-09-12T00:57:50.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb9001d990 con 0x556a72bdab60 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb980ed720 con 0x556a72bdab60 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 20 client.15882 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72c65800 0x556a72c67bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 4 client.15882 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556a72b47d00 con 0x556a72c65800 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.083+0000 7feb9ffff640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72c65800 0x556a72c67bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.084+0000 7feb9ffff640 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72c65800 0x556a72c67bf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0e8bcaf7c0cd6 server_cookie=c946ca54e8035a85 in_seq=0 out_seq=0 2024-09-12T00:57:50.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.084+0000 7feb86ffd640 10 client.15882 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7feb86ffd640 1 -- 172.21.15.16:0/790974420 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556a72b47d00 con 0x556a72c65800 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7feb86ffd640 10 client.15882 handle_command_reply: tid=0 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 client.15882 shutdown 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 msgr2=0x556a72bee020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feb980b86f0 tx=0x7feb980092d0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 shutdown_connections 2024-09-12T00:57:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556a72c65800 0x556a72c67bf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a72be5d40 0x556a72bea8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556a72bdb480 0x556a72bea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 --2- 172.21.15.16:0/790974420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556a72bdab60 0x556a72bee020 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 >> 172.21.15.16:0/790974420 conn(0x556a72bd70f0 msgr2=0x556a72be2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7feb86ffd640 0 client.15882 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 shutdown_connections 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 1 -- 172.21.15.16:0/790974420 wait complete. 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.085+0000 7febaffd2740 20 client.15882 trim_cache size 0 max 16384 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:50.088 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:57:50.089 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e7b9282e-c549-4f2c-8bd4-fa0f1663d976", 2024-09-12T00:57:50.089 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:57:50.089 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:50.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 msgr2=0x7feb940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 0x7feb940a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feba0005fd0 tx=0x7feba0014bf0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 msgr2=0x7feba8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feb98006010 tx=0x7feb980042f0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 shutdown_connections 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb940a2cf0 0x7feb940a51b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feba810c180 0x7feba819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feba8103670 0x7feba8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 --2- 172.21.15.16:0/2608715769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba8102d50 0x7feba8198560 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 >> 172.21.15.16:0/2608715769 conn(0x7feba80fc7d0 msgr2=0x7feba80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 shutdown_connections 2024-09-12T00:57:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:50.086+0000 7febadbf9640 1 -- 172.21.15.16:0/2608715769 wait complete. 2024-09-12T00:57:50.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:57:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:57:51.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:51 smithi016 ceph-mon[30972]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:51.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:51 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:57:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:51 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:51 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:57:51.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:51 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:51 smithi159 ceph-mon[35566]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:51 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:57:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:51 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:51 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:57:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:51 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:51 smithi099 ceph-mon[35654]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:51 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:57:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:51 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:51 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:57:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:51 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:57:51.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 -- 172.21.15.16:0/2539148489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b81036e0 msgr2=0x7fd6b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/2539148489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b81036e0 0x7fd6b8107b90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009930 tx=0x7fd6ac02a420 comp rx=0 tx=0).stop 2024-09-12T00:57:51.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 -- 172.21.15.16:0/2539148489 shutdown_connections 2024-09-12T00:57:51.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/2539148489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81082a0 0x7fd6b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/2539148489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b81036e0 0x7fd6b8107b90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/2539148489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b8102d30 0x7fd6b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 -- 172.21.15.16:0/2539148489 >> 172.21.15.16:0/2539148489 conn(0x7fd6b80fc7d0 msgr2=0x7fd6b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 -- 172.21.15.16:0/2539148489 shutdown_connections 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 -- 172.21.15.16:0/2539148489 wait complete. 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.578+0000 7fd6bdc15640 1 Processor -- start 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 -- start start 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 0x7fd6b8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 0x7fd6b8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81120a0 con 0x7fd6b8102d30 2024-09-12T00:57:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8111f20 con 0x7fd6b81082a0 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6bdc15640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8112220 con 0x7fd6b81036e0 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60418/0 (socket says 172.21.15.16:60418) 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 -- 172.21.15.16:0/7245905 learned_addr learned my addr 172.21.15.16:0/7245905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 0x7fd6b8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 0x7fd6b8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 -- 172.21.15.16:0/7245905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 msgr2=0x7fd6b8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 0x7fd6b8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 -- 172.21.15.16:0/7245905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 msgr2=0x7fd6b8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 0x7fd6b8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b77fe640 1 -- 172.21.15.16:0/7245905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b81a6b00 con 0x7fd6b8102d30 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 0x7fd6b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.579+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 0x7fd6b8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e9f0 tx=0x7fd6a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6b4ff9640 1 -- 172.21.15.16:0/7245905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a800cd80 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6b4ff9640 1 -- 172.21.15.16:0/7245905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a8010910 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b81a6df0 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6b4ff9640 1 -- 172.21.15.16:0/7245905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8018990 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.580+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81a7250 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.582+0000 7fd6b4ff9640 1 -- 172.21.15.16:0/7245905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd6a8010ac0 con 0x7fd6b8102d30 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.582+0000 7fd6bffee740 1 Processor -- start 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.582+0000 7fd6bffee740 1 -- start start 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345cb680 0x5638345cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638345262e0 con 0x5638345c04a0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638344cdce0 con 0x5638345cb680 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6bffee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5638345222a0 con 0x5638345c0dc0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b4ff9640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 0x7fd6a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60432/0 (socket says 172.21.15.16:60432) 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 -- 172.21.15.16:0/1818739291 learned_addr learned my addr 172.21.15.16:0/1818739291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 0x7fd6a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345cb680 0x5638345cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b4ff9640 1 -- 172.21.15.16:0/7245905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd6a80cf900 con 0x7fd6b8102d30 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 -- 172.21.15.16:0/1818739291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 msgr2=0x5638345cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 -- 172.21.15.16:0/1818739291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345cb680 msgr2=0x5638345cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345cb680 0x5638345cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 -- 172.21.15.16:0/1818739291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638344b2440 con 0x5638345c04a0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.583+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0012d90 tx=0x7fd6b00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedd1b6570f24085 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 0x7fd6a00a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010ca0 tx=0x7fd6b001d000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd69dffb640 1 -- 172.21.15.16:0/1818739291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0012580 con 0x5638345c04a0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd69dffb640 1 -- 172.21.15.16:0/1818739291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0012870 con 0x5638345c04a0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd69dffb640 1 -- 172.21.15.16:0/1818739291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0005410 con 0x5638345c04a0 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 -- 172.21.15.16:0/1818739291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 msgr2=0x5638345c0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0012d90 tx=0x7fd6b00107a0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 -- 172.21.15.16:0/1818739291 shutdown_connections 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345cb680 0x5638345cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 --2- 172.21.15.16:0/1818739291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345c0880 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 -- 172.21.15.16:0/1818739291 >> 172.21.15.16:0/1818739291 conn(0x5638345bca30 msgr2=0x5638345bce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.584+0000 7fd6bffee740 1 -- 172.21.15.16:0/1818739291 shutdown_connections 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 -- 172.21.15.16:0/1818739291 wait complete. 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 -- start start 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 Processor -- start 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638345ce1f0 con 0x5638345cb680 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5638344e1050 con 0x5638345c04a0 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.585+0000 7fd6bffee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5638345ce370 con 0x5638345c0dc0 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6bffee740 10 client.0 _ll_register_callbacks cb 0x563834521160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60434/0 (socket says 172.21.15.16:60434) 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 -- 172.21.15.16:0/3861535853 learned_addr learned my addr 172.21.15.16:0/3861535853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 -- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 msgr2=0x5638345c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 -- 172.21.15.16:0/3861535853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 msgr2=0x5638345cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 -- 172.21.15.16:0/3861535853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638342973c0 con 0x5638345cb680 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010fd0 tx=0x7fd6b000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de08e218ba5e2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0004bd0 con 0x5638345cb680 2024-09-12T00:57:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd6b0004d70 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.586+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b0012870 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6b003d050 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638345d9be0 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0012870 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6b002f590 con 0x5638345cb680 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 20 client.15894 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638345db0a0 0x5638345dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 4 client.15894 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563834533850 con 0x5638345db0a0 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.587+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638345db0a0 0x5638345dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.588+0000 7fd696ffd640 10 client.15894 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:51.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.588+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638345db0a0 0x5638345dd490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4730bb8686727e server_cookie=eb61e9092490bb2e in_seq=0 out_seq=0 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.588+0000 7fd696ffd640 1 -- 172.21.15.16:0/3861535853 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563834533850 con 0x5638345db0a0 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.588+0000 7fd696ffd640 10 client.15894 handle_command_reply: tid=0 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.588+0000 7fd6bffee740 1 client.15894 shutdown 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 msgr2=0x5638345c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010fd0 tx=0x7fd6b000ded0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 shutdown_connections 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638345db0a0 0x5638345dd490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345c8830 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 --2- 172.21.15.16:0/3861535853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd696ffd640 0 client.15894 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 >> 172.21.15.16:0/3861535853 conn(0x5638345bca30 msgr2=0x5638345c2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 shutdown_connections 2024-09-12T00:57:51.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 1 -- 172.21.15.16:0/3861535853 wait complete. 2024-09-12T00:57:51.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.589+0000 7fd6bffee740 20 client.15894 trim_cache size 0 max 16384 2024-09-12T00:57:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 Processor -- start 2024-09-12T00:57:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 -- start start 2024-09-12T00:57:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c0dc0 0x5638345e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345cb680 0x5638345e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81afaa0 con 0x5638345c04a0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b15e0 con 0x5638345c0dc0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6bffee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0170 con 0x5638345cb680 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60440/0 (socket says 172.21.15.16:60440) 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6b77fe640 1 -- 172.21.15.16:0/2280797121 learned_addr learned my addr 172.21.15.16:0/2280797121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c0dc0 0x5638345e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.606+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345cb680 0x5638345e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 -- 172.21.15.16:0/2280797121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345cb680 msgr2=0x5638345e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345cb680 0x5638345e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 -- 172.21.15.16:0/2280797121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c0dc0 msgr2=0x5638345e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c0dc0 0x5638345e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 -- 172.21.15.16:0/2280797121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638345c0bf0 con 0x5638345c04a0 2024-09-12T00:57:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000c00 tx=0x7fd6a801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916290e5f6a5f528 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd69d7fa640 1 -- 172.21.15.16:0/2280797121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80a4a60 con 0x5638345c04a0 2024-09-12T00:57:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd69d7fa640 1 -- 172.21.15.16:0/2280797121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a809a0f0 con 0x5638345c04a0 2024-09-12T00:57:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd69d7fa640 1 -- 172.21.15.16:0/2280797121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8018d90 con 0x5638345c04a0 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6bffee740 1 -- 172.21.15.16:0/2280797121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 msgr2=0x5638345ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.607+0000 7fd6bffee740 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8000c00 tx=0x7fd6a801a040 comp rx=0 tx=0).stop 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- 172.21.15.16:0/2280797121 shutdown_connections 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345cb680 0x5638345e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c0dc0 0x5638345e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- 172.21.15.16:0/2280797121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345c04a0 0x5638345ca5e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- 172.21.15.16:0/2280797121 >> 172.21.15.16:0/2280797121 conn(0x5638345bca30 msgr2=0x5638345c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- 172.21.15.16:0/2280797121 shutdown_connections 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- 172.21.15.16:0/2280797121 wait complete. 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- start start 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 Processor -- start 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0f10 con 0x5638345cb680 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81b0840 con 0x5638345c04a0 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81adaa0 con 0x5638345c0dc0 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.608+0000 7fd6bffee740 10 client.0 _ll_register_callbacks cb 0x7fd6a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:57:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60454/0 (socket says 172.21.15.16:60454) 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 -- 172.21.15.16:0/4162429286 learned_addr learned my addr 172.21.15.16:0/4162429286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b6ffd640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 -- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 msgr2=0x5638345d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345d1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 -- 172.21.15.16:0/4162429286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 msgr2=0x5638345d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 -- 172.21.15.16:0/4162429286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638345d1fd0 con 0x5638345cb680 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6b7fff640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0000f80 tx=0x7fd6b000ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dfda2ca75d9865 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0012580 con 0x5638345cb680 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd6b0012040 con 0x5638345cb680 2024-09-12T00:57:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.609+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a809a0f0 con 0x5638345cb680 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6b0012240 con 0x5638345cb680 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563834645be0 con 0x5638345cb680 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a809a0f0 con 0x5638345cb680 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6b0026a50 con 0x5638345cb680 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 20 client.15900 populate_metadata read hostname 'smithi016' 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638346462f0 0x563834648710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 4 client.15900 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56383452dd60 con 0x5638346462f0 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.610+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638346462f0 0x563834648710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:57:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6b77fe640 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638346462f0 0x563834648710 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaec3db04bdfaea server_cookie=89ccfde83b3175f4 in_seq=0 out_seq=0 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd696ffd640 10 client.15900 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd696ffd640 1 -- 172.21.15.16:0/4162429286 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56383452dd60 con 0x5638346462f0 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd696ffd640 10 client.15900 handle_command_reply: tid=0 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 client.15900 shutdown 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 msgr2=0x5638345d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0000f80 tx=0x7fd6b000ddb0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 shutdown_connections 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5638346462f0 0x563834648710 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638345cb680 0x5638345d17a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5638345c0dc0 0x5638345d1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 --2- 172.21.15.16:0/4162429286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5638345c04a0 0x5638345d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 >> 172.21.15.16:0/4162429286 conn(0x5638345bca30 msgr2=0x5638345c6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd696ffd640 0 client.15900 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:57:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 shutdown_connections 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 1 -- 172.21.15.16:0/4162429286 wait complete. 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.611+0000 7fd6bffee740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "e7b9282e-c549-4f2c-8bd4-fa0f1663d976": { 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e7b9282e-c549-4f2c-8bd4-fa0f1663d976", 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 msgr2=0x7fd6a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 0x7fd6a00a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010ca0 tx=0x7fd6b001d000 comp rx=0 tx=0).stop 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 msgr2=0x7fd6b8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e9f0 tx=0x7fd6a800eeb0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 shutdown_connections 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6a00a2cf0 0x7fd6a00a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6b81082a0 0x7fd6b8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6b81036e0 0x7fd6b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 --2- 172.21.15.16:0/7245905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6b8102d30 0x7fd6b8100990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 >> 172.21.15.16:0/7245905 conn(0x7fd6b80fc7d0 msgr2=0x7fd6b8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 shutdown_connections 2024-09-12T00:57:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:57:51.612+0000 7fd6bdc15640 1 -- 172.21.15.16:0/7245905 wait complete. 2024-09-12T00:57:52.336 INFO:tasks.cephfs.filesystem:scrub status for tag:e7b9282e-c549-4f2c-8bd4-fa0f1663d976 - {'path': '/', 'tag': 'e7b9282e-c549-4f2c-8bd4-fa0f1663d976', 'options': 'recursive,force'} 2024-09-12T00:57:53.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:53 smithi159 ceph-mon[35566]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:53.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:53 smithi016 ceph-mon[30972]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:53.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:53 smithi099 ceph-mon[35654]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:55.326 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:55 smithi099 ceph-mon[35654]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:55.326 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:55 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:57:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:55 smithi159 ceph-mon[35566]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:55 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:57:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:55 smithi016 ceph-mon[30972]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-12T00:57:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:55 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:57:57.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:57 smithi159 ceph-mon[35566]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:57.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:57 smithi016 ceph-mon[30972]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:57 smithi099 ceph-mon[35654]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-12T00:57:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:57:59 smithi159 ceph-mon[35566]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:57:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:57:59 smithi016 ceph-mon[30972]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:57:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:57:59 smithi099 ceph-mon[35654]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:01 smithi159 ceph-mon[35566]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T00:58:01.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:01 smithi016 ceph-mon[30972]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T00:58:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:01 smithi099 ceph-mon[35654]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T00:58:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:03 smithi159 ceph-mon[35566]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:03 smithi016 ceph-mon[30972]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:03 smithi099 ceph-mon[35654]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:05.189 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T00:58:05.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:58:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:05 smithi159 ceph-mon[35566]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:05 smithi016 ceph-mon[30972]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:05.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:05 smithi099 ceph-mon[35654]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-09-12T00:58:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.938+0000 7f26f9c20640 1 -- 172.21.15.16:0/4119304270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102d40 msgr2=0x7f26f4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.938+0000 7f26f9c20640 1 --2- 172.21.15.16:0/4119304270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102d40 0x7f26f4103120 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009900 tx=0x7f26e402a480 comp rx=0 tx=0).stop 2024-09-12T00:58:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 -- 172.21.15.16:0/4119304270 shutdown_connections 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 --2- 172.21.15.16:0/4119304270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f410c170 0x7f26f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 --2- 172.21.15.16:0/4119304270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4103660 0x7f26f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 --2- 172.21.15.16:0/4119304270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102d40 0x7f26f4103120 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 -- 172.21.15.16:0/4119304270 >> 172.21.15.16:0/4119304270 conn(0x7f26f40fc7a0 msgr2=0x7f26f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 -- 172.21.15.16:0/4119304270 shutdown_connections 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 -- 172.21.15.16:0/4119304270 wait complete. 2024-09-12T00:58:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 Processor -- start 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.939+0000 7f26f9c20640 1 -- start start 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102d40 0x7f26f419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4111f70 con 0x7f26f410c170 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4111df0 con 0x7f26f4103660 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f9c20640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41120f0 con 0x7f26f4102d40 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:55444/0 (socket says 172.21.15.16:55444) 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53798/0 (socket says 172.21.15.16:53798) 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f2ffd640 1 -- 172.21.15.16:0/3752029093 learned_addr learned my addr 172.21.15.16:0/3752029093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f37fe640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102d40 0x7f26f419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 -- 172.21.15.16:0/3752029093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102d40 msgr2=0x7f26f419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102d40 0x7f26f419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 -- 172.21.15.16:0/3752029093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 msgr2=0x7f26f419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 -- 172.21.15.16:0/3752029093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f419f680 con 0x7f26f410c170 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f2ffd640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.940+0000 7f26f3fff640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00ca30 tx=0x7f26ec00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.941+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007d90 con 0x7f26f410c170 2024-09-12T00:58:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.941+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec005020 con 0x7f26f410c170 2024-09-12T00:58:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.941+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f419f970 con 0x7f26f410c170 2024-09-12T00:58:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.941+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00f780 con 0x7f26f410c170 2024-09-12T00:58:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.941+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f41a7250 con 0x7f26f410c170 2024-09-12T00:58:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.942+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f4103120 con 0x7f26f410c170 2024-09-12T00:58:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.943+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26ec0051d0 con 0x7f26f410c170 2024-09-12T00:58:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.944+0000 7f26f0ff9640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.944+0000 7f26f37fe640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.944+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f26ec0cf520 con 0x7f26f410c170 2024-09-12T00:58:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.944+0000 7f26f37fe640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 0x7f26dc0a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:05.949+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26ec014070 con 0x7f26f410c170 2024-09-12T00:58:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.161+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f41a7530 con 0x7f26f410c170 2024-09-12T00:58:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.162+0000 7f26f0ff9640 1 -- 172.21.15.16:0/3752029093 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f26ec09bac0 con 0x7f26f410c170 2024-09-12T00:58:06.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:58:06.165 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 msgr2=0x7f26dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 0x7f26dc0a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40058d0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 msgr2=0x7f26f419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00ca30 tx=0x7f26ec00cef0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 shutdown_connections 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26dc0a2d70 0x7f26dc0a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f410c170 0x7f26f419efa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4103660 0x7f26f419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 --2- 172.21.15.16:0/3752029093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102d40 0x7f26f419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 >> 172.21.15.16:0/3752029093 conn(0x7f26f40fc7a0 msgr2=0x7f26f40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 shutdown_connections 2024-09-12T00:58:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:06.164+0000 7f26f9c20640 1 -- 172.21.15.16:0/3752029093 wait complete. 2024-09-12T00:58:06.899 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T00:58:06.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T00:58:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:07 smithi016 ceph-mon[30972]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-09-12T00:58:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:07 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3752029093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:07 smithi159 ceph-mon[35566]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-09-12T00:58:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:07 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3752029093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:07 smithi099 ceph-mon[35654]: pgmap v64: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-09-12T00:58:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:07 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3752029093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:07.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 -- 172.21.15.16:0/4215321006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044100a80 msgr2=0x7fc044100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/4215321006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044100a80 0x7fc044100ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc034009930 tx=0x7fc03402a420 comp rx=0 tx=0).stop 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 -- 172.21.15.16:0/4215321006 shutdown_connections 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/4215321006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044101420 0x7fc04410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/4215321006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044100a80 0x7fc044100ee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/4215321006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 0x7fc044106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 -- 172.21.15.16:0/4215321006 >> 172.21.15.16:0/4215321006 conn(0x7fc0440fc7a0 msgr2=0x7fc0440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.639+0000 7fc04c7b8640 1 -- 172.21.15.16:0/4215321006 shutdown_connections 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 -- 172.21.15.16:0/4215321006 wait complete. 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 Processor -- start 2024-09-12T00:58:07.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 -- start start 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044100a80 0x7fc0440fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 0x7fc0440fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc044110e80 con 0x7fc044101420 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc044110d00 con 0x7fc044106a80 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.640+0000 7fc04c7b8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc044111000 con 0x7fc044100a80 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53816/0 (socket says 172.21.15.16:53816) 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 -- 172.21.15.16:0/1202680520 learned_addr learned my addr 172.21.15.16:0/1202680520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc04a554640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044100a80 0x7fc0440fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc04ad55640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 0x7fc0440fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 -- 172.21.15.16:0/1202680520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044100a80 msgr2=0x7fc0440fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044100a80 0x7fc0440fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 -- 172.21.15.16:0/1202680520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 msgr2=0x7fc0440fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 0x7fc0440fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 -- 172.21.15.16:0/1202680520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441a68f0 con 0x7fc044101420 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc049d53640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc034009230 tx=0x7fc034004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc03b7fe640 1 -- 172.21.15.16:0/1202680520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03403f030 con 0x7fc044101420 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc03b7fe640 1 -- 172.21.15.16:0/1202680520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc034003ad0 con 0x7fc044101420 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441a6b80 con 0x7fc044101420 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc03b7fe640 1 -- 172.21.15.16:0/1202680520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034035070 con 0x7fc044101420 2024-09-12T00:58:07.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.641+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a7060 con 0x7fc044101420 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.644+0000 7fc03b7fe640 1 -- 172.21.15.16:0/1202680520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc034042030 con 0x7fc044101420 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 Processor -- start 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 -- start start 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96edfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e96e452a0 con 0x564e96eea940 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e96decb10 con 0x564e96ee0080 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04eb91740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e96e41260 con 0x564e96edf760 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53820/0 (socket says 172.21.15.16:53820) 2024-09-12T00:58:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc049d53640 1 -- 172.21.15.16:0/4237994852 learned_addr learned my addr 172.21.15.16:0/4237994852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04ad55640 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96edfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.645+0000 7fc04a554640 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc03b7fe640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 -- 172.21.15.16:0/4237994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 msgr2=0x564e96edfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96edfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 -- 172.21.15.16:0/4237994852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 msgr2=0x564e96eea400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eea400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 -- 172.21.15.16:0/4237994852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e96bb6b90 con 0x564e96eea940 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc04ad55640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc03b7fe640 1 -- 172.21.15.16:0/1202680520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc0340efa80 con 0x7fc044101420 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc049d53640 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ed170 tx=0x7fc0340b80a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251eb0476f14348b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc038ff9640 1 -- 172.21.15.16:0/4237994852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034049070 con 0x564e96eea940 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc038ff9640 1 -- 172.21.15.16:0/4237994852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0340bc6d0 con 0x564e96eea940 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc038ff9640 1 -- 172.21.15.16:0/4237994852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340f7040 con 0x564e96eea940 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.646+0000 7fc04ad55640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 0x7fc0300a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005fd0 tx=0x7fc03c004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- 172.21.15.16:0/4237994852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 msgr2=0x564e96eecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ed170 tx=0x7fc0340b80a0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- 172.21.15.16:0/4237994852 shutdown_connections 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96eea940 0x564e96eecd30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4237994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96edfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- 172.21.15.16:0/4237994852 >> 172.21.15.16:0/4237994852 conn(0x564e96edbcf0 msgr2=0x564e96edc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- 172.21.15.16:0/4237994852 shutdown_connections 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- 172.21.15.16:0/4237994852 wait complete. 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.647+0000 7fc04eb91740 1 -- start start 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 Processor -- start 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96ee7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 0x564e96ee82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e96eed2d0 con 0x564e96ee0080 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e96dcd6e0 con 0x564e96eea940 2024-09-12T00:58:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e96eed450 con 0x564e96edf760 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04eb91740 10 client.0 _ll_register_callbacks cb 0x564e96df1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53824/0 (socket says 172.21.15.16:53824) 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 -- 172.21.15.16:0/729148429 learned_addr learned my addr 172.21.15.16:0/729148429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04ad55640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 0x564e96ee82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04a554640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96ee7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 -- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 msgr2=0x564e96ee7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96ee7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 -- 172.21.15.16:0/729148429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 msgr2=0x564e96ee82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 0x564e96ee82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 -- 172.21.15.16:0/729148429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e96ef6760 con 0x564e96ee0080 2024-09-12T00:58:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc04ad55640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 0x564e96ee82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc049d53640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc03403d040 tx=0x7fc0340f3160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eb291114005dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.648+0000 7fc02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034049070 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc0340f3810 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0340bc6d0 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0340bbb10 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e96eed2d0 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0340bc6d0 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc034003c80 con 0x564e96ee0080 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:07.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 10 client.15918 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 20 client.15918 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96efc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 4 client.15918 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.649+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e96e529d0 con 0x564e96efa310 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.650+0000 7fc04a554640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96efc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.650+0000 7fc04a554640 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96efc730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7f62fa367b3902 server_cookie=751a370143bb6b4c in_seq=0 out_seq=0 2024-09-12T00:58:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.650+0000 7fc02dffb640 10 client.15918 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc02dffb640 1 -- 172.21.15.16:0/729148429 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e96e529d0 con 0x564e96efa310 2024-09-12T00:58:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc02dffb640 10 client.15918 handle_command_reply: tid=0 2024-09-12T00:58:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 client.15918 shutdown 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 msgr2=0x564e96ee7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc03403d040 tx=0x7fc0340f3160 comp rx=0 tx=0).stop 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 shutdown_connections 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96efc730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96eea940 0x564e96ee82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee7d80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc02dffb640 0 client.15918 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 --2- 172.21.15.16:0/729148429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96edf760 0x564e96ee7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 >> 172.21.15.16:0/729148429 conn(0x564e96edbcf0 msgr2=0x564e96ee19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 shutdown_connections 2024-09-12T00:58:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 1 -- 172.21.15.16:0/729148429 wait complete. 2024-09-12T00:58:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.651+0000 7fc04eb91740 20 client.15918 trim_cache size 0 max 16384 2024-09-12T00:58:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 Processor -- start 2024-09-12T00:58:07.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 -- start start 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 0x564e96edacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96f02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441af930 con 0x564e96ee0080 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b1470 con 0x564e96edf760 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc04eb91740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b0000 con 0x564e96eea940 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc049d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53840/0 (socket says 172.21.15.16:53840) 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.668+0000 7fc049d53640 1 -- 172.21.15.16:0/2696765145 learned_addr learned my addr 172.21.15.16:0/2696765145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc04a554640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 0x564e96edacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc04ad55640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96f02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 -- 172.21.15.16:0/2696765145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 msgr2=0x564e96f02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96f02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 -- 172.21.15.16:0/2696765145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 msgr2=0x564e96edacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 0x564e96edacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 -- 172.21.15.16:0/2696765145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e96edfe70 con 0x564e96ee0080 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc04a554640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 0x564e96edacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc049d53640 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc0340c4ce0 tx=0x7fc0340ba4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be7602e971b4fde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc02ffff640 1 -- 172.21.15.16:0/2696765145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034049050 con 0x564e96ee0080 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc02ffff640 1 -- 172.21.15.16:0/2696765145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0340bc6d0 con 0x564e96ee0080 2024-09-12T00:58:07.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.669+0000 7fc02ffff640 1 -- 172.21.15.16:0/2696765145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340bb930 con 0x564e96ee0080 2024-09-12T00:58:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- 172.21.15.16:0/2696765145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 msgr2=0x564e96ee8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc0340c4ce0 tx=0x7fc0340ba4e0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- 172.21.15.16:0/2696765145 shutdown_connections 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96f02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96ee0080 0x564e96ee8110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- 172.21.15.16:0/2696765145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96edf760 0x564e96edacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- 172.21.15.16:0/2696765145 >> 172.21.15.16:0/2696765145 conn(0x564e96edbcf0 msgr2=0x564e96ee98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- 172.21.15.16:0/2696765145 shutdown_connections 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- 172.21.15.16:0/2696765145 wait complete. 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- start start 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 Processor -- start 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eeea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96eeefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b0da0 con 0x564e96edf760 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441b06d0 con 0x564e96ee0080 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441ad910 con 0x564e96eea940 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.670+0000 7fc04eb91740 10 client.0 _ll_register_callbacks cb 0x7fc0340b8640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53854/0 (socket says 172.21.15.16:53854) 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 -- 172.21.15.16:0/4258080494 learned_addr learned my addr 172.21.15.16:0/4258080494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc049d53640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eeea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04ad55640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96eeefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 -- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 msgr2=0x564e96eeefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96eeefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 -- 172.21.15.16:0/4258080494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 msgr2=0x564e96eeea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eeea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 -- 172.21.15.16:0/4258080494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e96f641d0 con 0x564e96edf760 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04a554640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x564e96efcfa0 tx=0x7fc04000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241ef43647eb655 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040003f60 con 0x564e96edf760 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc04001d3b0 con 0x564e96edf760 2024-09-12T00:58:07.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0340bc6d0 con 0x564e96edf760 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc04002e050 con 0x564e96edf760 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04eb91740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.671+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e96f654b0 con 0x564e96edf760 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0340bc6d0 con 0x564e96edf760 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc040011a40 con 0x564e96edf760 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 10 client.15924 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 20 client.15924 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96f67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 4 client.15924 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T00:58:07.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x564e96e4cd20 con 0x564e96efa310 2024-09-12T00:58:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.672+0000 7fc049d53640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96f67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.673+0000 7fc049d53640 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96f67cb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49734497b76cac3b server_cookie=8938f3a4cf25f359 in_seq=0 out_seq=0 2024-09-12T00:58:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.673+0000 7fc02dffb640 10 client.15924 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:07.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc02dffb640 1 -- 172.21.15.16:0/4258080494 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564e96e4cd20 con 0x564e96efa310 2024-09-12T00:58:07.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc02dffb640 10 client.15924 handle_command_reply: tid=0 2024-09-12T00:58:07.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 client.15924 shutdown 2024-09-12T00:58:07.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 msgr2=0x564e96eee540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x564e96efcfa0 tx=0x7fc04000bcc0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 shutdown_connections 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e96efa310 0x564e96f67cb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e96eea940 0x564e96eeefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e96ee0080 0x564e96eeea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc02dffb640 0 client.15924 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 --2- 172.21.15.16:0/4258080494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e96edf760 0x564e96eee540 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 >> 172.21.15.16:0/4258080494 conn(0x564e96edbcf0 msgr2=0x564e96ee5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.687+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 shutdown_connections 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04eb91740 1 -- 172.21.15.16:0/4258080494 wait complete. 2024-09-12T00:58:07.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04eb91740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 6, 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bb97285d": { 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 6, 2024-09-12T00:58:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 133.4, 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:07.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:07.691 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 msgr2=0x7fc0300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 0x7fc0300a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc03c005fd0 tx=0x7fc03c004a40 comp rx=0 tx=0).stop 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 msgr2=0x7fc0440fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc034009230 tx=0x7fc034004270 comp rx=0 tx=0).stop 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 shutdown_connections 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc0300a2cf0 0x7fc0300a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc044106a80 0x7fc0440fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044101420 0x7fc0440fd300 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 --2- 172.21.15.16:0/1202680520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc044100a80 0x7fc0440fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 >> 172.21.15.16:0/1202680520 conn(0x7fc0440fc7a0 msgr2=0x7fc04406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 shutdown_connections 2024-09-12T00:58:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:07.688+0000 7fc04c7b8640 1 -- 172.21.15.16:0/1202680520 wait complete. 2024-09-12T00:58:08.509 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb97285d', quiesce took 3.3 seconds. Will release after: 10.100000000000001 2024-09-12T00:58:09.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:09 smithi159 ceph-mon[35566]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-09-12T00:58:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:09 smithi016 ceph-mon[30972]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-09-12T00:58:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:09 smithi099 ceph-mon[35654]: pgmap v65: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-09-12T00:58:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:11 smithi159 ceph-mon[35566]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-09-12T00:58:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:11 smithi016 ceph-mon[30972]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-09-12T00:58:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:11 smithi099 ceph-mon[35654]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 44 KiB/s wr, 33 op/s 2024-09-12T00:58:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:13 smithi159 ceph-mon[35566]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:13 smithi016 ceph-mon[30972]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:13 smithi099 ceph-mon[35654]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:58:15.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:58:15.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:58:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:58:15.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 37 KiB/s wr, 27 op/s 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:58:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:58:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:17 smithi159 ceph-mon[35566]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-09-12T00:58:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:17 smithi016 ceph-mon[30972]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-09-12T00:58:17.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:17 smithi099 ceph-mon[35654]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 31 op/s 2024-09-12T00:58:18.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:58:18.881 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:19.288 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:19 smithi016 ceph-mon[30972]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T00:58:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:19 smithi159 ceph-mon[35566]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 -- 172.21.15.16:0/3045394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0102ef0 msgr2=0x7f22e010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 --2- 172.21.15.16:0/3045394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0102ef0 0x7f22e010da40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009ca0 tx=0x7f22d802a950 comp rx=0 tx=0).stop 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 -- 172.21.15.16:0/3045394334 shutdown_connections 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 --2- 172.21.15.16:0/3045394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0102ef0 0x7f22e010da40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 --2- 172.21.15.16:0/3045394334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 0x7f22e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 --2- 172.21.15.16:0/3045394334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0100e20 0x7f22e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 -- 172.21.15.16:0/3045394334 >> 172.21.15.16:0/3045394334 conn(0x7f22e00fc7d0 msgr2=0x7f22e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 -- 172.21.15.16:0/3045394334 shutdown_connections 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 -- 172.21.15.16:0/3045394334 wait complete. 2024-09-12T00:58:19.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.371+0000 7f22e577d640 1 Processor -- start 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 -- start start 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 0x7f22e019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0102ef0 0x7f22e019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0110f80 con 0x7f22e0100e20 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0110e00 con 0x7f22e0102ef0 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22e577d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22e0111100 con 0x7f22e0102570 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52506/0 (socket says 172.21.15.16:52506) 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 -- 172.21.15.16:0/1902100937 learned_addr learned my addr 172.21.15.16:0/1902100937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22de7fc640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 0x7f22e019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22df7fe640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0102ef0 0x7f22e019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 -- 172.21.15.16:0/1902100937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 msgr2=0x7f22e019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 0x7f22e019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 -- 172.21.15.16:0/1902100937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0102ef0 msgr2=0x7f22e019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0102ef0 0x7f22e019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.372+0000 7f22deffd640 1 -- 172.21.15.16:0/1902100937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e019f5e0 con 0x7f22e0100e20 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22deffd640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f22d000c920 tx=0x7f22d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0007c70 con 0x7f22e0100e20 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e019f8d0 con 0x7f22e0100e20 2024-09-12T00:58:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e01a71b0 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d0007e10 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.373+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d000f450 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.374+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc005290 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.375+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f22d0004f80 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.376+0000 7f22c7fff640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 0x7f22c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.376+0000 7f22de7fc640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 0x7f22c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.376+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f22d00cf580 con 0x7f22e0100e20 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.376+0000 7f22de7fc640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 0x7f22c80a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f22e0105640 tx=0x7f22d4005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.380+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22d0014030 con 0x7f22e0100e20 2024-09-12T00:58:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:19 smithi099 ceph-mon[35654]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-12T00:58:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.594+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22cc005850 con 0x7f22e0100e20 2024-09-12T00:58:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.595+0000 7f22c7fff640 1 -- 172.21.15.16:0/1902100937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f22d009c1d0 con 0x7f22e0100e20 2024-09-12T00:58:19.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:58:19.595 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:58:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 msgr2=0x7f22c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 0x7f22c80a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f22e0105640 tx=0x7f22d4005f90 comp rx=0 tx=0).stop 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 msgr2=0x7f22e019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f22d000c920 tx=0x7f22d000cde0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 shutdown_connections 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22c80a2cf0 0x7f22c80a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e0102ef0 0x7f22e019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e0102570 0x7f22e019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 --2- 172.21.15.16:0/1902100937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e0100e20 0x7f22e019a9d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 >> 172.21.15.16:0/1902100937 conn(0x7f22e00fc7d0 msgr2=0x7f22e01036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 shutdown_connections 2024-09-12T00:58:19.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:19.597+0000 7f22e577d640 1 -- 172.21.15.16:0/1902100937 wait complete. 2024-09-12T00:58:20.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:20 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1902100937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:20.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:20 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1902100937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:20 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1902100937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:20.622 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id bb97285d --release --await' 2024-09-12T00:58:20.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id bb97285d --release --await 2024-09-12T00:58:20.885 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:21.280 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:21 smithi016 ceph-mon[30972]: pgmap v71: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-09-12T00:58:21.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:21 smithi159 ceph-mon[35566]: pgmap v71: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-09-12T00:58:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 -- 172.21.15.16:0/1258689569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4106a80 msgr2=0x7f50c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 --2- 172.21.15.16:0/1258689569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4106a80 0x7f50c4106e60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009930 tx=0x7f50b802a420 comp rx=0 tx=0).stop 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 -- 172.21.15.16:0/1258689569 shutdown_connections 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 --2- 172.21.15.16:0/1258689569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 --2- 172.21.15.16:0/1258689569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4100a80 0x7f50c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 --2- 172.21.15.16:0/1258689569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4106a80 0x7f50c4106e60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 -- 172.21.15.16:0/1258689569 >> 172.21.15.16:0/1258689569 conn(0x7f50c40fc7a0 msgr2=0x7f50c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 -- 172.21.15.16:0/1258689569 shutdown_connections 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.371+0000 7f50caa82640 1 -- 172.21.15.16:0/1258689569 wait complete. 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 Processor -- start 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 -- start start 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 0x7f50c419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4111fd0 con 0x7f50c4100a80 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4111e50 con 0x7f50c4101420 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50caa82640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4112150 con 0x7f50c4106a80 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52524/0 (socket says 172.21.15.16:52524) 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.372+0000 7f50c881e640 1 -- 172.21.15.16:0/1705403901 learned_addr learned my addr 172.21.15.16:0/1705403901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c901f640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 0x7f50c419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 -- 172.21.15.16:0/1705403901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 msgr2=0x7f50c419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c3fff640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 0x7f50c419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 -- 172.21.15.16:0/1705403901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 msgr2=0x7f50c4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 -- 172.21.15.16:0/1705403901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c419d4c0 con 0x7f50c4100a80 2024-09-12T00:58:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c901f640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 0x7f50c419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c3fff640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c881e640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009a50 tx=0x7f50b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c1ffb640 1 -- 172.21.15.16:0/1705403901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b803f030 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c419d750 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c1ffb640 1 -- 172.21.15.16:0/1705403901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b8003ce0 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50c1ffb640 1 -- 172.21.15.16:0/1705403901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8035070 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.373+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41a5090 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c1ffb640 1 -- 172.21.15.16:0/1705403901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f50b8042030 con 0x7f50c4100a80 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 Processor -- start 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 -- start start 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582b8128620 con 0x5582b81ce240 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582b80cfe50 con 0x5582b81c3060 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50cce5b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5582b81245e0 con 0x5582b81c3980 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52526/0 (socket says 172.21.15.16:52526) 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c3fff640 1 -- 172.21.15.16:0/1131992024 learned_addr learned my addr 172.21.15.16:0/1131992024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c881e640 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.376+0000 7f50c901f640 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c1ffb640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 0x7f50b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 -- 172.21.15.16:0/1131992024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 msgr2=0x5582b81cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 -- 172.21.15.16:0/1131992024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 msgr2=0x5582b81c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 -- 172.21.15.16:0/1131992024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b80c0d40 con 0x5582b81ce240 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c901f640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 0x7f50b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c1ffb640 1 -- 172.21.15.16:0/1705403901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f50b80efa90 con 0x7f50c4100a80 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50c3fff640 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f50b000bb40 tx=0x7f50b000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2652c2dfeda59f5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50aeffd640 1 -- 172.21.15.16:0/1131992024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0017420 con 0x5582b81ce240 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50aeffd640 1 -- 172.21.15.16:0/1131992024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b0017aa0 con 0x5582b81ce240 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.377+0000 7f50aeffd640 1 -- 172.21.15.16:0/1131992024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000fc10 con 0x5582b81ce240 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- 172.21.15.16:0/1131992024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 msgr2=0x5582b81d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50c901f640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 0x7f50b40a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00a4c0 tx=0x7f50bc00c040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f50b000bb40 tx=0x7f50b000d340 comp rx=0 tx=0).stop 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- 172.21.15.16:0/1131992024 shutdown_connections 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81ce240 0x5582b81d0630 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 --2- 172.21.15.16:0/1131992024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- 172.21.15.16:0/1131992024 >> 172.21.15.16:0/1131992024 conn(0x5582b81bf5f0 msgr2=0x5582b81bf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- 172.21.15.16:0/1131992024 shutdown_connections 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- 172.21.15.16:0/1131992024 wait complete. 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 -- start start 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.378+0000 7f50cce5b740 1 Processor -- start 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3980 0x5582b81cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582b81d0e10 con 0x5582b81c3060 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582b80b0f10 con 0x5582b81c3980 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5582b81d0f90 con 0x5582b81ce240 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50cce5b740 10 client.0 _ll_register_callbacks cb 0x5582b80d4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52532/0 (socket says 172.21.15.16:52532) 2024-09-12T00:58:21.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 -- 172.21.15.16:0/3613458970 learned_addr learned my addr 172.21.15.16:0/3613458970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c901f640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c3fff640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3980 0x5582b81cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 -- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 msgr2=0x5582b81cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 -- 172.21.15.16:0/3613458970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3980 msgr2=0x5582b81cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3980 0x5582b81cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.379+0000 7f50c881e640 1 -- 172.21.15.16:0/3613458970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b7e37580 con 0x5582b81c3060 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50c881e640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50b8031550 tx=0x7f50b803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a0222ce9c9fd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8049070 con 0x5582b81c3060 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f50b80b8650 con 0x5582b81c3060 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b0017aa0 con 0x5582b81c3060 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50b80b8280 con 0x5582b81c3060 2024-09-12T00:58:21.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582b7f01960 con 0x5582b81c3060 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b0017aa0 con 0x5582b81c3060 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f50b803b460 con 0x5582b81c3060 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 10 client.15942 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 20 client.15942 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b81e0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 4 client.15942 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.380+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b8135b90 con 0x5582b81ddc30 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.381+0000 7f50c3fff640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b81e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.381+0000 7f50a3fff640 10 client.15942 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.381+0000 7f50c3fff640 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b81e0080 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e26cf1f334bffb server_cookie=69c308dcdfeb5b9d in_seq=0 out_seq=0 2024-09-12T00:58:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50a3fff640 1 -- 172.21.15.16:0/3613458970 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582b8135b90 con 0x5582b81ddc30 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50a3fff640 10 client.15942 handle_command_reply: tid=0 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 client.15942 shutdown 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 msgr2=0x5582b81cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f50b8031550 tx=0x7f50b803b800 comp rx=0 tx=0).stop 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 shutdown_connections 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b81e0080 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3980 0x5582b81cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50a3fff640 0 client.15942 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 --2- 172.21.15.16:0/3613458970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81cb080 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 >> 172.21.15.16:0/3613458970 conn(0x5582b81bf5f0 msgr2=0x5582b81c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 shutdown_connections 2024-09-12T00:58:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 1 -- 172.21.15.16:0/3613458970 wait complete. 2024-09-12T00:58:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.382+0000 7f50cce5b740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.399+0000 7f50cce5b740 1 Processor -- start 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.399+0000 7f50cce5b740 1 -- start start 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81d65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81ce240 0x5582b81d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41ad830 con 0x5582b81c3060 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41af370 con 0x5582b81ce240 2024-09-12T00:58:21.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50cce5b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41adf00 con 0x5582b81c3980 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52534/0 (socket says 172.21.15.16:52534) 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 -- 172.21.15.16:0/2621090395 learned_addr learned my addr 172.21.15.16:0/2621090395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c901f640 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81ce240 0x5582b81d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c3fff640 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81d65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 -- 172.21.15.16:0/2621090395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 msgr2=0x5582b81d65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81d65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 -- 172.21.15.16:0/2621090395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81ce240 msgr2=0x5582b81d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81ce240 0x5582b81d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 -- 172.21.15.16:0/2621090395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b81bf420 con 0x5582b81c3060 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50c881e640 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50b80bbf70 tx=0x7f50b80efe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c743978f326302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.400+0000 7f50ae7fc640 1 -- 172.21.15.16:0/2621090395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8049070 con 0x5582b81c3060 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50ae7fc640 1 -- 172.21.15.16:0/2621090395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b80f7040 con 0x5582b81c3060 2024-09-12T00:58:21.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50ae7fc640 1 -- 172.21.15.16:0/2621090395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b80bb940 con 0x5582b81c3060 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 -- 172.21.15.16:0/2621090395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 msgr2=0x5582b81be640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f50b80bbf70 tx=0x7f50b80efe40 comp rx=0 tx=0).stop 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 -- 172.21.15.16:0/2621090395 shutdown_connections 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81ce240 0x5582b81d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81c3980 0x5582b81d65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 --2- 172.21.15.16:0/2621090395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3060 0x5582b81be640 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 -- 172.21.15.16:0/2621090395 >> 172.21.15.16:0/2621090395 conn(0x5582b81bf5f0 msgr2=0x5582b81bdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 -- 172.21.15.16:0/2621090395 shutdown_connections 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.401+0000 7f50cce5b740 1 -- 172.21.15.16:0/2621090395 wait complete. 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 -- start start 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 Processor -- start 2024-09-12T00:58:21.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41aeca0 con 0x5582b81c3980 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41ae5d0 con 0x5582b81c3060 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50c41ab810 con 0x5582b81ce240 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50cce5b740 10 client.0 _ll_register_callbacks cb 0x7f50b8003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52538/0 (socket says 172.21.15.16:52538) 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.402+0000 7f50c3fff640 1 -- 172.21.15.16:0/320975671 learned_addr learned my addr 172.21.15.16:0/320975671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c901f640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c881e640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 -- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 msgr2=0x5582b81cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 -- 172.21.15.16:0/320975671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 msgr2=0x5582b81ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 -- 172.21.15.16:0/320975671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b81c0780 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c881e640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50c3fff640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50b0019040 tx=0x7f50b000f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50bba0fb59d740f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b002f070 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b80f7040 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f50b000f630 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50b003b050 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50cce5b740 10 client.15948 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.403+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582b81c0280 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b80f7040 con 0x5582b81c3980 2024-09-12T00:58:21.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f50b00275e0 con 0x5582b81c3980 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 10 client.15948 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 10 client.15948 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 20 client.15948 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b824ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 4 client.15948 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb97285d", "release": true, "await": true}] 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb97285d", "release": true, "await": true}) -- 0x5582b81300a0 con 0x5582b81ddc30 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.404+0000 7f50c881e640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b824ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.405+0000 7f50c881e640 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b824ffd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf642ccf95e635a3 server_cookie=e7f739bc9005e610 in_seq=0 out_seq=0 2024-09-12T00:58:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.405+0000 7f50a3fff640 10 client.15948 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50a3fff640 1 -- 172.21.15.16:0/320975671 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5582b81300a0 con 0x5582b81ddc30 2024-09-12T00:58:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50a3fff640 10 client.15948 handle_command_reply: tid=0 2024-09-12T00:58:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 client.15948 shutdown 2024-09-12T00:58:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 msgr2=0x5582b81ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f50b0019040 tx=0x7f50b000f420 comp rx=0 tx=0).stop 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 shutdown_connections 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5582b81ddc30 0x5582b824ffd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582b81ce240 0x5582b81cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582b81c3980 0x5582b81ceba0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 --2- 172.21.15.16:0/320975671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582b81c3060 0x5582b81ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 >> 172.21.15.16:0/320975671 conn(0x5582b81bf5f0 msgr2=0x5582b81be750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.408+0000 7f50a3fff640 0 client.15948 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 shutdown_connections 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50cce5b740 1 -- 172.21.15.16:0/320975671 wait complete. 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50cce5b740 20 client.15948 trim_cache size 0 max 16384 2024-09-12T00:58:21.410 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 8, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "bb97285d": { 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 8, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 133.4, 2024-09-12T00:58:21.411 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:21.412 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 msgr2=0x7f50b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 0x7f50b40a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00a4c0 tx=0x7f50bc00c040 comp rx=0 tx=0).stop 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 msgr2=0x7f50c4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009a50 tx=0x7f50b8004270 comp rx=0 tx=0).stop 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 shutdown_connections 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f50b40a2cf0 0x7f50b40a51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50c4106a80 0x7f50c419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50c4101420 0x7f50c4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 --2- 172.21.15.16:0/1705403901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4100a80 0x7f50c4198510 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 >> 172.21.15.16:0/1705403901 conn(0x7f50c40fc7a0 msgr2=0x7f50c40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.409+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 shutdown_connections 2024-09-12T00:58:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:21.410+0000 7f50caa82640 1 -- 172.21.15.16:0/1705403901 wait complete. 2024-09-12T00:58:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:21 smithi099 ceph-mon[35654]: pgmap v71: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 14 op/s 2024-09-12T00:58:22.130 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb97285d', total seconds elapsed: 16.9 2024-09-12T00:58:22.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:58:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:23.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 -- 172.21.15.16:0/990550721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404102790 msgr2=0x7f44041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 --2- 172.21.15.16:0/990550721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404102790 0x7f44041067e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009900 tx=0x7f43f802a410 comp rx=0 tx=0).stop 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 -- 172.21.15.16:0/990550721 shutdown_connections 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 --2- 172.21.15.16:0/990550721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404106f10 0x7f440410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 --2- 172.21.15.16:0/990550721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404102790 0x7f44041067e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 --2- 172.21.15.16:0/990550721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404101de0 0x7f44041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 -- 172.21.15.16:0/990550721 >> 172.21.15.16:0/990550721 conn(0x7f44040fd8c0 msgr2=0x7f44040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 -- 172.21.15.16:0/990550721 shutdown_connections 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 -- 172.21.15.16:0/990550721 wait complete. 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.082+0000 7f4408bb3640 1 Processor -- start 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 -- start start 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 0x7f44041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 0x7f440419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4404111fb0 con 0x7f4404101de0 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4404111e30 con 0x7f4404102790 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4408bb3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4404112130 con 0x7f4404106f10 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52544/0 (socket says 172.21.15.16:52544) 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 -- 172.21.15.16:0/3059974154 learned_addr learned my addr 172.21.15.16:0/3059974154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 0x7f440419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4401d9b640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 0x7f44041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 -- 172.21.15.16:0/3059974154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 msgr2=0x7f440419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 0x7f440419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 -- 172.21.15.16:0/3059974154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 msgr2=0x7f44041989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 0x7f44041989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 -- 172.21.15.16:0/3059974154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440419d460 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4401d9b640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 0x7f44041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 0x7f440419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.083+0000 7f440259c640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d770 tx=0x7f43f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.084+0000 7f43f37fe640 1 -- 172.21.15.16:0/3059974154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4017070 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.084+0000 7f43f37fe640 1 -- 172.21.15.16:0/3059974154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f4005500 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.084+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440419d750 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.084+0000 7f43f37fe640 1 -- 172.21.15.16:0/3059974154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004e00 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.084+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44041a5030 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.086+0000 7f43f37fe640 1 -- 172.21.15.16:0/3059974154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f43f4004fa0 con 0x7f4404101de0 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.086+0000 7f440af8c740 1 Processor -- start 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 -- start start 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d696dc0 0x556b6d6a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b6d5fc2e0 con 0x556b6d6964a0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b6d5a3ce0 con 0x556b6d696dc0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440af8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b6d5f82a0 con 0x556b6d6a1680 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52556/0 (socket says 172.21.15.16:52556) 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f4402d9d640 1 -- 172.21.15.16:0/3973221277 learned_addr learned my addr 172.21.15.16:0/3973221277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f43f37fe640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 0x7f43ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f440259c640 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d696dc0 0x556b6d6a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f4401d9b640 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.087+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 0x7f43ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f43f37fe640 1 -- 172.21.15.16:0/3059974154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f43f40cefb0 con 0x7f4404101de0 2024-09-12T00:58:23.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 -- 172.21.15.16:0/3973221277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 msgr2=0x556b6d6a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 -- 172.21.15.16:0/3973221277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d696dc0 msgr2=0x556b6d6a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d696dc0 0x556b6d6a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 -- 172.21.15.16:0/3973221277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6d588440 con 0x556b6d6964a0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00cbf0 tx=0x7f43fc007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce396f330103d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f43f0ff9640 1 -- 172.21.15.16:0/3973221277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc015070 con 0x556b6d6964a0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f43f0ff9640 1 -- 172.21.15.16:0/3973221277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43fc016040 con 0x556b6d6964a0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 0x7f43ec0a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43fc01f000 tx=0x7f43fc01b000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.088+0000 7f43f0ff9640 1 -- 172.21.15.16:0/3973221277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc01e020 con 0x556b6d6964a0 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- 172.21.15.16:0/3973221277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 msgr2=0x556b6d696880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00cbf0 tx=0x7f43fc007680 comp rx=0 tx=0).stop 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- 172.21.15.16:0/3973221277 shutdown_connections 2024-09-12T00:58:23.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d696dc0 0x556b6d6a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 --2- 172.21.15.16:0/3973221277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6964a0 0x556b6d696880 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- 172.21.15.16:0/3973221277 >> 172.21.15.16:0/3973221277 conn(0x556b6d692a30 msgr2=0x556b6d692e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- 172.21.15.16:0/3973221277 shutdown_connections 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- 172.21.15.16:0/3973221277 wait complete. 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 -- start start 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.089+0000 7f440af8c740 1 Processor -- start 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d69e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b6d6a4250 con 0x556b6d696dc0 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b6d5b7050 con 0x556b6d6964a0 2024-09-12T00:58:23.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b6d6a43d0 con 0x556b6d6a1680 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440af8c740 10 client.0 _ll_register_callbacks cb 0x556b6d5f7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52560/0 (socket says 172.21.15.16:52560) 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 -- 172.21.15.16:0/150455306 learned_addr learned my addr 172.21.15.16:0/150455306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440259c640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4402d9d640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d69e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 -- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 msgr2=0x556b6d69e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d69e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 -- 172.21.15.16:0/150455306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 msgr2=0x556b6d6a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 -- 172.21.15.16:0/150455306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6d36d3c0 con 0x556b6d696dc0 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f440259c640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:23.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4402d9d640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d69e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f4401d9b640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009230 tx=0x7f43f80317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6c2ba047981bad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f43e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.090+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8040070 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f43f8004250 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43fc016040 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43f8035040 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b6d6afc70 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43fc016040 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43f8004b00 con 0x556b6d696dc0 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:23.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 20 client.15960 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d6b1080 0x556b6d6b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 4 client.15960 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.091+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6d609850 con 0x556b6d6b1080 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.092+0000 7f440259c640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d6b1080 0x556b6d6b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.092+0000 7f440259c640 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d6b1080 0x556b6d6b34e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bde9a44574c0051 server_cookie=f452956f625b0d7b in_seq=0 out_seq=0 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.092+0000 7f43e9ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f43e9ffb640 1 -- 172.21.15.16:0/150455306 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b6d609850 con 0x556b6d6b1080 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f43e9ffb640 10 client.15960 handle_command_reply: tid=0 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 client.15960 shutdown 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 msgr2=0x556b6d69e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009230 tx=0x7f43f80317e0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 shutdown_connections 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d6b1080 0x556b6d6b34e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d69e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d69e2a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f43e9ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 --2- 172.21.15.16:0/150455306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 >> 172.21.15.16:0/150455306 conn(0x556b6d692a30 msgr2=0x556b6d691b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 shutdown_connections 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 1 -- 172.21.15.16:0/150455306 wait complete. 2024-09-12T00:58:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.093+0000 7f440af8c740 20 client.15960 trim_cache size 0 max 16384 2024-09-12T00:58:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 Processor -- start 2024-09-12T00:58:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 -- start start 2024-09-12T00:58:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 0x556b6d6b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ad810 con 0x556b6d6a1680 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44041af350 con 0x556b6d6964a0 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440af8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44041adee0 con 0x556b6d696dc0 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f4402d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52562/0 (socket says 172.21.15.16:52562) 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f4402d9d640 1 -- 172.21.15.16:0/3028014435 learned_addr learned my addr 172.21.15.16:0/3028014435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f4401d9b640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 0x556b6d6b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.113+0000 7f440259c640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 -- 172.21.15.16:0/3028014435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 msgr2=0x556b6d6b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 0x556b6d6b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 -- 172.21.15.16:0/3028014435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 msgr2=0x556b6d6ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 -- 172.21.15.16:0/3028014435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6d69eb70 con 0x556b6d6a1680 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f440259c640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4401d9b640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 0x556b6d6b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f4402d9d640 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f43fc007b80 tx=0x7f43fc00c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c14b01e341be95e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f43ebfff640 1 -- 172.21.15.16:0/3028014435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc030070 con 0x556b6d6a1680 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f43ebfff640 1 -- 172.21.15.16:0/3028014435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43fc00a600 con 0x556b6d6a1680 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.114+0000 7f43ebfff640 1 -- 172.21.15.16:0/3028014435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc015040 con 0x556b6d6a1680 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- 172.21.15.16:0/3028014435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 msgr2=0x556b6d6b92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f43fc007b80 tx=0x7f43fc00c360 comp rx=0 tx=0).stop 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- 172.21.15.16:0/3028014435 shutdown_connections 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d6a1680 0x556b6d6b92f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d696dc0 0x556b6d6b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 --2- 172.21.15.16:0/3028014435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- 172.21.15.16:0/3028014435 >> 172.21.15.16:0/3028014435 conn(0x556b6d692a30 msgr2=0x556b6d69d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- 172.21.15.16:0/3028014435 shutdown_connections 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- 172.21.15.16:0/3028014435 wait complete. 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 -- start start 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.115+0000 7f440af8c740 1 Processor -- start 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44041aec80 con 0x556b6d696dc0 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ae5b0 con 0x556b6d6964a0 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44041ab7f0 con 0x556b6d6a1680 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440af8c740 10 client.0 _ll_register_callbacks cb 0x7f43f8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52568/0 (socket says 172.21.15.16:52568) 2024-09-12T00:58:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 -- 172.21.15.16:0/409980378 learned_addr learned my addr 172.21.15.16:0/409980378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f440259c640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4402d9d640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 -- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 msgr2=0x556b6d6a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 -- 172.21.15.16:0/409980378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 msgr2=0x556b6d6a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 -- 172.21.15.16:0/409980378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6d696bf0 con 0x556b6d696dc0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4402d9d640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f4401d9b640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009900 tx=0x7f43f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f170513cc40d7c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f43e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.116+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8040050 con 0x556b6d696dc0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f43f8004450 con 0x556b6d696dc0 2024-09-12T00:58:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43fc00a600 con 0x556b6d696dc0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43f8031a60 con 0x556b6d696dc0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 10 client.15966 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b6d71ba80 con 0x556b6d696dc0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43fc00a600 con 0x556b6d696dc0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43f8005000 con 0x556b6d696dc0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 10 client.15966 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 20 client.15966 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d71c1d0 0x556b6d71e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 4 client.15966 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.117+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b6d603d60 con 0x556b6d71c1d0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440259c640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d71c1d0 0x556b6d71e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440259c640 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d71c1d0 0x556b6d71e5f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19a077ba85d1f90 server_cookie=45aad4428e00b0c8 in_seq=0 out_seq=0 2024-09-12T00:58:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f43e9ffb640 10 client.15966 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f43e9ffb640 1 -- 172.21.15.16:0/409980378 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556b6d603d60 con 0x556b6d71c1d0 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f43e9ffb640 10 client.15966 handle_command_reply: tid=0 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 client.15966 shutdown 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 msgr2=0x556b6d6a6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009900 tx=0x7f43f802a410 comp rx=0 tx=0).stop 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 shutdown_connections 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b6d71c1d0 0x556b6d71e5f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f43e9ffb640 0 client.15966 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b6d6a1680 0x556b6d6a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b6d696dc0 0x556b6d6a6ff0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 --2- 172.21.15.16:0/409980378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b6d6964a0 0x556b6d6a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.118+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 >> 172.21.15.16:0/409980378 conn(0x556b6d692a30 msgr2=0x556b6d69fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 shutdown_connections 2024-09-12T00:58:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f440af8c740 1 -- 172.21.15.16:0/409980378 wait complete. 2024-09-12T00:58:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f440af8c740 20 client.15966 trim_cache size 0 max 16384 2024-09-12T00:58:23.127 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:23.127 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T00:58:23.127 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T00:58:23.127 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 msgr2=0x7f43ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 0x7f43ec0a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43fc01f000 tx=0x7f43fc01b000 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 msgr2=0x7f44041984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f43f400d770 tx=0x7f43f400dc30 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 shutdown_connections 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43ec0a2cf0 0x7f43ec0a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4404106f10 0x7f440419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4404102790 0x7f44041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 --2- 172.21.15.16:0/3059974154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4404101de0 0x7f44041984b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 >> 172.21.15.16:0/3059974154 conn(0x7f44040fd8c0 msgr2=0x7f44040feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 shutdown_connections 2024-09-12T00:58:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:23.119+0000 7f4408bb3640 1 -- 172.21.15.16:0/3059974154 wait complete. 2024-09-12T00:58:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:23 smithi159 ceph-mon[35566]: pgmap v72: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:23 smithi016 ceph-mon[30972]: pgmap v72: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:23 smithi099 ceph-mon[35654]: pgmap v72: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:23.894 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:58:23.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:58:24.156 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 -- 172.21.15.16:0/326096221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60101de0 msgr2=0x7f7d601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 --2- 172.21.15.16:0/326096221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60101de0 0x7f7d601021c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009900 tx=0x7f7d5c02a410 comp rx=0 tx=0).stop 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 -- 172.21.15.16:0/326096221 shutdown_connections 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 --2- 172.21.15.16:0/326096221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 0x7f7d6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 --2- 172.21.15.16:0/326096221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60102790 0x7f7d601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 --2- 172.21.15.16:0/326096221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60101de0 0x7f7d601021c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 -- 172.21.15.16:0/326096221 >> 172.21.15.16:0/326096221 conn(0x7f7d600fd8c0 msgr2=0x7f7d600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 -- 172.21.15.16:0/326096221 shutdown_connections 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.646+0000 7f7d67576640 1 -- 172.21.15.16:0/326096221 wait complete. 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 Processor -- start 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 -- start start 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60101de0 0x7f7d60198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 0x7f7d6019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60111f70 con 0x7f7d60102790 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60111df0 con 0x7f7d60101de0 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d67576640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601120f0 con 0x7f7d60106f10 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d64b11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d64b11640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52598/0 (socket says 172.21.15.16:52598) 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d64b11640 1 -- 172.21.15.16:0/719063140 learned_addr learned my addr 172.21.15.16:0/719063140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.647+0000 7f7d65312640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60101de0 0x7f7d60198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d65b13640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 0x7f7d6019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 -- 172.21.15.16:0/719063140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 msgr2=0x7f7d6019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 0x7f7d6019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 -- 172.21.15.16:0/719063140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60101de0 msgr2=0x7f7d60198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60101de0 0x7f7d60198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 -- 172.21.15.16:0/719063140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6019d340 con 0x7f7d60102790 2024-09-12T00:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d64b11640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c920 tx=0x7f7d5000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50007c70 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6019d630 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d50007e10 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50018890 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.648+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601a4f10 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.649+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d601021c0 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.650+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d50018a70 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.651+0000 7f7d567fc640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 0x7f7d4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.651+0000 7f7d65312640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 0x7f7d4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.651+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7d500cf4c0 con 0x7f7d60102790 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.651+0000 7f7d65312640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 0x7f7d4c0a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009230 tx=0x7f7d5c0058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:24.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.655+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d50014030 con 0x7f7d60102790 2024-09-12T00:58:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.866+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d60109b60 con 0x7f7d60102790 2024-09-12T00:58:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.867+0000 7f7d567fc640 1 -- 172.21.15.16:0/719063140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f7d5009c110 con 0x7f7d60102790 2024-09-12T00:58:24.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:58:24.866 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:58:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:58:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 msgr2=0x7f7d4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 0x7f7d4c0a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009230 tx=0x7f7d5c0058d0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 msgr2=0x7f7d601988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c920 tx=0x7f7d5000cde0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 shutdown_connections 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d4c0a2cf0 0x7f7d4c0a51b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d60106f10 0x7f7d6019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d60102790 0x7f7d601988d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 --2- 172.21.15.16:0/719063140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d60101de0 0x7f7d60198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 >> 172.21.15.16:0/719063140 conn(0x7f7d600fd8c0 msgr2=0x7f7d600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 shutdown_connections 2024-09-12T00:58:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:24.869+0000 7f7d67576640 1 -- 172.21.15.16:0/719063140 wait complete. 2024-09-12T00:58:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:25 smithi016 ceph-mon[30972]: pgmap v73: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/719063140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:25 smithi159 ceph-mon[35566]: pgmap v73: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/719063140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:25 smithi099 ceph-mon[35654]: pgmap v73: 97 pgs: 97 active+clean; 1.9 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-12T00:58:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/719063140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:25.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T00:58:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:26.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 -- 172.21.15.16:0/3312189069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 msgr2=0x7f27e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/3312189069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81021c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009900 tx=0x7f27dc02a410 comp rx=0 tx=0).stop 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 -- 172.21.15.16:0/3312189069 shutdown_connections 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/3312189069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8106f10 0x7f27e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/3312189069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8102790 0x7f27e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/3312189069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81021c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 -- 172.21.15.16:0/3312189069 >> 172.21.15.16:0/3312189069 conn(0x7f27e80fd8c0 msgr2=0x7f27e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 -- 172.21.15.16:0/3312189069 shutdown_connections 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.392+0000 7f27ef0f7640 1 -- 172.21.15.16:0/3312189069 wait complete. 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 Processor -- start 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 -- start start 2024-09-12T00:58:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8102790 0x7f27e8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8106f10 0x7f27e819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8111fc0 con 0x7f27e8101de0 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8111e40 con 0x7f27e8102790 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ef0f7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e8112140 con 0x7f27e8106f10 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52616/0 (socket says 172.21.15.16:52616) 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ece93640 1 -- 172.21.15.16:0/2975974639 learned_addr learned my addr 172.21.15.16:0/2975974639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27ed694640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8106f10 0x7f27e819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.393+0000 7f27e7fff640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8102790 0x7f27e8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 -- 172.21.15.16:0/2975974639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8106f10 msgr2=0x7f27e819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8106f10 0x7f27e819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 -- 172.21.15.16:0/2975974639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8102790 msgr2=0x7f27e8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8102790 0x7f27e8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 -- 172.21.15.16:0/2975974639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e819d390 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ece93640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27e5ffb640 1 -- 172.21.15.16:0/2975974639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc03f070 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e819d620 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27e5ffb640 1 -- 172.21.15.16:0/2975974639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc031c90 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27e5ffb640 1 -- 172.21.15.16:0/2975974639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc035030 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.394+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e81a4f60 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.396+0000 7f27e5ffb640 1 -- 172.21.15.16:0/2975974639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f27dc042070 con 0x7f27e8101de0 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 Processor -- start 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 -- start start 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece05b40 0x55a8ece04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 0x55a8ece11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8ecd6a2f0 con 0x55a8ece045d0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8ecd11cf0 con 0x55a8ece0f370 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27f14d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a8ecd662b0 con 0x55a8ece05b40 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27e5ffb640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 0x7f27d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52626/0 (socket says 172.21.15.16:52626) 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27ece93640 1 -- 172.21.15.16:0/643174844 learned_addr learned my addr 172.21.15.16:0/643174844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27ed694640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece05b40 0x55a8ece04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27ed694640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 0x7f27d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.397+0000 7f27e7fff640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 0x55a8ece11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27e5ffb640 1 -- 172.21.15.16:0/2975974639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f27dc0efc70 con 0x7f27e8101de0 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 -- 172.21.15.16:0/643174844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece05b40 msgr2=0x55a8ece04090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece05b40 0x55a8ece04090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 -- 172.21.15.16:0/643174844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 msgr2=0x55a8ece11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 0x55a8ece11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 -- 172.21.15.16:0/643174844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8eccf63a0 con 0x55a8ece045d0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27e7fff640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 0x55a8ece11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ece93640 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0bc430 tx=0x7f27dc04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69f9552f38d6b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27ed694640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 0x7f27d80a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007fd0 tx=0x7f27e0007520 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27d2ffd640 1 -- 172.21.15.16:0/643174844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc049070 con 0x55a8ece045d0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27d2ffd640 1 -- 172.21.15.16:0/643174844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc031e40 con 0x55a8ece045d0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.398+0000 7f27d2ffd640 1 -- 172.21.15.16:0/643174844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0b88e0 con 0x55a8ece045d0 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 -- 172.21.15.16:0/643174844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 msgr2=0x55a8ece04a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0bc430 tx=0x7f27dc04b000 comp rx=0 tx=0).stop 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 -- 172.21.15.16:0/643174844 shutdown_connections 2024-09-12T00:58:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece0f370 0x55a8ece11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece04a10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 --2- 172.21.15.16:0/643174844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece05b40 0x55a8ece04090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 -- 172.21.15.16:0/643174844 >> 172.21.15.16:0/643174844 conn(0x55a8ecdcd310 msgr2=0x55a8ece00c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 -- 172.21.15.16:0/643174844 shutdown_connections 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.399+0000 7f27f14d0740 1 -- 172.21.15.16:0/643174844 wait complete. 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 -- start start 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 Processor -- start 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece0c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a8ece11e30 con 0x55a8ece045d0 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8ecd25060 con 0x55a8ece05b40 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a8ece11fb0 con 0x55a8ece0f370 2024-09-12T00:58:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27f14d0740 10 client.0 _ll_register_callbacks cb 0x55a8eca788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27ece93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52642/0 (socket says 172.21.15.16:52642) 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27ece93640 1 -- 172.21.15.16:0/1443768323 learned_addr learned my addr 172.21.15.16:0/1443768323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27ed694640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.400+0000 7f27e7fff640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece0c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 -- 172.21.15.16:0/1443768323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 msgr2=0x55a8ece0c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece0c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 -- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 msgr2=0x55a8ece0c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece0c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 -- 172.21.15.16:0/1443768323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ecadb3c0 con 0x55a8ece045d0 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27e7fff640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece0c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ed694640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece0c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27ece93640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ed4a0 tx=0x7f27dc03b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddf723dba0f8fe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0d5060 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f27dc0b88e0 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8ecd77860 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27dc049070 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 10 client.15984 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8ece11e30 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a8ecd77860 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc0c0170 con 0x55a8ece045d0 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 10 client.15984 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 20 client.15984 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece20e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 4 client.15984 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.401+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8ecd71d70 con 0x55a8ece1ea80 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.402+0000 7f27e7fff640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece20e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.402+0000 7f27e7fff640 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece20e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5255cf8e36050419 server_cookie=9669ab948d5afc41 in_seq=0 out_seq=0 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.402+0000 7f27c7fff640 10 client.15984 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27c7fff640 1 -- 172.21.15.16:0/1443768323 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8ecd71d70 con 0x55a8ece1ea80 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27c7fff640 10 client.15984 handle_command_reply: tid=0 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 client.15984 shutdown 2024-09-12T00:58:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 msgr2=0x55a8ece0ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ed4a0 tx=0x7f27dc03b990 comp rx=0 tx=0).stop 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 shutdown_connections 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece20e90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece0c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27c7fff640 0 client.15984 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece0c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 --2- 172.21.15.16:0/1443768323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece045d0 0x55a8ece0ed90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 >> 172.21.15.16:0/1443768323 conn(0x55a8ecdcd310 msgr2=0x55a8ece06990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 shutdown_connections 2024-09-12T00:58:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 1 -- 172.21.15.16:0/1443768323 wait complete. 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.403+0000 7f27f14d0740 20 client.15984 trim_cache size 0 max 16384 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.416+0000 7f27f14d0740 1 Processor -- start 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.416+0000 7f27f14d0740 1 -- start start 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.416+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece045d0 0x55a8ece1b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27f14d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81ad7a0 con 0x55a8ece05b40 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27f14d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81af2e0 con 0x55a8ece045d0 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27f14d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81ade70 con 0x55a8ece0f370 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52646/0 (socket says 172.21.15.16:52646) 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 -- 172.21.15.16:0/685787276 learned_addr learned my addr 172.21.15.16:0/685787276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27ece93640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece045d0 0x55a8ece1b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 -- 172.21.15.16:0/685787276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 msgr2=0x55a8ece07660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27ed694640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece07660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 -- 172.21.15.16:0/685787276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece045d0 msgr2=0x55a8ece1b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece045d0 0x55a8ece1b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 -- 172.21.15.16:0/685787276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ece0f100 con 0x55a8ece05b40 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27ed694640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece07660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27e7fff640 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005d90 tx=0x7f27d4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5531f6a531a95a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27d27fc640 1 -- 172.21.15.16:0/685787276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4005020 con 0x55a8ece05b40 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27d27fc640 1 -- 172.21.15.16:0/685787276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d40047a0 con 0x55a8ece05b40 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.417+0000 7f27d27fc640 1 -- 172.21.15.16:0/685787276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d401d890 con 0x55a8ece05b40 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- 172.21.15.16:0/685787276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 msgr2=0x55a8ece07120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f27d4005d90 tx=0x7f27d4010800 comp rx=0 tx=0).stop 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- 172.21.15.16:0/685787276 shutdown_connections 2024-09-12T00:58:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece0f370 0x55a8ece07660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece05b40 0x55a8ece07120 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 --2- 172.21.15.16:0/685787276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece045d0 0x55a8ece1b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- 172.21.15.16:0/685787276 >> 172.21.15.16:0/685787276 conn(0x55a8ecdcd310 msgr2=0x55a8ece0b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- 172.21.15.16:0/685787276 shutdown_connections 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- 172.21.15.16:0/685787276 wait complete. 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 -- start start 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.418+0000 7f27f14d0740 1 Processor -- start 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece045d0 0x55a8ece179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81aec10 con 0x55a8ece0f370 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81ae540 con 0x55a8ece05b40 2024-09-12T00:58:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e81ab780 con 0x55a8ece045d0 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27f14d0740 10 client.0 _ll_register_callbacks cb 0x7f27d400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27ed694640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27ed694640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52650/0 (socket says 172.21.15.16:52650) 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27ed694640 1 -- 172.21.15.16:0/2277964041 learned_addr learned my addr 172.21.15.16:0/2277964041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27ece93640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece045d0 0x55a8ece179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 -- 172.21.15.16:0/2277964041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece045d0 msgr2=0x55a8ece179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece045d0 0x55a8ece179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 -- 172.21.15.16:0/2277964041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 msgr2=0x55a8ece092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27e7fff640 1 -- 172.21.15.16:0/2277964041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ece0c9c0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.419+0000 7f27ed694640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27e7fff640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece08db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a8ece21330 tx=0x7f27d400a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abea216e80cbf2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4005020 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d40047a0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f27d40051c0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27d40053e0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 10 client.25493 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8ece894d0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d40047a0 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27d4030020 con 0x55a8ece05b40 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 10 client.25493 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 10 client.25493 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:58:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 20 client.25493 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 4 client.25493 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.420+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8eccee170 con 0x55a8ece1ea80 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27ece93640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27ece93640 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece8bb80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ecfc838a3851691d server_cookie=5a2baab868fd25ec in_seq=0 out_seq=0 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27c7fff640 10 client.25493 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27c7fff640 1 -- 172.21.15.16:0/2277964041 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8eccee170 con 0x55a8ece1ea80 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27c7fff640 10 client.25493 handle_command_reply: tid=0 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 client.25493 shutdown 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 msgr2=0x55a8ece08db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece08db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a8ece21330 tx=0x7f27d400a6c0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 shutdown_connections 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a8ece1ea80 0x55a8ece8bb80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8ece0f370 0x55a8ece092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8ece05b40 0x55a8ece08db0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27c7fff640 0 client.25493 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 --2- 172.21.15.16:0/2277964041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8ece045d0 0x55a8ece179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.421+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 >> 172.21.15.16:0/2277964041 conn(0x55a8ecdcd310 msgr2=0x55a8ece0d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 shutdown_connections 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27f14d0740 1 -- 172.21.15.16:0/2277964041 wait complete. 2024-09-12T00:58:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27f14d0740 20 client.25493 trim_cache size 0 max 16384 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 msgr2=0x7f27d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 0x7f27d80a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007fd0 tx=0x7f27e0007520 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 msgr2=0x7f27e81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc003ba0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 shutdown_connections 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d80a2cf0 0x7f27d80a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e8106f10 0x7f27e819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e8102790 0x7f27e8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 --2- 172.21.15.16:0/2975974639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e8101de0 0x7f27e81983e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 >> 172.21.15.16:0/2975974639 conn(0x7f27e80fd8c0 msgr2=0x7f27e80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 shutdown_connections 2024-09-12T00:58:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:26.422+0000 7f27ef0f7640 1 -- 172.21.15.16:0/2975974639 wait complete. 2024-09-12T00:58:27.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T00:58:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:27 smithi159 ceph-mon[35566]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-12T00:58:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:27 smithi016 ceph-mon[30972]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-12T00:58:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:27 smithi099 ceph-mon[35654]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-12T00:58:28.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 -- 172.21.15.16:0/3090448123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8106a80 msgr2=0x7fa8c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 --2- 172.21.15.16:0/3090448123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8106a80 0x7fa8c8106e60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009930 tx=0x7fa8b802a420 comp rx=0 tx=0).stop 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 -- 172.21.15.16:0/3090448123 shutdown_connections 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 --2- 172.21.15.16:0/3090448123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 0x7fa8c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 --2- 172.21.15.16:0/3090448123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8100a80 0x7fa8c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 --2- 172.21.15.16:0/3090448123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8106a80 0x7fa8c8106e60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 -- 172.21.15.16:0/3090448123 >> 172.21.15.16:0/3090448123 conn(0x7fa8c80fc7a0 msgr2=0x7fa8c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 -- 172.21.15.16:0/3090448123 shutdown_connections 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.959+0000 7fa8cd007640 1 -- 172.21.15.16:0/3090448123 wait complete. 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 Processor -- start 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 -- start start 2024-09-12T00:58:28.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 0x7fa8c8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8106a80 0x7fa8c819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8110ea0 con 0x7fa8c8100a80 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8110d20 con 0x7fa8c8106a80 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8cd007640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8111020 con 0x7fa8c8101420 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52670/0 (socket says 172.21.15.16:52670) 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/2566758427 learned_addr learned my addr 172.21.15.16:0/2566758427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8bffff640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 0x7fa8c8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c759e640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8106a80 0x7fa8c819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/2566758427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 msgr2=0x7fa8c8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 0x7fa8c8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/2566758427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8106a80 msgr2=0x7fa8c819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8106a80 0x7fa8c819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.960+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/2566758427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c819d3e0 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009a50 tx=0x7fa8b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8c4d99640 1 -- 172.21.15.16:0/2566758427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b803f030 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c819d670 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8c4d99640 1 -- 172.21.15.16:0/2566758427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8003ce0 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8c4d99640 1 -- 172.21.15.16:0/2566758427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035070 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.961+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81a4fb0 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.963+0000 7fa8c4d99640 1 -- 172.21.15.16:0/2566758427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa8b8042030 con 0x7fa8c8100a80 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.963+0000 7fa8cf3e0740 1 Processor -- start 2024-09-12T00:58:28.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 -- start start 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7cb40 0x562ed0e7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7b5d0 0x562ed0e7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ed0de12f0 con 0x562ed0e86370 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ed0d88cf0 con 0x562ed0e7cb40 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ed0ddd2b0 con 0x562ed0e7b5d0 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8c4d99640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 0x7fa8b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52678/0 (socket says 172.21.15.16:52678) 2024-09-12T00:58:28.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8bffff640 1 -- 172.21.15.16:0/3120345829 learned_addr learned my addr 172.21.15.16:0/3120345829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8c759e640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 0x7fa8b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7b5d0 0x562ed0e7ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8c759e640 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7cb40 0x562ed0e7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8c4d99640 1 -- 172.21.15.16:0/2566758427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa8b80efac0 con 0x7fa8c8100a80 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8bffff640 1 -- 172.21.15.16:0/3120345829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7b5d0 msgr2=0x562ed0e7ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.964+0000 7fa8bffff640 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7b5d0 0x562ed0e7ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bffff640 1 -- 172.21.15.16:0/3120345829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7cb40 msgr2=0x562ed0e7b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bffff640 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7cb40 0x562ed0e7b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bffff640 1 -- 172.21.15.16:0/3120345829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed0d6d3a0 con 0x562ed0e86370 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8c759e640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 0x7fa8b40a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0014590 tx=0x7fa8c0006090 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bffff640 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000cc40 tx=0x7fa8b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eaf18b2d84f282 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bd7fa640 1 -- 172.21.15.16:0/3120345829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0014070 con 0x562ed0e86370 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bd7fa640 1 -- 172.21.15.16:0/3120345829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b00047a0 con 0x562ed0e86370 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.965+0000 7fa8bd7fa640 1 -- 172.21.15.16:0/3120345829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0010760 con 0x562ed0e86370 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/3120345829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 msgr2=0x562ed0e88760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000cc40 tx=0x7fa8b000e720 comp rx=0 tx=0).stop 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/3120345829 shutdown_connections 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e86370 0x562ed0e88760 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7b5d0 0x562ed0e7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/3120345829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7cb40 0x562ed0e7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/3120345829 >> 172.21.15.16:0/3120345829 conn(0x562ed0e44310 msgr2=0x562ed0e77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/3120345829 shutdown_connections 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/3120345829 wait complete. 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 -- start start 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.966+0000 7fa8cf3e0740 1 Processor -- start 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7b5d0 0x562ed0e85d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 0x562ed0e835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e89070 con 0x562ed0e7cb40 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ed0d9c060 con 0x562ed0e7b5d0 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ed0e891f0 con 0x562ed0e86370 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 10 client.0 _ll_register_callbacks cb 0x562ed0aef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52690/0 (socket says 172.21.15.16:52690) 2024-09-12T00:58:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 -- 172.21.15.16:0/204883741 learned_addr learned my addr 172.21.15.16:0/204883741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7b5d0 0x562ed0e85d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8c759e640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 0x562ed0e835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 -- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 msgr2=0x562ed0e835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 0x562ed0e835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 -- 172.21.15.16:0/204883741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7b5d0 msgr2=0x562ed0e85d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7b5d0 0x562ed0e85d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 -- 172.21.15.16:0/204883741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed0b523c0 con 0x562ed0e7cb40 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8c759e640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 0x562ed0e835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8bffff640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0005da0 tx=0x7fa8b0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cffcbe647484387 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0027060 con 0x562ed0e7cb40 2024-09-12T00:58:28.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa8b0018620 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.967+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b00047a0 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8b0014070 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 10 client.16002 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ed0e94ad0 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b00047a0 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa8b000f6d0 con 0x562ed0e7cb40 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 10 client.16002 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 10 client.16002 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 20 client.16002 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0e95ed0 0x562ed0e982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 4 client.16002 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ed0dee860 con 0x562ed0e95ed0 2024-09-12T00:58:28.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.968+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0e95ed0 0x562ed0e982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8ae7fc640 10 client.16002 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0e95ed0 0x562ed0e982c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e0c949d3faffeea server_cookie=aef1bbbc44f47aa1 in_seq=0 out_seq=0 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/204883741 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ed0dee860 con 0x562ed0e95ed0 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8ae7fc640 10 client.16002 handle_command_reply: tid=0 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8cf3e0740 1 client.16002 shutdown 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 msgr2=0x562ed0e82fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.969+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0005da0 tx=0x7fa8b0010cf0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 shutdown_connections 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e86370 0x562ed0e835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0e95ed0 0x562ed0e982c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7cb40 0x562ed0e82fe0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/204883741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e7b5d0 0x562ed0e85d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 >> 172.21.15.16:0/204883741 conn(0x562ed0e44310 msgr2=0x562ed0e7d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8ae7fc640 0 client.16002 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 shutdown_connections 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/204883741 wait complete. 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.970+0000 7fa8cf3e0740 20 client.16002 trim_cache size 0 max 16384 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 Processor -- start 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 -- start start 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e7e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ad790 con 0x562ed0e7b5d0 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81af2d0 con 0x562ed0e86370 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ade60 con 0x562ed0e7cb40 2024-09-12T00:58:28.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52700/0 (socket says 172.21.15.16:52700) 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/482108080 learned_addr learned my addr 172.21.15.16:0/482108080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8bffff640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e7e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8c759e640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.983+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/482108080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 msgr2=0x562ed0e7e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e7e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/482108080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 msgr2=0x562ed0e7e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e7e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/482108080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ed0e891b0 con 0x562ed0e7b5d0 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c759e640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031550 tx=0x7fa8b803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2bdf396bebc361 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8bcff9640 1 -- 172.21.15.16:0/482108080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8049040 con 0x562ed0e7b5d0 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8bcff9640 1 -- 172.21.15.16:0/482108080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b80c0360 con 0x562ed0e7b5d0 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.984+0000 7fa8bcff9640 1 -- 172.21.15.16:0/482108080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b803b460 con 0x562ed0e7b5d0 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/482108080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 msgr2=0x562ed0e92310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031550 tx=0x7fa8b803b800 comp rx=0 tx=0).stop 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/482108080 shutdown_connections 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e7e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/482108080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e92310 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/482108080 >> 172.21.15.16:0/482108080 conn(0x562ed0e44310 msgr2=0x562ed0e820d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/482108080 shutdown_connections 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/482108080 wait complete. 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 -- start start 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.985+0000 7fa8cf3e0740 1 Processor -- start 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81aec00 con 0x562ed0e7b5d0 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ae530 con 0x562ed0e86370 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c81ab770 con 0x562ed0e7cb40 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 10 client.0 _ll_register_callbacks cb 0x7fa8b000ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52708/0 (socket says 172.21.15.16:52708) 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/688871857 learned_addr learned my addr 172.21.15.16:0/688871857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c759e640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8bffff640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 msgr2=0x562ed0e89f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e89f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/688871857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 msgr2=0x562ed0e8a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e8a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 -- 172.21.15.16:0/688871857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ed0e8acd0 con 0x562ed0e7b5d0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8bffff640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8c6d9d640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8032c00 tx=0x7fa8b803bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d758ef78c2656b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8085020 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa8b80b8690 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.986+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b80c0360 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8b8049040 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ed0f008c0 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b80c0360 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa8b80c4790 con 0x562ed0e7b5d0 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:28.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 20 client.16008 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0f00fb0 0x562ed0f033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 4 client.16008 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ed0de8d70 con 0x562ed0f00fb0 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.987+0000 7fa8bffff640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0f00fb0 0x562ed0f033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8bffff640 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0f00fb0 0x562ed0f033d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d3d84a04ea6c721 server_cookie=1fd952d4ea3552e4 in_seq=0 out_seq=0 2024-09-12T00:58:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8ae7fc640 10 client.16008 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8ae7fc640 1 -- 172.21.15.16:0/688871857 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ed0de8d70 con 0x562ed0f00fb0 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8ae7fc640 10 client.16008 handle_command_reply: tid=0 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 client.16008 shutdown 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 msgr2=0x562ed0e8ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8032c00 tx=0x7fa8b803bf10 comp rx=0 tx=0).stop 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 shutdown_connections 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ed0e86370 0x562ed0e8a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ed0e7cb40 0x562ed0e89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x562ed0f00fb0 0x562ed0f033d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 --2- 172.21.15.16:0/688871857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ed0e7b5d0 0x562ed0e8ec30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8ae7fc640 0 client.16008 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 >> 172.21.15.16:0/688871857 conn(0x562ed0e44310 msgr2=0x562ed0e84840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 shutdown_connections 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 1 -- 172.21.15.16:0/688871857 wait complete. 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.988+0000 7fa8cf3e0740 20 client.16008 trim_cache size 0 max 16384 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 msgr2=0x7fa8b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 0x7fa8b40a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0014590 tx=0x7fa8c0006090 comp rx=0 tx=0).stop 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 msgr2=0x7fa8c8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009a50 tx=0x7fa8b8004270 comp rx=0 tx=0).stop 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 shutdown_connections 2024-09-12T00:58:28.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8b40a2cf0 0x7fa8b40a51b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8c8106a80 0x7fa8c819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8c8101420 0x7fa8c8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 --2- 172.21.15.16:0/2566758427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8100a80 0x7fa8c8198430 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 >> 172.21.15.16:0/2566758427 conn(0x7fa8c80fc7a0 msgr2=0x7fa8c806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 shutdown_connections 2024-09-12T00:58:28.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:27.989+0000 7fa8cd007640 1 -- 172.21.15.16:0/2566758427 wait complete. 2024-09-12T00:58:28.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:58:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:29 smithi159 ceph-mon[35566]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T00:58:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:29 smithi016 ceph-mon[30972]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T00:58:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:29 smithi099 ceph-mon[35654]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 9 op/s 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4024320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c101de0 msgr2=0x7f2f2c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4024320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1021c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009900 tx=0x7f2f1c02a410 comp rx=0 tx=0).stop 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4024320934 shutdown_connections 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4024320934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 0x7f2f2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4024320934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c102790 0x7f2f2c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4024320934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1021c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4024320934 >> 172.21.15.16:0/4024320934 conn(0x7f2f2c0fd8c0 msgr2=0x7f2f2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4024320934 shutdown_connections 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4024320934 wait complete. 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.526+0000 7f2f31f5e640 1 Processor -- start 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 -- start start 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 0x7f2f2c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c111fc0 con 0x7f2f2c102790 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c111e40 con 0x7f2f2c106f10 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f31f5e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c112140 con 0x7f2f2c101de0 2024-09-12T00:58:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46624/0 (socket says 172.21.15.16:46624) 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 -- 172.21.15.16:0/4045908974 learned_addr learned my addr 172.21.15.16:0/4045908974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 0x7f2f2c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 -- 172.21.15.16:0/4045908974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 msgr2=0x7f2f2c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 -- 172.21.15.16:0/4045908974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 msgr2=0x7f2f2c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 0x7f2f2c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2affd640 1 -- 172.21.15.16:0/4045908974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c19d450 con 0x7f2f2c102790 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.527+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c890 tx=0x7f2f2000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f28ff9640 1 -- 172.21.15.16:0/4045908974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20004220 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c19d740 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f28ff9640 1 -- 172.21.15.16:0/4045908974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f200043c0 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f28ff9640 1 -- 172.21.15.16:0/4045908974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200049e0 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.528+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1a5020 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.530+0000 7f2f28ff9640 1 -- 172.21.15.16:0/4045908974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2f200079b0 con 0x7f2f2c102790 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.530+0000 7f2f34337740 1 Processor -- start 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.530+0000 7f2f34337740 1 -- start start 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c12b40 0x55a518c11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f28ff9640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 0x7f2f140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a518b772f0 con 0x55a518c20780 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a518b1ecf0 con 0x55a518c12b40 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f34337740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a518b732b0 con 0x55a518c115d0 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 0x7f2f140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46640/0 (socket says 172.21.15.16:46640) 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 -- 172.21.15.16:0/785964556 learned_addr learned my addr 172.21.15.16:0/785964556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f28ff9640 1 -- 172.21.15.16:0/4045908974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f200cf590 con 0x7f2f2c102790 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c12b40 0x55a518c11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2affd640 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 -- 172.21.15.16:0/785964556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 msgr2=0x55a518c11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 -- 172.21.15.16:0/785964556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c12b40 msgr2=0x55a518c11090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c12b40 0x55a518c11090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 -- 172.21.15.16:0/785964556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a518b033a0 con 0x55a518c20780 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400aeb0 tx=0x7f2f2400a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c715c88c31eef92b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.531+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 0x7f2f140a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c000c00 tx=0x7f2f1c005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f11ffb640 1 -- 172.21.15.16:0/785964556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24018070 con 0x55a518c20780 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f11ffb640 1 -- 172.21.15.16:0/785964556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f24003e50 con 0x55a518c20780 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f11ffb640 1 -- 172.21.15.16:0/785964556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24004280 con 0x55a518c20780 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 -- 172.21.15.16:0/785964556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 msgr2=0x55a518c22b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400aeb0 tx=0x7f2f2400a770 comp rx=0 tx=0).stop 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 -- 172.21.15.16:0/785964556 shutdown_connections 2024-09-12T00:58:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c20780 0x55a518c22b70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 --2- 172.21.15.16:0/785964556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c12b40 0x55a518c11090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 -- 172.21.15.16:0/785964556 >> 172.21.15.16:0/785964556 conn(0x55a518bda310 msgr2=0x55a518c0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 -- 172.21.15.16:0/785964556 shutdown_connections 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.532+0000 7f2f34337740 1 -- 172.21.15.16:0/785964556 wait complete. 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 -- start start 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 Processor -- start 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c20190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c1da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a518c23480 con 0x55a518c12b40 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a518b32060 con 0x55a518c20780 2024-09-12T00:58:29.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a518c23600 con 0x55a518c115d0 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f34337740 10 client.0 _ll_register_callbacks cb 0x55a5188858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f2b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c20190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46648/0 (socket says 172.21.15.16:46648) 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f2affd640 1 -- 172.21.15.16:0/4277318695 learned_addr learned my addr 172.21.15.16:0/4277318695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.533+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c1da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 -- 172.21.15.16:0/4277318695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 msgr2=0x55a518c20190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c20190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 -- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 msgr2=0x55a518c1da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c1da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 -- 172.21.15.16:0/4277318695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5188e83c0 con 0x55a518c12b40 2024-09-12T00:58:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f2affd640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009ba10 tx=0x7f2f2009b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881e75d56f1ca762 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2001d040 con 0x55a518c12b40 2024-09-12T00:58:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f20007ca0 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f24003e50 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f200ae5c0 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f34337740 10 client.16020 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a518c2ef50 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f24003e50 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.534+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f200ba2b0 con 0x55a518c12b40 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 10 client.16020 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 20 client.16020 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c303d0 0x55a518c327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 4 client.16020 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a518b84860 con 0x55a518c303d0 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c303d0 0x55a518c327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c303d0 0x55a518c327e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5259b110b389fb56 server_cookie=b45d81d4fc7a4cc5 in_seq=0 out_seq=0 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.535+0000 7f2f0affd640 10 client.16020 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f0affd640 1 -- 172.21.15.16:0/4277318695 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a518b84860 con 0x55a518c303d0 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f0affd640 10 client.16020 handle_command_reply: tid=0 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 client.16020 shutdown 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 msgr2=0x55a518c1d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009ba10 tx=0x7f2f2009b3e0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 shutdown_connections 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c303d0 0x55a518c327e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c1da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1d440 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f0affd640 0 client.16020 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:58:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 --2- 172.21.15.16:0/4277318695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c20190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 >> 172.21.15.16:0/4277318695 conn(0x55a518bda310 msgr2=0x55a518c13960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 shutdown_connections 2024-09-12T00:58:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 1 -- 172.21.15.16:0/4277318695 wait complete. 2024-09-12T00:58:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.536+0000 7f2f34337740 20 client.16020 trim_cache size 0 max 16384 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.549+0000 7f2f34337740 1 Processor -- start 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.549+0000 7f2f34337740 1 -- start start 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c2c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1ab600 con 0x55a518c12b40 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1ad140 con 0x55a518c20780 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f34337740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1abcd0 con 0x55a518c115d0 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46660/0 (socket says 172.21.15.16:46660) 2024-09-12T00:58:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 -- 172.21.15.16:0/1286450498 learned_addr learned my addr 172.21.15.16:0/1286450498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c2c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 -- 172.21.15.16:0/1286450498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 msgr2=0x55a518c2c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c2c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 -- 172.21.15.16:0/1286450498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 msgr2=0x55a518c385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 -- 172.21.15.16:0/1286450498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a518c1e3e0 con 0x55a518c12b40 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009b780 tx=0x7f2f20099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50acf60bd4109e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f117fa640 1 -- 172.21.15.16:0/1286450498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2001d040 con 0x55a518c12b40 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f117fa640 1 -- 172.21.15.16:0/1286450498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2009fe30 con 0x55a518c12b40 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.550+0000 7f2f117fa640 1 -- 172.21.15.16:0/1286450498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200aea00 con 0x55a518c12b40 2024-09-12T00:58:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- 172.21.15.16:0/1286450498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 msgr2=0x55a518c380b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2f2009b780 tx=0x7f2f20099f90 comp rx=0 tx=0).stop 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- 172.21.15.16:0/1286450498 shutdown_connections 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c20780 0x55a518c385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c380b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 --2- 172.21.15.16:0/1286450498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c115d0 0x55a518c2c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- 172.21.15.16:0/1286450498 >> 172.21.15.16:0/1286450498 conn(0x55a518bda310 msgr2=0x55a518c1c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- 172.21.15.16:0/1286450498 shutdown_connections 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- 172.21.15.16:0/1286450498 wait complete. 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.551+0000 7f2f34337740 1 -- start start 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 Processor -- start 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c115d0 0x55a518c1a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 0x55a518c1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1aca70 con 0x55a518c12b40 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1ac3a0 con 0x55a518c115d0 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c1a95e0 con 0x55a518c20780 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f34337740 10 client.0 _ll_register_callbacks cb 0x7f2f20004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46662/0 (socket says 172.21.15.16:46662) 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2affd640 1 -- 172.21.15.16:0/1064333739 learned_addr learned my addr 172.21.15.16:0/1064333739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c115d0 0x55a518c1a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 0x55a518c1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2affd640 1 -- 172.21.15.16:0/1064333739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 msgr2=0x55a518c1b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.552+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 0x55a518c1b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f2affd640 1 -- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c115d0 msgr2=0x55a518c1a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c115d0 0x55a518c1a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f2affd640 1 -- 172.21.15.16:0/1064333739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a518c14120 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f2bfff640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 0x55a518c1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f2affd640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2f20097ee0 tx=0x7f2f2009b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238bcf7b51969250 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2001d040 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2f200cf310 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2009fe30 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f2009c180 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f34337740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a518c9b390 con 0x55a518c12b40 2024-09-12T00:58:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f2009fe30 con 0x55a518c12b40 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f200aee40 con 0x55a518c12b40 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f34337740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.553+0000 7f2f34337740 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f34337740 20 client.16026 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c9b810 0x55a518c9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f34337740 4 client.16026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a518b7ed70 con 0x55a518c9b810 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c9b810 0x55a518c9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f2b7fe640 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c9b810 0x55a518c9dc30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5531f1e72f76818 server_cookie=f0e8cf8c4584ad5d in_seq=0 out_seq=0 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f0affd640 10 client.16026 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.554+0000 7f2f0affd640 1 -- 172.21.15.16:0/1064333739 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a518b7ed70 con 0x55a518c9b810 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f0affd640 10 client.16026 handle_command_reply: tid=0 2024-09-12T00:58:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 client.16026 shutdown 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 msgr2=0x55a518c1abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2f20097ee0 tx=0x7f2f2009b3e0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 shutdown_connections 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55a518c9b810 0x55a518c9dc30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a518c20780 0x55a518c1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a518c12b40 0x55a518c1abb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 --2- 172.21.15.16:0/1064333739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a518c115d0 0x55a518c1a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 >> 172.21.15.16:0/1064333739 conn(0x55a518bda310 msgr2=0x55a518c0c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f0affd640 0 client.16026 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 shutdown_connections 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 1 -- 172.21.15.16:0/1064333739 wait complete. 2024-09-12T00:58:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f34337740 20 client.16026 trim_cache size 0 max 16384 2024-09-12T00:58:29.565 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:58:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 msgr2=0x7f2f140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 0x7f2f140a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c000c00 tx=0x7f2f1c005a20 comp rx=0 tx=0).stop 2024-09-12T00:58:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 msgr2=0x7f2f2c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.555+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000c890 tx=0x7f2f2000cd50 comp rx=0 tx=0).stop 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 shutdown_connections 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2f140a2cf0 0x7f2f140a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f2c106f10 0x7f2f2c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c102790 0x7f2f2c1989e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 --2- 172.21.15.16:0/4045908974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f2c101de0 0x7f2f2c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 >> 172.21.15.16:0/4045908974 conn(0x7f2f2c0fd8c0 msgr2=0x7f2f2c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 shutdown_connections 2024-09-12T00:58:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:29.556+0000 7f2f31f5e640 1 -- 172.21.15.16:0/4045908974 wait complete. 2024-09-12T00:58:30.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:58:30.678 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 -- 172.21.15.16:0/1826127466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100e30 msgr2=0x7fe140104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 --2- 172.21.15.16:0/1826127466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100e30 0x7fe140104e80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe130009900 tx=0x7fe13002a410 comp rx=0 tx=0).stop 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 -- 172.21.15.16:0/1826127466 shutdown_connections 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 --2- 172.21.15.16:0/1826127466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe1401055b0 0x7fe14010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 --2- 172.21.15.16:0/1826127466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100e30 0x7fe140104e80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 --2- 172.21.15.16:0/1826127466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe140100480 0x7fe140100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 -- 172.21.15.16:0/1826127466 >> 172.21.15.16:0/1826127466 conn(0x7fe140077210 msgr2=0x7fe1400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 -- 172.21.15.16:0/1826127466 shutdown_connections 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.161+0000 7fe148573640 1 -- 172.21.15.16:0/1826127466 wait complete. 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 Processor -- start 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 -- start start 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe140100e30 0x7fe1401989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1401055b0 0x7fe14019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010fd90 con 0x7fe140100480 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010fc10 con 0x7fe140100e30 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe148573640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010ff10 con 0x7fe1401055b0 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.162+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46676/0 (socket says 172.21.15.16:46676) 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 -- 172.21.15.16:0/57341148 learned_addr learned my addr 172.21.15.16:0/57341148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe146b10640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1401055b0 0x7fe14019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe145b0e640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe140100e30 0x7fe1401989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 -- 172.21.15.16:0/57341148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1401055b0 msgr2=0x7fe14019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1401055b0 0x7fe14019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 -- 172.21.15.16:0/57341148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe140100e30 msgr2=0x7fe1401989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe140100e30 0x7fe1401989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 -- 172.21.15.16:0/57341148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14019d430 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe14630f640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00dbc0 tx=0x7fe13c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe1377fe640 1 -- 172.21.15.16:0/57341148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c005220 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14019d720 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe1377fe640 1 -- 172.21.15.16:0/57341148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c00ae50 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe1377fe640 1 -- 172.21.15.16:0/57341148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00e870 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.163+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1401a5000 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe1377fe640 1 -- 172.21.15.16:0/57341148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe13c00ea10 con 0x7fe140100480 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 Processor -- start 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 -- start start 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37212090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f3721f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f371782f0 con 0x558f372125d0 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558f3711fcf0 con 0x558f3721d370 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14a94c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f371742b0 con 0x558f37213b40 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46684/0 (socket says 172.21.15.16:46684) 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe14630f640 1 -- 172.21.15.16:0/3213534289 learned_addr learned my addr 172.21.15.16:0/3213534289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.166+0000 7fe146b10640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37212090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe145b0e640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f3721f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe1377fe640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 0x7fe12c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 -- 172.21.15.16:0/3213534289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 msgr2=0x558f37212090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37212090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 -- 172.21.15.16:0/3213534289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 msgr2=0x558f3721f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f3721f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 -- 172.21.15.16:0/3213534289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f371043a0 con 0x558f372125d0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe145b0e640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f3721f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe146b10640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 0x7fe12c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe1377fe640 1 -- 172.21.15.16:0/57341148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe13c0cf370 con 0x7fe140100480 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe14630f640 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ccb70 tx=0x7fe13c0666b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd71785bd367b4b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe146b10640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 0x7fe12c0a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe138007fc0 tx=0x7fe138004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe134ff9640 1 -- 172.21.15.16:0/3213534289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c09b5c0 con 0x558f372125d0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe134ff9640 1 -- 172.21.15.16:0/3213534289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c024050 con 0x558f372125d0 2024-09-12T00:58:31.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.167+0000 7fe134ff9640 1 -- 172.21.15.16:0/3213534289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c09cd00 con 0x558f372125d0 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 -- 172.21.15.16:0/3213534289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 msgr2=0x558f37212a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ccb70 tx=0x7fe13c0666b0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 -- 172.21.15.16:0/3213534289 shutdown_connections 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f3721f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37212a10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 --2- 172.21.15.16:0/3213534289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37212090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 -- 172.21.15.16:0/3213534289 >> 172.21.15.16:0/3213534289 conn(0x558f371db310 msgr2=0x558f3720ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 -- 172.21.15.16:0/3213534289 shutdown_connections 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.168+0000 7fe14a94c740 1 -- 172.21.15.16:0/3213534289 wait complete. 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 -- start start 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 Processor -- start 2024-09-12T00:58:31.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 0x558f3721cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f3721a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558f37220010 con 0x558f37213b40 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558f37133060 con 0x558f372125d0 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f37220190 con 0x558f3721d370 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14a94c740 10 client.0 _ll_register_callbacks cb 0x558f36e868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46694/0 (socket says 172.21.15.16:46694) 2024-09-12T00:58:31.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 -- 172.21.15.16:0/1893023381 learned_addr learned my addr 172.21.15.16:0/1893023381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14630f640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 0x558f3721cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe146b10640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f3721a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 -- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 msgr2=0x558f3721a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f3721a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 -- 172.21.15.16:0/1893023381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 msgr2=0x558f3721cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 0x558f3721cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe145b0e640 1 -- 172.21.15.16:0/1893023381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f36ee93c0 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.169+0000 7fe14630f640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 0x558f3721cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe13002a930 tx=0x7fe1300319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa76d049330dda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130035040 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe130004300 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c024050 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe130005020 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe14a94c740 10 client.16038 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f3722ba70 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c024050 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe1300035f0 con 0x558f37213b40 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe14a94c740 10 client.16038 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:31.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.170+0000 7fe14a94c740 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14a94c740 20 client.16038 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3722f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14a94c740 4 client.16038 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f37185860 con 0x558f3722ce80 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14630f640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3722f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe129ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:58:31.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.171+0000 7fe14630f640 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3722f300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef43d560dfd0189a server_cookie=83928aac05546c39 in_seq=0 out_seq=0 2024-09-12T00:58:31.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe129ffb640 1 -- 172.21.15.16:0/1893023381 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f37185860 con 0x558f3722ce80 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe129ffb640 10 client.16038 handle_command_reply: tid=0 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 client.16038 shutdown 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 msgr2=0x558f37219fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe13002a930 tx=0x7fe1300319a0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 shutdown_connections 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f3721a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f37213b40 0x558f37219fe0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3722f300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe129ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1893023381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f372125d0 0x558f3721cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 >> 172.21.15.16:0/1893023381 conn(0x558f371db310 msgr2=0x558f37214960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 shutdown_connections 2024-09-12T00:58:31.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.172+0000 7fe14a94c740 1 -- 172.21.15.16:0/1893023381 wait complete. 2024-09-12T00:58:31.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.173+0000 7fe14a94c740 20 client.16038 trim_cache size 0 max 16384 2024-09-12T00:58:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 Processor -- start 2024-09-12T00:58:31.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 -- start start 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37225800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f37225d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401ad7c0 con 0x558f372125d0 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401af300 con 0x558f3721d370 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14a94c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401ade90 con 0x558f37213b40 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46696/0 (socket says 172.21.15.16:46696) 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.186+0000 7fe14630f640 1 -- 172.21.15.16:0/455321294 learned_addr learned my addr 172.21.15.16:0/455321294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe145b0e640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37225800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe146b10640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f37225d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 -- 172.21.15.16:0/455321294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 msgr2=0x558f37225800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37225800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 -- 172.21.15.16:0/455321294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 msgr2=0x558f37225d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f37225d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 -- 172.21.15.16:0/455321294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3720e4f0 con 0x558f372125d0 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe146b10640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f37225d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe14630f640 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0cccb0 tx=0x7fe13c097e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b82a7e34faf56ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe12bfff640 1 -- 172.21.15.16:0/455321294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c09b3e0 con 0x558f372125d0 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe12bfff640 1 -- 172.21.15.16:0/455321294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c09bce0 con 0x558f372125d0 2024-09-12T00:58:31.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.187+0000 7fe12bfff640 1 -- 172.21.15.16:0/455321294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c099e50 con 0x558f372125d0 2024-09-12T00:58:31.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- 172.21.15.16:0/455321294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 msgr2=0x558f3721bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0cccb0 tx=0x7fe13c097e50 comp rx=0 tx=0).stop 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- 172.21.15.16:0/455321294 shutdown_connections 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f3721d370 0x558f37225d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f37213b40 0x558f37225800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 --2- 172.21.15.16:0/455321294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f3721bf50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- 172.21.15.16:0/455321294 >> 172.21.15.16:0/455321294 conn(0x558f371db310 msgr2=0x558f3721b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- 172.21.15.16:0/455321294 shutdown_connections 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- 172.21.15.16:0/455321294 wait complete. 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 -- start start 2024-09-12T00:58:31.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.188+0000 7fe14a94c740 1 Processor -- start 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 0x558f37217ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f37216270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401aec30 con 0x558f372125d0 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401ae560 con 0x558f37213b40 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1401ab7a0 con 0x558f3721d370 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14a94c740 10 client.0 _ll_register_callbacks cb 0x7fe130009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46698/0 (socket says 172.21.15.16:46698) 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 -- 172.21.15.16:0/1644740971 learned_addr learned my addr 172.21.15.16:0/1644740971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:31.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 0x558f37217ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe146b10640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f37216270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 -- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 msgr2=0x558f37216270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f37216270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 -- 172.21.15.16:0/1644740971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 msgr2=0x558f37217ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 0x558f37217ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 -- 172.21.15.16:0/1644740971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f37216a80 con 0x558f372125d0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 0x558f37217ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe14630f640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe13c098450 tx=0x7fe13c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf4aee3a03bb4cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.189+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c09b5c0 con 0x558f372125d0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe13c00ed00 con 0x558f372125d0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f3717fd70 con 0x558f372125d0 2024-09-12T00:58:31.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe13c09fd80 con 0x558f372125d0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 10 client.16044 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558f3729c5f0 con 0x558f372125d0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558f3717fd70 con 0x558f372125d0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe13c00b040 con 0x558f372125d0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 10 client.16044 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 20 client.16044 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3729e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 4 client.16044 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f370fc170 con 0x558f3722ce80 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.190+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3729e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe129ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe145b0e640 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3729e7d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89c606bb28b1fbeb server_cookie=1344adf536df3ee2 in_seq=0 out_seq=0 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe129ffb640 1 -- 172.21.15.16:0/1644740971 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f370fc170 con 0x558f3722ce80 2024-09-12T00:58:31.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe129ffb640 10 client.16044 handle_command_reply: tid=0 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 client.16044 shutdown 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 msgr2=0x558f37217580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe13c098450 tx=0x7fe13c005670 comp rx=0 tx=0).stop 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 shutdown_connections 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f3721d370 0x558f37216270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558f3722ce80 0x558f3729e7d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f37213b40 0x558f37217ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 --2- 172.21.15.16:0/1644740971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558f372125d0 0x558f37217580 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 >> 172.21.15.16:0/1644740971 conn(0x558f371db310 msgr2=0x558f3721c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe129ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 shutdown_connections 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 1 -- 172.21.15.16:0/1644740971 wait complete. 2024-09-12T00:58:31.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.191+0000 7fe14a94c740 20 client.16044 trim_cache size 0 max 16384 2024-09-12T00:58:31.196 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:58:31.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 msgr2=0x7fe12c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 0x7fe12c0a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe138007fc0 tx=0x7fe138004a40 comp rx=0 tx=0).stop 2024-09-12T00:58:31.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 msgr2=0x7fe140198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:31.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00dbc0 tx=0x7fe13c00a620 comp rx=0 tx=0).stop 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 shutdown_connections 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe12c0a2cf0 0x7fe12c0a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1401055b0 0x7fe14019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe140100e30 0x7fe1401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 --2- 172.21.15.16:0/57341148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140100480 0x7fe140198480 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 >> 172.21.15.16:0/57341148 conn(0x7fe140077210 msgr2=0x7fe140103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 shutdown_connections 2024-09-12T00:58:31.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:31.192+0000 7fe148573640 1 -- 172.21.15.16:0/57341148 wait complete. 2024-09-12T00:58:31.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:31 smithi159 ceph-mon[35566]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-12T00:58:31.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:31 smithi016 ceph-mon[30972]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-12T00:58:31.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:31 smithi099 ceph-mon[35654]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-12T00:58:31.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:58:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 -- 172.21.15.16:0/3228128410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c106a80 msgr2=0x7fc99c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3228128410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c106a80 0x7fc99c106e60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc998009930 tx=0x7fc99802a420 comp rx=0 tx=0).stop 2024-09-12T00:58:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 -- 172.21.15.16:0/3228128410 shutdown_connections 2024-09-12T00:58:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3228128410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 0x7fc99c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3228128410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c100a80 0x7fc99c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3228128410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c106a80 0x7fc99c106e60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 -- 172.21.15.16:0/3228128410 >> 172.21.15.16:0/3228128410 conn(0x7fc99c0fc7a0 msgr2=0x7fc99c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 -- 172.21.15.16:0/3228128410 shutdown_connections 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.687+0000 7fc9a440b640 1 -- 172.21.15.16:0/3228128410 wait complete. 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 Processor -- start 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 -- start start 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 0x7fc99c10e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c106a80 0x7fc99c10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111fa0 con 0x7fc99c100a80 2024-09-12T00:58:32.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111e20 con 0x7fc99c106a80 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a440b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112120 con 0x7fc99c101420 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a21a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46718/0 (socket says 172.21.15.16:46718) 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a21a7640 1 -- 172.21.15.16:0/3846912853 learned_addr learned my addr 172.21.15.16:0/3846912853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c106a80 0x7fc99c10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.688+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 0x7fc99c10e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 -- 172.21.15.16:0/3846912853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 msgr2=0x7fc99c10e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 0x7fc99c10e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 -- 172.21.15.16:0/3846912853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c106a80 msgr2=0x7fc99c10ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c106a80 0x7fc99c10ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 -- 172.21.15.16:0/3846912853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1a68c0 con 0x7fc99c100a80 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc998004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9937fe640 1 -- 172.21.15.16:0/3846912853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99803f030 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9937fe640 1 -- 172.21.15.16:0/3846912853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc998003ad0 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1a6b50 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9937fe640 1 -- 172.21.15.16:0/3846912853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998035070 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.689+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a7030 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.691+0000 7fc9937fe640 1 -- 172.21.15.16:0/3846912853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc998042030 con 0x7fc99c100a80 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 Processor -- start 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 -- start start 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 0x559eb2d37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 0x559eb2d37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559eb2c9d2f0 con 0x559eb2d42370 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eb2c44cf0 con 0x559eb2d375d0 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559eb2c992b0 con 0x559eb2d38b40 2024-09-12T00:58:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a19a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a29a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 0x559eb2d37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.692+0000 7fc9a19a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46734/0 (socket says 172.21.15.16:46734) 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 -- 172.21.15.16:0/2264523165 learned_addr learned my addr 172.21.15.16:0/2264523165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 0x559eb2d37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9937fe640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 0x7fc9880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 -- 172.21.15.16:0/2264523165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 msgr2=0x559eb2d37090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 0x559eb2d37090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 -- 172.21.15.16:0/2264523165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 msgr2=0x559eb2d37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 0x559eb2d37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 -- 172.21.15.16:0/2264523165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb2c293a0 con 0x559eb2d42370 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 0x559eb2d37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 0x559eb2d37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 0x7fc9880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9937fe640 1 -- 172.21.15.16:0/3846912853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc9980ef890 con 0x7fc99c100a80 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00aca0 tx=0x7fc98c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcfd4058393d0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc990dea640 1 -- 172.21.15.16:0/2264523165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01d070 con 0x559eb2d42370 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 0x7fc9880a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc994005fd0 tx=0x7fc994004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc990dea640 1 -- 172.21.15.16:0/2264523165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c018b20 con 0x559eb2d42370 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.693+0000 7fc990dea640 1 -- 172.21.15.16:0/2264523165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c010c00 con 0x559eb2d42370 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2264523165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 msgr2=0x559eb2d44760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00aca0 tx=0x7fc98c00f790 comp rx=0 tx=0).stop 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2264523165 shutdown_connections 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d44760 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d375d0 0x559eb2d37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2264523165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d38b40 0x559eb2d37090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2264523165 >> 172.21.15.16:0/2264523165 conn(0x559eb2d00310 msgr2=0x559eb2d33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2264523165 shutdown_connections 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.694+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2264523165 wait complete. 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 -- start start 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 Processor -- start 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d3efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559eb2d45070 con 0x559eb2d42370 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eb2c58060 con 0x559eb2d38b40 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559eb2d451f0 con 0x559eb2d375d0 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a67e4740 10 client.0 _ll_register_callbacks cb 0x559eb29ab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46736/0 (socket says 172.21.15.16:46736) 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a29a8640 1 -- 172.21.15.16:0/741396145 learned_addr learned my addr 172.21.15.16:0/741396145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.695+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d3efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 -- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 msgr2=0x559eb2d41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d41d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 -- 172.21.15.16:0/741396145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 msgr2=0x559eb2d3efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d3efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 -- 172.21.15.16:0/741396145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb2a0e3c0 con 0x559eb2d42370 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d3efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc994013490 tx=0x7fc994004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e644ab00b35e6dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994003970 con 0x559eb2d42370 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc994018070 con 0x559eb2d42370 2024-09-12T00:58:32.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c018b20 con 0x559eb2d42370 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc994003b10 con 0x559eb2d42370 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a67e4740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eb2d50b30 con 0x559eb2d42370 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c018b20 con 0x559eb2d42370 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.696+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc99400f040 con 0x559eb2d42370 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 20 client.16056 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2d52090 0x559eb2d54510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 4 client.16056 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb2caa860 con 0x559eb2d52090 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2d52090 0x559eb2d54510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2d52090 0x559eb2d54510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7353694384fbb68d server_cookie=36078b91fa9ac8ec in_seq=0 out_seq=0 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.697+0000 7fc985ffb640 10 client.16056 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc985ffb640 1 -- 172.21.15.16:0/741396145 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eb2caa860 con 0x559eb2d52090 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc985ffb640 10 client.16056 handle_command_reply: tid=0 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 client.16056 shutdown 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 msgr2=0x559eb2d3f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc994013490 tx=0x7fc994004ab0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 shutdown_connections 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2d52090 0x559eb2d54510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d3f530 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d3efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/741396145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 >> 172.21.15.16:0/741396145 conn(0x559eb2d00310 msgr2=0x559eb2d39960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc985ffb640 0 client.16056 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 shutdown_connections 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 1 -- 172.21.15.16:0/741396145 wait complete. 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.698+0000 7fc9a67e4740 20 client.16056 trim_cache size 0 max 16384 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 Processor -- start 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 -- start start 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d42370 0x559eb2d3a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1af9f0 con 0x559eb2d38b40 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b1530 con 0x559eb2d42370 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b00c0 con 0x559eb2d375d0 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46748/0 (socket says 172.21.15.16:46748) 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 -- 172.21.15.16:0/3137102858 learned_addr learned my addr 172.21.15.16:0/3137102858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d42370 0x559eb2d3a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 -- 172.21.15.16:0/3137102858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 msgr2=0x559eb2d4e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 -- 172.21.15.16:0/3137102858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d42370 msgr2=0x559eb2d3a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d42370 0x559eb2d3a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 -- 172.21.15.16:0/3137102858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb2d32310 con 0x559eb2d38b40 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.712+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc98c01b040 tx=0x7fc98c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b01349e8c480752 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc987fff640 1 -- 172.21.15.16:0/3137102858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01d040 con 0x559eb2d38b40 2024-09-12T00:58:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc987fff640 1 -- 172.21.15.16:0/3137102858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c005520 con 0x559eb2d38b40 2024-09-12T00:58:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc987fff640 1 -- 172.21.15.16:0/3137102858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c023490 con 0x559eb2d38b40 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 -- 172.21.15.16:0/3137102858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 msgr2=0x559eb2d3a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc98c01b040 tx=0x7fc98c005e40 comp rx=0 tx=0).stop 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 -- 172.21.15.16:0/3137102858 shutdown_connections 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d42370 0x559eb2d3a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d38b40 0x559eb2d3a120 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/3137102858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 -- 172.21.15.16:0/3137102858 >> 172.21.15.16:0/3137102858 conn(0x559eb2d00310 msgr2=0x559eb2d3ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 -- 172.21.15.16:0/3137102858 shutdown_connections 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.713+0000 7fc9a67e4740 1 -- 172.21.15.16:0/3137102858 wait complete. 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 -- start start 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 Processor -- start 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d45d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b0e60 con 0x559eb2d42370 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1b0790 con 0x559eb2d38b40 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1ad9d0 con 0x559eb2d375d0 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a67e4740 10 client.0 _ll_register_callbacks cb 0x7fc994009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a29a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46762/0 (socket says 172.21.15.16:46762) 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a29a8640 1 -- 172.21.15.16:0/2735982172 learned_addr learned my addr 172.21.15.16:0/2735982172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a19a6640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d45d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a29a8640 1 -- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 msgr2=0x559eb2d4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.714+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a29a8640 1 -- 172.21.15.16:0/2735982172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 msgr2=0x559eb2d45d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d45d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a29a8640 1 -- 172.21.15.16:0/2735982172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb2d34960 con 0x559eb2d42370 2024-09-12T00:58:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a29a8640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098b0 tx=0x7fc994004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd812e2de2165e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940040c0 con 0x559eb2d42370 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc99401e800 con 0x559eb2d42370 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98c005520 con 0x559eb2d42370 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc99403d050 con 0x559eb2d42370 2024-09-12T00:58:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eb2dbc9f0 con 0x559eb2d42370 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c005520 con 0x559eb2d42370 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc99402f970 con 0x559eb2d42370 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 20 client.16062 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.715+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2dbd160 0x559eb2dbf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc9a67e4740 4 client.16062 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eb2ca4d70 con 0x559eb2dbd160 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2dbd160 0x559eb2dbf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc9a21a7640 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2dbd160 0x559eb2dbf580 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b446b7ae5008fd server_cookie=7486f421bdf1b79a in_seq=0 out_seq=0 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc985ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc985ffb640 1 -- 172.21.15.16:0/2735982172 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eb2ca4d70 con 0x559eb2dbd160 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc985ffb640 10 client.16062 handle_command_reply: tid=0 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.716+0000 7fc9a67e4740 1 client.16062 shutdown 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 msgr2=0x559eb2d46280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc9940098b0 tx=0x7fc994004920 comp rx=0 tx=0).stop 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 shutdown_connections 2024-09-12T00:58:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559eb2dbd160 0x559eb2dbf580 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb2d42370 0x559eb2d46280 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb2d38b40 0x559eb2d45d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc985ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 --2- 172.21.15.16:0/2735982172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb2d375d0 0x559eb2d4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 >> 172.21.15.16:0/2735982172 conn(0x559eb2d00310 msgr2=0x559eb2d324e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 shutdown_connections 2024-09-12T00:58:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 1 -- 172.21.15.16:0/2735982172 wait complete. 2024-09-12T00:58:32.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a67e4740 20 client.16062 trim_cache size 0 max 16384 2024-09-12T00:58:32.736 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 msgr2=0x7fc9880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 0x7fc9880a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc994005fd0 tx=0x7fc994004a40 comp rx=0 tx=0).stop 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 msgr2=0x7fc99c10e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc998009230 tx=0x7fc998004270 comp rx=0 tx=0).stop 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 shutdown_connections 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc9880a2cf0 0x7fc9880a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.717+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c106a80 0x7fc99c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.718+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc99c101420 0x7fc99c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.718+0000 7fc9a440b640 1 --2- 172.21.15.16:0/3846912853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc99c100a80 0x7fc99c10e3e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.718+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 >> 172.21.15.16:0/3846912853 conn(0x7fc99c0fc7a0 msgr2=0x7fc99c0fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.718+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 shutdown_connections 2024-09-12T00:58:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:32.718+0000 7fc9a440b640 1 -- 172.21.15.16:0/3846912853 wait complete. 2024-09-12T00:58:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:33 smithi159 ceph-mon[35566]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:33.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:33 smithi016 ceph-mon[30972]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:33 smithi099 ceph-mon[35654]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:34.506 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:58:34.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:58:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:35.185 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:35 smithi016 ceph-mon[30972]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.272+0000 7f754e29f640 1 -- 172.21.15.16:0/3644484049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100350 msgr2=0x7f7548100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.272+0000 7f754e29f640 1 --2- 172.21.15.16:0/3644484049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100350 0x7f7548100730 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7538009930 tx=0x7f753802a420 comp rx=0 tx=0).stop 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 -- 172.21.15.16:0/3644484049 shutdown_connections 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 --2- 172.21.15.16:0/3644484049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548106f50 0x7f754810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 --2- 172.21.15.16:0/3644484049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548100d00 0x7f7548106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 --2- 172.21.15.16:0/3644484049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100350 0x7f7548100730 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 -- 172.21.15.16:0/3644484049 >> 172.21.15.16:0/3644484049 conn(0x7f7548077210 msgr2=0x7f75480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 -- 172.21.15.16:0/3644484049 shutdown_connections 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 -- 172.21.15.16:0/3644484049 wait complete. 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 Processor -- start 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.273+0000 7f754e29f640 1 -- start start 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548100350 0x7f7548198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 0x7f754819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7548111ff0 con 0x7f7548100d00 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7548111e70 con 0x7f7548106f50 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7548112170 con 0x7f7548100350 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46772/0 (socket says 172.21.15.16:46772) 2024-09-12T00:58:35.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 -- 172.21.15.16:0/1600357292 learned_addr learned my addr 172.21.15.16:0/1600357292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f7547fff640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548100350 0x7f7548198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754c83c640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 0x7f754819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 -- 172.21.15.16:0/1600357292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548100350 msgr2=0x7f7548198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548100350 0x7f7548198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 -- 172.21.15.16:0/1600357292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 msgr2=0x7f754819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 0x7f754819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 -- 172.21.15.16:0/1600357292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754819d520 con 0x7f7548100d00 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754c83c640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 0x7f754819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75477fe640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c970 tx=0x7f753c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75457fa640 1 -- 172.21.15.16:0/1600357292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c007bb0 con 0x7f7548100d00 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754819d810 con 0x7f7548100d00 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75457fa640 1 -- 172.21.15.16:0/1600357292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753c007d50 con 0x7f7548100d00 2024-09-12T00:58:35.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f75457fa640 1 -- 172.21.15.16:0/1600357292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c00f450 con 0x7f7548100d00 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.274+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a50f0 con 0x7f7548100d00 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f75457fa640 1 -- 172.21.15.16:0/1600357292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f753c004f90 con 0x7f7548100d00 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 Processor -- start 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 -- start start 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056abb60 0x5634056abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056b6d40 0x5634056b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563405611280 con 0x5634056ac480 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5634055b8b10 con 0x5634056abb60 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.277+0000 7f7550678740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56340560d240 con 0x5634056b6d40 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f75457fa640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 0x7f75300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46786/0 (socket says 172.21.15.16:46786) 2024-09-12T00:58:35.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 -- 172.21.15.16:0/2615946911 learned_addr learned my addr 172.21.15.16:0/2615946911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f754c83c640 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056abb60 0x5634056abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f754c83c640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 0x7f75300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f75457fa640 1 -- 172.21.15.16:0/1600357292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f753c0cf0c0 con 0x7f7548100d00 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f75477fe640 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056b6d40 0x5634056b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 -- 172.21.15.16:0/2615946911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056b6d40 msgr2=0x5634056b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056b6d40 0x5634056b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 -- 172.21.15.16:0/2615946911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056abb60 msgr2=0x5634056abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056abb60 0x5634056abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 -- 172.21.15.16:0/2615946911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634055c51f0 con 0x5634056ac480 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f7547fff640 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7538009930 tx=0x7f7538005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377ea315d810a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.278+0000 7f754c83c640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 0x7f75300a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7540013b30 tx=0x7f7540005c60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f752e7fc640 1 -- 172.21.15.16:0/2615946911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538040070 con 0x5634056ac480 2024-09-12T00:58:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f752e7fc640 1 -- 172.21.15.16:0/2615946911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7538004490 con 0x5634056ac480 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f752e7fc640 1 -- 172.21.15.16:0/2615946911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538035040 con 0x5634056ac480 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 -- 172.21.15.16:0/2615946911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 msgr2=0x5634056b6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7538009930 tx=0x7f7538005c00 comp rx=0 tx=0).stop 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 -- 172.21.15.16:0/2615946911 shutdown_connections 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056b6d40 0x5634056b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b6800 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 --2- 172.21.15.16:0/2615946911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056abb60 0x5634056abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 -- 172.21.15.16:0/2615946911 >> 172.21.15.16:0/2615946911 conn(0x5634056a80f0 msgr2=0x5634056a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 -- 172.21.15.16:0/2615946911 shutdown_connections 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.279+0000 7f7550678740 1 -- 172.21.15.16:0/2615946911 wait complete. 2024-09-12T00:58:35.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 -- start start 2024-09-12T00:58:35.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 Processor -- start 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056abb60 0x5634056b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634056b9910 con 0x5634056ac480 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5634055cbf10 con 0x5634056b6d40 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5634056b9a90 con 0x5634056abb60 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7550678740 10 client.0 _ll_register_callbacks cb 0x563405594a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f75477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f75477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46800/0 (socket says 172.21.15.16:46800) 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f75477fe640 1 -- 172.21.15.16:0/3095751776 learned_addr learned my addr 172.21.15.16:0/3095751776 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.280+0000 7f7547fff640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056abb60 0x5634056b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f754c83c640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 -- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056abb60 msgr2=0x5634056b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056abb60 0x5634056b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 -- 172.21.15.16:0/3095751776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 msgr2=0x5634056b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 -- 172.21.15.16:0/3095751776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634053826a0 con 0x5634056ac480 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75477fe640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f753c00afd0 tx=0x7f753c09b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70897d9de307e55a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75277fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c01d030 con 0x5634056ac480 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f753c09ab00 con 0x5634056ac480 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7538004490 con 0x5634056ac480 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f753c09bdb0 con 0x5634056ac480 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f7550678740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634056c5450 con 0x5634056ac480 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.281+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7538004490 con 0x5634056ac480 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f753c0bb040 con 0x5634056ac480 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 20 client.16074 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634056c67a0 0x5634056c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 4 client.16074 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56340561e7f0 con 0x5634056c67a0 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.282+0000 7f7547fff640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634056c67a0 0x5634056c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.283+0000 7f75277fe640 10 client.16074 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.283+0000 7f7547fff640 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634056c67a0 0x5634056c8bb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d860be12109074 server_cookie=101f8674caeb0f03 in_seq=0 out_seq=0 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.283+0000 7f75277fe640 1 -- 172.21.15.16:0/3095751776 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56340561e7f0 con 0x5634056c67a0 2024-09-12T00:58:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.283+0000 7f75277fe640 10 client.16074 handle_command_reply: tid=0 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.283+0000 7f7550678740 1 client.16074 shutdown 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 msgr2=0x5634056b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f753c00afd0 tx=0x7f753c09b5b0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 shutdown_connections 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634056c67a0 0x5634056c8bb0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056ac480 0x5634056b4120 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 --2- 172.21.15.16:0/3095751776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056abb60 0x5634056b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f75277fe640 0 client.16074 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 >> 172.21.15.16:0/3095751776 conn(0x5634056a80f0 msgr2=0x5634056ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 shutdown_connections 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 1 -- 172.21.15.16:0/3095751776 wait complete. 2024-09-12T00:58:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.284+0000 7f7550678740 20 client.16074 trim_cache size 0 max 16384 2024-09-12T00:58:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 Processor -- start 2024-09-12T00:58:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 -- start start 2024-09-12T00:58:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75481ad8e0 con 0x5634056abb60 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75481af420 con 0x5634056b6d40 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7550678740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75481adfb0 con 0x5634056ac480 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46808/0 (socket says 172.21.15.16:46808) 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 -- 172.21.15.16:0/1302866304 learned_addr learned my addr 172.21.15.16:0/1302866304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 -- 172.21.15.16:0/1302866304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 msgr2=0x5634056ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f754c83c640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f75477fe640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 -- 172.21.15.16:0/1302866304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 msgr2=0x5634056ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f7547fff640 1 -- 172.21.15.16:0/1302866304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634056ac2b0 con 0x5634056abb60 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f754c83c640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.302+0000 7f75477fe640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7547fff640 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7538009930 tx=0x7f7538005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7de996a6edd9843 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f752dffb640 1 -- 172.21.15.16:0/1302866304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538040050 con 0x5634056abb60 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f752dffb640 1 -- 172.21.15.16:0/1302866304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75380049a0 con 0x5634056abb60 2024-09-12T00:58:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f752dffb640 1 -- 172.21.15.16:0/1302866304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538035040 con 0x5634056abb60 2024-09-12T00:58:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 -- 172.21.15.16:0/1302866304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 msgr2=0x5634056b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7538009930 tx=0x7f7538005fd0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 -- 172.21.15.16:0/1302866304 shutdown_connections 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 --2- 172.21.15.16:0/1302866304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056b5a40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 -- 172.21.15.16:0/1302866304 >> 172.21.15.16:0/1302866304 conn(0x5634056a80f0 msgr2=0x5634056b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 -- 172.21.15.16:0/1302866304 shutdown_connections 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.303+0000 7f7550678740 1 -- 172.21.15.16:0/1302866304 wait complete. 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 -- start start 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 Processor -- start 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75481aed50 con 0x5634056abb60 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75481ae680 con 0x5634056b6d40 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75481ab8c0 con 0x5634056ac480 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7550678740 10 client.0 _ll_register_callbacks cb 0x7f753c0048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7547fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46814/0 (socket says 172.21.15.16:46814) 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f7547fff640 1 -- 172.21.15.16:0/3942022466 learned_addr learned my addr 172.21.15.16:0/3942022466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.304+0000 7f75477fe640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f754c83c640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 -- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 msgr2=0x5634056bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 -- 172.21.15.16:0/3942022466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 msgr2=0x5634056bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 -- 172.21.15.16:0/3942022466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563405730650 con 0x5634056abb60 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f754c83c640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7547fff640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f75380092a0 tx=0x7f7538005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d9a28fdd1910f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f75277fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753805c060 con 0x5634056abb60 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7538004450 con 0x5634056abb60 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75380049a0 con 0x5634056abb60 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7538040070 con 0x5634056abb60 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7550678740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:35.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.305+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634057318a0 con 0x5634056abb60 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75380049a0 con 0x5634056abb60 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f753804e450 con 0x5634056abb60 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 20 client.16080 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563405731b20 0x563405733f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 4 client.16080 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563405618d00 con 0x563405731b20 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.306+0000 7f75477fe640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563405731b20 0x563405733f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.307+0000 7f75277fe640 10 client.16080 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.307+0000 7f75477fe640 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563405731b20 0x563405733f10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cbeca8850e03e3 server_cookie=533d3346e0ef8381 in_seq=0 out_seq=0 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f75277fe640 1 -- 172.21.15.16:0/3942022466 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563405618d00 con 0x563405731b20 2024-09-12T00:58:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f75277fe640 10 client.16080 handle_command_reply: tid=0 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 client.16080 shutdown 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 msgr2=0x5634056bac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f75380092a0 tx=0x7f7538005c00 comp rx=0 tx=0).stop 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 shutdown_connections 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563405731b20 0x563405733f10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634056b6d40 0x5634056bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5634056ac480 0x5634056bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 --2- 172.21.15.16:0/3942022466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634056abb60 0x5634056bac10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 >> 172.21.15.16:0/3942022466 conn(0x5634056a80f0 msgr2=0x5634056a7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f75277fe640 0 client.16080 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 shutdown_connections 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 1 -- 172.21.15.16:0/3942022466 wait complete. 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.308+0000 7f7550678740 20 client.16080 trim_cache size 0 max 16384 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "93e760be-da8c-4bed-8559-585d1b0dc159", 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:58:35.312 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 msgr2=0x7f75300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 0x7f75300a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7540013b30 tx=0x7f7540005c60 comp rx=0 tx=0).stop 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 msgr2=0x7f7548198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c970 tx=0x7f753c00ce30 comp rx=0 tx=0).stop 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 shutdown_connections 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75300a2cf0 0x7f75300a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7548106f50 0x7f754819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7548100d00 0x7f7548198ab0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 --2- 172.21.15.16:0/1600357292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7548100350 0x7f7548198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 >> 172.21.15.16:0/1600357292 conn(0x7f7548077210 msgr2=0x7f7548105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 shutdown_connections 2024-09-12T00:58:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:35.309+0000 7f754e29f640 1 -- 172.21.15.16:0/1600357292 wait complete. 2024-09-12T00:58:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:35 smithi159 ceph-mon[35566]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:35.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:35 smithi099 ceph-mon[35654]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-12T00:58:36.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:58:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:36.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:36 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:58:36.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:36 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:36 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:58:36.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:36 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:36 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:58:36.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:36 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:36 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:58:36.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:36 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:36 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:58:36.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:36 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:36 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:58:36.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:36 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 -- 172.21.15.16:0/625174298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e41036e0 msgr2=0x7fa2e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 --2- 172.21.15.16:0/625174298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e41036e0 0x7fa2e4107b90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009930 tx=0x7fa2d802a420 comp rx=0 tx=0).stop 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 -- 172.21.15.16:0/625174298 shutdown_connections 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 --2- 172.21.15.16:0/625174298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41082a0 0x7fa2e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 --2- 172.21.15.16:0/625174298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e41036e0 0x7fa2e4107b90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 --2- 172.21.15.16:0/625174298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e4102d30 0x7fa2e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 -- 172.21.15.16:0/625174298 >> 172.21.15.16:0/625174298 conn(0x7fa2e40fc7d0 msgr2=0x7fa2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 -- 172.21.15.16:0/625174298 shutdown_connections 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 -- 172.21.15.16:0/625174298 wait complete. 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.864+0000 7fa2e957b640 1 Processor -- start 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 -- start start 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41036e0 0x7fa2e4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e41082a0 0x7fa2e419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41120c0 con 0x7fa2e4102d30 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4111f40 con 0x7fa2e41036e0 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e957b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4112240 con 0x7fa2e41082a0 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46828/0 (socket says 172.21.15.16:46828) 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/1195472311 learned_addr learned my addr 172.21.15.16:0/1195472311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41036e0 0x7fa2e4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e41082a0 0x7fa2e419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/1195472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e41082a0 msgr2=0x7fa2e419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e41082a0 0x7fa2e419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/1195472311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41036e0 msgr2=0x7fa2e4198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41036e0 0x7fa2e4198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.865+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/1195472311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e419d5f0 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400ba50 tx=0x7fa2d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2cbfff640 1 -- 172.21.15.16:0/1195472311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4019070 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2cbfff640 1 -- 172.21.15.16:0/1195472311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d400cbe0 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e419d8e0 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2cbfff640 1 -- 172.21.15.16:0/1195472311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40076e0 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.866+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e41a51c0 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.868+0000 7fa2cbfff640 1 -- 172.21.15.16:0/1195472311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2d401c070 con 0x7fa2e4102d30 2024-09-12T00:58:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.868+0000 7fa2eb954740 1 Processor -- start 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 -- start start 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c646780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c657e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56312c5ac2e0 con 0x56312c646cc0 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56312c553ce0 con 0x56312c6463a0 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2eb954740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56312c5a82a0 con 0x56312c655a90 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46836/0 (socket says 172.21.15.16:46836) 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3685500719 learned_addr learned my addr 172.21.15.16:0/3685500719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c646780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2cbfff640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 0x7fa2cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c657e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 0x7fa2cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2cbfff640 1 -- 172.21.15.16:0/1195472311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa2d40ce970 con 0x7fa2e4102d30 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3685500719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 msgr2=0x56312c657e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c657e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3685500719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 msgr2=0x56312c646780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c646780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3685500719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312c538440 con 0x56312c646cc0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c657e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.869+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c646780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa2d409ad20 tx=0x7fa2d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab3a361e48a266f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2c97fa640 1 -- 172.21.15.16:0/3685500719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40bb070 con 0x56312c646cc0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 0x7fa2cc0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005d00 tx=0x7fa2dc004730 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2c97fa640 1 -- 172.21.15.16:0/3685500719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d409b020 con 0x56312c646cc0 2024-09-12T00:58:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2c97fa640 1 -- 172.21.15.16:0/3685500719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40a3750 con 0x56312c646cc0 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2eb954740 1 -- 172.21.15.16:0/3685500719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 msgr2=0x56312c655550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.870+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa2d409ad20 tx=0x7fa2d4004a40 comp rx=0 tx=0).stop 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- 172.21.15.16:0/3685500719 shutdown_connections 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c657e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c655550 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3685500719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c646780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- 172.21.15.16:0/3685500719 >> 172.21.15.16:0/3685500719 conn(0x56312c642a30 msgr2=0x56312c642e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- 172.21.15.16:0/3685500719 shutdown_connections 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- 172.21.15.16:0/3685500719 wait complete. 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- start start 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 Processor -- start 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c655420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c652c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56312c658660 con 0x56312c646cc0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56312c567050 con 0x56312c6463a0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56312c6587e0 con 0x56312c655a90 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2eb954740 10 client.0 _ll_register_callbacks cb 0x56312c5a7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2e27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46846/0 (socket says 172.21.15.16:46846) 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2e27fc640 1 -- 172.21.15.16:0/445818317 learned_addr learned my addr 172.21.15.16:0/445818317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.871+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c652c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c655420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 -- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 msgr2=0x56312c652c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c652c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 -- 172.21.15.16:0/445818317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 msgr2=0x56312c655420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c655420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 -- 172.21.15.16:0/445818317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312c31d3c0 con 0x56312c646cc0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c655420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009900 tx=0x7fa2d8032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cecef343ae57269 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8035040 con 0x56312c646cc0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa2d8004040 con 0x56312c646cc0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d409b020 con 0x56312c646cc0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2d80319f0 con 0x56312c646cc0 2024-09-12T00:58:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2eb954740 10 client.16092 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.872+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56312c664090 con 0x56312c646cc0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d409b020 con 0x56312c646cc0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2d8004a40 con 0x56312c646cc0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 10 client.16092 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 20 client.16092 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c665550 0x56312c667990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 4 client.16092 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56312c5b9850 con 0x56312c665550 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c665550 0x56312c667990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2c6ff5640 10 client.16092 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.873+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c665550 0x56312c667990 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9b533cea80a36e server_cookie=e86d3f4096f60ab1 in_seq=0 out_seq=0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/445818317 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56312c5b9850 con 0x56312c665550 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2c6ff5640 10 client.16092 handle_command_reply: tid=0 2024-09-12T00:58:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 client.16092 shutdown 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 msgr2=0x56312c6526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009900 tx=0x7fa2d8032f50 comp rx=0 tx=0).stop 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 shutdown_connections 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c665550 0x56312c667990 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c655a90 0x56312c652c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c646cc0 0x56312c6526d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 --2- 172.21.15.16:0/445818317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c655420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2c6ff5640 0 client.16092 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.874+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 >> 172.21.15.16:0/445818317 conn(0x56312c642a30 msgr2=0x56312c648740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.875+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 shutdown_connections 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.875+0000 7fa2eb954740 1 -- 172.21.15.16:0/445818317 wait complete. 2024-09-12T00:58:36.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.875+0000 7fa2eb954740 20 client.16092 trim_cache size 0 max 16384 2024-09-12T00:58:36.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 Processor -- start 2024-09-12T00:58:36.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 -- start start 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c66d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 0x56312c66d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41ab770 con 0x56312c6463a0 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41ad2b0 con 0x56312c655a90 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2eb954740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41abe40 con 0x56312c646cc0 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46856/0 (socket says 172.21.15.16:46856) 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3796740796 learned_addr learned my addr 172.21.15.16:0/3796740796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c66d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 0x56312c66d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3796740796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 msgr2=0x56312c66d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c66d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3796740796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 msgr2=0x56312c66d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 0x56312c66d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e2ffd640 1 -- 172.21.15.16:0/3796740796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312c657e80 con 0x56312c6463a0 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 0x56312c66d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.892+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c66d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4097430 tx=0x7fa2d40973c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c286e97f45cb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2c8ff9640 1 -- 172.21.15.16:0/3796740796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4097710 con 0x56312c6463a0 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2c8ff9640 1 -- 172.21.15.16:0/3796740796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d40bb040 con 0x56312c6463a0 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2c8ff9640 1 -- 172.21.15.16:0/3796740796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40cc280 con 0x56312c6463a0 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2eb954740 1 -- 172.21.15.16:0/3796740796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 msgr2=0x56312c661900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.893+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4097430 tx=0x7fa2d40973c0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- 172.21.15.16:0/3796740796 shutdown_connections 2024-09-12T00:58:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c655a90 0x56312c66d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c66d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3796740796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c6463a0 0x56312c661900 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- 172.21.15.16:0/3796740796 >> 172.21.15.16:0/3796740796 conn(0x56312c642a30 msgr2=0x56312c654750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- 172.21.15.16:0/3796740796 shutdown_connections 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- 172.21.15.16:0/3796740796 wait complete. 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- start start 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 Processor -- start 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c65e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c659650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41acbe0 con 0x56312c655a90 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41ac510 con 0x56312c6463a0 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41a9750 con 0x56312c646cc0 2024-09-12T00:58:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.894+0000 7fa2eb954740 10 client.0 _ll_register_callbacks cb 0x7fa2d8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46866/0 (socket says 172.21.15.16:46866) 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 -- 172.21.15.16:0/3023150925 learned_addr learned my addr 172.21.15.16:0/3023150925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e27fc640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c659650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c65e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 -- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 msgr2=0x56312c659650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c659650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 -- 172.21.15.16:0/3023150925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 msgr2=0x56312c65e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c65e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 -- 172.21.15.16:0/3023150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56312c65a470 con 0x56312c655a90 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2e37fe640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc012700 tx=0x7fa2dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8604476fedcd4b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc005220 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa2dc0053c0 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d40bb040 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.895+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc0055c0 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56312c6d0320 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d40bb040 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2dc029030 con 0x56312c655a90 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 20 client.16098 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c6d0a70 0x56312c6d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 4 client.16098 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56312c5b3d60 con 0x56312c6d0a70 2024-09-12T00:58:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.896+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c6d0a70 0x56312c6d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2e2ffd640 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c6d0a70 0x56312c6d2e90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972f55a8ec88701 server_cookie=d5c71fd230db561f in_seq=0 out_seq=0 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2c6ff5640 10 client.16098 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2c6ff5640 1 -- 172.21.15.16:0/3023150925 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56312c5b3d60 con 0x56312c6d0a70 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2c6ff5640 10 client.16098 handle_command_reply: tid=0 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 client.16098 shutdown 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 msgr2=0x56312c659c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc012700 tx=0x7fa2dc0046c0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 shutdown_connections 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56312c6d0a70 0x56312c6d2e90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56312c655a90 0x56312c659c40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2c6ff5640 0 client.16098 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56312c646cc0 0x56312c659650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 --2- 172.21.15.16:0/3023150925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56312c6463a0 0x56312c65e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 >> 172.21.15.16:0/3023150925 conn(0x56312c642a30 msgr2=0x56312c653f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 shutdown_connections 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 1 -- 172.21.15.16:0/3023150925 wait complete. 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.897+0000 7fa2eb954740 20 client.16098 trim_cache size 0 max 16384 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "93e760be-da8c-4bed-8559-585d1b0dc159": { 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "93e760be-da8c-4bed-8559-585d1b0dc159", 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T00:58:36.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:36.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:36.904 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:36.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 msgr2=0x7fa2cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 0x7fa2cc0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005d00 tx=0x7fa2dc004730 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 msgr2=0x7fa2e4198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400ba50 tx=0x7fa2d400bf10 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 shutdown_connections 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2cc0a2cf0 0x7fa2cc0a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2e41082a0 0x7fa2e419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2e41036e0 0x7fa2e4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 --2- 172.21.15.16:0/1195472311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e4102d30 0x7fa2e4198640 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 >> 172.21.15.16:0/1195472311 conn(0x7fa2e40fc7d0 msgr2=0x7fa2e4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 shutdown_connections 2024-09-12T00:58:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:36.898+0000 7fa2e957b640 1 -- 172.21.15.16:0/1195472311 wait complete. 2024-09-12T00:58:37.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:37 smithi016 ceph-mon[30972]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-12T00:58:37.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:37 smithi099 ceph-mon[35654]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-12T00:58:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:37 smithi159 ceph-mon[35566]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-12T00:58:37.694 INFO:tasks.cephfs.filesystem:scrub status for tag:93e760be-da8c-4bed-8559-585d1b0dc159 - {'path': '/', 'tag': '93e760be-da8c-4bed-8559-585d1b0dc159', 'options': 'recursive,force'} 2024-09-12T00:58:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:39 smithi016 ceph-mon[30972]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T00:58:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:39 smithi099 ceph-mon[35654]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T00:58:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:39 smithi159 ceph-mon[35566]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-12T00:58:40.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:40 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:58:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:40 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:58:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:40 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:58:41.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:41 smithi016 ceph-mon[30972]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-09-12T00:58:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:41 smithi099 ceph-mon[35654]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-09-12T00:58:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:41 smithi159 ceph-mon[35566]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 19 op/s 2024-09-12T00:58:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:43 smithi016 ceph-mon[30972]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:43 smithi099 ceph-mon[35654]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:43 smithi159 ceph-mon[35566]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:45 smithi016 ceph-mon[30972]: pgmap v83: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:45 smithi099 ceph-mon[35654]: pgmap v83: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:45.593 INFO:tasks.workunit.client.0.smithi016.stdout:1/600: open fd = 3 2024-09-12T00:58:45.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:2/600: open fd = 3 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:3/600: open fd = 3 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:4/600: open fd = 3 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:5/600: open fd = 3 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:6/600: open fd = 3 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:7/600: open fd = 3 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:8/600: open fd = 3 2024-09-12T00:58:45.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:9/600: open fd = 3 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:10/600: open fd = 3 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:11/600: open fd = 3 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:12/600: open fd = 3 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:13/600: open fd = 3 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:14/600: open fd = 3 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.597 INFO:tasks.workunit.client.0.smithi016.stdout:15/600: open fd = 3 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:16/600: open fd = 3 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:17/600: open fd = 3 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:18/600: open fd = 3 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.598 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:19/600: open fd = 3 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:20/600: open fd = 3 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:21/600: open fd = 3 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:22/600: open fd = 3 2024-09-12T00:58:45.599 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:23/600: open fd = 3 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:24/600: open fd = 3 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:25/600: open fd = 3 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.600 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:26/600: open fd = 3 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:27/600: open fd = 3 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:28/600: open fd = 3 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:29/600: open fd = 3 2024-09-12T00:58:45.601 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:30/600: open fd = 3 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:31/600: open fd = 3 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:32/600: open fd = 3 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.602 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:33/600: open fd = 3 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:34/600: open fd = 3 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:35/600: open fd = 3 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:36/600: open fd = 3 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.603 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:37/600: open fd = 3 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:38/600: open fd = 3 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:39/600: open fd = 3 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.604 INFO:tasks.workunit.client.0.smithi016.stdout:40/600: open fd = 3 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:41/600: open fd = 3 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:42/600: open fd = 3 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:43/600: open fd = 3 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.605 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:44/600: open fd = 3 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:45/600: open fd = 3 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:46/600: open fd = 3 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.606 INFO:tasks.workunit.client.0.smithi016.stdout:47/600: open fd = 3 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:48/600: open fd = 3 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:49/600: open fd = 3 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:50/600: open fd = 3 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.607 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:51/600: open fd = 3 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:52/600: open fd = 3 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:53/600: open fd = 3 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.608 INFO:tasks.workunit.client.0.smithi016.stdout:54/600: open fd = 3 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:55/600: open fd = 3 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:56/600: open fd = 3 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:57/600: open fd = 3 2024-09-12T00:58:45.609 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:58/600: open fd = 3 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:59/600: open fd = 3 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:60/600: open fd = 3 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.610 INFO:tasks.workunit.client.0.smithi016.stdout:61/600: open fd = 3 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:62/600: open fd = 3 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:63/600: open fd = 3 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T00:58:45.611 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T00:58:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:45 smithi159 ceph-mon[35566]: pgmap v83: 97 pgs: 97 active+clean; 2.3 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T00:58:47.018 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T00:58:47.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:58:47.291 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:47.392 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:47 smithi016 ceph-mon[30972]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:47.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:47 smithi099 ceph-mon[35654]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:47.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:47 smithi159 ceph-mon[35566]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 -- 172.21.15.16:0/3471093370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68101de0 msgr2=0x7fed681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 --2- 172.21.15.16:0/3471093370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68101de0 0x7fed681021c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fed64009900 tx=0x7fed6402a410 comp rx=0 tx=0).stop 2024-09-12T00:58:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 -- 172.21.15.16:0/3471093370 shutdown_connections 2024-09-12T00:58:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 --2- 172.21.15.16:0/3471093370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 --2- 172.21.15.16:0/3471093370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68102790 0x7fed681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 --2- 172.21.15.16:0/3471093370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68101de0 0x7fed681021c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 -- 172.21.15.16:0/3471093370 >> 172.21.15.16:0/3471093370 conn(0x7fed680fd8c0 msgr2=0x7fed680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 -- 172.21.15.16:0/3471093370 shutdown_connections 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.793+0000 7fed6f760640 1 -- 172.21.15.16:0/3471093370 wait complete. 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 Processor -- start 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 -- start start 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 0x7fed68198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111f70 con 0x7fed68102790 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed68111df0 con 0x7fed68101de0 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6f760640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed681120f0 con 0x7fed68106f10 2024-09-12T00:58:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6ccfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6ccfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33632/0 (socket says 172.21.15.16:33632) 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6ccfb640 1 -- 172.21.15.16:0/398298944 learned_addr learned my addr 172.21.15.16:0/398298944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6dcfd640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6d4fc640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 0x7fed68198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.794+0000 7fed6ccfb640 1 -- 172.21.15.16:0/398298944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 msgr2=0x7fed6819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6ccfb640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6ccfb640 1 -- 172.21.15.16:0/398298944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 msgr2=0x7fed68198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6ccfb640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 0x7fed68198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6ccfb640 1 -- 172.21.15.16:0/398298944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6819d2b0 con 0x7fed68102790 2024-09-12T00:58:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6d4fc640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 0x7fed68198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6dcfd640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6ccfb640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c920 tx=0x7fed5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58007c70 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6819d5a0 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed58007e10 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58018890 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.795+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed681a4e80 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.796+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed681021c0 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.797+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fed58018a70 con 0x7fed68102790 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.798+0000 7fed5e7fc640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 0x7fed540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.798+0000 7fed6d4fc640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 0x7fed540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.798+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fed580cf620 con 0x7fed68102790 2024-09-12T00:58:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.798+0000 7fed6d4fc640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 0x7fed540a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed64032d50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:47.802+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed58014030 con 0x7fed68102790 2024-09-12T00:58:48.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.017+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed6802d7c0 con 0x7fed68102790 2024-09-12T00:58:48.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.018+0000 7fed5e7fc640 1 -- 172.21.15.16:0/398298944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fed5800de20 con 0x7fed68102790 2024-09-12T00:58:48.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:58:48.017 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:58:48.017 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 msgr2=0x7fed540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 0x7fed540a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed64032d50 comp rx=0 tx=0).stop 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 msgr2=0x7fed68198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c920 tx=0x7fed5800cde0 comp rx=0 tx=0).stop 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 shutdown_connections 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed540a2cf0 0x7fed540a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed68106f10 0x7fed6819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:48.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed68102790 0x7fed68198840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:48.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 --2- 172.21.15.16:0/398298944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed68101de0 0x7fed68198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:48.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 >> 172.21.15.16:0/398298944 conn(0x7fed680fd8c0 msgr2=0x7fed680fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:48.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.021+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 shutdown_connections 2024-09-12T00:58:48.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:48.022+0000 7fed6f760640 1 -- 172.21.15.16:0/398298944 wait complete. 2024-09-12T00:58:48.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:48 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/398298944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:48.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:48 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/398298944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:48.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:48 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/398298944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:58:48.835 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T00:58:48.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T00:58:49.096 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:58:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:49 smithi099 ceph-mon[35654]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T00:58:49.545 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:49 smithi016 ceph-mon[30972]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T00:58:49.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:49 smithi159 ceph-mon[35566]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 -- 172.21.15.16:0/3913911752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 msgr2=0x7ff0141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 --2- 172.21.15.16:0/3913911752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff0141067e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff004009900 tx=0x7ff00402a410 comp rx=0 tx=0).stop 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 -- 172.21.15.16:0/3913911752 shutdown_connections 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 --2- 172.21.15.16:0/3913911752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014106f10 0x7ff01410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 --2- 172.21.15.16:0/3913911752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff0141067e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 --2- 172.21.15.16:0/3913911752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014101de0 0x7ff0141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 -- 172.21.15.16:0/3913911752 >> 172.21.15.16:0/3913911752 conn(0x7ff0140fd8c0 msgr2=0x7ff0140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 -- 172.21.15.16:0/3913911752 shutdown_connections 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.638+0000 7ff01b158640 1 -- 172.21.15.16:0/3913911752 wait complete. 2024-09-12T00:58:49.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 Processor -- start 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 -- start start 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 0x7ff014198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014106f10 0x7ff01419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff014111f70 con 0x7ff014102790 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff014111df0 con 0x7ff014101de0 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.639+0000 7ff01b158640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141120f0 con 0x7ff014106f10 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48988/0 (socket says 172.21.15.16:48988) 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 -- 172.21.15.16:0/4067012065 learned_addr learned my addr 172.21.15.16:0/4067012065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff0196f5640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014106f10 0x7ff01419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff018ef4640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 0x7ff014198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 -- 172.21.15.16:0/4067012065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014106f10 msgr2=0x7ff01419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014106f10 0x7ff01419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 -- 172.21.15.16:0/4067012065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 msgr2=0x7ff014198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 0x7ff014198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 -- 172.21.15.16:0/4067012065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01419d4c0 con 0x7ff014102790 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff018ef4640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 0x7ff014198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff00bfff640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff004006010 tx=0x7ff0040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff009ffb640 1 -- 172.21.15.16:0/4067012065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00403f030 con 0x7ff014102790 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01419d750 con 0x7ff014102790 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff009ffb640 1 -- 172.21.15.16:0/4067012065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff004003ce0 con 0x7ff014102790 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.640+0000 7ff009ffb640 1 -- 172.21.15.16:0/4067012065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004035070 con 0x7ff014102790 2024-09-12T00:58:49.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.641+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a5090 con 0x7ff014102790 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff009ffb640 1 -- 172.21.15.16:0/4067012065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff004042030 con 0x7ff014102790 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 Processor -- start 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 -- start start 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641f898e2a0 con 0x5641f8a2cb70 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f8935b10 con 0x5641f8a37d50 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff01d531740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641f898a260 con 0x5641f8a2d490 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff0196f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff0196f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49000/0 (socket says 172.21.15.16:49000) 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.643+0000 7ff0196f5640 1 -- 172.21.15.16:0/1854797438 learned_addr learned my addr 172.21.15.16:0/1854797438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff018ef4640 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff00bfff640 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 -- 172.21.15.16:0/1854797438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 msgr2=0x5641f8a37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 -- 172.21.15.16:0/1854797438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 msgr2=0x5641f8a3a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 -- 172.21.15.16:0/1854797438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f86ffb90 con 0x5641f8a2cb70 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff009ffb640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 0x7feffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff0196f5640 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ae10 tx=0x7ff00c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834647f0e95da71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff018ef4640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 0x7feffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7ff009ffb640 1 -- 172.21.15.16:0/4067012065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff0040efa50 con 0x7ff014102790 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7feffaffd640 1 -- 172.21.15.16:0/1854797438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c003b70 con 0x5641f8a2cb70 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7feffaffd640 1 -- 172.21.15.16:0/1854797438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff00c003d10 con 0x5641f8a2cb70 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.644+0000 7feffaffd640 1 -- 172.21.15.16:0/1854797438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c010b00 con 0x5641f8a2cb70 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- 172.21.15.16:0/1854797438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 msgr2=0x5641f8a2cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00ae10 tx=0x7ff00c00f790 comp rx=0 tx=0).stop 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff018ef4640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 0x7feffc0a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5641f8a29f60 tx=0x7ff01000b5a0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- 172.21.15.16:0/1854797438 shutdown_connections 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 --2- 172.21.15.16:0/1854797438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a2cf50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- 172.21.15.16:0/1854797438 >> 172.21.15.16:0/1854797438 conn(0x5641f8a29100 msgr2=0x5641f8a294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- 172.21.15.16:0/1854797438 shutdown_connections 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- 172.21.15.16:0/1854797438 wait complete. 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 -- start start 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.645+0000 7ff01d531740 1 Processor -- start 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a2cb70 0x5641f8a34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a37d50 0x5641f8a356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641f8a3a920 con 0x5641f8a2d490 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f89166e0 con 0x5641f8a2cb70 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641f8a3aaa0 con 0x5641f8a37d50 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff01d531740 10 client.0 _ll_register_callbacks cb 0x5641f893aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49002/0 (socket says 172.21.15.16:49002) 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 -- 172.21.15.16:0/3129474051 learned_addr learned my addr 172.21.15.16:0/3129474051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff0196f5640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a37d50 0x5641f8a356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff018ef4640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a2cb70 0x5641f8a34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 -- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a37d50 msgr2=0x5641f8a356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a37d50 0x5641f8a356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 -- 172.21.15.16:0/3129474051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a2cb70 msgr2=0x5641f8a34c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a2cb70 0x5641f8a34c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 -- 172.21.15.16:0/3129474051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f8a43e10 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7ff00bfff640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff004004650 tx=0x7ff004003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eceac20a9ecec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.646+0000 7feff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004049070 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff0040dabe0 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00c003d10 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0040b8300 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641f8a467a0 con 0x5641f8a2d490 2024-09-12T00:58:49.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff00c003d10 con 0x5641f8a2d490 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0040bc2c0 con 0x5641f8a2d490 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 10 client.16116 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 20 client.16116 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8a47b90 0x5641f8a49f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 4 client.16116 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f899b9d0 con 0x5641f8a47b90 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.647+0000 7ff018ef4640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8a47b90 0x5641f8a49f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.648+0000 7ff018ef4640 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8a47b90 0x5641f8a49f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495d82f30f0b8518 server_cookie=73c362e872194fb2 in_seq=0 out_seq=0 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.648+0000 7feff3fff640 10 client.16116 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:49.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.648+0000 7feff3fff640 1 -- 172.21.15.16:0/3129474051 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641f899b9d0 con 0x5641f8a47b90 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7feff3fff640 10 client.16116 handle_command_reply: tid=0 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 client.16116 shutdown 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 msgr2=0x5641f8a35180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff004004650 tx=0x7ff004003e90 comp rx=0 tx=0).stop 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 shutdown_connections 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8a47b90 0x5641f8a49f80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a37d50 0x5641f8a356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2d490 0x5641f8a35180 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7feff3fff640 0 client.16116 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 --2- 172.21.15.16:0/3129474051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a2cb70 0x5641f8a34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 >> 172.21.15.16:0/3129474051 conn(0x5641f8a29100 msgr2=0x5641f8a23e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 shutdown_connections 2024-09-12T00:58:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 1 -- 172.21.15.16:0/3129474051 wait complete. 2024-09-12T00:58:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.649+0000 7ff01d531740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 Processor -- start 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 -- start start 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a33510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a2f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ab6e0 con 0x5641f8a2cb70 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ad220 con 0x5641f8a37d50 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff01d531740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141abdb0 con 0x5641f8a2d490 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff018ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff018ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49008/0 (socket says 172.21.15.16:49008) 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.666+0000 7ff018ef4640 1 -- 172.21.15.16:0/1812262224 learned_addr learned my addr 172.21.15.16:0/1812262224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff0196f5640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a2f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff00bfff640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a33510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 -- 172.21.15.16:0/1812262224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 msgr2=0x5641f8a33510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a33510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 -- 172.21.15.16:0/1812262224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 msgr2=0x5641f8a2f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a2f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 -- 172.21.15.16:0/1812262224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f8a2c9a0 con 0x5641f8a2cb70 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff00bfff640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7ff018ef4640 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff010010500 tx=0x7ff010015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06153d0aa089d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7feffa7fc640 1 -- 172.21.15.16:0/1812262224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010005220 con 0x5641f8a2cb70 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7feffa7fc640 1 -- 172.21.15.16:0/1812262224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff010004c80 con 0x5641f8a2cb70 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.667+0000 7feffa7fc640 1 -- 172.21.15.16:0/1812262224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100288f0 con 0x5641f8a2cb70 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- 172.21.15.16:0/1812262224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 msgr2=0x5641f8a36b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff010010500 tx=0x7ff010015000 comp rx=0 tx=0).stop 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- 172.21.15.16:0/1812262224 shutdown_connections 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a2f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 --2- 172.21.15.16:0/1812262224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a36b10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- 172.21.15.16:0/1812262224 >> 172.21.15.16:0/1812262224 conn(0x5641f8a29100 msgr2=0x5641f8a23e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- 172.21.15.16:0/1812262224 shutdown_connections 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- 172.21.15.16:0/1812262224 wait complete. 2024-09-12T00:58:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 -- start start 2024-09-12T00:58:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.668+0000 7ff01d531740 1 Processor -- start 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141acb50 con 0x5641f8a2cb70 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ac480 con 0x5641f8a37d50 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a96c0 con 0x5641f8a2d490 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff01d531740 10 client.0 _ll_register_callbacks cb 0x7ff0040044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49010/0 (socket says 172.21.15.16:49010) 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 -- 172.21.15.16:0/2618301022 learned_addr learned my addr 172.21.15.16:0/2618301022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff00bfff640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff0196f5640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 -- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 msgr2=0x5641f8a40230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a40230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 -- 172.21.15.16:0/2618301022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 msgr2=0x5641f8a3ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 -- 172.21.15.16:0/2618301022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f8a20260 con 0x5641f8a2cb70 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff0196f5640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7ff018ef4640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b930 tx=0x7ff010004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87327ca56a630e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:58:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.669+0000 7feff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010005200 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff0100053a0 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff010004c80 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff010004a90 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641f8ab2d80 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff010004c80 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff010038020 con 0x5641f8a2cb70 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 10 client.16122 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:58:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 20 client.16122 populate_metadata read hostname 'smithi016' 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8ab3490 0x5641f8ab58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 4 client.16122 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.670+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x5641f8995d20 con 0x5641f8ab3490 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.671+0000 7ff00bfff640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8ab3490 0x5641f8ab58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.671+0000 7ff00bfff640 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8ab3490 0x5641f8ab58b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50ed0542ae69f50 server_cookie=d48a650741b7752c in_seq=0 out_seq=0 2024-09-12T00:58:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.671+0000 7feff3fff640 10 client.16122 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7feff3fff640 1 -- 172.21.15.16:0/2618301022 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5641f8995d20 con 0x5641f8ab3490 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7feff3fff640 10 client.16122 handle_command_reply: tid=0 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 client.16122 shutdown 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 msgr2=0x5641f8a3fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff01000b930 tx=0x7ff010004e30 comp rx=0 tx=0).stop 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 shutdown_connections 2024-09-12T00:58:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5641f8ab3490 0x5641f8ab58b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f8a37d50 0x5641f8a3ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f8a2d490 0x5641f8a40230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 --2- 172.21.15.16:0/2618301022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f8a2cb70 0x5641f8a3fcf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7feff3fff640 0 client.16122 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 >> 172.21.15.16:0/2618301022 conn(0x5641f8a29100 msgr2=0x5641f8a23e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 shutdown_connections 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.684+0000 7ff01d531740 1 -- 172.21.15.16:0/2618301022 wait complete. 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01d531740 20 client.16122 trim_cache size 0 max 16384 2024-09-12T00:58:49.684 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 10, 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "5605cf54": { 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 10, 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:58:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.7, 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:49.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:49.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:58:49.687 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:58:49.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 msgr2=0x7feffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 0x7feffc0a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5641f8a29f60 tx=0x7ff01000b5a0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 msgr2=0x7ff014198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff004006010 tx=0x7ff0040042f0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 shutdown_connections 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feffc0a2cf0 0x7feffc0a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff014106f10 0x7ff01419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff014102790 0x7ff014198a50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 --2- 172.21.15.16:0/4067012065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff014101de0 0x7ff014198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 >> 172.21.15.16:0/4067012065 conn(0x7ff0140fd8c0 msgr2=0x7ff0140fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 shutdown_connections 2024-09-12T00:58:49.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:58:49.685+0000 7ff01b158640 1 -- 172.21.15.16:0/4067012065 wait complete. 2024-09-12T00:58:50.415 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5605cf54', quiesce took 3.4 seconds. Will release after: 21.3 2024-09-12T00:58:51.418 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:51 smithi159 ceph-mon[35566]: pgmap v86: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:51.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:51 smithi099 ceph-mon[35654]: pgmap v86: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:51 smithi016 ceph-mon[30972]: pgmap v86: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 19 op/s 2024-09-12T00:58:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:53 smithi099 ceph-mon[35654]: pgmap v87: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:53 smithi159 ceph-mon[35566]: pgmap v87: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:53 smithi016 ceph-mon[30972]: pgmap v87: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:55 smithi099 ceph-mon[35654]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:55.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:55 smithi159 ceph-mon[35566]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:55 smithi016 ceph-mon[30972]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-09-12T00:58:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:57 smithi099 ceph-mon[35654]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T00:58:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:57 smithi159 ceph-mon[35566]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T00:58:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:57 smithi016 ceph-mon[30972]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-09-12T00:58:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:58:59 smithi099 ceph-mon[35654]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T00:58:59.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:58:59 smithi159 ceph-mon[35566]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T00:58:59.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:58:59 smithi016 ceph-mon[30972]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-12T00:59:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:01 smithi099 ceph-mon[35654]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-12T00:59:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:01 smithi159 ceph-mon[35566]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-12T00:59:01.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:01 smithi016 ceph-mon[30972]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-12T00:59:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:03 smithi099 ceph-mon[35654]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:03 smithi159 ceph-mon[35566]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:03.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:03 smithi016 ceph-mon[30972]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:05.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:05 smithi099 ceph-mon[35654]: pgmap v93: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:05 smithi159 ceph-mon[35566]: pgmap v93: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:05 smithi016 ceph-mon[30972]: pgmap v93: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-12T00:59:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:07 smithi099 ceph-mon[35654]: pgmap v94: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-12T00:59:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:07 smithi159 ceph-mon[35566]: pgmap v94: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-12T00:59:07.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:07 smithi016 ceph-mon[30972]: pgmap v94: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-12T00:59:07.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:59:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 -- 172.21.15.16:0/2109263521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe734103660 msgr2=0x7fe73410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 --2- 172.21.15.16:0/2109263521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe734103660 0x7fe73410bb50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe724009900 tx=0x7fe72402a480 comp rx=0 tx=0).stop 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 -- 172.21.15.16:0/2109263521 shutdown_connections 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 --2- 172.21.15.16:0/2109263521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73410c170 0x7fe73410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 --2- 172.21.15.16:0/2109263521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe734103660 0x7fe73410bb50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 --2- 172.21.15.16:0/2109263521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 0x7fe734103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 -- 172.21.15.16:0/2109263521 >> 172.21.15.16:0/2109263521 conn(0x7fe7340fc7a0 msgr2=0x7fe7340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 -- 172.21.15.16:0/2109263521 shutdown_connections 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.521+0000 7fe73c621640 1 -- 172.21.15.16:0/2109263521 wait complete. 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 Processor -- start 2024-09-12T00:59:08.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 -- start start 2024-09-12T00:59:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 0x7fe7341983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe734103660 0x7fe734198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe734111f70 con 0x7fe73410c170 2024-09-12T00:59:08.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe734111df0 con 0x7fe734102d40 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.522+0000 7fe73c621640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341120f0 con 0x7fe734103660 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73a3bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 0x7fe7341983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41158/0 (socket says 172.21.15.16:41158) 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 -- 172.21.15.16:0/2389331687 learned_addr learned my addr 172.21.15.16:0/2389331687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe739bbc640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe734103660 0x7fe734198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 -- 172.21.15.16:0/2389331687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe734103660 msgr2=0x7fe734198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe734103660 0x7fe734198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 -- 172.21.15.16:0/2389331687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 msgr2=0x7fe7341983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 0x7fe7341983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 -- 172.21.15.16:0/2389331687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73419d380 con 0x7fe73410c170 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73abbe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00eaa0 tx=0x7fe72c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe72b7fe640 1 -- 172.21.15.16:0/2389331687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c004d20 con 0x7fe73410c170 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe72b7fe640 1 -- 172.21.15.16:0/2389331687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c005020 con 0x7fe73410c170 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73419d670 con 0x7fe73410c170 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe72b7fe640 1 -- 172.21.15.16:0/2389331687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0116d0 con 0x7fe73410c170 2024-09-12T00:59:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.523+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7341a4f50 con 0x7fe73410c170 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe72b7fe640 1 -- 172.21.15.16:0/2389331687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe72c0051d0 con 0x7fe73410c170 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 Processor -- start 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 -- start start 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e55880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e60680 0x55d961e62a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe72b7fe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 0x7fe7200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d961dbb2e0 con 0x55d961e55dc0 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d961d62ce0 con 0x55d961e60680 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.526+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d961db72a0 con 0x55d961e554a0 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41162/0 (socket says 172.21.15.16:41162) 2024-09-12T00:59:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 -- 172.21.15.16:0/864269734 learned_addr learned my addr 172.21.15.16:0/864269734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73abbe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 0x7fe7200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe739bbc640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e55880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe739bbc640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e60680 0x55d961e62a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe72b7fe640 1 -- 172.21.15.16:0/2389331687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe72c0cf4e0 con 0x7fe73410c170 2024-09-12T00:59:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 -- 172.21.15.16:0/864269734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 msgr2=0x55d961e55880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e55880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 -- 172.21.15.16:0/864269734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e60680 msgr2=0x55d961e62a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e60680 0x55d961e62a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 -- 172.21.15.16:0/864269734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d961d47440 con 0x55d961e55dc0 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe739bbc640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e55880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe73000ad00 tx=0x7fe73000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9d39838fe19a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe73abbe640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 0x7fe7200a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0a4110 tx=0x7fe72c06ea40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe728ff9640 1 -- 172.21.15.16:0/864269734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73001d070 con 0x55d961e55dc0 2024-09-12T00:59:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe728ff9640 1 -- 172.21.15.16:0/864269734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe730018b20 con 0x55d961e55dc0 2024-09-12T00:59:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.527+0000 7fe728ff9640 1 -- 172.21.15.16:0/864269734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730010bc0 con 0x55d961e55dc0 2024-09-12T00:59:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 -- 172.21.15.16:0/864269734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 msgr2=0x55d961e60140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe73000ad00 tx=0x7fe73000f790 comp rx=0 tx=0).stop 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 -- 172.21.15.16:0/864269734 shutdown_connections 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e60680 0x55d961e62a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e60140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/864269734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e55880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 -- 172.21.15.16:0/864269734 >> 172.21.15.16:0/864269734 conn(0x55d961e51a30 msgr2=0x55d961e51e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 -- 172.21.15.16:0/864269734 shutdown_connections 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.528+0000 7fe73e9fa740 1 -- 172.21.15.16:0/864269734 wait complete. 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 -- start start 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 Processor -- start 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e5ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 0x55d961e5d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 0x55d961e5d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d961e63250 con 0x55d961e60680 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d961d76050 con 0x55d961e55dc0 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d961e633d0 con 0x55d961e554a0 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe73e9fa740 10 client.0 _ll_register_callbacks cb 0x55d961db6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 0x55d961e5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 0x55d961e5d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41164/0 (socket says 172.21.15.16:41164) 2024-09-12T00:59:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe739bbc640 1 -- 172.21.15.16:0/572159324 learned_addr learned my addr 172.21.15.16:0/572159324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.529+0000 7fe739bbc640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e5ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 0x55d961e5d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 -- 172.21.15.16:0/572159324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 msgr2=0x55d961e5d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 0x55d961e5d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 -- 172.21.15.16:0/572159324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 msgr2=0x55d961e5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 0x55d961e5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 -- 172.21.15.16:0/572159324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d961b2c3c0 con 0x55d961e554a0 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 0x55d961e5d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe739bbc640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e5ffa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe724005ec0 tx=0x7fe7240058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b287c27d93c73518 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe71dffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724035070 con 0x55d961e554a0 2024-09-12T00:59:08.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe730018b20 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe724004100 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe724004300 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe73e9fa740 10 client.25651 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.530+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d961e77430 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe730018b20 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe72404b450 con 0x55d961e554a0 2024-09-12T00:59:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 10 client.25651 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 20 client.25651 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e78840 0x55d961e7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 4 client.25651 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d961dc8850 con 0x55d961e78840 2024-09-12T00:59:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.531+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e78840 0x55d961e7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe71dffb640 10 client.25651 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e78840 0x55d961e7ac30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4996c81eedff080b server_cookie=71bc563e4094a67a in_seq=0 out_seq=0 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe71dffb640 1 -- 172.21.15.16:0/572159324 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d961dc8850 con 0x55d961e78840 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe71dffb640 10 client.25651 handle_command_reply: tid=0 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe73e9fa740 1 client.25651 shutdown 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 msgr2=0x55d961e5ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.532+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e5ffa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe724005ec0 tx=0x7fe7240058d0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 shutdown_connections 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e78840 0x55d961e7ac30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e60680 0x55d961e5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e55dc0 0x55d961e5d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/572159324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e5ffa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe71dffb640 0 client.25651 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 >> 172.21.15.16:0/572159324 conn(0x55d961e51a30 msgr2=0x55d961e57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 shutdown_connections 2024-09-12T00:59:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 1 -- 172.21.15.16:0/572159324 wait complete. 2024-09-12T00:59:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.533+0000 7fe73e9fa740 20 client.25651 trim_cache size 0 max 16384 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 Processor -- start 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 -- start start 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e51a30 0x55d961e80460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e554a0 0x55d961e809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ab600 con 0x55d961e55dc0 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ad140 con 0x55d961e554a0 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341abcd0 con 0x55d961e51a30 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41174/0 (socket says 172.21.15.16:41174) 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 -- 172.21.15.16:0/1148996577 learned_addr learned my addr 172.21.15.16:0/1148996577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e51a30 0x55d961e80460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e554a0 0x55d961e809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 -- 172.21.15.16:0/1148996577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e51a30 msgr2=0x55d961e80460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e51a30 0x55d961e80460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 -- 172.21.15.16:0/1148996577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e554a0 msgr2=0x55d961e809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e554a0 0x55d961e809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.550+0000 7fe739bbc640 1 -- 172.21.15.16:0/1148996577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d961e526c0 con 0x55d961e55dc0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe739bbc640 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe724001eb0 tx=0x7fe724000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6093b57b80925382 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe71ffff640 1 -- 172.21.15.16:0/1148996577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724035070 con 0x55d961e55dc0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe71ffff640 1 -- 172.21.15.16:0/1148996577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7240044a0 con 0x55d961e55dc0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe71ffff640 1 -- 172.21.15.16:0/1148996577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72404ba40 con 0x55d961e55dc0 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 -- 172.21.15.16:0/1148996577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 msgr2=0x55d961e5a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe724001eb0 tx=0x7fe724000bc0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 -- 172.21.15.16:0/1148996577 shutdown_connections 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e5a4b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e554a0 0x55d961e809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/1148996577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e51a30 0x55d961e80460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 -- 172.21.15.16:0/1148996577 >> 172.21.15.16:0/1148996577 conn(0x55d961e5d8f0 msgr2=0x55d961e5bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 -- 172.21.15.16:0/1148996577 shutdown_connections 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.551+0000 7fe73e9fa740 1 -- 172.21.15.16:0/1148996577 wait complete. 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 -- start start 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 Processor -- start 2024-09-12T00:59:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e51a30 0x55d961e6eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341aca70 con 0x55d961e55dc0 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341ac3a0 con 0x55d961e51a30 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7341a95e0 con 0x55d961e554a0 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73e9fa740 10 client.0 _ll_register_callbacks cb 0x7fe724009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe739bbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41190/0 (socket says 172.21.15.16:41190) 2024-09-12T00:59:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe739bbc640 1 -- 172.21.15.16:0/726061293 learned_addr learned my addr 172.21.15.16:0/726061293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.552+0000 7fe739bbc640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e51a30 0x55d961e6eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 -- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 msgr2=0x55d961e6f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e6f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 -- 172.21.15.16:0/726061293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e51a30 msgr2=0x55d961e6eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e51a30 0x55d961e6eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 -- 172.21.15.16:0/726061293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d961ee2730 con 0x55d961e55dc0 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe739bbc640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe724031740 tx=0x7fe724031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27542ace703cb6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe71dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724035070 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe72404ba40 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7240044a0 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe724005820 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe73e9fa740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d961ee39c0 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.553+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7240044a0 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe72406b450 con 0x55d961e55dc0 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 20 client.16140 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e60680 0x55d961ee60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 4 client.16140 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:59:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d961dc2d60 con 0x55d961e60680 2024-09-12T00:59:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e60680 0x55d961ee60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe71dffb640 10 client.16140 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.554+0000 7fe73a3bd640 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e60680 0x55d961ee60b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811b6064543f58e1 server_cookie=e6557ef5d4b211dd in_seq=0 out_seq=0 2024-09-12T00:59:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe71dffb640 1 -- 172.21.15.16:0/726061293 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d961dc2d60 con 0x55d961e60680 2024-09-12T00:59:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe71dffb640 10 client.16140 handle_command_reply: tid=0 2024-09-12T00:59:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 client.16140 shutdown 2024-09-12T00:59:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 msgr2=0x55d961e6f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe724031740 tx=0x7fe724031b70 comp rx=0 tx=0).stop 2024-09-12T00:59:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 shutdown_connections 2024-09-12T00:59:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d961e60680 0x55d961ee60b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d961e55dc0 0x55d961e6f930 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d961e554a0 0x55d961e6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe71dffb640 0 client.16140 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 --2- 172.21.15.16:0/726061293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d961e51a30 0x55d961e6eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 >> 172.21.15.16:0/726061293 conn(0x55d961e5d8f0 msgr2=0x55d961e5eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 shutdown_connections 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 1 -- 172.21.15.16:0/726061293 wait complete. 2024-09-12T00:59:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73e9fa740 20 client.16140 trim_cache size 0 max 16384 2024-09-12T00:59:08.565 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T00:59:08.565 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T00:59:08.565 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 msgr2=0x7fe7200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 0x7fe7200a5230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0a4110 tx=0x7fe72c06ea40 comp rx=0 tx=0).stop 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 msgr2=0x7fe73419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.555+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00eaa0 tx=0x7fe72c00eea0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 shutdown_connections 2024-09-12T00:59:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7200a2d70 0x7fe7200a5230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73410c170 0x7fe73419cca0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe734103660 0x7fe734198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 --2- 172.21.15.16:0/2389331687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe734102d40 0x7fe7341983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 >> 172.21.15.16:0/2389331687 conn(0x7fe7340fc7a0 msgr2=0x7fe7340fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 shutdown_connections 2024-09-12T00:59:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:08.556+0000 7fe73c621640 1 -- 172.21.15.16:0/2389331687 wait complete. 2024-09-12T00:59:09.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:09 smithi016 ceph-mon[30972]: pgmap v95: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:09.419 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:59:09.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:59:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:09 smithi099 ceph-mon[35654]: pgmap v95: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:09 smithi159 ceph-mon[35566]: pgmap v95: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 -- 172.21.15.16:0/440210992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0106f10 msgr2=0x7fa6d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/440210992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0106f10 0x7fa6d010ea90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009ca0 tx=0x7fa6c802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 -- 172.21.15.16:0/440210992 shutdown_connections 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/440210992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0106f10 0x7fa6d010ea90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/440210992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0102790 0x7fa6d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/440210992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0101de0 0x7fa6d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 -- 172.21.15.16:0/440210992 >> 172.21.15.16:0/440210992 conn(0x7fa6d00fd8c0 msgr2=0x7fa6d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 -- 172.21.15.16:0/440210992 shutdown_connections 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.152+0000 7fa6d75b1640 1 -- 172.21.15.16:0/440210992 wait complete. 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.153+0000 7fa6d75b1640 1 Processor -- start 2024-09-12T00:59:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.153+0000 7fa6d75b1640 1 -- start start 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.153+0000 7fa6d75b1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0101de0 0x7fa6d01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.153+0000 7fa6d75b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0106f10 0x7fa6d019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0111fc0 con 0x7fa6d0102790 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0111e40 con 0x7fa6d0101de0 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d0112140 con 0x7fa6d0106f10 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41214/0 (socket says 172.21.15.16:41214) 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 -- 172.21.15.16:0/2383439199 learned_addr learned my addr 172.21.15.16:0/2383439199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d534d640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0101de0 0x7fa6d01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d5b4e640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0106f10 0x7fa6d019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 -- 172.21.15.16:0/2383439199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0106f10 msgr2=0x7fa6d019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0106f10 0x7fa6d019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 -- 172.21.15.16:0/2383439199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0101de0 msgr2=0x7fa6d01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0101de0 0x7fa6d01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 -- 172.21.15.16:0/2383439199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d019d380 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d4b4c640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c890 tx=0x7fa6c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004220 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d019d670 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c00043c0 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00049e0 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.154+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a4f50 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.155+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d01021c0 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.157+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6c00079b0 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.157+0000 7fa6c67fc640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 0x7fa6bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.157+0000 7fa6d534d640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 0x7fa6bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.157+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa6c00cf4f0 con 0x7fa6d0102790 2024-09-12T00:59:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.158+0000 7fa6d534d640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 0x7fa6bc0a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.161+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6c0014030 con 0x7fa6d0102790 2024-09-12T00:59:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.374+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d002d7c0 con 0x7fa6d0102790 2024-09-12T00:59:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.375+0000 7fa6c67fc640 1 -- 172.21.15.16:0/2383439199 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa6c009c140 con 0x7fa6d0102790 2024-09-12T00:59:10.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:59:10.375 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:59:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.377+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 msgr2=0x7fa6bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.377+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 0x7fa6bc0a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc005850 comp rx=0 tx=0).stop 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.377+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 msgr2=0x7fa6d0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.377+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000c890 tx=0x7fa6c000cd50 comp rx=0 tx=0).stop 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 shutdown_connections 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6bc0a2cf0 0x7fa6bc0a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0106f10 0x7fa6d019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d0102790 0x7fa6d0198910 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 --2- 172.21.15.16:0/2383439199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0101de0 0x7fa6d01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 >> 172.21.15.16:0/2383439199 conn(0x7fa6d00fd8c0 msgr2=0x7fa6d00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 shutdown_connections 2024-09-12T00:59:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:10.378+0000 7fa6d75b1640 1 -- 172.21.15.16:0/2383439199 wait complete. 2024-09-12T00:59:11.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T00:59:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:11 smithi099 ceph-mon[35654]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-12T00:59:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2383439199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:11 smithi159 ceph-mon[35566]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-12T00:59:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2383439199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:11.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:11 smithi016 ceph-mon[30972]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-09-12T00:59:11.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2383439199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:11.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 -- 172.21.15.16:0/3406341661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c10c170 msgr2=0x7fb52c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 --2- 172.21.15.16:0/3406341661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c10c170 0x7fb52c10e600 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb524009ca0 tx=0x7fb52402a950 comp rx=0 tx=0).stop 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 -- 172.21.15.16:0/3406341661 shutdown_connections 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 --2- 172.21.15.16:0/3406341661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c10c170 0x7fb52c10e600 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 --2- 172.21.15.16:0/3406341661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c1036f0 0x7fb52c10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 --2- 172.21.15.16:0/3406341661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 0x7fb52c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.005+0000 7fb5310c6640 1 -- 172.21.15.16:0/3406341661 >> 172.21.15.16:0/3406341661 conn(0x7fb52c0fc790 msgr2=0x7fb52c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.006+0000 7fb5310c6640 1 -- 172.21.15.16:0/3406341661 shutdown_connections 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.006+0000 7fb5310c6640 1 -- 172.21.15.16:0/3406341661 wait complete. 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.006+0000 7fb5310c6640 1 Processor -- start 2024-09-12T00:59:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 -- start start 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 0x7fb52c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c10c170 0x7fb52c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c111f60 con 0x7fb52c1036f0 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c111de0 con 0x7fb52c10c170 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb5310c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1120e0 con 0x7fb52c102dd0 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41230/0 (socket says 172.21.15.16:41230) 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 -- 172.21.15.16:0/475511021 learned_addr learned my addr 172.21.15.16:0/475511021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52b59e640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c10c170 0x7fb52c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 0x7fb52c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 -- 172.21.15.16:0/475511021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 msgr2=0x7fb52c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 0x7fb52c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 -- 172.21.15.16:0/475511021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c10c170 msgr2=0x7fb52c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c10c170 0x7fb52c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 -- 172.21.15.16:0/475511021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19d490 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.007+0000 7fb52a59c640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c890 tx=0x7fb52000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.008+0000 7fb513fff640 1 -- 172.21.15.16:0/475511021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520004220 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.008+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19d780 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.008+0000 7fb513fff640 1 -- 172.21.15.16:0/475511021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5200043c0 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.008+0000 7fb513fff640 1 -- 172.21.15.16:0/475511021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520004a00 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.008+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a5060 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.010+0000 7fb513fff640 1 -- 172.21.15.16:0/475511021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb5200079b0 con 0x7fb52c1036f0 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.010+0000 7fb53349f740 1 Processor -- start 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.010+0000 7fb53349f740 1 -- start start 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 0x5592c452a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452a5d0 0x5592c452aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592c44902f0 con 0x5592c4535370 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592c4437cf0 con 0x5592c452bb40 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb53349f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5592c448c2b0 con 0x5592c452a5d0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41232/0 (socket says 172.21.15.16:41232) 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 0x5592c452a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 -- 172.21.15.16:0/315297770 learned_addr learned my addr 172.21.15.16:0/315297770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb513fff640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452a5d0 0x5592c452aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52b59e640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb513fff640 1 -- 172.21.15.16:0/475511021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb5200cf120 con 0x7fb52c1036f0 2024-09-12T00:59:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 -- 172.21.15.16:0/315297770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452a5d0 msgr2=0x5592c452aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452a5d0 0x5592c452aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 -- 172.21.15.16:0/315297770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 msgr2=0x5592c452a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 0x5592c452a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 -- 172.21.15.16:0/315297770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c441c3a0 con 0x5592c4535370 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52b59e640 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 0x5592c452a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.011+0000 7fb52a59c640 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb520097ad0 tx=0x7fb5200054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f3bb768f648e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb5117fa640 1 -- 172.21.15.16:0/315297770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52001d050 con 0x5592c4535370 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb5117fa640 1 -- 172.21.15.16:0/315297770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52009bd70 con 0x5592c4535370 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb5117fa640 1 -- 172.21.15.16:0/315297770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200a42e0 con 0x5592c4535370 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb52b59e640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb52402ae70 tx=0x7fb52402a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 -- 172.21.15.16:0/315297770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 msgr2=0x5592c4537760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb520097ad0 tx=0x7fb5200054a0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 -- 172.21.15.16:0/315297770 shutdown_connections 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c4535370 0x5592c4537760 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452a5d0 0x5592c452aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 --2- 172.21.15.16:0/315297770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452bb40 0x5592c452a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 -- 172.21.15.16:0/315297770 >> 172.21.15.16:0/315297770 conn(0x5592c44f3310 msgr2=0x5592c4526c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 -- 172.21.15.16:0/315297770 shutdown_connections 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.012+0000 7fb53349f740 1 -- 172.21.15.16:0/315297770 wait complete. 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 -- start start 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 Processor -- start 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c4534e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c4532850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592c4538010 con 0x5592c452bb40 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592c444b060 con 0x5592c452a5d0 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5592c4538190 con 0x5592c4535370 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb53349f740 10 client.0 _ll_register_callbacks cb 0x5592c419e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41244/0 (socket says 172.21.15.16:41244) 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb52a59c640 1 -- 172.21.15.16:0/1760041207 learned_addr learned my addr 172.21.15.16:0/1760041207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.013+0000 7fb52b59e640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c4532850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c4534e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 -- 172.21.15.16:0/1760041207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 msgr2=0x5592c4532850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c4532850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 -- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 msgr2=0x5592c4534e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c4534e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 -- 172.21.15.16:0/1760041207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c42013c0 con 0x5592c452bb40 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cc950 tx=0x7fb52009fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b1e53d85bc0643 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200a42e0 con 0x5592c452bb40 2024-09-12T00:59:12.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb520007ca0 con 0x5592c452bb40 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52009bd70 con 0x5592c452bb40 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb520084bb0 con 0x5592c452bb40 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb53349f740 10 client.16158 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592c4543b60 con 0x5592c452bb40 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52009bd70 con 0x5592c452bb40 2024-09-12T00:59:12.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.014+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb52009ac50 con 0x5592c452bb40 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 10 client.16158 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 20 client.16158 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c4544eb0 0x5592c45472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 4 client.16158 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:12.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c449d860 con 0x5592c4544eb0 2024-09-12T00:59:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c4544eb0 0x5592c45472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb50eff5640 10 client.16158 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.015+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c4544eb0 0x5592c45472a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80846f726b51dc52 server_cookie=866df6ada1066c16 in_seq=0 out_seq=0 2024-09-12T00:59:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb50eff5640 1 -- 172.21.15.16:0/1760041207 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592c449d860 con 0x5592c4544eb0 2024-09-12T00:59:12.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb50eff5640 10 client.16158 handle_command_reply: tid=0 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 client.16158 shutdown 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 msgr2=0x5592c45320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cc950 tx=0x7fb52009fa20 comp rx=0 tx=0).stop 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 shutdown_connections 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c4544eb0 0x5592c45472a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c4532850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c45320c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 --2- 172.21.15.16:0/1760041207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c4534e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 >> 172.21.15.16:0/1760041207 conn(0x5592c44f3310 msgr2=0x5592c452c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb50eff5640 0 client.16158 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 shutdown_connections 2024-09-12T00:59:12.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 1 -- 172.21.15.16:0/1760041207 wait complete. 2024-09-12T00:59:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.016+0000 7fb53349f740 20 client.16158 trim_cache size 0 max 16384 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 Processor -- start 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 -- start start 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c45310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c452d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ab5f0 con 0x5592c452bb40 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ad130 con 0x5592c452a5d0 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.029+0000 7fb53349f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1abcc0 con 0x5592c4535370 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41256/0 (socket says 172.21.15.16:41256) 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 -- 172.21.15.16:0/3984868969 learned_addr learned my addr 172.21.15.16:0/3984868969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52b59e640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c452d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c45310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 -- 172.21.15.16:0/3984868969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 msgr2=0x5592c452d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c452d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 -- 172.21.15.16:0/3984868969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 msgr2=0x5592c45310a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c45310a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 -- 172.21.15.16:0/3984868969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c4532bc0 con 0x5592c452bb40 2024-09-12T00:59:12.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c45310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb52a59c640 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cf0f0 tx=0x7fb520000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0129b5aeae441b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb510ff9640 1 -- 172.21.15.16:0/3984868969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52001d030 con 0x5592c452bb40 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb510ff9640 1 -- 172.21.15.16:0/3984868969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52009bd70 con 0x5592c452bb40 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.030+0000 7fb510ff9640 1 -- 172.21.15.16:0/3984868969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520060b70 con 0x5592c452bb40 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- 172.21.15.16:0/3984868969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 msgr2=0x5592c452d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb5200cf0f0 tx=0x7fb520000bc0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- 172.21.15.16:0/3984868969 shutdown_connections 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c4535370 0x5592c452d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452bb40 0x5592c452d120 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 --2- 172.21.15.16:0/3984868969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c452a5d0 0x5592c45310a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- 172.21.15.16:0/3984868969 >> 172.21.15.16:0/3984868969 conn(0x5592c44f3310 msgr2=0x5592c4530990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- 172.21.15.16:0/3984868969 shutdown_connections 2024-09-12T00:59:12.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- 172.21.15.16:0/3984868969 wait complete. 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 -- start start 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.031+0000 7fb53349f740 1 Processor -- start 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452bb40 0x5592c4547820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c4535370 0x5592c45ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1aca60 con 0x5592c452a5d0 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1ac390 con 0x5592c4535370 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1a95d0 con 0x5592c452bb40 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb53349f740 10 client.0 _ll_register_callbacks cb 0x7fb524009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41258/0 (socket says 172.21.15.16:41258) 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 -- 172.21.15.16:0/1964037721 learned_addr learned my addr 172.21.15.16:0/1964037721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:12.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52b59e640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c4535370 0x5592c45ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452bb40 0x5592c4547820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 -- 172.21.15.16:0/1964037721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452bb40 msgr2=0x5592c4547820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452bb40 0x5592c4547820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 -- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c4535370 msgr2=0x5592c45ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c4535370 0x5592c45ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 -- 172.21.15.16:0/1964037721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c45add20 con 0x5592c452a5d0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb52ad9d640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb51c004700 tx=0x7fb51c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afac29eae13a2051 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb50eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.032+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c004230 con 0x5592c452a5d0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb51c0043d0 con 0x5592c452a5d0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52009bd70 con 0x5592c452a5d0 2024-09-12T00:59:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb51c02c050 con 0x5592c452a5d0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592c45aef30 con 0x5592c452a5d0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb52009bd70 con 0x5592c452a5d0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb51c01d7d0 con 0x5592c452a5d0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 10 client.16164 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 20 client.16164 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c45af6b0 0x5592c45b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 4 client.16164 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.033+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592c4497d70 con 0x5592c45af6b0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c45af6b0 0x5592c45b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb52a59c640 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c45af6b0 0x5592c45b1ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4dad7d953b6ae258 server_cookie=730f67c9a491db03 in_seq=0 out_seq=0 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb50eff5640 10 client.16164 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:12.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb50eff5640 1 -- 172.21.15.16:0/1964037721 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592c4497d70 con 0x5592c45af6b0 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb50eff5640 10 client.16164 handle_command_reply: tid=0 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 client.16164 shutdown 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 msgr2=0x5592c45472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb51c004700 tx=0x7fb51c011810 comp rx=0 tx=0).stop 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 shutdown_connections 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5592c45af6b0 0x5592c45b1ad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592c4535370 0x5592c45ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb50eff5640 0 client.16164 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592c452bb40 0x5592c4547820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 --2- 172.21.15.16:0/1964037721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592c452a5d0 0x5592c45472e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 >> 172.21.15.16:0/1964037721 conn(0x5592c44f3310 msgr2=0x5592c45311b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 shutdown_connections 2024-09-12T00:59:12.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.034+0000 7fb53349f740 1 -- 172.21.15.16:0/1964037721 wait complete. 2024-09-12T00:59:12.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb53349f740 20 client.16164 trim_cache size 0 max 16384 2024-09-12T00:59:12.045 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:12.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 msgr2=0x7fb5140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb52402ae70 tx=0x7fb52402a950 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 msgr2=0x7fb52c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c890 tx=0x7fb52000cd50 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 shutdown_connections 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c10c170 0x7fb52c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c1036f0 0x7fb52c198a20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 --2- 172.21.15.16:0/475511021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102dd0 0x7fb52c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 >> 172.21.15.16:0/475511021 conn(0x7fb52c0fc790 msgr2=0x7fb52c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 shutdown_connections 2024-09-12T00:59:12.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:12.035+0000 7fb5310c6640 1 -- 172.21.15.16:0/475511021 wait complete. 2024-09-12T00:59:12.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T00:59:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:13.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:13 smithi016 ceph-mon[30972]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 -- 172.21.15.16:0/1131112598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 msgr2=0x7f164c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:13.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 --2- 172.21.15.16:0/1131112598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1021c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f163c009900 tx=0x7f163c02a3f0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 -- 172.21.15.16:0/1131112598 shutdown_connections 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 --2- 172.21.15.16:0/1131112598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c106f10 0x7f164c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 --2- 172.21.15.16:0/1131112598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c102790 0x7f164c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 --2- 172.21.15.16:0/1131112598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1021c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 -- 172.21.15.16:0/1131112598 >> 172.21.15.16:0/1131112598 conn(0x7f164c0fd8c0 msgr2=0x7f164c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 -- 172.21.15.16:0/1131112598 shutdown_connections 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.472+0000 7f165254f640 1 -- 172.21.15.16:0/1131112598 wait complete. 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 Processor -- start 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 -- start start 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c102790 0x7f164c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:13.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c106f10 0x7f164c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f164c111f70 con 0x7f164c101de0 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f164c111df0 con 0x7f164c106f10 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.473+0000 7f165254f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f164c1120f0 con 0x7f164c102790 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41266/0 (socket says 172.21.15.16:41266) 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 -- 172.21.15.16:0/3734458071 learned_addr learned my addr 172.21.15.16:0/3734458071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f1650aec640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c106f10 0x7f164c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164b7fe640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c102790 0x7f164c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 -- 172.21.15.16:0/3734458071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c102790 msgr2=0x7f164c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c102790 0x7f164c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 -- 172.21.15.16:0/3734458071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c106f10 msgr2=0x7f164c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c106f10 0x7f164c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 -- 172.21.15.16:0/3734458071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c19d380 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f164bfff640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c03f030 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f163c003ce0 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c19d610 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c035070 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.474+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c1a4f50 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.475+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164c1032c0 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.477+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f163c042030 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.478+0000 7f16497fa640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 0x7f16340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.478+0000 7f164b7fe640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 0x7f16340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.478+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f163c0efc30 con 0x7f164c101de0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.478+0000 7f164b7fe640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 0x7f16340a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f164c0fdce0 tx=0x7f164000b410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:13.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.482+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f163c0bc290 con 0x7f164c101de0 2024-09-12T00:59:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:13 smithi099 ceph-mon[35654]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:13.534 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:13 smithi159 ceph-mon[35566]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.707+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f164c10a0f0 con 0x7f164c101de0 2024-09-12T00:59:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.708+0000 7f16497fa640 1 -- 172.21.15.16:0/3734458071 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f163c0c0590 con 0x7f164c101de0 2024-09-12T00:59:13.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:59:13.708 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:59:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 msgr2=0x7f16340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 0x7f16340a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f164c0fdce0 tx=0x7f164000b410 comp rx=0 tx=0).stop 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 msgr2=0x7f164c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c0042f0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 shutdown_connections 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16340a2cf0 0x7f16340a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f164c106f10 0x7f164c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f164c102790 0x7f164c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 --2- 172.21.15.16:0/3734458071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164c101de0 0x7f164c1983d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 >> 172.21.15.16:0/3734458071 conn(0x7f164c0fd8c0 msgr2=0x7f164c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 shutdown_connections 2024-09-12T00:59:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:13.710+0000 7f165254f640 1 -- 172.21.15.16:0/3734458071 wait complete. 2024-09-12T00:59:14.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:14 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3734458071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:14.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:14 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3734458071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:14.561 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 5605cf54 --release --await' 2024-09-12T00:59:14.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 5605cf54 --release --await 2024-09-12T00:59:14.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:14 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3734458071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:14.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 -- 172.21.15.16:0/3675847469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 msgr2=0x7fdac81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/3675847469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac81067f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009900 tx=0x7fdab802a410 comp rx=0 tx=0).stop 2024-09-12T00:59:14.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 -- 172.21.15.16:0/3675847469 shutdown_connections 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/3675847469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8106f20 0x7fdac810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/3675847469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac81067f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/3675847469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8101df0 0x7fdac81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 -- 172.21.15.16:0/3675847469 >> 172.21.15.16:0/3675847469 conn(0x7fdac80fd8f0 msgr2=0x7fdac80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 -- 172.21.15.16:0/3675847469 shutdown_connections 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.751+0000 7fdacf2f1640 1 -- 172.21.15.16:0/3675847469 wait complete. 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 Processor -- start 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 -- start start 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8101df0 0x7fdac8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8106f20 0x7fdac819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111fa0 con 0x7fdac81027a0 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8111e20 con 0x7fdac8106f20 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacf2f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8112120 con 0x7fdac8101df0 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacc88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacc88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41282/0 (socket says 172.21.15.16:41282) 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacc88c640 1 -- 172.21.15.16:0/1379396973 learned_addr learned my addr 172.21.15.16:0/1379396973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacd88e640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8106f20 0x7fdac819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.752+0000 7fdacd08d640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8101df0 0x7fdac8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 -- 172.21.15.16:0/1379396973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8101df0 msgr2=0x7fdac8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8101df0 0x7fdac8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 -- 172.21.15.16:0/1379396973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8106f20 msgr2=0x7fdac819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8106f20 0x7fdac819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 -- 172.21.15.16:0/1379396973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819d510 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacc88c640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005ab0 tx=0x7fdab80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdabe7fc640 1 -- 172.21.15.16:0/1379396973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab803f030 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819d7a0 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdabe7fc640 1 -- 172.21.15.16:0/1379396973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab8003ce0 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdabe7fc640 1 -- 172.21.15.16:0/1379396973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8035070 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.753+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a50e0 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdabe7fc640 1 -- 172.21.15.16:0/1379396973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdab8042030 con 0x7fdac81027a0 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 Processor -- start 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 -- start start 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe19a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e3fd7f2f0 con 0x559e3fe1ab40 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e3fd26cf0 con 0x559e3fe195d0 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdad16ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e3fd7b2b0 con 0x559e3fe24370 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.757+0000 7fdacd88e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41286/0 (socket says 172.21.15.16:41286) 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 -- 172.21.15.16:0/900759505 learned_addr learned my addr 172.21.15.16:0/900759505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd08d640 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe19a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacc88c640 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 -- 172.21.15.16:0/900759505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 msgr2=0x559e3fe26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe26760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 -- 172.21.15.16:0/900759505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 msgr2=0x559e3fe19a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe19a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 -- 172.21.15.16:0/900759505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3fd0b3a0 con 0x559e3fe1ab40 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdabe7fc640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 0x7fdab40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd88e640 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdac0007f40 tx=0x7fdac000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada09fcae235d0f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdacd08d640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 0x7fdab40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdabe7fc640 1 -- 172.21.15.16:0/1379396973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fdab80efc50 con 0x7fdac81027a0 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.758+0000 7fdab37fe640 1 -- 172.21.15.16:0/900759505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0010900 con 0x559e3fe1ab40 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.759+0000 7fdab37fe640 1 -- 172.21.15.16:0/900759505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac00183f0 con 0x559e3fe1ab40 2024-09-12T00:59:14.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.759+0000 7fdab37fe640 1 -- 172.21.15.16:0/900759505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0017600 con 0x559e3fe1ab40 2024-09-12T00:59:14.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.759+0000 7fdad16ca740 1 -- 172.21.15.16:0/900759505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 msgr2=0x559e3fe19090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.759+0000 7fdacd08d640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 0x7fdab40a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x559e3fe16ac0 tx=0x7fdac4009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.759+0000 7fdad16ca740 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdac0007f40 tx=0x7fdac000f850 comp rx=0 tx=0).stop 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 -- 172.21.15.16:0/900759505 shutdown_connections 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 --2- 172.21.15.16:0/900759505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe19090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 -- 172.21.15.16:0/900759505 >> 172.21.15.16:0/900759505 conn(0x559e3fde2310 msgr2=0x559e3fe15c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 -- 172.21.15.16:0/900759505 shutdown_connections 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 -- 172.21.15.16:0/900759505 wait complete. 2024-09-12T00:59:14.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 -- start start 2024-09-12T00:59:14.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 Processor -- start 2024-09-12T00:59:14.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.760+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe23d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e3fe27070 con 0x559e3fe1ab40 2024-09-12T00:59:14.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e3fd3a060 con 0x559e3fe195d0 2024-09-12T00:59:14.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e3fe271f0 con 0x559e3fe24370 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdad16ca740 10 client.0 _ll_register_callbacks cb 0x559e3fa8d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacc88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe23d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacc88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41290/0 (socket says 172.21.15.16:41290) 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacc88c640 1 -- 172.21.15.16:0/3664737519 learned_addr learned my addr 172.21.15.16:0/3664737519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd88e640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 -- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 msgr2=0x559e3fe21570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe21570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 -- 172.21.15.16:0/3664737519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 msgr2=0x559e3fe20fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 -- 172.21.15.16:0/3664737519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3faf03c0 con 0x559e3fe195d0 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:14.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe23d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdac4014970 tx=0x7fdac40047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f468cf3248e72b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.761+0000 7fdab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4025070 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac00183f0 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fdac4021600 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdac4011e50 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 10 client.25625 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e3fe36f80 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac00183f0 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdac4020070 con 0x559e3fe195d0 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 10 client.25625 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 10 client.25625 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.762+0000 7fdad16ca740 20 client.25625 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:14.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fe38390 0x559e3fe3a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdad16ca740 4 client.25625 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3fd8c860 con 0x559e3fe38390 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fe38390 0x559e3fe3a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fe38390 0x559e3fe3a7a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3152f7a0b6b36bc9 server_cookie=618ddacd49f9d291 in_seq=0 out_seq=0 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.763+0000 7fdab0ff9640 10 client.25625 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3664737519 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e3fd8c860 con 0x559e3fe38390 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdab0ff9640 10 client.25625 handle_command_reply: tid=0 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 client.25625 shutdown 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 msgr2=0x559e3fe23d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe23d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdac4014970 tx=0x7fdac40047d0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 shutdown_connections 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fe21570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fe38390 0x559e3fe3a7a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe1ab40 0x559e3fe20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3664737519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe195d0 0x559e3fe23d20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 >> 172.21.15.16:0/3664737519 conn(0x559e3fde2310 msgr2=0x559e3fe1b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.764+0000 7fdab0ff9640 0 client.25625 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.765+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 shutdown_connections 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.765+0000 7fdad16ca740 1 -- 172.21.15.16:0/3664737519 wait complete. 2024-09-12T00:59:14.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.765+0000 7fdad16ca740 20 client.25625 trim_cache size 0 max 16384 2024-09-12T00:59:14.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 Processor -- start 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 -- start start 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe1ab40 0x559e3fe20890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 0x559e3fe1f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ab600 con 0x559e3fe195d0 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ad140 con 0x559e3fe24370 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdad16ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81abcd0 con 0x559e3fe1ab40 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdacd08d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdacd08d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41300/0 (socket says 172.21.15.16:41300) 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.779+0000 7fdacd08d640 1 -- 172.21.15.16:0/3778237904 learned_addr learned my addr 172.21.15.16:0/3778237904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe1ab40 0x559e3fe20890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd88e640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 0x559e3fe1f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 -- 172.21.15.16:0/3778237904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe1ab40 msgr2=0x559e3fe20890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe1ab40 0x559e3fe20890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 -- 172.21.15.16:0/3778237904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 msgr2=0x559e3fe1f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 0x559e3fe1f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 -- 172.21.15.16:0/3778237904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3fe218a0 con 0x559e3fe195d0 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd88e640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 0x559e3fe1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdac4016040 tx=0x7fdac4006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e642b89133bf4b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdab2ffd640 1 -- 172.21.15.16:0/3778237904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4039020 con 0x559e3fe195d0 2024-09-12T00:59:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdab2ffd640 1 -- 172.21.15.16:0/3778237904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac401b070 con 0x559e3fe195d0 2024-09-12T00:59:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.780+0000 7fdab2ffd640 1 -- 172.21.15.16:0/3778237904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4025040 con 0x559e3fe195d0 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 -- 172.21.15.16:0/3778237904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 msgr2=0x559e3fe347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdac4016040 tx=0x7fdac4006b10 comp rx=0 tx=0).stop 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 -- 172.21.15.16:0/3778237904 shutdown_connections 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe24370 0x559e3fe1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe1ab40 0x559e3fe20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3778237904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe347a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.781+0000 7fdad16ca740 1 -- 172.21.15.16:0/3778237904 >> 172.21.15.16:0/3778237904 conn(0x559e3fde2310 msgr2=0x559e3fe1c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.782+0000 7fdad16ca740 1 -- 172.21.15.16:0/3778237904 shutdown_connections 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.782+0000 7fdad16ca740 1 -- 172.21.15.16:0/3778237904 wait complete. 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.782+0000 7fdad16ca740 1 -- start start 2024-09-12T00:59:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.782+0000 7fdad16ca740 1 Processor -- start 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 0x559e3fe2c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fea1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81aca70 con 0x559e3fe195d0 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81ac3a0 con 0x559e3fe1ab40 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81a9600 con 0x559e3fe24370 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdad16ca740 10 client.0 _ll_register_callbacks cb 0x7fdac4009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41310/0 (socket says 172.21.15.16:41310) 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 -- 172.21.15.16:0/3252559521 learned_addr learned my addr 172.21.15.16:0/3252559521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 0x559e3fe2c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd88e640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fea1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 -- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 msgr2=0x559e3fea1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fea1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 -- 172.21.15.16:0/3252559521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 msgr2=0x559e3fe2c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 0x559e3fe2c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd08d640 1 -- 172.21.15.16:0/3252559521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3fea1620 con 0x559e3fe195d0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacd88e640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fea1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.783+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 0x559e3fe2c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdacd08d640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdac4029ba0 tx=0x7fdac4029f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a16f416498c7b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdab0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4045020 con 0x559e3fe195d0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fdac4021960 con 0x559e3fe195d0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac401b070 con 0x559e3fe195d0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdac4050050 con 0x559e3fe195d0 2024-09-12T00:59:14.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdad16ca740 10 client.16188 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e3fea28e0 con 0x559e3fe195d0 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.784+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac401b070 con 0x559e3fe195d0 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdac4020040 con 0x559e3fe195d0 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 10 client.16188 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 10 client.16188 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 20 client.16188 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fea2f30 0x559e3fea5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 4 client.16188 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:14.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e3fd86d70 con 0x559e3fea2f30 2024-09-12T00:59:14.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.785+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fea2f30 0x559e3fea5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdab0ff9640 10 client.16188 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdacc88c640 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fea2f30 0x559e3fea5320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3246c787c090fc2f server_cookie=c44cc48406c3b400 in_seq=0 out_seq=0 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdab0ff9640 1 -- 172.21.15.16:0/3252559521 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e3fd86d70 con 0x559e3fea2f30 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdab0ff9640 10 client.16188 handle_command_reply: tid=0 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 client.16188 shutdown 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 msgr2=0x559e3fe2be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdac4029ba0 tx=0x7fdac4029f20 comp rx=0 tx=0).stop 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 shutdown_connections 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e3fe24370 0x559e3fea1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e3fe1ab40 0x559e3fe2c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559e3fea2f30 0x559e3fea5320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 --2- 172.21.15.16:0/3252559521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e3fe195d0 0x559e3fe2be70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdab0ff9640 0 client.16188 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 >> 172.21.15.16:0/3252559521 conn(0x559e3fde2310 msgr2=0x559e3fe1f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 shutdown_connections 2024-09-12T00:59:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.786+0000 7fdad16ca740 1 -- 172.21.15.16:0/3252559521 wait complete. 2024-09-12T00:59:14.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdad16ca740 20 client.16188 trim_cache size 0 max 16384 2024-09-12T00:59:14.792 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 msgr2=0x7fdab40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 0x7fdab40a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x559e3fe16ac0 tx=0x7fdac4009110 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 msgr2=0x7fdac8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005ab0 tx=0x7fdab80042f0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 shutdown_connections 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdab40a2d70 0x7fdab40a5230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdac8106f20 0x7fdac819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac81027a0 0x7fdac8198aa0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 --2- 172.21.15.16:0/1379396973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8101df0 0x7fdac8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:14.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.787+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 >> 172.21.15.16:0/1379396973 conn(0x7fdac80fd8f0 msgr2=0x7fdac80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.788+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 shutdown_connections 2024-09-12T00:59:14.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:14.788+0000 7fdacf2f1640 1 -- 172.21.15.16:0/1379396973 wait complete. 2024-09-12T00:59:15.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:15 smithi016 ceph-mon[30972]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:15.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:15 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:59:15.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:59:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:15.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:15 smithi099 ceph-mon[35654]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:15 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:59:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:15 smithi159 ceph-mon[35566]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T00:59:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:15 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T00:59:16.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 -- 172.21.15.16:0/3802051539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 msgr2=0x7f19cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 --2- 172.21.15.16:0/3802051539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10da40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009ca0 tx=0x7f19c402a950 comp rx=0 tx=0).stop 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 -- 172.21.15.16:0/3802051539 shutdown_connections 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 --2- 172.21.15.16:0/3802051539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10da40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 --2- 172.21.15.16:0/3802051539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc102570 0x7f19cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 --2- 172.21.15.16:0/3802051539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc100e20 0x7f19cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 -- 172.21.15.16:0/3802051539 >> 172.21.15.16:0/3802051539 conn(0x7f19cc0fc7d0 msgr2=0x7f19cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 -- 172.21.15.16:0/3802051539 shutdown_connections 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.013+0000 7f19d2856640 1 -- 172.21.15.16:0/3802051539 wait complete. 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.014+0000 7f19d2856640 1 Processor -- start 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.014+0000 7f19d2856640 1 -- start start 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 0x7f19cc1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc102570 0x7f19cc1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1120a0 con 0x7f19cc102ef0 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc111f20 con 0x7f19cc100e20 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d2856640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc112220 con 0x7f19cc102570 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41332/0 (socket says 172.21.15.16:41332) 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19cbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 0x7f19cc1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 -- 172.21.15.16:0/1719377228 learned_addr learned my addr 172.21.15.16:0/1719377228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc102570 0x7f19cc1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 -- 172.21.15.16:0/1719377228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc102570 msgr2=0x7f19cc1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc102570 0x7f19cc1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 -- 172.21.15.16:0/1719377228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 msgr2=0x7f19cc1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 0x7f19cc1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 -- 172.21.15.16:0/1719377228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc10d2b0 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19cbfff640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 0x7f19cc1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19d0df3640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f19c4004af0 tx=0x7f19c402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.015+0000 7f19c9ffb640 1 -- 172.21.15.16:0/1719377228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4034040 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.016+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc10d540 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.016+0000 7f19c9ffb640 1 -- 172.21.15.16:0/1719377228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19c4030920 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.016+0000 7f19c9ffb640 1 -- 172.21.15.16:0/1719377228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4004820 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.016+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc1ab530 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.019+0000 7f19c9ffb640 1 -- 172.21.15.16:0/1719377228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19c4037020 con 0x7f19cc102ef0 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.019+0000 7f19d4c2f740 1 Processor -- start 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.019+0000 7f19d4c2f740 1 -- start start 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19c9ffb640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 0x7f19b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23980 0x564c57b2dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c57a88620 con 0x564c57b2e240 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c57a2fe50 con 0x564c57b23060 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c57a845e0 con 0x564c57b23980 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 0x7f19b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19c9ffb640 1 -- 172.21.15.16:0/1719377228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f19c40f00e0 con 0x7f19cc102ef0 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:54008/0 (socket says 172.21.15.16:54008) 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41336/0 (socket says 172.21.15.16:41336) 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 -- 172.21.15.16:0/1216304753 learned_addr learned my addr 172.21.15.16:0/1216304753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23980 0x564c57b2dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 -- 172.21.15.16:0/1216304753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23980 msgr2=0x564c57b2dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23980 0x564c57b2dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 -- 172.21.15.16:0/1216304753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 msgr2=0x564c57b30630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19cbfff640 1 -- 172.21.15.16:0/1216304753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c57a20d40 con 0x564c57b23060 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19d0df3640 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.020+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 0x7f19b80a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19cc10df50 tx=0x7f19b40127c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19cbfff640 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0114f0 tx=0x7f19bc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4072ac654f5f504 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19c27fc640 1 -- 172.21.15.16:0/1216304753 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc004020 con 0x564c57b23060 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19c27fc640 1 -- 172.21.15.16:0/1216304753 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19bc0041c0 con 0x564c57b23060 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19c27fc640 1 -- 172.21.15.16:0/1216304753 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00f920 con 0x564c57b23060 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1216304753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 msgr2=0x564c57b23440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0114f0 tx=0x7f19bc0119b0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1216304753 shutdown_connections 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57b30630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23980 0x564c57b2dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1216304753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b23440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1216304753 >> 172.21.15.16:0/1216304753 conn(0x564c57b1f5f0 msgr2=0x564c57b1f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1216304753 shutdown_connections 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.021+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1216304753 wait complete. 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 -- start start 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 Processor -- start 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b2e240 0x564c57b2bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c57b30e10 con 0x564c57b23980 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c57a10f10 con 0x564c57b23060 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c57b30f90 con 0x564c57b2e240 2024-09-12T00:59:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19d4c2f740 10 client.0 _ll_register_callbacks cb 0x564c57a34db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41338/0 (socket says 172.21.15.16:41338) 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.022+0000 7f19c3fff640 1 -- 172.21.15.16:0/4102761975 learned_addr learned my addr 172.21.15.16:0/4102761975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19cbfff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19d0df3640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b2e240 0x564c57b2bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 -- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b2e240 msgr2=0x564c57b2bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b2e240 0x564c57b2bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 -- 172.21.15.16:0/4102761975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 msgr2=0x564c57b2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 -- 172.21.15.16:0/4102761975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c57797580 con 0x564c57b23980 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19cbfff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19c3fff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f19b4010cf0 tx=0x7f19b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14759e501a3925a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4024070 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19bc0041c0 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f19b401e470 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19b4030050 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19d4c2f740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.023+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c57861960 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19bc0041c0 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f19b401c040 con 0x564c57b23980 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 10 client.16200 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 20 client.16200 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57b3e170 0x564c57b40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 4 client.16200 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c57a95b90 con 0x564c57b3e170 2024-09-12T00:59:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19cbfff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57b3e170 0x564c57b40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19ab7fe640 10 client.16200 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.024+0000 7f19cbfff640 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57b3e170 0x564c57b40630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778f3ca45b7177ef server_cookie=c340eb3f292895fb in_seq=0 out_seq=0 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19ab7fe640 1 -- 172.21.15.16:0/4102761975 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c57a95b90 con 0x564c57b3e170 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19ab7fe640 10 client.16200 handle_command_reply: tid=0 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 client.16200 shutdown 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 msgr2=0x564c57b2b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f19b4010cf0 tx=0x7f19b4004730 comp rx=0 tx=0).stop 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 shutdown_connections 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57b3e170 0x564c57b40630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b2e240 0x564c57b2bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b2b660 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19ab7fe640 0 client.16200 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/4102761975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23060 0x564c57b2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 >> 172.21.15.16:0/4102761975 conn(0x564c57b1f5f0 msgr2=0x564c57b252d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.025+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 shutdown_connections 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.026+0000 7f19d4c2f740 1 -- 172.21.15.16:0/4102761975 wait complete. 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.026+0000 7f19d4c2f740 20 client.16200 trim_cache size 0 max 16384 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 Processor -- start 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 -- start start 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b2e240 0x564c57b45f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b3c60 con 0x564c57b23980 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b57a0 con 0x564c57b2e240 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b4330 con 0x564c57b23060 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41354/0 (socket says 172.21.15.16:41354) 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 -- 172.21.15.16:0/1498400956 learned_addr learned my addr 172.21.15.16:0/1498400956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19cbfff640 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 -- 172.21.15.16:0/1498400956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 msgr2=0x564c57b26420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b26420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 -- 172.21.15.16:0/1498400956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b2e240 msgr2=0x564c57b45f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b2e240 0x564c57b45f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 -- 172.21.15.16:0/1498400956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c57b2d060 con 0x564c57b23980 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.043+0000 7f19c3fff640 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f19b4004700 tx=0x7f19b4005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639e53855082e1df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19c1ffb640 1 -- 172.21.15.16:0/1498400956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b402d020 con 0x564c57b23980 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19c1ffb640 1 -- 172.21.15.16:0/1498400956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b400dba0 con 0x564c57b23980 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19c1ffb640 1 -- 172.21.15.16:0/1498400956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b4024040 con 0x564c57b23980 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1498400956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 msgr2=0x564c57b1e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f19b4004700 tx=0x7f19b4005ec0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1498400956 shutdown_connections 2024-09-12T00:59:16.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b2e240 0x564c57b45f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b23980 0x564c57b1e640 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/1498400956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b26420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1498400956 >> 172.21.15.16:0/1498400956 conn(0x564c57b1f5f0 msgr2=0x564c57b1df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1498400956 shutdown_connections 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.044+0000 7f19d4c2f740 1 -- 172.21.15.16:0/1498400956 wait complete. 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 -- start start 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 Processor -- start 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b32ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23980 0x564c57b33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b50d0 con 0x564c57b2e240 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b4a00 con 0x564c57b23980 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc1b1c40 con 0x564c57b23060 2024-09-12T00:59:16.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d4c2f740 10 client.0 _ll_register_callbacks cb 0x7f19b40109f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.045+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41370/0 (socket says 172.21.15.16:41370) 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 -- 172.21.15.16:0/720003835 learned_addr learned my addr 172.21.15.16:0/720003835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19c3fff640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23980 0x564c57b33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19cbfff640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b32ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 -- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 msgr2=0x564c57b32ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b32ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 -- 172.21.15.16:0/720003835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23980 msgr2=0x564c57b33010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23980 0x564c57b33010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 -- 172.21.15.16:0/720003835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c57b237b0 con 0x564c57b2e240 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d0df3640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b8c40 tx=0x7f19c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b50a4b729bf51f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:16.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c402c040 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f19c40b8e50 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19b400dba0 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f19c40bc5b0 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d4c2f740 10 client.16206 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c57ba9910 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.046+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b400dba0 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f19c404d030 con 0x564c57b2e240 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 10 client.16206 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 10 client.16206 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 20 client.16206 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57ba9fe0 0x564c57bac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 4 client.16206 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5605cf54", "release": true, "await": true}] 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5605cf54", "release": true, "await": true}) -- 0x564c57a900a0 con 0x564c57ba9fe0 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19cbfff640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57ba9fe0 0x564c57bac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:16.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19cbfff640 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57ba9fe0 0x564c57bac3d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec438264d75566f server_cookie=213dfb91d3a6d1f4 in_seq=0 out_seq=0 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.047+0000 7f19ab7fe640 10 client.16206 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.050+0000 7f19ab7fe640 1 -- 172.21.15.16:0/720003835 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564c57a900a0 con 0x564c57ba9fe0 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.050+0000 7f19ab7fe640 10 client.16206 handle_command_reply: tid=0 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.050+0000 7f19d4c2f740 1 client.16206 shutdown 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 msgr2=0x564c57ba0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b8c40 tx=0x7f19c40095c0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 shutdown_connections 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c57ba9fe0 0x564c57bac3d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c57b2e240 0x564c57ba0ed0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c57b23980 0x564c57b33010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 --2- 172.21.15.16:0/720003835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c57b23060 0x564c57b32ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19ab7fe640 0 client.16206 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:16.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 >> 172.21.15.16:0/720003835 conn(0x564c57b1f5f0 msgr2=0x564c57b2b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 12, 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "5605cf54": { 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 12, 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:59:16.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 144.7, 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:16.102 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 shutdown_connections 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 1 -- 172.21.15.16:0/720003835 wait complete. 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d4c2f740 20 client.16206 trim_cache size 0 max 16384 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 msgr2=0x7f19b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 0x7f19b80a5230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f19cc10df50 tx=0x7f19b40127c0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 msgr2=0x7f19cc10ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.051+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f19c4004af0 tx=0x7f19c402f5b0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 shutdown_connections 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19b80a2d70 0x7f19b80a5230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19cc102ef0 0x7f19cc10ca80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19cc102570 0x7f19cc1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 --2- 172.21.15.16:0/1719377228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19cc100e20 0x7f19cc1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 >> 172.21.15.16:0/1719377228 conn(0x7f19cc0fc7d0 msgr2=0x7f19cc0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 shutdown_connections 2024-09-12T00:59:16.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:16.052+0000 7f19d2856640 1 -- 172.21.15.16:0/1719377228 wait complete. 2024-09-12T00:59:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:16 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:59:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:16 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:59:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:16 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:59:16.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:16 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:59:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:16 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:59:16.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:16 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:59:16.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:16 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T00:59:16.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:16 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T00:59:16.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:16 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T00:59:16.964 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:16.979 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5605cf54', total seconds elapsed: 30.0 2024-09-12T00:59:17.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:17 smithi016 ceph-mon[30972]: pgmap v99: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-12T00:59:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 -- 172.21.15.16:0/2460671207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0106f10 msgr2=0x7f83d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 --2- 172.21.15.16:0/2460671207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0106f10 0x7f83d010ea90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009ca0 tx=0x7f83c802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 -- 172.21.15.16:0/2460671207 shutdown_connections 2024-09-12T00:59:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 --2- 172.21.15.16:0/2460671207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0106f10 0x7f83d010ea90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 --2- 172.21.15.16:0/2460671207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0102790 0x7f83d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 --2- 172.21.15.16:0/2460671207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 0x7f83d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 -- 172.21.15.16:0/2460671207 >> 172.21.15.16:0/2460671207 conn(0x7f83d00fd8c0 msgr2=0x7f83d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 -- 172.21.15.16:0/2460671207 shutdown_connections 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.448+0000 7f83d8657640 1 -- 172.21.15.16:0/2460671207 wait complete. 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 Processor -- start 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 -- start start 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 0x7f83d0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0106f10 0x7f83d019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0111f70 con 0x7f83d0102790 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0111df0 con 0x7f83d0106f10 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d8657640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01120f0 con 0x7f83d0101de0 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41392/0 (socket says 172.21.15.16:41392) 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 -- 172.21.15.16:0/585443520 learned_addr learned my addr 172.21.15.16:0/585443520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d63f3640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 0x7f83d0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 -- 172.21.15.16:0/585443520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 msgr2=0x7f83d0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0106f10 0x7f83d019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 0x7f83d0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 -- 172.21.15.16:0/585443520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0106f10 msgr2=0x7f83d019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0106f10 0x7f83d019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.449+0000 7f83d5bf2640 1 -- 172.21.15.16:0/585443520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d019d2f0 con 0x7f83d0102790 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c920 tx=0x7f83c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83c77fe640 1 -- 172.21.15.16:0/585443520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0007c70 con 0x7f83d0102790 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83c77fe640 1 -- 172.21.15.16:0/585443520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c0007e10 con 0x7f83d0102790 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d019d5e0 con 0x7f83d0102790 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83c77fe640 1 -- 172.21.15.16:0/585443520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0018890 con 0x7f83d0102790 2024-09-12T00:59:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.450+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01a4ec0 con 0x7f83d0102790 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.452+0000 7f83c77fe640 1 -- 172.21.15.16:0/585443520 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83c0018a70 con 0x7f83d0102790 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 Processor -- start 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 -- start start 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6aeab40 0x5618d6ae9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6af6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5618d6a4f2f0 con 0x5618d6aeab40 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618d69f6cf0 con 0x5618d6af4370 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83daa30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618d6a4b2b0 con 0x5618d6ae95d0 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83c77fe640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 0x7f83bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d6bf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6aeab40 0x5618d6ae9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:33602/0 (socket says 172.21.15.16:33602) 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d63f3640 1 -- 172.21.15.16:0/1369920446 learned_addr learned my addr 172.21.15.16:0/1369920446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83c77fe640 1 -- 172.21.15.16:0/585443520 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83c00cf600 con 0x7f83d0102790 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 0x7f83bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.453+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6af6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 -- 172.21.15.16:0/1369920446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 msgr2=0x5618d6af6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6af6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 -- 172.21.15.16:0/1369920446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6aeab40 msgr2=0x5618d6ae9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6aeab40 0x5618d6ae9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 -- 172.21.15.16:0/1369920446 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d69db3a0 con 0x5618d6ae95d0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 0x7f83bc0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009bf0 tx=0x7f83c80095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00bcf0 tx=0x7f83cc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed5c82d2f3d4210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83c4ff9640 1 -- 172.21.15.16:0/1369920446 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc015070 con 0x5618d6ae95d0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83c4ff9640 1 -- 172.21.15.16:0/1369920446 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc01a940 con 0x5618d6ae95d0 2024-09-12T00:59:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.454+0000 7f83c4ff9640 1 -- 172.21.15.16:0/1369920446 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc011a10 con 0x5618d6ae95d0 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 -- 172.21.15.16:0/1369920446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 msgr2=0x5618d6ae9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00bcf0 tx=0x7f83cc00f730 comp rx=0 tx=0).stop 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 -- 172.21.15.16:0/1369920446 shutdown_connections 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6af6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6ae9a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 --2- 172.21.15.16:0/1369920446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6aeab40 0x5618d6ae9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 -- 172.21.15.16:0/1369920446 >> 172.21.15.16:0/1369920446 conn(0x5618d6ab2310 msgr2=0x5618d6ae5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 -- 172.21.15.16:0/1369920446 shutdown_connections 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.455+0000 7f83daa30740 1 -- 172.21.15.16:0/1369920446 wait complete. 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 -- start start 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 Processor -- start 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6af3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 0x5618d6af0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5618d6af7070 con 0x5618d6af4370 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618d6a0a060 con 0x5618d6aeab40 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618d6af71f0 con 0x5618d6ae95d0 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83daa30740 10 client.0 _ll_register_callbacks cb 0x5618d675d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41418/0 (socket says 172.21.15.16:41418) 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 -- 172.21.15.16:0/1042203495 learned_addr learned my addr 172.21.15.16:0/1042203495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6af3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 0x5618d6af0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 -- 172.21.15.16:0/1042203495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 msgr2=0x5618d6af3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6af3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 -- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 msgr2=0x5618d6af0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 0x5618d6af0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d6bf4640 1 -- 172.21.15.16:0/1042203495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d67c03c0 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.456+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 0x5618d6af0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f83c8035000 tx=0x7f83c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f911a2e805917a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c803cca0 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83c8004040 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc01a940 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83c803ce40 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83daa30740 10 client.16218 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618d6b02ae0 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc01a940 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83c804a070 con 0x5618d6af4370 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83daa30740 10 client.16218 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.457+0000 7f83daa30740 10 client.16218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:59:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83daa30740 20 client.16218 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b03ed0 0x5618d6b06300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83daa30740 4 client.16218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d6a5c860 con 0x5618d6b03ed0 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b03ed0 0x5618d6b06300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83d63f3640 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b03ed0 0x5618d6b06300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac5013c24944ba04 server_cookie=783458978878e5d6 in_seq=0 out_seq=0 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.458+0000 7f83b9ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83b9ffb640 1 -- 172.21.15.16:0/1042203495 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618d6a5c860 con 0x5618d6b03ed0 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83b9ffb640 10 client.16218 handle_command_reply: tid=0 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 client.16218 shutdown 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 msgr2=0x5618d6af15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f83c8035000 tx=0x7f83c802a8c0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 shutdown_connections 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b03ed0 0x5618d6b06300 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6af4370 0x5618d6af15e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6aeab40 0x5618d6af0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83b9ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 --2- 172.21.15.16:0/1042203495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6ae95d0 0x5618d6af3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 >> 172.21.15.16:0/1042203495 conn(0x5618d6ab2310 msgr2=0x5618d6aeb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 shutdown_connections 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 1 -- 172.21.15.16:0/1042203495 wait complete. 2024-09-12T00:59:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.459+0000 7f83daa30740 20 client.16218 trim_cache size 0 max 16384 2024-09-12T00:59:17.550 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:17 smithi099 ceph-mon[35654]: pgmap v99: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.472+0000 7f83daa30740 1 Processor -- start 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.472+0000 7f83daa30740 1 -- start start 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.472+0000 7f83daa30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83daa30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6af2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83daa30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83daa30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ab550 con 0x5618d6ae95d0 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83daa30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ad090 con 0x5618d6af4370 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83daa30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01abc20 con 0x5618d6aeab40 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41426/0 (socket says 172.21.15.16:41426) 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 -- 172.21.15.16:0/2311146567 learned_addr learned my addr 172.21.15.16:0/2311146567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6af2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 -- 172.21.15.16:0/2311146567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 msgr2=0x5618d6af2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6af2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 -- 172.21.15.16:0/2311146567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 msgr2=0x5618d6aec120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aec120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 -- 172.21.15.16:0/2311146567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d6af40b0 con 0x5618d6ae95d0 2024-09-12T00:59:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6af2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83d63f3640 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0115c0 tx=0x7f83cc00b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13db66f50348dba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83bbfff640 1 -- 172.21.15.16:0/2311146567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc015050 con 0x5618d6ae95d0 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83bbfff640 1 -- 172.21.15.16:0/2311146567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc004680 con 0x5618d6ae95d0 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.473+0000 7f83bbfff640 1 -- 172.21.15.16:0/2311146567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc011850 con 0x5618d6ae95d0 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- 172.21.15.16:0/2311146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 msgr2=0x5618d6aeff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0115c0 tx=0x7f83cc00b5e0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- 172.21.15.16:0/2311146567 shutdown_connections 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aec120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6af2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 --2- 172.21.15.16:0/2311146567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6aeff70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- 172.21.15.16:0/2311146567 >> 172.21.15.16:0/2311146567 conn(0x5618d6ab2310 msgr2=0x5618d6af2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- 172.21.15.16:0/2311146567 shutdown_connections 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- 172.21.15.16:0/2311146567 wait complete. 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 -- start start 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.474+0000 7f83daa30740 1 Processor -- start 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6aedb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ac9c0 con 0x5618d6ae95d0 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ac2f0 con 0x5618d6af4370 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01a9550 con 0x5618d6aeab40 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83daa30740 10 client.0 _ll_register_callbacks cb 0x7f83c8003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41438/0 (socket says 172.21.15.16:41438) 2024-09-12T00:59:17.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 -- 172.21.15.16:0/137224465 learned_addr learned my addr 172.21.15.16:0/137224465 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d6bf4640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6aedb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 -- 172.21.15.16:0/137224465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 msgr2=0x5618d6aedb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6aedb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 -- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 msgr2=0x5618d6aee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 -- 172.21.15.16:0/137224465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d6aee690 con 0x5618d6ae95d0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83d63f3640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006010 tx=0x7f83cc00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5518e2f95558a682 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.475+0000 7f83b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc035020 con 0x5618d6ae95d0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83cc005450 con 0x5618d6ae95d0 2024-09-12T00:59:17.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc004680 con 0x5618d6ae95d0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83cc015020 con 0x5618d6ae95d0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 10 client.16224 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618d6b6e9a0 con 0x5618d6ae95d0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc004680 con 0x5618d6ae95d0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83cc038020 con 0x5618d6ae95d0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 10 client.16224 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 20 client.16224 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b6ef40 0x5618d6b71360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 4 client.16224 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.476+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618d6a56d70 con 0x5618d6b6ef40 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b6ef40 0x5618d6b71360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83d5bf2640 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b6ef40 0x5618d6b71360 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=952d65c7da372860 server_cookie=d05226d641b51d in_seq=0 out_seq=0 2024-09-12T00:59:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83b9ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83b9ffb640 1 -- 172.21.15.16:0/137224465 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618d6a56d70 con 0x5618d6b6ef40 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83b9ffb640 10 client.16224 handle_command_reply: tid=0 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 client.16224 shutdown 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 msgr2=0x5618d6afc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006010 tx=0x7f83cc00fce0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 shutdown_connections 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618d6b6ef40 0x5618d6b71360 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618d6af4370 0x5618d6aee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618d6aeab40 0x5618d6aedb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 --2- 172.21.15.16:0/137224465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618d6ae95d0 0x5618d6afc9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 >> 172.21.15.16:0/137224465 conn(0x5618d6ab2310 msgr2=0x5618d6af3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83b9ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 shutdown_connections 2024-09-12T00:59:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.477+0000 7f83daa30740 1 -- 172.21.15.16:0/137224465 wait complete. 2024-09-12T00:59:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83daa30740 20 client.16224 trim_cache size 0 max 16384 2024-09-12T00:59:17.561 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 msgr2=0x7f83bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 0x7f83bc0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009bf0 tx=0x7f83c80095c0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 msgr2=0x7f83d0198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c920 tx=0x7f83c000cde0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 shutdown_connections 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83bc0a2cf0 0x7f83bc0a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0106f10 0x7f83d019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0102790 0x7f83d0198880 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 --2- 172.21.15.16:0/585443520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0101de0 0x7f83d0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 >> 172.21.15.16:0/585443520 conn(0x7f83d00fd8c0 msgr2=0x7f83d00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 shutdown_connections 2024-09-12T00:59:17.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:17.478+0000 7f83d8657640 1 -- 172.21.15.16:0/585443520 wait complete. 2024-09-12T00:59:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:17 smithi159 ceph-mon[35566]: pgmap v99: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-12T00:59:18.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:59:18.460 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 -- 172.21.15.16:0/1294810535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120102d30 msgr2=0x7f0120103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 --2- 172.21.15.16:0/1294810535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120102d30 0x7f0120103110 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0110009930 tx=0x7f011002a420 comp rx=0 tx=0).stop 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 -- 172.21.15.16:0/1294810535 shutdown_connections 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 --2- 172.21.15.16:0/1294810535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 --2- 172.21.15.16:0/1294810535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01201036e0 0x7f0120107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 --2- 172.21.15.16:0/1294810535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120102d30 0x7f0120103110 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 -- 172.21.15.16:0/1294810535 >> 172.21.15.16:0/1294810535 conn(0x7f01200fc7d0 msgr2=0x7f01200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 -- 172.21.15.16:0/1294810535 shutdown_connections 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.955+0000 7f0126905640 1 -- 172.21.15.16:0/1294810535 wait complete. 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.956+0000 7f0126905640 1 Processor -- start 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.956+0000 7f0126905640 1 -- start start 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.956+0000 7f0126905640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.956+0000 7f0126905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01201036e0 0x7f0120198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.956+0000 7f0126905640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f0126905640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01201120a0 con 0x7f01201036e0 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f0126905640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111f20 con 0x7f0120102d30 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f0126905640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0120112220 con 0x7f01201082a0 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:44446/0 (socket says 172.21.15.16:44446) 2024-09-12T00:59:18.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 -- 172.21.15.16:0/458235186 learned_addr learned my addr 172.21.15.16:0/458235186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f0124ea2640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011f7fe640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01201036e0 0x7f0120198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 -- 172.21.15.16:0/458235186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 msgr2=0x7f012019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 -- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01201036e0 msgr2=0x7f0120198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01201036e0 0x7f0120198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 -- 172.21.15.16:0/458235186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012019d570 con 0x7f0120102d30 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f0124ea2640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.957+0000 7f011ffff640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0110009230 tx=0x7f0110004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.958+0000 7f011d7fa640 1 -- 172.21.15.16:0/458235186 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011003f030 con 0x7f0120102d30 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.958+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012019d800 con 0x7f0120102d30 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.958+0000 7f011d7fa640 1 -- 172.21.15.16:0/458235186 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110003ad0 con 0x7f0120102d30 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.958+0000 7f011d7fa640 1 -- 172.21.15.16:0/458235186 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035070 con 0x7f0120102d30 2024-09-12T00:59:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.958+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a5140 con 0x7f0120102d30 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 Processor -- start 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 -- start start 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ce952f0 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ce3ccf0 con 0x55ad6cf30b40 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.960+0000 7f0128cde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ce912b0 con 0x55ad6cf3a370 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f0124ea2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f0124ea2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:44448/0 (socket says 172.21.15.16:44448) 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f0124ea2640 1 -- 172.21.15.16:0/3678932074 learned_addr learned my addr 172.21.15.16:0/3678932074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011d7fa640 1 -- 172.21.15.16:0/458235186 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0110042030 con 0x7f0120102d30 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 -- 172.21.15.16:0/3678932074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 msgr2=0x55ad6cf2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011ffff640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 -- 172.21.15.16:0/3678932074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 msgr2=0x55ad6cf2fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 -- 172.21.15.16:0/3678932074 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6ce213a0 con 0x55ad6cf3a370 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f0124ea2640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011ffff640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.961+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3c760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f012006a370 tx=0x7f011400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd4414692c4b059 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f01067fc640 1 -- 172.21.15.16:0/3678932074 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114005220 con 0x55ad6cf3a370 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f01067fc640 1 -- 172.21.15.16:0/3678932074 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114004c80 con 0x55ad6cf3a370 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f01067fc640 1 -- 172.21.15.16:0/3678932074 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114018830 con 0x55ad6cf3a370 2024-09-12T00:59:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f011d7fa640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 0x7f01080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0124ea2640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 0x7f01080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f011d7fa640 1 -- 172.21.15.16:0/458235186 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f01100efa10 con 0x7f0120102d30 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 -- 172.21.15.16:0/3678932074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 msgr2=0x55ad6cf3c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0124ea2640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 0x7f01080a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0118009fd0 tx=0x7f0118009500 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3c760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f012006a370 tx=0x7f011400f790 comp rx=0 tx=0).stop 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 -- 172.21.15.16:0/3678932074 shutdown_connections 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3c760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 --2- 172.21.15.16:0/3678932074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 -- 172.21.15.16:0/3678932074 >> 172.21.15.16:0/3678932074 conn(0x55ad6cef8310 msgr2=0x55ad6cf2bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.962+0000 7f0128cde740 1 -- 172.21.15.16:0/3678932074 shutdown_connections 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 -- 172.21.15.16:0/3678932074 wait complete. 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 -- start start 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 Processor -- start 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad6cf3d070 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ce50060 con 0x55ad6cf3a370 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad6cf3d1f0 con 0x55ad6cf30b40 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.963+0000 7f0128cde740 10 client.0 _ll_register_callbacks cb 0x55ad6cba38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:18.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46514/0 (socket says 172.21.15.16:46514) 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 -- 172.21.15.16:0/88364573 learned_addr learned my addr 172.21.15.16:0/88364573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f0124ea2640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 -- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 msgr2=0x55ad6cf36ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf36ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 -- 172.21.15.16:0/88364573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 msgr2=0x55ad6cf375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 -- 172.21.15.16:0/88364573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6cc063c0 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011f7fe640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f0124ea2640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f011ffff640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0110009a50 tx=0x7f01100bbd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0542cbaab4c5e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f00ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110049050 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0110003c80 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.964+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0114004c80 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01100bc600 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 10 client.16233 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad6cf48a50 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0114004c80 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f011003bc80 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 10 client.16233 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 20 client.16233 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cf49e40 0x55ad6cf4c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 4 client.16233 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6cea2860 con 0x55ad6cf49e40 2024-09-12T00:59:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.965+0000 7f011f7fe640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cf49e40 0x55ad6cf4c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.966+0000 7f011f7fe640 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cf49e40 0x55ad6cf4c290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=990322d255bc30d8 server_cookie=2381e6333c594b76 in_seq=0 out_seq=0 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.966+0000 7f00ff7fe640 10 client.16233 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.966+0000 7f00ff7fe640 1 -- 172.21.15.16:0/88364573 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad6cea2860 con 0x55ad6cf49e40 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.966+0000 7f00ff7fe640 10 client.16233 handle_command_reply: tid=0 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 client.16233 shutdown 2024-09-12T00:59:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 msgr2=0x55ad6cf39d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0110009a50 tx=0x7f01100bbd90 comp rx=0 tx=0).stop 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 shutdown_connections 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cf49e40 0x55ad6cf4c290 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 --2- 172.21.15.16:0/88364573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf39d40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 >> 172.21.15.16:0/88364573 conn(0x55ad6cef8310 msgr2=0x55ad6cf31930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f00ff7fe640 0 client.16233 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 shutdown_connections 2024-09-12T00:59:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 1 -- 172.21.15.16:0/88364573 wait complete. 2024-09-12T00:59:18.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.967+0000 7f0128cde740 20 client.16233 trim_cache size 0 max 16384 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 Processor -- start 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 -- start start 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf32120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ad7f0 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01201af330 con 0x55ad6cf30b40 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0128cde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01201adec0 con 0x55ad6cf3a370 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46518/0 (socket says 172.21.15.16:46518) 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f011ffff640 1 -- 172.21.15.16:0/3976803927 learned_addr learned my addr 172.21.15.16:0/3976803927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f0124ea2640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf32120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.993+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 -- 172.21.15.16:0/3976803927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 msgr2=0x55ad6cf32120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf32120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 -- 172.21.15.16:0/3976803927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 msgr2=0x55ad6cf46310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf46310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 -- 172.21.15.16:0/3976803927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6cf30760 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011f7fe640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f011ffff640 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f01100ed420 tx=0x7f01100bbe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db00919918af1401 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:18.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f0105ffb640 1 -- 172.21.15.16:0/3976803927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110049070 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f0105ffb640 1 -- 172.21.15.16:0/3976803927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01100c02b0 con 0x55ad6cf2f5d0 2024-09-12T00:59:18.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.994+0000 7f0105ffb640 1 -- 172.21.15.16:0/3976803927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100bbf00 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- 172.21.15.16:0/3976803927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 msgr2=0x55ad6cf2be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f01100ed420 tx=0x7f01100bbe00 comp rx=0 tx=0).stop 2024-09-12T00:59:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- 172.21.15.16:0/3976803927 shutdown_connections 2024-09-12T00:59:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf3a370 0x55ad6cf32120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf30b40 0x55ad6cf46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 --2- 172.21.15.16:0/3976803927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf2be10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- 172.21.15.16:0/3976803927 >> 172.21.15.16:0/3976803927 conn(0x55ad6cef8310 msgr2=0x55ad6cf2b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- 172.21.15.16:0/3976803927 shutdown_connections 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- 172.21.15.16:0/3976803927 wait complete. 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 -- start start 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.995+0000 7f0128cde740 1 Processor -- start 2024-09-12T00:59:19.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01201aec60 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ae590 con 0x55ad6cf3a370 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01201ab7d0 con 0x55ad6cf30b40 2024-09-12T00:59:19.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0128cde740 10 client.0 _ll_register_callbacks cb 0x7f0110031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46534/0 (socket says 172.21.15.16:46534) 2024-09-12T00:59:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 -- 172.21.15.16:0/72319672 learned_addr learned my addr 172.21.15.16:0/72319672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f0124ea2640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:19.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011f7fe640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 -- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 msgr2=0x55ad6cf3d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf3d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 -- 172.21.15.16:0/72319672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 msgr2=0x55ad6cf3dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 -- 172.21.15.16:0/72319672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6cfb2cb0 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f011ffff640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01100bbc00 tx=0x7f01100bbd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d9708d4d747d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.996+0000 7f00ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110049070 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0110031dd0 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01100c02b0 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01100bbf00 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 10 client.16239 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad6cfb2cb0 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01100c4b70 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f01100c46e0 con 0x55ad6cf2f5d0 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 10 client.16239 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 10 client.16239 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:59:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 20 client.16239 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cfb8650 0x55ad6cfbaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 4 client.16239 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.997+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f01100c02b0 con 0x55ad6cfb8650 2024-09-12T00:59:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f011f7fe640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cfb8650 0x55ad6cfbaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f011f7fe640 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cfb8650 0x55ad6cfbaa40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10a2547c60287434 server_cookie=49c772626330ca40 in_seq=0 out_seq=0 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f00ff7fe640 10 client.16239 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f00ff7fe640 1 -- 172.21.15.16:0/72319672 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f01100c02b0 con 0x55ad6cfb8650 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f00ff7fe640 10 client.16239 handle_command_reply: tid=0 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f0128cde740 1 client.16239 shutdown 2024-09-12T00:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 msgr2=0x55ad6cf3d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f01100bbc00 tx=0x7f01100bbd90 comp rx=0 tx=0).stop 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 shutdown_connections 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.998+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ad6cfb8650 0x55ad6cfbaa40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad6cf3a370 0x55ad6cf3dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad6cf30b40 0x55ad6cf3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 --2- 172.21.15.16:0/72319672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad6cf2f5d0 0x55ad6cf3d440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 >> 172.21.15.16:0/72319672 conn(0x55ad6cef8310 msgr2=0x55ad6cf2a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f00ff7fe640 0 client.16239 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 shutdown_connections 2024-09-12T00:59:19.014 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 1 -- 172.21.15.16:0/72319672 wait complete. 2024-09-12T00:59:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0128cde740 20 client.16239 trim_cache size 0 max 16384 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 msgr2=0x7f01080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 0x7f01080a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0118009fd0 tx=0x7f0118009500 comp rx=0 tx=0).stop 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 msgr2=0x7f01201985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0110009230 tx=0x7f0110004270 comp rx=0 tx=0).stop 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 shutdown_connections 2024-09-12T00:59:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f01080a2cf0 0x7f01080a51b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01201082a0 0x7f012019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01201036e0 0x7f0120198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 --2- 172.21.15.16:0/458235186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0120102d30 0x7f01201985c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:18.999+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 >> 172.21.15.16:0/458235186 conn(0x7f01200fc7d0 msgr2=0x7f0120105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:19.000+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 shutdown_connections 2024-09-12T00:59:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:19.000+0000 7f0126905640 1 -- 172.21.15.16:0/458235186 wait complete. 2024-09-12T00:59:19.522 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:19 smithi016 ceph-mon[30972]: pgmap v100: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T00:59:19.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:19 smithi099 ceph-mon[35654]: pgmap v100: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T00:59:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:19 smithi159 ceph-mon[35566]: pgmap v100: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-12T00:59:19.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:59:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/1357432932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac106f50 msgr2=0x7f6fac10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/1357432932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac106f50 0x7f6fac10eaf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009cd0 tx=0x7f6fa402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/1357432932 shutdown_connections 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/1357432932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac106f50 0x7f6fac10eaf0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/1357432932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fac100ca0 0x7f6fac106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/1357432932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac1002f0 0x7f6fac1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/1357432932 >> 172.21.15.16:0/1357432932 conn(0x7f6fac077210 msgr2=0x7f6fac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/1357432932 shutdown_connections 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/1357432932 wait complete. 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.556+0000 7f6fb1fcb640 1 Processor -- start 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 -- start start 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 0x7f6fac1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fac106f50 0x7f6fac101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac111fa0 con 0x7f6fac1002f0 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac111e20 con 0x7f6fac106f50 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fb1fcb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac112120 con 0x7f6fac100ca0 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46552/0 (socket says 172.21.15.16:46552) 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 -- 172.21.15.16:0/3059241422 learned_addr learned my addr 172.21.15.16:0/3059241422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6faaffd640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 0x7f6fac1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 -- 172.21.15.16:0/3059241422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 msgr2=0x7f6fac1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 0x7f6fac1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 -- 172.21.15.16:0/3059241422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fac106f50 msgr2=0x7f6fac101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fac106f50 0x7f6fac101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6fab7fe640 1 -- 172.21.15.16:0/3059241422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac1a6920 con 0x7f6fac1002f0 2024-09-12T00:59:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.557+0000 7f6faaffd640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 0x7f6fac1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00d950 tx=0x7f6f9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fa8ff9640 1 -- 172.21.15.16:0/3059241422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c017070 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fa8ff9640 1 -- 172.21.15.16:0/3059241422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c009c30 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac1a6c10 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fa8ff9640 1 -- 172.21.15.16:0/3059241422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c00f6b0 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.558+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac1a71a0 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.560+0000 7f6fa8ff9640 1 -- 172.21.15.16:0/3059241422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f9c0109d0 con 0x7f6fac1002f0 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.560+0000 7f6fb43a4740 1 Processor -- start 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 -- start start 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616db99e2f0 con 0x5616dba39b40 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5616db945cf0 con 0x5616dba43370 2024-09-12T00:59:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5616db99a2b0 con 0x5616dba385d0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fa8ff9640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 0x7f6f940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46560/0 (socket says 172.21.15.16:46560) 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 -- 172.21.15.16:0/1612805371 learned_addr learned my addr 172.21.15.16:0/1612805371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 0x7f6f940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fa8ff9640 1 -- 172.21.15.16:0/3059241422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6f9c0cf060 con 0x7f6fac1002f0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6faaffd640 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 -- 172.21.15.16:0/1612805371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 msgr2=0x5616dba38a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba38a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 -- 172.21.15.16:0/1612805371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 msgr2=0x5616dba45760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba45760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.561+0000 7f6fabfff640 1 -- 172.21.15.16:0/1612805371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616db92a3a0 con 0x5616dba39b40 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.562+0000 7f6fabfff640 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005e30 tx=0x7f6fa4031d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5913c10adbb8be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.562+0000 7f6fabfff640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 0x7f6f940a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402ae00 tx=0x7f6fa403f000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.562+0000 7f6f91ffb640 1 -- 172.21.15.16:0/1612805371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4009860 con 0x5616dba39b40 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.562+0000 7f6f91ffb640 1 -- 172.21.15.16:0/1612805371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4036050 con 0x5616dba39b40 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.562+0000 7f6f91ffb640 1 -- 172.21.15.16:0/1612805371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa403c410 con 0x5616dba39b40 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1612805371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 msgr2=0x5616dba38090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4005e30 tx=0x7f6fa4031d50 comp rx=0 tx=0).stop 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1612805371 shutdown_connections 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba45760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba38a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1612805371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba38090 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1612805371 >> 172.21.15.16:0/1612805371 conn(0x5616dba01310 msgr2=0x5616dba34c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1612805371 shutdown_connections 2024-09-12T00:59:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1612805371 wait complete. 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 -- start start 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.563+0000 7f6fb43a4740 1 Processor -- start 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba40650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616dba46010 con 0x5616dba39b40 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5616db959060 con 0x5616dba43370 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5616dba46190 con 0x5616dba385d0 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 10 client.0 _ll_register_callbacks cb 0x5616db6ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46562/0 (socket says 172.21.15.16:46562) 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 -- 172.21.15.16:0/3561076068 learned_addr learned my addr 172.21.15.16:0/3561076068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fabfff640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba40650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 -- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 msgr2=0x5616dba42d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba42d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 -- 172.21.15.16:0/3561076068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 msgr2=0x5616dba40650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba40650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 -- 172.21.15.16:0/3561076068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616db70f3c0 con 0x5616dba39b40 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6faaffd640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000bec0 tx=0x7f6fa0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305a9f5bd9bed97e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6f8affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0004230 con 0x5616dba39b40 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6fa00043d0 con 0x5616dba39b40 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.564+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa4036050 con 0x5616dba39b40 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fa0010760 con 0x5616dba39b40 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616dba51b40 con 0x5616dba39b40 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4036050 con 0x5616dba39b40 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fa0003ce0 con 0x5616dba39b40 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 20 client.16251 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dba52fa0 0x5616dba553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 4 client.16251 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616db9ab860 con 0x5616dba52fa0 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.565+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dba52fa0 0x5616dba553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.566+0000 7f6f8affd640 10 client.16251 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.566+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dba52fa0 0x5616dba553c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd54d60a64215d4 server_cookie=71685d5aaff2083b in_seq=0 out_seq=0 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.566+0000 7f6f8affd640 1 -- 172.21.15.16:0/3561076068 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616db9ab860 con 0x5616dba52fa0 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.566+0000 7f6f8affd640 10 client.16251 handle_command_reply: tid=0 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.566+0000 7f6fb43a4740 1 client.16251 shutdown 2024-09-12T00:59:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 msgr2=0x5616dba40040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000bec0 tx=0x7f6fa0012820 comp rx=0 tx=0).stop 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 shutdown_connections 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dba52fa0 0x5616dba553c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba43370 0x5616dba40650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba40040 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/3561076068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba385d0 0x5616dba42d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 >> 172.21.15.16:0/3561076068 conn(0x5616dba01310 msgr2=0x5616dba3a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6f8affd640 0 client.16251 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 shutdown_connections 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 1 -- 172.21.15.16:0/3561076068 wait complete. 2024-09-12T00:59:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.567+0000 7f6fb43a4740 20 client.16251 trim_cache size 0 max 16384 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 Processor -- start 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 -- start start 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 0x5616dba4f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1afa50 con 0x5616dba385d0 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1b1590 con 0x5616dba39b40 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1b0120 con 0x5616dba43370 2024-09-12T00:59:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46566/0 (socket says 172.21.15.16:46566) 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 -- 172.21.15.16:0/977848285 learned_addr learned my addr 172.21.15.16:0/977848285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fabfff640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 -- 172.21.15.16:0/977848285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 msgr2=0x5616dba3f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6faaffd640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 0x5616dba4f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 -- 172.21.15.16:0/977848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 msgr2=0x5616dba4f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 0x5616dba4f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.580+0000 7f6fab7fe640 1 -- 172.21.15.16:0/977848285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dba409c0 con 0x5616dba385d0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fabfff640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6faaffd640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 0x5616dba4f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009610 tx=0x7f6f9c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4e53e59794bc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6f917fa640 1 -- 172.21.15.16:0/977848285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c020040 con 0x5616dba385d0 2024-09-12T00:59:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6f917fa640 1 -- 172.21.15.16:0/977848285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c0a4220 con 0x5616dba385d0 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6f917fa640 1 -- 172.21.15.16:0/977848285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c09ac60 con 0x5616dba385d0 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 -- 172.21.15.16:0/977848285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 msgr2=0x5616dba34d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009610 tx=0x7f6f9c00bf60 comp rx=0 tx=0).stop 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 -- 172.21.15.16:0/977848285 shutdown_connections 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba39b40 0x5616dba4f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.581+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/977848285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba385d0 0x5616dba34d70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- 172.21.15.16:0/977848285 >> 172.21.15.16:0/977848285 conn(0x5616dba01310 msgr2=0x5616dba3e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- 172.21.15.16:0/977848285 shutdown_connections 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- 172.21.15.16:0/977848285 wait complete. 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- start start 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 Processor -- start 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba385d0 0x5616dba4b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1b0ec0 con 0x5616dba39b40 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1b07f0 con 0x5616dba385d0 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1ada30 con 0x5616dba43370 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.582+0000 7f6fb43a4740 10 client.0 _ll_register_callbacks cb 0x7f6fa000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46582/0 (socket says 172.21.15.16:46582) 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 -- 172.21.15.16:0/1384778591 learned_addr learned my addr 172.21.15.16:0/1384778591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba385d0 0x5616dba4b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6fabfff640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 -- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 msgr2=0x5616dba3d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 -- 172.21.15.16:0/1384778591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba385d0 msgr2=0x5616dba4b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba385d0 0x5616dba4b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 -- 172.21.15.16:0/1384778591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dba3d6e0 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6faaffd640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x5616dba41680 tx=0x7f6fa000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5461c64a6ce5945 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6f8affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0000e60 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c0a4220 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6fa001bce0 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fa001b920 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6fb43a4740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.583+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616dbabda10 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9c0a4220 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6fa0004680 con 0x5616dba39b40 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 20 client.16257 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dbabe1c0 0x5616dbac05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 4 client.16257 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616db9a5d70 con 0x5616dbabe1c0 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.584+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dbabe1c0 0x5616dbac05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fab7fe640 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dbabe1c0 0x5616dbac05e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb68d389508f4fac server_cookie=735da231c3123709 in_seq=0 out_seq=0 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6f8affd640 10 client.16257 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6f8affd640 1 -- 172.21.15.16:0/1384778591 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616db9a5d70 con 0x5616dbabe1c0 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6f8affd640 10 client.16257 handle_command_reply: tid=0 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 client.16257 shutdown 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 msgr2=0x5616dba3cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x5616dba41680 tx=0x7f6fa000b8d0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 shutdown_connections 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616dbabe1c0 0x5616dbac05e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616dba43370 0x5616dba3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616dba39b40 0x5616dba3cbb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 --2- 172.21.15.16:0/1384778591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616dba385d0 0x5616dba4b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 >> 172.21.15.16:0/1384778591 conn(0x5616dba01310 msgr2=0x5616dba3f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6f8affd640 0 client.16257 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 shutdown_connections 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 1 -- 172.21.15.16:0/1384778591 wait complete. 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.585+0000 7f6fb43a4740 20 client.16257 trim_cache size 0 max 16384 2024-09-12T00:59:20.591 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:20.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 msgr2=0x7f6f940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 0x7f6f940a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fa402ae00 tx=0x7f6fa403f000 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 msgr2=0x7f6fac101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00d950 tx=0x7f6f9c00de10 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 shutdown_connections 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f940a2d70 0x7f6f940a5230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fac106f50 0x7f6fac101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fac100ca0 0x7f6fac1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 --2- 172.21.15.16:0/3059241422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fac1002f0 0x7f6fac101270 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 >> 172.21.15.16:0/3059241422 conn(0x7f6fac077210 msgr2=0x7f6fac105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 shutdown_connections 2024-09-12T00:59:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:20.586+0000 7f6fb1fcb640 1 -- 172.21.15.16:0/3059241422 wait complete. 2024-09-12T00:59:21.468 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:21 smithi016 ceph-mon[30972]: pgmap v101: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-09-12T00:59:21.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:21 smithi099 ceph-mon[35654]: pgmap v101: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-09-12T00:59:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:21 smithi159 ceph-mon[35566]: pgmap v101: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 14 op/s 2024-09-12T00:59:22.498 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:59:22.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:59:22.770 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 -- 172.21.15.16:0/1090654144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150101410 msgr2=0x7f615010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 --2- 172.21.15.16:0/1090654144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150101410 0x7f615010d940 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6148009cd0 tx=0x7f614802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 -- 172.21.15.16:0/1090654144 shutdown_connections 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 --2- 172.21.15.16:0/1090654144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150101410 0x7f615010d940 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 --2- 172.21.15.16:0/1090654144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150100a70 0x7f6150100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 --2- 172.21.15.16:0/1090654144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150106a70 0x7f6150106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 -- 172.21.15.16:0/1090654144 >> 172.21.15.16:0/1090654144 conn(0x7f61500fc790 msgr2=0x7f61500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 -- 172.21.15.16:0/1090654144 shutdown_connections 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.251+0000 7f6155fb6640 1 -- 172.21.15.16:0/1090654144 wait complete. 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 Processor -- start 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 -- start start 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150101410 0x7f6150102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106a70 0x7f6150103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6150110e70 con 0x7f6150100a70 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6150110cf0 con 0x7f6150106a70 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f6155fb6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6150110ff0 con 0x7f6150101410 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46588/0 (socket says 172.21.15.16:46588) 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f614f7fe640 1 -- 172.21.15.16:0/219991573 learned_addr learned my addr 172.21.15.16:0/219991573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f614effd640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150101410 0x7f6150102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.252+0000 7f614ffff640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106a70 0x7f6150103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 -- 172.21.15.16:0/219991573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150101410 msgr2=0x7f6150102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150101410 0x7f6150102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 -- 172.21.15.16:0/219991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106a70 msgr2=0x7f6150103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106a70 0x7f6150103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 -- 172.21.15.16:0/219991573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61501a68f0 con 0x7f6150100a70 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614f7fe640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f614000e9f0 tx=0x7f614000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614cff9640 1 -- 172.21.15.16:0/219991573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614000cd80 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614cff9640 1 -- 172.21.15.16:0/219991573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6140005500 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61501a6be0 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f614cff9640 1 -- 172.21.15.16:0/219991573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140010690 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.253+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a70d0 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.255+0000 7f614cff9640 1 -- 172.21.15.16:0/219991573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f61400046c0 con 0x7f6150100a70 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 Processor -- start 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 -- start start 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b84af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b858130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a58b7b0280 con 0x55a58b84b480 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a58b757b10 con 0x55a58b855d40 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f615838f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a58b7ac240 con 0x55a58b84ab60 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614cff9640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 0x7f61380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614ffff640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 0x7f61380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46602/0 (socket says 172.21.15.16:46602) 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614f7fe640 1 -- 172.21.15.16:0/851595373 learned_addr learned my addr 172.21.15.16:0/851595373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614effd640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b858130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614ffff640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b84af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.256+0000 7f614cff9640 1 -- 172.21.15.16:0/219991573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f61400cf880 con 0x7f6150100a70 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 -- 172.21.15.16:0/851595373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 msgr2=0x55a58b84af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b84af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 -- 172.21.15.16:0/851595373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 msgr2=0x55a58b855800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 -- 172.21.15.16:0/851595373 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58b7641f0 con 0x55a58b855d40 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614f7fe640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614ffff640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 0x7f61380a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6148009a60 tx=0x7f6148033000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f614effd640 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b858130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6144009b70 tx=0x7f6144011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eda0f2060c75860 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f6135ffb640 1 -- 172.21.15.16:0/851595373 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144004050 con 0x55a58b855d40 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f6135ffb640 1 -- 172.21.15.16:0/851595373 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61440041f0 con 0x55a58b855d40 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.257+0000 7f6135ffb640 1 -- 172.21.15.16:0/851595373 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440197a0 con 0x55a58b855d40 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- 172.21.15.16:0/851595373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 msgr2=0x55a58b858130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b858130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6144009b70 tx=0x7f6144011810 comp rx=0 tx=0).stop 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- 172.21.15.16:0/851595373 shutdown_connections 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b858130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b855800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 --2- 172.21.15.16:0/851595373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b84af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- 172.21.15.16:0/851595373 >> 172.21.15.16:0/851595373 conn(0x55a58b8470f0 msgr2=0x55a58b8474d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- 172.21.15.16:0/851595373 shutdown_connections 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- 172.21.15.16:0/851595373 wait complete. 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 -- start start 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.258+0000 7f615838f740 1 Processor -- start 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b8557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b8532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a58b858910 con 0x55a58b84b480 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a58b76af10 con 0x55a58b84ab60 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a58b858a90 con 0x55a58b855d40 2024-09-12T00:59:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f615838f740 10 client.0 _ll_register_callbacks cb 0x55a58b733a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46616/0 (socket says 172.21.15.16:46616) 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 -- 172.21.15.16:0/3616907623 learned_addr learned my addr 172.21.15.16:0/3616907623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b8557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614ffff640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b8532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 -- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 msgr2=0x55a58b8532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b8532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 -- 172.21.15.16:0/3616907623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 msgr2=0x55a58b8557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b8557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 -- 172.21.15.16:0/3616907623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58b5216a0 con 0x55a58b84b480 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f614effd640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f614401b040 tx=0x7f6144009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd475527b734e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f612effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.259+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144027070 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6144003a00 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61440041f0 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6144033050 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 10 client.16269 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a58b864450 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61440041f0 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6144004cc0 con 0x55a58b84b480 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 10 client.16269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 20 client.16269 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b865800 0x55a58b867bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 4 client.16269 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a58b7bd7f0 con 0x55a58b865800 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.260+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b865800 0x55a58b867bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.261+0000 7f612effd640 10 client.16269 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.261+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b865800 0x55a58b867bf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567721c374b82c4d server_cookie=2e0d09ce7bca33ba in_seq=0 out_seq=0 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f612effd640 1 -- 172.21.15.16:0/3616907623 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a58b7bd7f0 con 0x55a58b865800 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f612effd640 10 client.16269 handle_command_reply: tid=0 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 client.16269 shutdown 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 msgr2=0x55a58b852b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f614401b040 tx=0x7f6144009850 comp rx=0 tx=0).stop 2024-09-12T00:59:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 shutdown_connections 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b865800 0x55a58b867bf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b8532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b852b30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f612effd640 0 client.16269 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 --2- 172.21.15.16:0/3616907623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b8557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 >> 172.21.15.16:0/3616907623 conn(0x55a58b8470f0 msgr2=0x55a58b84ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 shutdown_connections 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 1 -- 172.21.15.16:0/3616907623 wait complete. 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.262+0000 7f615838f740 20 client.16269 trim_cache size 0 max 16384 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 Processor -- start 2024-09-12T00:59:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 -- start start 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b847930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b86d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61501afa90 con 0x55a58b84b480 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61501b15d0 con 0x55a58b855d40 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.279+0000 7f615838f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61501b0160 con 0x55a58b84ab60 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46618/0 (socket says 172.21.15.16:46618) 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 -- 172.21.15.16:0/3253283599 learned_addr learned my addr 172.21.15.16:0/3253283599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b847930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614ffff640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b86d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 -- 172.21.15.16:0/3253283599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 msgr2=0x55a58b847930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b847930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 -- 172.21.15.16:0/3253283599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 msgr2=0x55a58b86d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b86d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 -- 172.21.15.16:0/3253283599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58b853630 con 0x55a58b84b480 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b847930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f614effd640 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6144000c00 tx=0x7f61440097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fab46e8cd20ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f61357fa640 1 -- 172.21.15.16:0/3253283599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144027040 con 0x55a58b84b480 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f61357fa640 1 -- 172.21.15.16:0/3253283599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6144030030 con 0x55a58b84b480 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.280+0000 7f61357fa640 1 -- 172.21.15.16:0/3253283599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144019e40 con 0x55a58b84b480 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- 172.21.15.16:0/3253283599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 msgr2=0x55a58b86d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6144000c00 tx=0x7f61440097a0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- 172.21.15.16:0/3253283599 shutdown_connections 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b855d40 0x55a58b86d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b86d360 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 --2- 172.21.15.16:0/3253283599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b84ab60 0x55a58b847930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- 172.21.15.16:0/3253283599 >> 172.21.15.16:0/3253283599 conn(0x55a58b8470f0 msgr2=0x55a58b851c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- 172.21.15.16:0/3253283599 shutdown_connections 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- 172.21.15.16:0/3253283599 wait complete. 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.281+0000 7f615838f740 1 -- start start 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 Processor -- start 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b85dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b85ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61501b0f00 con 0x55a58b84b480 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61501b0830 con 0x55a58b84ab60 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ada70 con 0x55a58b855d40 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f615838f740 10 client.0 _ll_register_callbacks cb 0x7f614000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46628/0 (socket says 172.21.15.16:46628) 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 -- 172.21.15.16:0/3752505013 learned_addr learned my addr 172.21.15.16:0/3752505013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614ffff640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b85ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b85dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 -- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 msgr2=0x55a58b85ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b85ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 -- 172.21.15.16:0/3752505013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 msgr2=0x55a58b85dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b85dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614effd640 1 -- 172.21.15.16:0/3752505013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58b85e8c0 con 0x55a58b84b480 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614ffff640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b85ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.282+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b85dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f614effd640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6144011850 tx=0x7f614401ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f108d4fb32d8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144027040 con 0x55a58b84b480 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6144030030 con 0x55a58b84b480 2024-09-12T00:59:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6144005040 con 0x55a58b84b480 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6144048050 con 0x55a58b84b480 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f615838f740 10 client.16275 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a58b8d08f0 con 0x55a58b84b480 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6144030030 con 0x55a58b84b480 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.283+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6144005340 con 0x55a58b84b480 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 10 client.16275 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 20 client.16275 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b8d0f50 0x55a58b8d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 4 client.16275 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:59:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a58b7b7d00 con 0x55a58b8d0f50 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b8d0f50 0x55a58b8d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f614f7fe640 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b8d0f50 0x55a58b8d3370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b303ed26c182d7a server_cookie=1c51712ddc76650 in_seq=0 out_seq=0 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.284+0000 7f612effd640 10 client.16275 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f612effd640 1 -- 172.21.15.16:0/3752505013 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a58b7b7d00 con 0x55a58b8d0f50 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f612effd640 10 client.16275 handle_command_reply: tid=0 2024-09-12T00:59:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 client.16275 shutdown 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 msgr2=0x55a58b85e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6144011850 tx=0x7f614401ae20 comp rx=0 tx=0).stop 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 shutdown_connections 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a58b8d0f50 0x55a58b8d3370 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a58b855d40 0x55a58b85ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a58b84b480 0x55a58b85e250 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f612effd640 0 client.16275 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 --2- 172.21.15.16:0/3752505013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a58b84ab60 0x55a58b85dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.285+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 >> 172.21.15.16:0/3752505013 conn(0x55a58b8470f0 msgr2=0x55a58b84c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 shutdown_connections 2024-09-12T00:59:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f615838f740 1 -- 172.21.15.16:0/3752505013 wait complete. 2024-09-12T00:59:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f615838f740 20 client.16275 trim_cache size 0 max 16384 2024-09-12T00:59:23.306 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:23.306 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:59:23.306 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "1e546680-efae-40c7-ac5a-3fd8ef32c643", 2024-09-12T00:59:23.306 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:59:23.306 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 msgr2=0x7f61380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 0x7f61380a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6148009a60 tx=0x7f6148033000 comp rx=0 tx=0).stop 2024-09-12T00:59:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 msgr2=0x7f6150102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f614000e9f0 tx=0x7f614000edf0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 shutdown_connections 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61380a2cf0 0x7f61380a51b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106a70 0x7f6150103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6150101410 0x7f6150102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 --2- 172.21.15.16:0/219991573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150100a70 0x7f6150102a90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.286+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 >> 172.21.15.16:0/219991573 conn(0x7f61500fc790 msgr2=0x7f6150104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.287+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 shutdown_connections 2024-09-12T00:59:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:23.287+0000 7f6155fb6640 1 -- 172.21.15.16:0/219991573 wait complete. 2024-09-12T00:59:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:23 smithi099 ceph-mon[35654]: pgmap v102: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:23 smithi159 ceph-mon[35566]: pgmap v102: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:23 smithi016 ceph-mon[30972]: pgmap v102: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:24.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:59:24.477 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:24 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:59:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:24 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:24 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:59:24.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:24 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:24 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:59:24.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:24 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:24 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:59:24.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:24 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:24 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:59:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:24 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:24 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:59:24.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:24 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:24.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 -- 172.21.15.16:0/643826379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300101420 msgr2=0x7f330010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 --2- 172.21.15.16:0/643826379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300101420 0x7f330010d950 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009cd0 tx=0x7f32f802a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 -- 172.21.15.16:0/643826379 shutdown_connections 2024-09-12T00:59:24.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 --2- 172.21.15.16:0/643826379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300101420 0x7f330010d950 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 --2- 172.21.15.16:0/643826379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300100a80 0x7f3300100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 --2- 172.21.15.16:0/643826379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300106a80 0x7f3300106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 -- 172.21.15.16:0/643826379 >> 172.21.15.16:0/643826379 conn(0x7f33000fc7a0 msgr2=0x7f33000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 -- 172.21.15.16:0/643826379 shutdown_connections 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.943+0000 7f3304f1a640 1 -- 172.21.15.16:0/643826379 wait complete. 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 Processor -- start 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 -- start start 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300101420 0x7f3300198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 0x7f330019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3300110e80 con 0x7f3300100a80 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3300110d00 con 0x7f3300106a80 2024-09-12T00:59:24.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f3304f1a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3300111000 con 0x7f3300101420 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46644/0 (socket says 172.21.15.16:46644) 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f32fe59c640 1 -- 172.21.15.16:0/2117211333 learned_addr learned my addr 172.21.15.16:0/2117211333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f32f7fff640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300101420 0x7f3300198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.944+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 0x7f330019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 -- 172.21.15.16:0/2117211333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300101420 msgr2=0x7f3300198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300101420 0x7f3300198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 -- 172.21.15.16:0/2117211333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 msgr2=0x7f330019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 0x7f330019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 -- 172.21.15.16:0/2117211333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330019d3b0 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 0x7f330019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d770 tx=0x7f32f000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32f77fe640 1 -- 172.21.15.16:0/2117211333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32f77fe640 1 -- 172.21.15.16:0/2117211333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f0009820 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330019d6a0 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f32f77fe640 1 -- 172.21.15.16:0/2117211333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0004cd0 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.945+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33001a4f80 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.947+0000 7f32f77fe640 1 -- 172.21.15.16:0/2117211333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f32f00099d0 con 0x7f3300100a80 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 Processor -- start 2024-09-12T00:59:24.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 -- start start 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f64443140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64445a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f6439e2e0 con 0x560f644384a0 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f64345ce0 con 0x560f64443680 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f33072f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f6439a2a0 con 0x560f64438dc0 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f32f77fe640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 0x7f32ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f32fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f32fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46658/0 (socket says 172.21.15.16:46658) 2024-09-12T00:59:24.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.948+0000 7f32fed9d640 1 -- 172.21.15.16:0/799511083 learned_addr learned my addr 172.21.15.16:0/799511083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fe59c640 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f64443140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 0x7f32ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32f77fe640 1 -- 172.21.15.16:0/2117211333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f32f00cee80 con 0x7f3300100a80 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32f7fff640 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64445a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 -- 172.21.15.16:0/799511083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 msgr2=0x560f64443140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f64443140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 -- 172.21.15.16:0/799511083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 msgr2=0x560f64445a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64445a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 -- 172.21.15.16:0/799511083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6432a440 con 0x560f644384a0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32fed9d640 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009430 tx=0x7f32f8031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32ce84c25dcbe89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32f4ff9640 1 -- 172.21.15.16:0/799511083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f804f070 con 0x560f644384a0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32f4ff9640 1 -- 172.21.15.16:0/799511083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f8036040 con 0x560f644384a0 2024-09-12T00:59:24.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.949+0000 7f32f4ff9640 1 -- 172.21.15.16:0/799511083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f80309c0 con 0x560f644384a0 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 -- 172.21.15.16:0/799511083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 msgr2=0x560f64438880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009430 tx=0x7f32f8031c00 comp rx=0 tx=0).stop 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 0x7f32ec0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f32f804c000 tx=0x7f32f8048000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 -- 172.21.15.16:0/799511083 shutdown_connections 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64445a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f64443140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 --2- 172.21.15.16:0/799511083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64438880 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 -- 172.21.15.16:0/799511083 >> 172.21.15.16:0/799511083 conn(0x560f64434a30 msgr2=0x560f64434e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 -- 172.21.15.16:0/799511083 shutdown_connections 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.950+0000 7f33072f3740 1 -- 172.21.15.16:0/799511083 wait complete. 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 -- start start 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 Processor -- start 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f644402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64440830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f64446250 con 0x560f644384a0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f64359050 con 0x560f64443680 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f644463d0 con 0x560f64438dc0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f33072f3740 10 client.0 _ll_register_callbacks cb 0x560f64399160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46664/0 (socket says 172.21.15.16:46664) 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 -- 172.21.15.16:0/3582576230 learned_addr learned my addr 172.21.15.16:0/3582576230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32f7fff640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f644402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fed9d640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64440830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 -- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 msgr2=0x560f644402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f644402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 -- 172.21.15.16:0/3582576230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 msgr2=0x560f64440830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64440830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.951+0000 7f32fe59c640 1 -- 172.21.15.16:0/3582576230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6410f3c0 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32fe59c640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f32f0097d40 tx=0x7f32f0065ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20df68a85322dd94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0020040 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f32f0016170 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f8036040 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32f009f720 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f64451c90 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f8036040 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32f00a3b50 con 0x560f644384a0 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:24.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 20 client.16287 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f64453150 0x560f644555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 4 client.16287 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.952+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f643ab850 con 0x560f64453150 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.953+0000 7f32f7fff640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f64453150 0x560f644555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.953+0000 7f32e5ffb640 10 client.16287 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.953+0000 7f32f7fff640 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f64453150 0x560f644555a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea9346bf2bccc52 server_cookie=b7cae939f603e92 in_seq=0 out_seq=0 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f32e5ffb640 1 -- 172.21.15.16:0/3582576230 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f643ab850 con 0x560f64453150 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f32e5ffb640 10 client.16287 handle_command_reply: tid=0 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 client.16287 shutdown 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 msgr2=0x560f64443000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f32f0097d40 tx=0x7f32f0065ef0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 shutdown_connections 2024-09-12T00:59:24.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f64453150 0x560f644555a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f64440830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f64438dc0 0x560f644402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 --2- 172.21.15.16:0/3582576230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f644384a0 0x560f64443000 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 >> 172.21.15.16:0/3582576230 conn(0x560f64434a30 msgr2=0x560f6443a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f32e5ffb640 0 client.16287 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 shutdown_connections 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 1 -- 172.21.15.16:0/3582576230 wait complete. 2024-09-12T00:59:24.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.954+0000 7f33072f3740 20 client.16287 trim_cache size 0 max 16384 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.971+0000 7f33072f3740 1 Processor -- start 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 -- start start 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f6445ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6443d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33001ad710 con 0x560f64438dc0 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33001af250 con 0x560f64443680 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f33072f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33001adde0 con 0x560f644384a0 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46678/0 (socket says 172.21.15.16:46678) 2024-09-12T00:59:24.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 -- 172.21.15.16:0/479613916 learned_addr learned my addr 172.21.15.16:0/479613916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32fe59c640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f6445ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 -- 172.21.15.16:0/479613916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 msgr2=0x560f6445ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f6445ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32fed9d640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6443d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 -- 172.21.15.16:0/479613916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 msgr2=0x560f6443d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6443d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 -- 172.21.15.16:0/479613916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6443f260 con 0x560f64438dc0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32fe59c640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f6445ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.972+0000 7f32f7fff640 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005f10 tx=0x7f32e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea17018af074556f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f32e7fff640 1 -- 172.21.15.16:0/479613916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8022030 con 0x560f64438dc0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f32e7fff640 1 -- 172.21.15.16:0/479613916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e8005500 con 0x560f64438dc0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f32e7fff640 1 -- 172.21.15.16:0/479613916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e801d680 con 0x560f64438dc0 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 -- 172.21.15.16:0/479613916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 msgr2=0x560f6445b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005f10 tx=0x7f32e8011810 comp rx=0 tx=0).stop 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 -- 172.21.15.16:0/479613916 shutdown_connections 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6443d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f6445b360 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 --2- 172.21.15.16:0/479613916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f6445ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 -- 172.21.15.16:0/479613916 >> 172.21.15.16:0/479613916 conn(0x560f64434a30 msgr2=0x560f64442330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 -- 172.21.15.16:0/479613916 shutdown_connections 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.973+0000 7f33072f3740 1 -- 172.21.15.16:0/479613916 wait complete. 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 -- start start 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 Processor -- start 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f64449640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6444a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33001aeb80 con 0x560f64438dc0 2024-09-12T00:59:24.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33001ae4b0 con 0x560f64443680 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33001ab6f0 con 0x560f644384a0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f33072f3740 10 client.0 _ll_register_callbacks cb 0x7f32f009b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46688/0 (socket says 172.21.15.16:46688) 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f32f7fff640 1 -- 172.21.15.16:0/2136251607 learned_addr learned my addr 172.21.15.16:0/2136251607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.974+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f64449640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6444a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 -- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 msgr2=0x560f64449640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f64449640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 -- 172.21.15.16:0/2136251607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 msgr2=0x560f6444a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6444a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 -- 172.21.15.16:0/2136251607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f644c1460 con 0x560f64438dc0 2024-09-12T00:59:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f64449640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32fed9d640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6444a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32f7fff640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005250 tx=0x7f32e8005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd1133f674b69d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e802a030 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f32e80046c0 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e8005500 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32e8022040 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f33072f3740 10 client.16293 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f644c26c0 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.975+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e8005500 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32e80314d0 con 0x560f64438dc0 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 10 client.16293 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 20 client.16293 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f644c2db0 0x560f644c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 4 client.16293 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f643a5d60 con 0x560f644c2db0 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f644c2db0 0x560f644c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f32e5ffb640 10 client.16293 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.976+0000 7f32fe59c640 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f644c2db0 0x560f644c51d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8156fd8766e3ae3c server_cookie=16756a271342fcf6 in_seq=0 out_seq=0 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f32e5ffb640 1 -- 172.21.15.16:0/2136251607 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f643a5d60 con 0x560f644c2db0 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f32e5ffb640 10 client.16293 handle_command_reply: tid=0 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 client.16293 shutdown 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 msgr2=0x560f64449b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f32e8005250 tx=0x7f32e8005280 comp rx=0 tx=0).stop 2024-09-12T00:59:24.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 shutdown_connections 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f644c2db0 0x560f644c51d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f64443680 0x560f6444a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f64438dc0 0x560f64449b80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f32e5ffb640 0 client.16293 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 --2- 172.21.15.16:0/2136251607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f644384a0 0x560f64449640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 >> 172.21.15.16:0/2136251607 conn(0x560f64434a30 msgr2=0x560f64433350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 shutdown_connections 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 1 -- 172.21.15.16:0/2136251607 wait complete. 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f33072f3740 20 client.16293 trim_cache size 0 max 16384 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:24.988 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 msgr2=0x7f32ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 0x7f32ec0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f32f804c000 tx=0x7f32f8048000 comp rx=0 tx=0).stop 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 msgr2=0x7f3300198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.977+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d770 tx=0x7f32f000db70 comp rx=0 tx=0).stop 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 shutdown_connections 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32ec0a2cf0 0x7f32ec0a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3300106a80 0x7f330019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3300101420 0x7f3300198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 --2- 172.21.15.16:0/2117211333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3300100a80 0x7f3300198400 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 >> 172.21.15.16:0/2117211333 conn(0x7f33000fc7a0 msgr2=0x7f330006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 shutdown_connections 2024-09-12T00:59:24.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:24.978+0000 7f3304f1a640 1 -- 172.21.15.16:0/2117211333 wait complete. 2024-09-12T00:59:25.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:25 smithi099 ceph-mon[35654]: pgmap v103: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:25 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:59:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:25 smithi159 ceph-mon[35566]: pgmap v103: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:25 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:59:25.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:25 smithi016 ceph-mon[30972]: pgmap v103: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 10 op/s 2024-09-12T00:59:25.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:25 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:59:25.781 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T00:59:25.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T00:59:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- 172.21.15.16:0/2946442736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 msgr2=0x7f611410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 --2- 172.21.15.16:0/2946442736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f611410d950 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f610c009cd0 tx=0x7f610c02a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- 172.21.15.16:0/2946442736 shutdown_connections 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 --2- 172.21.15.16:0/2946442736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f611410d950 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 --2- 172.21.15.16:0/2946442736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 0x7f6114100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 --2- 172.21.15.16:0/2946442736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f6114106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- 172.21.15.16:0/2946442736 >> 172.21.15.16:0/2946442736 conn(0x7f61140fc7a0 msgr2=0x7f61140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- 172.21.15.16:0/2946442736 shutdown_connections 2024-09-12T00:59:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- 172.21.15.16:0/2946442736 wait complete. 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 Processor -- start 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.525+0000 7f611b947640 1 -- start start 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 0x7f61141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f611419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6114110ea0 con 0x7f6114101420 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6114110d20 con 0x7f6114100a80 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6114111020 con 0x7f6114106a80 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46706/0 (socket says 172.21.15.16:46706) 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 -- 172.21.15.16:0/2858007164 learned_addr learned my addr 172.21.15.16:0/2858007164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f61196e3640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 0x7f61141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6119ee4640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f611419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 -- 172.21.15.16:0/2858007164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 msgr2=0x7f611419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f611419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 -- 172.21.15.16:0/2858007164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 msgr2=0x7f61141983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 0x7f61141983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 -- 172.21.15.16:0/2858007164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611419d3a0 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6119ee4640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f611419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f6118ee2640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f610400c920 tx=0x7f610400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104007bb0 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6104007d50 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.526+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611419d690 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.527+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61040187d0 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.527+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61141a4f70 con 0x7f6114101420 2024-09-12T00:59:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.528+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6114100ee0 con 0x7f6114101420 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.529+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f610400f450 con 0x7f6114101420 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.529+0000 7f610a7fc640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 0x7f61000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.530+0000 7f61196e3640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 0x7f61000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.530+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f61040cf7f0 con 0x7f6114101420 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.530+0000 7f61196e3640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 0x7f61000a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6110009990 tx=0x7f6110009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.533+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6104014030 con 0x7f6114101420 2024-09-12T00:59:26.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.743+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6114199750 con 0x7f6114101420 2024-09-12T00:59:26.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.744+0000 7f610a7fc640 1 -- 172.21.15.16:0/2858007164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f6114199750 con 0x7f6114101420 2024-09-12T00:59:26.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T00:59:26.743 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T00:59:26.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 msgr2=0x7f61000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 0x7f61000a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6110009990 tx=0x7f6110009110 comp rx=0 tx=0).stop 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 msgr2=0x7f6114198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f610400c920 tx=0x7f610400cd20 comp rx=0 tx=0).stop 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 shutdown_connections 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f61000a2cf0 0x7f61000a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6114106a80 0x7f611419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114101420 0x7f6114198930 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 --2- 172.21.15.16:0/2858007164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6114100a80 0x7f61141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:26.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 >> 172.21.15.16:0/2858007164 conn(0x7f61140fc7a0 msgr2=0x7f611406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:26.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.746+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 shutdown_connections 2024-09-12T00:59:26.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:26.747+0000 7f611b947640 1 -- 172.21.15.16:0/2858007164 wait complete. 2024-09-12T00:59:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:27 smithi016 ceph-mon[30972]: pgmap v104: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T00:59:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:27 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2858007164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:27 smithi099 ceph-mon[35654]: pgmap v104: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T00:59:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:27 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2858007164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:27.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T00:59:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:27 smithi159 ceph-mon[35566]: pgmap v104: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-12T00:59:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:27 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2858007164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T00:59:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 -- 172.21.15.16:0/2197395043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 msgr2=0x7f75181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 --2- 172.21.15.16:0/2197395043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f75181021c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f750802a3f0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 -- 172.21.15.16:0/2197395043 shutdown_connections 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 --2- 172.21.15.16:0/2197395043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518106f10 0x7f751810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 --2- 172.21.15.16:0/2197395043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518102790 0x7f75181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 --2- 172.21.15.16:0/2197395043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f75181021c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 -- 172.21.15.16:0/2197395043 >> 172.21.15.16:0/2197395043 conn(0x7f75180fd8c0 msgr2=0x7f75180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 -- 172.21.15.16:0/2197395043 shutdown_connections 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.302+0000 7f751c884640 1 -- 172.21.15.16:0/2197395043 wait complete. 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 Processor -- start 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 -- start start 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518102790 0x7f7518198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518106f10 0x7f751819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111fb0 con 0x7f7518101de0 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111e30 con 0x7f7518106f10 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751c884640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7518112130 con 0x7f7518102790 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46720/0 (socket says 172.21.15.16:46720) 2024-09-12T00:59:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.303+0000 7f751659c640 1 -- 172.21.15.16:0/1884007562 learned_addr learned my addr 172.21.15.16:0/1884007562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f7516d9d640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518106f10 0x7f751819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f750ffff640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518102790 0x7f7518198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 -- 172.21.15.16:0/1884007562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518102790 msgr2=0x7f7518198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518102790 0x7f7518198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 -- 172.21.15.16:0/1884007562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518106f10 msgr2=0x7f751819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518106f10 0x7f751819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 -- 172.21.15.16:0/1884007562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751819d4c0 con 0x7f7518101de0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751659c640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f750f7fe640 1 -- 172.21.15.16:0/1884007562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750803f030 con 0x7f7518101de0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751819d750 con 0x7f7518101de0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f750f7fe640 1 -- 172.21.15.16:0/1884007562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7508003ce0 con 0x7f7518101de0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f750f7fe640 1 -- 172.21.15.16:0/1884007562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508035070 con 0x7f7518101de0 2024-09-12T00:59:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.304+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a5090 con 0x7f7518101de0 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.306+0000 7f750f7fe640 1 -- 172.21.15.16:0/1884007562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7508042030 con 0x7f7518101de0 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 Processor -- start 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 -- start start 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3394162f0 con 0x55d3394b05d0 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d3393bdcf0 con 0x55d3394b1b40 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751ec5d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d3394122b0 con 0x55d3394bb370 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f750ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46732/0 (socket says 172.21.15.16:46732) 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f751659c640 1 -- 172.21.15.16:0/3007543674 learned_addr learned my addr 172.21.15.16:0/3007543674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.307+0000 7f750f7fe640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 0x7f75040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f7516d9d640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f750f7fe640 1 -- 172.21.15.16:0/1884007562 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f75080efb90 con 0x7f7518101de0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 -- 172.21.15.16:0/3007543674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 msgr2=0x55d3394bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 -- 172.21.15.16:0/3007543674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 msgr2=0x55d3394b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 -- 172.21.15.16:0/3007543674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3393a23a0 con 0x55d3394b05d0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f7516d9d640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f750ffff640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f7516d9d640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 0x7f75040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f751659c640 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f75080b8600 tx=0x7f750802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2554823c091302ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f750cff9640 1 -- 172.21.15.16:0/3007543674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508049070 con 0x55d3394b05d0 2024-09-12T00:59:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f750cff9640 1 -- 172.21.15.16:0/3007543674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750803b580 con 0x55d3394b05d0 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.308+0000 7f750cff9640 1 -- 172.21.15.16:0/3007543674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080bc1a0 con 0x55d3394b05d0 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 -- 172.21.15.16:0/3007543674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 msgr2=0x55d3394b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f75080b8600 tx=0x7f750802a3f0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f7516d9d640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 0x7f75040a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7510005c40 tx=0x7f7510004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 -- 172.21.15.16:0/3007543674 shutdown_connections 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394b0a10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3007543674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 -- 172.21.15.16:0/3007543674 >> 172.21.15.16:0/3007543674 conn(0x55d339479310 msgr2=0x55d3394acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 -- 172.21.15.16:0/3007543674 shutdown_connections 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.309+0000 7f751ec5d740 1 -- 172.21.15.16:0/3007543674 wait complete. 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 -- start start 2024-09-12T00:59:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 Processor -- start 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394b05d0 0x55d3394b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394bb370 0x55d3394b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3394bde30 con 0x55d3394b1b40 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d3393d1060 con 0x55d3394bb370 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d3394bdfb0 con 0x55d3394b05d0 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751ec5d740 10 client.0 _ll_register_callbacks cb 0x55d3391248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46746/0 (socket says 172.21.15.16:46746) 2024-09-12T00:59:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 -- 172.21.15.16:0/2295981871 learned_addr learned my addr 172.21.15.16:0/2295981871 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f751659c640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394b05d0 0x55d3394b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f7516d9d640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394bb370 0x55d3394b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 -- 172.21.15.16:0/2295981871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394b05d0 msgr2=0x55d3394b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394b05d0 0x55d3394b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 -- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394bb370 msgr2=0x55d3394b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.310+0000 7f750ffff640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394bb370 0x55d3394b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f750ffff640 1 -- 172.21.15.16:0/2295981871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3391873c0 con 0x55d3394b1b40 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f750ffff640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f750000d440 tx=0x7f750000d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fe26bdb34bccc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f74fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750001c060 con 0x55d3394b1b40 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7500015750 con 0x55d3394b1b40 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d339423860 con 0x55d3394b1b40 2024-09-12T00:59:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7500015950 con 0x55d3394b1b40 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f751ec5d740 10 client.16311 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d3394c9880 con 0x55d3394b1b40 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.311+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d339423860 con 0x55d3394b1b40 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7500017480 con 0x55d3394b1b40 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 10 client.16311 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 20 client.16311 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d3394cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 4 client.16311 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d33941dd70 con 0x55d3394caed0 2024-09-12T00:59:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751659c640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d3394cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f751659c640 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d3394cd2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a55f11d173c34e2 server_cookie=ab90b526a489cdcd in_seq=0 out_seq=0 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.312+0000 7f74fdffb640 10 client.16311 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f74fdffb640 1 -- 172.21.15.16:0/2295981871 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d33941dd70 con 0x55d3394caed0 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f74fdffb640 10 client.16311 handle_command_reply: tid=0 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 client.16311 shutdown 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 msgr2=0x55d3394b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f750000d440 tx=0x7f750000d900 comp rx=0 tx=0).stop 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 shutdown_connections 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d3394cd2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394bb370 0x55d3394b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f74fdffb640 0 client.16311 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394b8630 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2295981871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394b05d0 0x55d3394b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 >> 172.21.15.16:0/2295981871 conn(0x55d339479310 msgr2=0x55d3394b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 shutdown_connections 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.313+0000 7f751ec5d740 1 -- 172.21.15.16:0/2295981871 wait complete. 2024-09-12T00:59:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.314+0000 7f751ec5d740 20 client.16311 trim_cache size 0 max 16384 2024-09-12T00:59:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.326+0000 7f751ec5d740 1 Processor -- start 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.326+0000 7f751ec5d740 1 -- start start 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b05d0 0x55d3394d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394b9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ad840 con 0x55d3394b1b40 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75181af380 con 0x55d3394b05d0 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751ec5d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75181adf10 con 0x55d3394bb370 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46750/0 (socket says 172.21.15.16:46750) 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 -- 172.21.15.16:0/3988474743 learned_addr learned my addr 172.21.15.16:0/3988474743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f751659c640 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b05d0 0x55d3394d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f7516d9d640 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 -- 172.21.15.16:0/3988474743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 msgr2=0x55d3394b9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394b9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 -- 172.21.15.16:0/3988474743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b05d0 msgr2=0x55d3394d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b05d0 0x55d3394d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 -- 172.21.15.16:0/3988474743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3394ad960 con 0x55d3394b1b40 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f750ffff640 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f75000179c0 tx=0x7f7500017690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4559e00509c61c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.327+0000 7f74fffff640 1 -- 172.21.15.16:0/3988474743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500006ca0 con 0x55d3394b1b40 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f74fffff640 1 -- 172.21.15.16:0/3988474743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7500017a40 con 0x55d3394b1b40 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f74fffff640 1 -- 172.21.15.16:0/3988474743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500021880 con 0x55d3394b1b40 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- 172.21.15.16:0/3988474743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 msgr2=0x55d3394d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f75000179c0 tx=0x7f7500017690 comp rx=0 tx=0).stop 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- 172.21.15.16:0/3988474743 shutdown_connections 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b1b40 0x55d3394d2f70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 --2- 172.21.15.16:0/3988474743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b05d0 0x55d3394d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- 172.21.15.16:0/3988474743 >> 172.21.15.16:0/3988474743 conn(0x55d339479310 msgr2=0x55d3394b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- 172.21.15.16:0/3988474743 shutdown_connections 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- 172.21.15.16:0/3988474743 wait complete. 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.328+0000 7f751ec5d740 1 -- start start 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 Processor -- start 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aecb0 con 0x55d3394b05d0 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ae5e0 con 0x55d3394b1b40 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ab820 con 0x55d3394bb370 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751ec5d740 10 client.0 _ll_register_callbacks cb 0x7f750000b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46764/0 (socket says 172.21.15.16:46764) 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 -- 172.21.15.16:0/2114093247 learned_addr learned my addr 172.21.15.16:0/2114093247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f7516d9d640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f750ffff640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 -- 172.21.15.16:0/2114093247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 msgr2=0x55d3394bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 -- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 msgr2=0x55d3394bea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394bea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f751659c640 1 -- 172.21.15.16:0/2114093247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3394bf640 con 0x55d3394b05d0 2024-09-12T00:59:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f7516d9d640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.329+0000 7f750ffff640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394bea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751659c640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f75080bbea0 tx=0x7f75080bbce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af38e8a445822dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508049040 con 0x55d3394b05d0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f75080b8890 con 0x55d3394b05d0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7500017a40 con 0x55d3394b05d0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75080b8440 con 0x55d3394b05d0 2024-09-12T00:59:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 10 client.16317 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d339535920 con 0x55d3394b05d0 2024-09-12T00:59:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7500017a40 con 0x55d3394b05d0 2024-09-12T00:59:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75080ed590 con 0x55d3394b05d0 2024-09-12T00:59:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 10 client.16317 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 10 client.16317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 20 client.16317 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d339537f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 4 client.16317 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.330+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d33939a170 con 0x55d3394caed0 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f750ffff640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d339537f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f750ffff640 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d339537f40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c5a8393c3810ff server_cookie=577d2a7456e94384 in_seq=0 out_seq=0 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f74fdffb640 10 client.16317 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f74fdffb640 1 -- 172.21.15.16:0/2114093247 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d33939a170 con 0x55d3394caed0 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f74fdffb640 10 client.16317 handle_command_reply: tid=0 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 client.16317 shutdown 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 msgr2=0x55d3394c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f75080bbea0 tx=0x7f75080bbce0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 shutdown_connections 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d3394caed0 0x55d339537f40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3394bb370 0x55d3394bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3394b1b40 0x55d3394bea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.331+0000 7f751ec5d740 1 --2- 172.21.15.16:0/2114093247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3394b05d0 0x55d3394c36e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f74fdffb640 0 client.16317 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 >> 172.21.15.16:0/2114093247 conn(0x55d339479310 msgr2=0x55d3394b9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 shutdown_connections 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751ec5d740 1 -- 172.21.15.16:0/2114093247 wait complete. 2024-09-12T00:59:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751ec5d740 20 client.16317 trim_cache size 0 max 16384 2024-09-12T00:59:28.343 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 msgr2=0x7f75040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 0x7f75040a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7510005c40 tx=0x7f7510004a40 comp rx=0 tx=0).stop 2024-09-12T00:59:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 msgr2=0x7f7518198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7508006010 tx=0x7f75080042f0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 shutdown_connections 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75040a2cf0 0x7f75040a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7518106f10 0x7f751819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7518102790 0x7f7518198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 --2- 172.21.15.16:0/1884007562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7518101de0 0x7f7518198510 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 >> 172.21.15.16:0/1884007562 conn(0x7f75180fd8c0 msgr2=0x7f75180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 shutdown_connections 2024-09-12T00:59:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:28.332+0000 7f751c884640 1 -- 172.21.15.16:0/1884007562 wait complete. 2024-09-12T00:59:29.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T00:59:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:29 smithi016 ceph-mon[30972]: pgmap v105: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T00:59:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:29 smithi099 ceph-mon[35654]: pgmap v105: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T00:59:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:29 smithi159 ceph-mon[35566]: pgmap v105: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T00:59:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 -- 172.21.15.16:0/3079616224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 msgr2=0x7fa948105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/3079616224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa948105f80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa938009900 tx=0x7fa93802a480 comp rx=0 tx=0).stop 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 -- 172.21.15.16:0/3079616224 shutdown_connections 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/3079616224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948102110 0x7fa94810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/3079616224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948101790 0x7fa948101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/3079616224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa948105f80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 -- 172.21.15.16:0/3079616224 >> 172.21.15.16:0/3079616224 conn(0x7fa948077210 msgr2=0x7fa948077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 -- 172.21.15.16:0/3079616224 shutdown_connections 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.888+0000 7fa94d7c6640 1 -- 172.21.15.16:0/3079616224 wait complete. 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 Processor -- start 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 -- start start 2024-09-12T00:59:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 0x7fa9481984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948102110 0x7fa948198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa948110f10 con 0x7fa948105ba0 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa948110d90 con 0x7fa948102110 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa94d7c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa948111090 con 0x7fa948101790 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 0x7fa9481984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948102110 0x7fa948198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52550/0 (socket says 172.21.15.16:52550) 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa9477fe640 1 -- 172.21.15.16:0/177997334 learned_addr learned my addr 172.21.15.16:0/177997334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.889+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 0x7fa9481984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:51350/0 (socket says 172.21.15.16:51350) 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 -- 172.21.15.16:0/177997334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 msgr2=0x7fa9481984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 0x7fa9481984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 -- 172.21.15.16:0/177997334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948102110 msgr2=0x7fa948198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948102110 0x7fa948198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 -- 172.21.15.16:0/177997334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94819d4a0 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa9477fe640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b970 tx=0x7fa94000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa944ff9640 1 -- 172.21.15.16:0/177997334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940019070 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa944ff9640 1 -- 172.21.15.16:0/177997334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94000cb40 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa944ff9640 1 -- 172.21.15.16:0/177997334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940007650 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94819d790 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.890+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a5070 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.892+0000 7fa944ff9640 1 -- 172.21.15.16:0/177997334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa94001c070 con 0x7fa948105ba0 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.892+0000 7fa94fb9f740 1 Processor -- start 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 -- start start 2024-09-12T00:59:29.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed748090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed753370 0x5586ed755760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586ed6ae2f0 con 0x5586ed7485d0 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586ed655cf0 con 0x5586ed749b40 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586ed6aa2b0 con 0x5586ed753370 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa944ff9640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 0x7fa9340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52560/0 (socket says 172.21.15.16:52560) 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa946ffd640 1 -- 172.21.15.16:0/3975316316 learned_addr learned my addr 172.21.15.16:0/3975316316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa9477fe640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 0x7fa9340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed748090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed753370 0x5586ed755760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.893+0000 7fa944ff9640 1 -- 172.21.15.16:0/177997334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa9400ce870 con 0x7fa948105ba0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 -- 172.21.15.16:0/3975316316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed753370 msgr2=0x5586ed755760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed753370 0x5586ed755760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 -- 172.21.15.16:0/3975316316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 msgr2=0x5586ed748090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed748090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 -- 172.21.15.16:0/3975316316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ed63a3a0 con 0x5586ed7485d0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed748090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa9477fe640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 0x7fa9340a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa94009ac80 tx=0x7fa940004b50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa946ffd640 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa938009230 tx=0x7fa938004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7625ae536f42e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa93d7fa640 1 -- 172.21.15.16:0/3975316316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938040070 con 0x5586ed7485d0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa93d7fa640 1 -- 172.21.15.16:0/3975316316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9380047a0 con 0x5586ed7485d0 2024-09-12T00:59:29.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.894+0000 7fa93d7fa640 1 -- 172.21.15.16:0/3975316316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938035040 con 0x5586ed7485d0 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- 172.21.15.16:0/3975316316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 msgr2=0x5586ed748a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa938009230 tx=0x7fa938004310 comp rx=0 tx=0).stop 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- 172.21.15.16:0/3975316316 shutdown_connections 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed753370 0x5586ed755760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed748a10 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/3975316316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed748090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- 172.21.15.16:0/3975316316 >> 172.21.15.16:0/3975316316 conn(0x5586ed711310 msgr2=0x5586ed744c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- 172.21.15.16:0/3975316316 shutdown_connections 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- 172.21.15.16:0/3975316316 wait complete. 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 -- start start 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.895+0000 7fa94fb9f740 1 Processor -- start 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed752d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed74fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586ed756010 con 0x5586ed753370 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586ed669060 con 0x5586ed749b40 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586ed756190 con 0x5586ed7485d0 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 10 client.0 _ll_register_callbacks cb 0x5586ed3bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52568/0 (socket says 172.21.15.16:52568) 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 -- 172.21.15.16:0/1575200626 learned_addr learned my addr 172.21.15.16:0/1575200626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa946ffd640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed74fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed752d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 -- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 msgr2=0x5586ed752d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed752d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 -- 172.21.15.16:0/1575200626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 msgr2=0x5586ed74fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed74fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 -- 172.21.15.16:0/1575200626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ed41f3c0 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa930011fd0 tx=0x7fa930004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618381ba7b679bae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa92e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930004220 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa9300043c0 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.896+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9380047a0 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa930005020 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586ed761ae0 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9380047a0 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa930022060 con 0x5586ed753370 2024-09-12T00:59:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 10 client.16329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 20 client.16329 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed763040 0x5586ed765430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 4 client.16329 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586ed6bb860 con 0x5586ed763040 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.897+0000 7fa946ffd640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed763040 0x5586ed765430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.898+0000 7fa92e7fc640 10 client.16329 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.898+0000 7fa946ffd640 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed763040 0x5586ed765430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=127c905615de0f79 server_cookie=ab60e6589fe49c10 in_seq=0 out_seq=0 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa92e7fc640 1 -- 172.21.15.16:0/1575200626 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586ed6bb860 con 0x5586ed763040 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa92e7fc640 10 client.16329 handle_command_reply: tid=0 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 client.16329 shutdown 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 msgr2=0x5586ed7505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa930011fd0 tx=0x7fa930004730 comp rx=0 tx=0).stop 2024-09-12T00:59:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 shutdown_connections 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed763040 0x5586ed765430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed753370 0x5586ed7505e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed749b40 0x5586ed74fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa92e7fc640 0 client.16329 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/1575200626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed752d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 >> 172.21.15.16:0/1575200626 conn(0x5586ed711310 msgr2=0x5586ed74a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 shutdown_connections 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 1 -- 172.21.15.16:0/1575200626 wait complete. 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.899+0000 7fa94fb9f740 20 client.16329 trim_cache size 0 max 16384 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 Processor -- start 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 -- start start 2024-09-12T00:59:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed73fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 0x5586ed75f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ad870 con 0x5586ed7485d0 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481af3b0 con 0x5586ed753370 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.912+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481adf40 con 0x5586ed749b40 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed73fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed73fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52576/0 (socket says 172.21.15.16:52576) 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93e7fc640 1 -- 172.21.15.16:0/2569395902 learned_addr learned my addr 172.21.15.16:0/2569395902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 0x5586ed75f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 -- 172.21.15.16:0/2569395902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 msgr2=0x5586ed74b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 -- 172.21.15.16:0/2569395902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 msgr2=0x5586ed73fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed73fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 -- 172.21.15.16:0/2569395902 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586ed744de0 con 0x5586ed749b40 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa946ffd640 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 0x5586ed75f2b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa938005ec0 tx=0x7fa938005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604da993943b2a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93cff9640 1 -- 172.21.15.16:0/2569395902 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938040040 con 0x5586ed749b40 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93cff9640 1 -- 172.21.15.16:0/2569395902 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938031c20 con 0x5586ed749b40 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.913+0000 7fa93cff9640 1 -- 172.21.15.16:0/2569395902 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938035040 con 0x5586ed749b40 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- 172.21.15.16:0/2569395902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 msgr2=0x5586ed75f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 0x5586ed75f2b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa938005ec0 tx=0x7fa938005d50 comp rx=0 tx=0).stop 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- 172.21.15.16:0/2569395902 shutdown_connections 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed749b40 0x5586ed75f2b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/2569395902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed7485d0 0x5586ed73fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- 172.21.15.16:0/2569395902 >> 172.21.15.16:0/2569395902 conn(0x5586ed711310 msgr2=0x5586ed74f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- 172.21.15.16:0/2569395902 shutdown_connections 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- 172.21.15.16:0/2569395902 wait complete. 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.914+0000 7fa94fb9f740 1 -- start start 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 Processor -- start 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed75b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481aece0 con 0x5586ed749b40 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ae610 con 0x5586ed753370 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ab850 con 0x5586ed7485d0 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa94fb9f740 10 client.0 _ll_register_callbacks cb 0x7fa93000ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa93e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed75b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52584/0 (socket says 172.21.15.16:52584) 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 -- 172.21.15.16:0/189130533 learned_addr learned my addr 172.21.15.16:0/189130533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 -- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 msgr2=0x5586ed75b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed75b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 -- 172.21.15.16:0/189130533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 msgr2=0x5586ed74cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa946ffd640 1 -- 172.21.15.16:0/189130533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586ed74d480 con 0x5586ed749b40 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.915+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed75b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa946ffd640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa938005de0 tx=0x7fa938031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b05e81055abff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938058060 con 0x5586ed749b40 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa938005310 con 0x5586ed749b40 2024-09-12T00:59:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa938031c20 con 0x5586ed749b40 2024-09-12T00:59:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa938035040 con 0x5586ed749b40 2024-09-12T00:59:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa94fb9f740 10 client.16335 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586ed7cd9a0 con 0x5586ed749b40 2024-09-12T00:59:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938031c20 con 0x5586ed749b40 2024-09-12T00:59:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.916+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa93804f440 con 0x5586ed749b40 2024-09-12T00:59:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 10 client.16335 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 10 client.16335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 20 client.16335 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed7cdc70 0x5586ed7d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 4 client.16335 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586ed6b5d70 con 0x5586ed7cdc70 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed7cdc70 0x5586ed7d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa93e7fc640 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed7cdc70 0x5586ed7d0060 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a06dfaf72739a226 server_cookie=10e3a5a4df91a3ac in_seq=0 out_seq=0 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa92e7fc640 10 client.16335 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa92e7fc640 1 -- 172.21.15.16:0/189130533 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586ed6b5d70 con 0x5586ed7cdc70 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa92e7fc640 10 client.16335 handle_command_reply: tid=0 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.917+0000 7fa94fb9f740 1 client.16335 shutdown 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 msgr2=0x5586ed75bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa938005de0 tx=0x7fa938031400 comp rx=0 tx=0).stop 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 shutdown_connections 2024-09-12T00:59:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586ed753370 0x5586ed74cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5586ed7cdc70 0x5586ed7d0060 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586ed749b40 0x5586ed75bd10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 --2- 172.21.15.16:0/189130533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586ed7485d0 0x5586ed75b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 >> 172.21.15.16:0/189130533 conn(0x5586ed711310 msgr2=0x5586ed7434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa92e7fc640 0 client.16335 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 shutdown_connections 2024-09-12T00:59:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 1 -- 172.21.15.16:0/189130533 wait complete. 2024-09-12T00:59:29.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94fb9f740 20 client.16335 trim_cache size 0 max 16384 2024-09-12T00:59:29.934 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 msgr2=0x7fa9340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 0x7fa9340a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa94009ac80 tx=0x7fa940004b50 comp rx=0 tx=0).stop 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 msgr2=0x7fa94819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.918+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa94000b970 tx=0x7fa94000be30 comp rx=0 tx=0).stop 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 shutdown_connections 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9340a2cf0 0x7fa9340a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa948105ba0 0x7fa94819cdc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa948102110 0x7fa948198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 --2- 172.21.15.16:0/177997334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa948101790 0x7fa9481984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 >> 172.21.15.16:0/177997334 conn(0x7fa948077210 msgr2=0x7fa948100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 shutdown_connections 2024-09-12T00:59:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:29.919+0000 7fa94d7c6640 1 -- 172.21.15.16:0/177997334 wait complete. 2024-09-12T00:59:30.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T00:59:31.042 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:31.446 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:31 smithi016 ceph-mon[30972]: pgmap v106: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T00:59:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:31 smithi099 ceph-mon[35654]: pgmap v106: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 -- 172.21.15.16:0/2410227827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c101420 msgr2=0x7fbb9c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 --2- 172.21.15.16:0/2410227827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c101420 0x7fbb9c10d950 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009cd0 tx=0x7fbb9402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 -- 172.21.15.16:0/2410227827 shutdown_connections 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 --2- 172.21.15.16:0/2410227827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c101420 0x7fbb9c10d950 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 --2- 172.21.15.16:0/2410227827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c100a80 0x7fbb9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 --2- 172.21.15.16:0/2410227827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 0x7fbb9c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 -- 172.21.15.16:0/2410227827 >> 172.21.15.16:0/2410227827 conn(0x7fbb9c0fc7a0 msgr2=0x7fbb9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 -- 172.21.15.16:0/2410227827 shutdown_connections 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 -- 172.21.15.16:0/2410227827 wait complete. 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.531+0000 7fbba196b640 1 Processor -- start 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 -- start start 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 0x7fbb9c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 0x7fbb9c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c110e80 con 0x7fbb9c100a80 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c110d00 con 0x7fbb9c101420 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbba196b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c111000 con 0x7fbb9c106a80 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52610/0 (socket says 172.21.15.16:52610) 2024-09-12T00:59:31.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 -- 172.21.15.16:0/2636501300 learned_addr learned my addr 172.21.15.16:0/2636501300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 0x7fbb9c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 -- 172.21.15.16:0/2636501300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 msgr2=0x7fbb9c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 0x7fbb9c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 0x7fbb9c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 -- 172.21.15.16:0/2636501300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 msgr2=0x7fbb9c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 0x7fbb9c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 -- 172.21.15.16:0/2636501300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c19d440 con 0x7fbb9c100a80 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 0x7fbb9c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.532+0000 7fbb9affd640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d770 tx=0x7fbb8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.533+0000 7fbba0969640 1 -- 172.21.15.16:0/2636501300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c017070 con 0x7fbb9c100a80 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.533+0000 7fbba0969640 1 -- 172.21.15.16:0/2636501300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c005500 con 0x7fbb9c100a80 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.533+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c19d730 con 0x7fbb9c100a80 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.533+0000 7fbba0969640 1 -- 172.21.15.16:0/2636501300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004e00 con 0x7fbb9c100a80 2024-09-12T00:59:31.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.533+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1a5010 con 0x7fbb9c100a80 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.535+0000 7fbba0969640 1 -- 172.21.15.16:0/2636501300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbb8c004fa0 con 0x7fbb9c100a80 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.535+0000 7fbba3d44740 1 Processor -- start 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.535+0000 7fbba3d44740 1 -- start start 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba783e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cba77a02f0 con 0x55cba783e9e0 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cba7747cf0 con 0x55cba783ff50 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba3d44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cba779c2b0 con 0x55cba7849780 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52616/0 (socket says 172.21.15.16:52616) 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 -- 172.21.15.16:0/3462680977 learned_addr learned my addr 172.21.15.16:0/3462680977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:31.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba783e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba0969640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 0x7fbb840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 0x7fbb840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 -- 172.21.15.16:0/3462680977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 msgr2=0x55cba784bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 -- 172.21.15.16:0/3462680977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 msgr2=0x55cba783e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba783e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9affd640 1 -- 172.21.15.16:0/3462680977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba772c3a0 con 0x55cba783e9e0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.536+0000 7fbba0969640 1 -- 172.21.15.16:0/2636501300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fbb8c0ceef0 con 0x7fbb9c100a80 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbb9affd640 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc5e0 tx=0x7fbb8c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85761eb8061a172 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3462680977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c020070 con 0x55cba783e9e0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 0x7fbb840a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb94004af0 tx=0x7fbb9402a8e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3462680977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c0046c0 con 0x55cba783e9e0 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbb81ffb640 1 -- 172.21.15.16:0/3462680977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c018d60 con 0x55cba783e9e0 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbba3d44740 1 -- 172.21.15.16:0/3462680977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 msgr2=0x55cba783ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.537+0000 7fbba3d44740 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc5e0 tx=0x7fbb8c022000 comp rx=0 tx=0).stop 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 -- 172.21.15.16:0/3462680977 shutdown_connections 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba783ee20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 --2- 172.21.15.16:0/3462680977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba783e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 -- 172.21.15.16:0/3462680977 >> 172.21.15.16:0/3462680977 conn(0x55cba7803310 msgr2=0x55cba783b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 -- 172.21.15.16:0/3462680977 shutdown_connections 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 -- 172.21.15.16:0/3462680977 wait complete. 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 -- start start 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.538+0000 7fbba3d44740 1 Processor -- start 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7849130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba78463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cba784c3c0 con 0x55cba7849780 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cba775b060 con 0x55cba783ff50 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cba784c540 con 0x55cba783e9e0 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 10 client.0 _ll_register_callbacks cb 0x55cba74ae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52632/0 (socket says 172.21.15.16:52632) 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/2614418698 learned_addr learned my addr 172.21.15.16:0/2614418698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba78463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/2614418698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 msgr2=0x55cba7849130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7849130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 msgr2=0x55cba78463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba78463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 -- 172.21.15.16:0/2614418698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba75113c0 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba78463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb9b7fe640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009cd0 tx=0x7fbb940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c392618cf03e7135 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb7affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9404e020 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fbb94004250 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.539+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cba77ad860 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb94059050 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 10 client.16347 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba7857e60 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cba77ad860 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb94030070 con 0x55cba7849780 2024-09-12T00:59:31.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 10 client.16347 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 20 client.16347 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78593c0 0x55cba785b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 4 client.16347 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cba77a7d70 con 0x55cba78593c0 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.540+0000 7fbb9affd640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78593c0 0x55cba785b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.541+0000 7fbb9affd640 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78593c0 0x55cba785b860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6ccd700e93f161c server_cookie=42463f989adab629 in_seq=0 out_seq=0 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.541+0000 7fbb7affd640 10 client.16347 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbb7affd640 1 -- 172.21.15.16:0/2614418698 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cba77a7d70 con 0x55cba78593c0 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbb7affd640 10 client.16347 handle_command_reply: tid=0 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 client.16347 shutdown 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 msgr2=0x55cba7846980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009cd0 tx=0x7fbb940095c0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 shutdown_connections 2024-09-12T00:59:31.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78593c0 0x55cba785b860 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba7849780 0x55cba7846980 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba78463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 --2- 172.21.15.16:0/2614418698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7849130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 >> 172.21.15.16:0/2614418698 conn(0x55cba7803310 msgr2=0x55cba7840d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbb7affd640 0 client.16347 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 shutdown_connections 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 1 -- 172.21.15.16:0/2614418698 wait complete. 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.542+0000 7fbba3d44740 20 client.16347 trim_cache size 0 max 16384 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 Processor -- start 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 -- start start 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7855660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba7849780 0x55cba7861130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1ab5e0 con 0x55cba783ff50 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1ad120 con 0x55cba7849780 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbba3d44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1abcb0 con 0x55cba783e9e0 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52636/0 (socket says 172.21.15.16:52636) 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 -- 172.21.15.16:0/1310323008 learned_addr learned my addr 172.21.15.16:0/1310323008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9affd640 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7855660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 -- 172.21.15.16:0/1310323008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 msgr2=0x55cba7855660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7855660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 -- 172.21.15.16:0/1310323008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba7849780 msgr2=0x55cba7861130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba7849780 0x55cba7861130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.555+0000 7fbb9a7fc640 1 -- 172.21.15.16:0/1310323008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba7841530 con 0x55cba783ff50 2024-09-12T00:59:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c105200 tx=0x7fbb90011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e706cb0e41e29b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbb817fa640 1 -- 172.21.15.16:0/1310323008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9001e070 con 0x55cba783ff50 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbb817fa640 1 -- 172.21.15.16:0/1310323008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb90019ad0 con 0x55cba783ff50 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbb817fa640 1 -- 172.21.15.16:0/1310323008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000bc80 con 0x55cba783ff50 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 -- 172.21.15.16:0/1310323008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 msgr2=0x55cba783dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c105200 tx=0x7fbb90011810 comp rx=0 tx=0).stop 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 -- 172.21.15.16:0/1310323008 shutdown_connections 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba7849780 0x55cba7861130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783ff50 0x55cba783dfe0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 --2- 172.21.15.16:0/1310323008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba783e9e0 0x55cba7855660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 -- 172.21.15.16:0/1310323008 >> 172.21.15.16:0/1310323008 conn(0x55cba7803310 msgr2=0x55cba7844cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 -- 172.21.15.16:0/1310323008 shutdown_connections 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.556+0000 7fbba3d44740 1 -- 172.21.15.16:0/1310323008 wait complete. 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 -- start start 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 Processor -- start 2024-09-12T00:59:31.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba784d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1aca50 con 0x55cba783e9e0 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1ac380 con 0x55cba783ff50 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a95e0 con 0x55cba7849780 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.557+0000 7fbba3d44740 10 client.0 _ll_register_callbacks cb 0x7fbb94004f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52644/0 (socket says 172.21.15.16:52644) 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 -- 172.21.15.16:0/178996798 learned_addr learned my addr 172.21.15.16:0/178996798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 -- 172.21.15.16:0/178996798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 msgr2=0x55cba784d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 -- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 msgr2=0x55cba784d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba784d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba784d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 -- 172.21.15.16:0/178996798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba784def0 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba784d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb9affd640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc720 tx=0x7fbb8c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21967261320558ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb7affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c020040 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fbb8c005290 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb90019ad0 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb8c09bb40 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbba3d44740 10 client.16353 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cba78c4070 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.558+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb90019ad0 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb8c0cc8f0 con 0x55cba783e9e0 2024-09-12T00:59:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 10 client.16353 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 10 client.16353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 20 client.16353 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78c4400 0x55cba78c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 4 client.16353 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cba7724170 con 0x55cba78c4400 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78c4400 0x55cba78c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbb7affd640 10 client.16353 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.559+0000 7fbb9a7fc640 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78c4400 0x55cba78c67f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb4598209f1c380 server_cookie=f9cd01909d137949 in_seq=0 out_seq=0 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbb7affd640 1 -- 172.21.15.16:0/178996798 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cba7724170 con 0x55cba78c4400 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbb7affd640 10 client.16353 handle_command_reply: tid=0 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 client.16353 shutdown 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 msgr2=0x55cba7851e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0cc720 tx=0x7fbb8c005bb0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 shutdown_connections 2024-09-12T00:59:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cba78c4400 0x55cba78c67f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cba7849780 0x55cba784d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbb7affd640 0 client.16353 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cba783ff50 0x55cba784d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 --2- 172.21.15.16:0/178996798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cba783e9e0 0x55cba7851e80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 >> 172.21.15.16:0/178996798 conn(0x55cba7803310 msgr2=0x55cba7847c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 shutdown_connections 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 1 -- 172.21.15.16:0/178996798 wait complete. 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba3d44740 20 client.16353 trim_cache size 0 max 16384 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 msgr2=0x7fbb840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 0x7fbb840a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbb94004af0 tx=0x7fbb9402a8e0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 msgr2=0x7fbb9c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.560+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d770 tx=0x7fbb8c00dc30 comp rx=0 tx=0).stop 2024-09-12T00:59:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 shutdown_connections 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb840a2cf0 0x7fbb840a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb9c106a80 0x7fbb9c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb9c101420 0x7fbb9c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 --2- 172.21.15.16:0/2636501300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb9c100a80 0x7fbb9c198490 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 >> 172.21.15.16:0/2636501300 conn(0x7fbb9c0fc7a0 msgr2=0x7fbb9c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 shutdown_connections 2024-09-12T00:59:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:31.561+0000 7fbba196b640 1 -- 172.21.15.16:0/2636501300 wait complete. 2024-09-12T00:59:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:31 smithi159 ceph-mon[35566]: pgmap v106: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-09-12T00:59:32.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T00:59:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 -- 172.21.15.16:0/3298207067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc1036e0 msgr2=0x7f80fc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 --2- 172.21.15.16:0/3298207067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc1036e0 0x7f80fc107b90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009930 tx=0x7f80f002a420 comp rx=0 tx=0).stop 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 -- 172.21.15.16:0/3298207067 shutdown_connections 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 --2- 172.21.15.16:0/3298207067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 --2- 172.21.15.16:0/3298207067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc1036e0 0x7f80fc107b90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 --2- 172.21.15.16:0/3298207067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc102d30 0x7f80fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 -- 172.21.15.16:0/3298207067 >> 172.21.15.16:0/3298207067 conn(0x7f80fc0fc7d0 msgr2=0x7f80fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 -- 172.21.15.16:0/3298207067 shutdown_connections 2024-09-12T00:59:33.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 -- 172.21.15.16:0/3298207067 wait complete. 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.100+0000 7f810311c640 1 Processor -- start 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 -- start start 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc1036e0 0x7f80fc1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc112060 con 0x7f80fc102d30 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc111ee0 con 0x7f80fc1082a0 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f810311c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1121e0 con 0x7f80fc1036e0 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52660/0 (socket says 172.21.15.16:52660) 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 -- 172.21.15.16:0/2180254039 learned_addr learned my addr 172.21.15.16:0/2180254039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f80fbfff640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc1036e0 0x7f80fc1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f81016b9640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 -- 172.21.15.16:0/2180254039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc1036e0 msgr2=0x7f80fc1a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc1036e0 0x7f80fc1a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 -- 172.21.15.16:0/2180254039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 msgr2=0x7f80fc19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 -- 172.21.15.16:0/2180254039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc19ace0 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f81016b9640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.101+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00d950 tx=0x7f80ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.102+0000 7f80f9ffb640 1 -- 172.21.15.16:0/2180254039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec017030 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.102+0000 7f80f9ffb640 1 -- 172.21.15.16:0/2180254039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec004020 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.102+0000 7f80f9ffb640 1 -- 172.21.15.16:0/2180254039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0047a0 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.102+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc19afd0 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.102+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc1a72c0 con 0x7f80fc102d30 2024-09-12T00:59:33.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.104+0000 7f80f9ffb640 1 -- 172.21.15.16:0/2180254039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f80ec0041d0 con 0x7f80fc102d30 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.104+0000 7f81054f5740 1 Processor -- start 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.104+0000 7f81054f5740 1 -- start start 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f80f9ffb640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 0x7f80e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f35d0 0x5619cb5f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb600760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619cb5592f0 con 0x5619cb5f4b40 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5619cb500cf0 con 0x5619cb5f35d0 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81054f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619cb5552b0 con 0x5619cb5fe370 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81016b9640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 0x7f80e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f80f9ffb640 1 -- 172.21.15.16:0/2180254039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f80ec0cef20 con 0x7f80fc102d30 2024-09-12T00:59:33.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52676/0 (socket says 172.21.15.16:52676) 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 -- 172.21.15.16:0/3242521320 learned_addr learned my addr 172.21.15.16:0/3242521320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f80fbfff640 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f35d0 0x5619cb5f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 -- 172.21.15.16:0/3242521320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 msgr2=0x5619cb600760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb600760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 -- 172.21.15.16:0/3242521320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f35d0 msgr2=0x5619cb5f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f35d0 0x5619cb5f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 -- 172.21.15.16:0/3242521320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cb4e53a0 con 0x5619cb5f4b40 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009610 tx=0x7f80ec09b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1f550fb596337c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f81016b9640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 0x7f80e40a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80f4005c40 tx=0x7f80f40097a0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.105+0000 7f80e2ffd640 1 -- 172.21.15.16:0/3242521320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0b4610 con 0x5619cb5f4b40 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f80e2ffd640 1 -- 172.21.15.16:0/3242521320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec09bb10 con 0x5619cb5f4b40 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f80e2ffd640 1 -- 172.21.15.16:0/3242521320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0b49d0 con 0x5619cb5f4b40 2024-09-12T00:59:33.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 -- 172.21.15.16:0/3242521320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 msgr2=0x5619cb5f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009610 tx=0x7f80ec09b2a0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 -- 172.21.15.16:0/3242521320 shutdown_connections 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb600760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f35d0 0x5619cb5f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 --2- 172.21.15.16:0/3242521320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f4b40 0x5619cb5f3090 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 -- 172.21.15.16:0/3242521320 >> 172.21.15.16:0/3242521320 conn(0x5619cb5bc310 msgr2=0x5619cb5efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 -- 172.21.15.16:0/3242521320 shutdown_connections 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.106+0000 7f81054f5740 1 -- 172.21.15.16:0/3242521320 wait complete. 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 -- start start 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 Processor -- start 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb5fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5619cb601010 con 0x5619cb5fe370 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5619cb514060 con 0x5619cb5f4b40 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619cb601190 con 0x5619cb5f35d0 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.107+0000 7f81054f5740 10 client.0 _ll_register_callbacks cb 0x5619cb2678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52680/0 (socket says 172.21.15.16:52680) 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 -- 172.21.15.16:0/2968218683 learned_addr learned my addr 172.21.15.16:0/2968218683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f80fbfff640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb5fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 -- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 msgr2=0x5619cb5fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 -- 172.21.15.16:0/2968218683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 msgr2=0x5619cb5fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb5fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 -- 172.21.15.16:0/2968218683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cb2ca3c0 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81016b9640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f80f40126f0 tx=0x7f80f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b635e0383d6be52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f80dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4023070 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f80f4004d20 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ec09bb10 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f80f402e050 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81054f5740 10 client.16365 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.108+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619cb60cb50 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ec09bb10 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f80f4005550 con 0x5619cb5fe370 2024-09-12T00:59:33.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 10 client.16365 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 20 client.16365 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb60df60 0x5619cb6103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 4 client.16365 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619cb566860 con 0x5619cb60df60 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.109+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb60df60 0x5619cb6103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.110+0000 7f8100eb8640 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb60df60 0x5619cb6103e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e055a26f2ae37955 server_cookie=788271bf95eba3b6 in_seq=0 out_seq=0 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.110+0000 7f80dbfff640 10 client.16365 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.110+0000 7f80dbfff640 1 -- 172.21.15.16:0/2968218683 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619cb566860 con 0x5619cb60df60 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.110+0000 7f80dbfff640 10 client.16365 handle_command_reply: tid=0 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.110+0000 7f81054f5740 1 client.16365 shutdown 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 msgr2=0x5619cb5fb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f80f40126f0 tx=0x7f80f4004ab0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 shutdown_connections 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb5fb7e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb60df60 0x5619cb6103e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb5fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f80dbfff640 0 client.16365 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 --2- 172.21.15.16:0/2968218683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 >> 172.21.15.16:0/2968218683 conn(0x5619cb5bc310 msgr2=0x5619cb5f5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 shutdown_connections 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 1 -- 172.21.15.16:0/2968218683 wait complete. 2024-09-12T00:59:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.111+0000 7f81054f5740 20 client.16365 trim_cache size 0 max 16384 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 Processor -- start 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 -- start start 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb615d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1afa70 con 0x5619cb5fe370 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b15b0 con 0x5619cb5f4b40 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.124+0000 7f81054f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b0140 con 0x5619cb5f35d0 2024-09-12T00:59:33.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52694/0 (socket says 172.21.15.16:52694) 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 -- 172.21.15.16:0/4250841588 learned_addr learned my addr 172.21.15.16:0/4250841588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f80fbfff640 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb615d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f8100eb8640 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 -- 172.21.15.16:0/4250841588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 msgr2=0x5619cb5efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 -- 172.21.15.16:0/4250841588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 msgr2=0x5619cb615d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb615d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 -- 172.21.15.16:0/4250841588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cb610a60 con 0x5619cb5fe370 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f81016b9640 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f80f40092f0 tx=0x7f80f4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9069ede1f3936ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f80e27fc640 1 -- 172.21.15.16:0/4250841588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4036030 con 0x5619cb5fe370 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f80e27fc640 1 -- 172.21.15.16:0/4250841588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4016070 con 0x5619cb5fe370 2024-09-12T00:59:33.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.125+0000 7f80e27fc640 1 -- 172.21.15.16:0/4250841588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4023070 con 0x5619cb5fe370 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- 172.21.15.16:0/4250841588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 msgr2=0x5619cb616240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f80f40092f0 tx=0x7f80f4009a50 comp rx=0 tx=0).stop 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- 172.21.15.16:0/4250841588 shutdown_connections 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5fe370 0x5619cb616240 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb615d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 --2- 172.21.15.16:0/4250841588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5f35d0 0x5619cb5efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- 172.21.15.16:0/4250841588 >> 172.21.15.16:0/4250841588 conn(0x5619cb5bc310 msgr2=0x5619cb5ef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- 172.21.15.16:0/4250841588 shutdown_connections 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- 172.21.15.16:0/4250841588 wait complete. 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 -- start start 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.126+0000 7f81054f5740 1 Processor -- start 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb602d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb677cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b0ee0 con 0x5619cb5f35d0 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1b0810 con 0x5619cb5f4b40 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc1ada50 con 0x5619cb5fe370 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81054f5740 10 client.0 _ll_register_callbacks cb 0x7f80f4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52704/0 (socket says 172.21.15.16:52704) 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 -- 172.21.15.16:0/3389901672 learned_addr learned my addr 172.21.15.16:0/3389901672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f81016b9640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb677cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f80fbfff640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb602d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 -- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 msgr2=0x5619cb677cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb677cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 -- 172.21.15.16:0/3389901672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 msgr2=0x5619cb602d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb602d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f8100eb8640 1 -- 172.21.15.16:0/3389901672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cb678210 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.127+0000 7f80fbfff640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb602d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f8100eb8640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f80ec097870 tx=0x7f80ec005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3626873b60e886 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec0b8070 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f80ec0cc8c0 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f4016070 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f80ec0ccac0 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 10 client.16371 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619cb679410 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f4016070 con 0x5619cb5f35d0 2024-09-12T00:59:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f80ec0afce0 con 0x5619cb5f35d0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 10 client.16371 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 10 client.16371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 20 client.16371 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb6796f0 0x5619cb67bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 4 client.16371 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.128+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619cb560d70 con 0x5619cb6796f0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80fbfff640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb6796f0 0x5619cb67bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80fbfff640 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb6796f0 0x5619cb67bae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=733bb7a65e3f0813 server_cookie=37bd87fd1960829b in_seq=0 out_seq=0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80dbfff640 10 client.16371 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80dbfff640 1 -- 172.21.15.16:0/3389901672 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619cb560d70 con 0x5619cb6796f0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80dbfff640 10 client.16371 handle_command_reply: tid=0 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 client.16371 shutdown 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 msgr2=0x5619cb6027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f80ec097870 tx=0x7f80ec005bf0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 shutdown_connections 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619cb5fe370 0x5619cb677cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5619cb6796f0 0x5619cb67bae0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5619cb5f4b40 0x5619cb602d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 --2- 172.21.15.16:0/3389901672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619cb5f35d0 0x5619cb6027c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f80dbfff640 0 client.16371 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.129+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 >> 172.21.15.16:0/3389901672 conn(0x5619cb5bc310 msgr2=0x5619cb5fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 shutdown_connections 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f81054f5740 1 -- 172.21.15.16:0/3389901672 wait complete. 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f81054f5740 20 client.16371 trim_cache size 0 max 16384 2024-09-12T00:59:33.135 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 msgr2=0x7f80e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 0x7f80e40a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80f4005c40 tx=0x7f80f40097a0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 msgr2=0x7f80fc1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00d950 tx=0x7f80ec00de10 comp rx=0 tx=0).stop 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 shutdown_connections 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80e40a2cf0 0x7f80e40a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80fc1082a0 0x7f80fc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80fc1036e0 0x7f80fc1a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 --2- 172.21.15.16:0/2180254039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80fc102d30 0x7f80fc1a03f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 >> 172.21.15.16:0/2180254039 conn(0x7f80fc0fc7d0 msgr2=0x7f80fc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 shutdown_connections 2024-09-12T00:59:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:33.130+0000 7f810311c640 1 -- 172.21.15.16:0/2180254039 wait complete. 2024-09-12T00:59:33.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:33 smithi016 ceph-mon[30972]: pgmap v107: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:33 smithi099 ceph-mon[35654]: pgmap v107: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:33 smithi159 ceph-mon[35566]: pgmap v107: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:33.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T00:59:34.162 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 -- 172.21.15.16:0/456573525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 msgr2=0x7f60d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/456573525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4101200 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f60c4009900 tx=0x7f60c402a480 comp rx=0 tx=0).stop 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 -- 172.21.15.16:0/456573525 shutdown_connections 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/456573525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 0x7f60d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/456573525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 0x7f60d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/456573525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4101200 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 -- 172.21.15.16:0/456573525 >> 172.21.15.16:0/456573525 conn(0x7f60d40fc7d0 msgr2=0x7f60d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 -- 172.21.15.16:0/456573525 shutdown_connections 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.650+0000 7f60d8c4c640 1 -- 172.21.15.16:0/456573525 wait complete. 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 Processor -- start 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 -- start start 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 0x7f60d41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 0x7f60d419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4110f80 con 0x7f60d4100e20 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4110e00 con 0x7f60d4102ef0 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d8c4c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60d4111100 con 0x7f60d4102570 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52716/0 (socket says 172.21.15.16:52716) 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60d259c640 1 -- 172.21.15.16:0/3573008834 learned_addr learned my addr 172.21.15.16:0/3573008834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.651+0000 7f60cbfff640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 0x7f60d41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 0x7f60d419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 -- 172.21.15.16:0/3573008834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 msgr2=0x7f60d41979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 0x7f60d41979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 -- 172.21.15.16:0/3573008834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 msgr2=0x7f60d419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 0x7f60d419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 -- 172.21.15.16:0/3573008834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d419c410 con 0x7f60d4100e20 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d259c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006010 tx=0x7f60c4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60cb7fe640 1 -- 172.21.15.16:0/3573008834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c403f070 con 0x7f60d4100e20 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d419c6a0 con 0x7f60d4100e20 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60cb7fe640 1 -- 172.21.15.16:0/3573008834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60c4031e30 con 0x7f60d4100e20 2024-09-12T00:59:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60cb7fe640 1 -- 172.21.15.16:0/3573008834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4035030 con 0x7f60d4100e20 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.652+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d41a3fe0 con 0x7f60d4100e20 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.654+0000 7f60cb7fe640 1 -- 172.21.15.16:0/3573008834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60c4042070 con 0x7f60d4100e20 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 Processor -- start 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 -- start start 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d32dae12f0 con 0x55d32db7cb40 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d32da88cf0 con 0x55d32db7b5d0 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60db025740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d32dadd2b0 con 0x55d32db86370 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52718/0 (socket says 172.21.15.16:52718) 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60d2d9d640 1 -- 172.21.15.16:0/4175239043 learned_addr learned my addr 172.21.15.16:0/4175239043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60cbfff640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60d259c640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db7ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.655+0000 7f60cb7fe640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 0x7f60c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 -- 172.21.15.16:0/4175239043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 msgr2=0x55d32db88760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db88760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 -- 172.21.15.16:0/4175239043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 msgr2=0x55d32db7ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db7ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 -- 172.21.15.16:0/4175239043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32da6d3a0 con 0x55d32db7cb40 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d259c640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 0x7f60c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60cb7fe640 1 -- 172.21.15.16:0/3573008834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f60c40efe80 con 0x7f60d4100e20 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f60cc008f40 tx=0x7f60cc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfca6bc045aa1d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60c8ff9640 1 -- 172.21.15.16:0/4175239043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc022070 con 0x55d32db7cb40 2024-09-12T00:59:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60c8ff9640 1 -- 172.21.15.16:0/4175239043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc011b70 con 0x55d32db7cb40 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60c8ff9640 1 -- 172.21.15.16:0/4175239043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc01cbb0 con 0x55d32db7cb40 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.656+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 0x7f60c00a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60cc01f000 tx=0x7f60cc020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- 172.21.15.16:0/4175239043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 msgr2=0x55d32db7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f60cc008f40 tx=0x7f60cc00e7e0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- 172.21.15.16:0/4175239043 shutdown_connections 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db88760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db7ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 --2- 172.21.15.16:0/4175239043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7b090 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- 172.21.15.16:0/4175239043 >> 172.21.15.16:0/4175239043 conn(0x55d32db44310 msgr2=0x55d32db77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- 172.21.15.16:0/4175239043 shutdown_connections 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- 172.21.15.16:0/4175239043 wait complete. 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 -- start start 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.657+0000 7f60db025740 1 Processor -- start 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db82fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d32db8b280 con 0x55d32db86370 2024-09-12T00:59:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d32da9c060 con 0x55d32db7b5d0 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d32db8b400 con 0x55d32db7cb40 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60db025740 10 client.0 _ll_register_callbacks cb 0x55d32d7ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52724/0 (socket says 172.21.15.16:52724) 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 -- 172.21.15.16:0/2657751611 learned_addr learned my addr 172.21.15.16:0/2657751611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60cbfff640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db82fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d259c640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 -- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 msgr2=0x55d32db82fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db82fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 -- 172.21.15.16:0/2657751611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 msgr2=0x55d32db85d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db85d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 -- 172.21.15.16:0/2657751611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32d8523c0 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d259c640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f60cc004cb0 tx=0x7f60cc01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878ba4daa3bbcd3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.658+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc030060 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc011b70 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f60cc011d20 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60cc022030 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 10 client.16383 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d32db96cb0 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60cc011b70 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60cc019040 con 0x55d32db86370 2024-09-12T00:59:34.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 10 client.16383 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 20 client.16383 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32db980b0 0x55d32db9a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 4 client.16383 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.659+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d32daee860 con 0x55d32db980b0 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.660+0000 7f60d259c640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32db980b0 0x55d32db9a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.660+0000 7f60d259c640 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32db980b0 0x55d32db9a4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8cf189cc577ed15 server_cookie=94c774ef9280bfe2 in_seq=0 out_seq=0 2024-09-12T00:59:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.660+0000 7f60b9ffb640 10 client.16383 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60b9ffb640 1 -- 172.21.15.16:0/2657751611 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d32daee860 con 0x55d32db980b0 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60b9ffb640 10 client.16383 handle_command_reply: tid=0 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 client.16383 shutdown 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 msgr2=0x55d32db83530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f60cc004cb0 tx=0x7f60cc01cb40 comp rx=0 tx=0).stop 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 shutdown_connections 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32db980b0 0x55d32db9a4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db86370 0x55d32db83530 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db82fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 --2- 172.21.15.16:0/2657751611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60b9ffb640 0 client.16383 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 >> 172.21.15.16:0/2657751611 conn(0x55d32db44310 msgr2=0x55d32db7d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 shutdown_connections 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 1 -- 172.21.15.16:0/2657751611 wait complete. 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.661+0000 7f60db025740 20 client.16383 trim_cache size 0 max 16384 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 Processor -- start 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 -- start start 2024-09-12T00:59:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db81880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41ab5b0 con 0x55d32db7cb40 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41ad0f0 con 0x55d32db7b5d0 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60db025740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41abc80 con 0x55d32db86370 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52730/0 (socket says 172.21.15.16:52730) 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 -- 172.21.15.16:0/165802174 learned_addr learned my addr 172.21.15.16:0/165802174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60d259c640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db81880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 -- 172.21.15.16:0/165802174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 msgr2=0x55d32db7e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db7e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 -- 172.21.15.16:0/165802174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 msgr2=0x55d32db81880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db81880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60cbfff640 1 -- 172.21.15.16:0/165802174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32db7eba0 con 0x55d32db7cb40 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.674+0000 7f60d259c640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db81880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60cbfff640 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ffee0 tx=0x7f60bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d845a9ba2a00edbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60bbfff640 1 -- 172.21.15.16:0/165802174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00ce50 con 0x55d32db7cb40 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60bbfff640 1 -- 172.21.15.16:0/165802174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60bc005020 con 0x55d32db7cb40 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60bbfff640 1 -- 172.21.15.16:0/165802174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc004c40 con 0x55d32db7cb40 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 -- 172.21.15.16:0/165802174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 msgr2=0x55d32db7e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f60d40ffee0 tx=0x7f60bc011810 comp rx=0 tx=0).stop 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 -- 172.21.15.16:0/165802174 shutdown_connections 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db86370 0x55d32db7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7cb40 0x55d32db7e120 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 --2- 172.21.15.16:0/165802174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db7b5d0 0x55d32db81880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 -- 172.21.15.16:0/165802174 >> 172.21.15.16:0/165802174 conn(0x55d32db44310 msgr2=0x55d32db85030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 -- 172.21.15.16:0/165802174 shutdown_connections 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.675+0000 7f60db025740 1 -- 172.21.15.16:0/165802174 wait complete. 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 -- start start 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 Processor -- start 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db8c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db86370 0x55d32db8ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41aca20 con 0x55d32db7b5d0 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41ac350 con 0x55d32db86370 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60d41a95b0 con 0x55d32db7cb40 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60db025740 10 client.0 _ll_register_callbacks cb 0x7f60cc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52740/0 (socket says 172.21.15.16:52740) 2024-09-12T00:59:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60d259c640 1 -- 172.21.15.16:0/3328285572 learned_addr learned my addr 172.21.15.16:0/3328285572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60d2d9d640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db86370 0x55d32db8ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.676+0000 7f60cbfff640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db8c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 -- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 msgr2=0x55d32db8c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db8c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 -- 172.21.15.16:0/3328285572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db86370 msgr2=0x55d32db8ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db86370 0x55d32db8ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 -- 172.21.15.16:0/3328285572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32dc01c50 con 0x55d32db7b5d0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60d259c640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f60c40b8d40 tx=0x7f60c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a83c7f8396d6b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4049040 con 0x55d32db7b5d0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60bc005020 con 0x55d32db7b5d0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f60c40ed820 con 0x55d32db7b5d0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60c40a5020 con 0x55d32db7b5d0 2024-09-12T00:59:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60db025740 10 client.16389 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d32dc02f00 con 0x55d32db7b5d0 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60bc005020 con 0x55d32db7b5d0 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60c40c4b50 con 0x55d32db7b5d0 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60db025740 10 client.16389 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.677+0000 7f60db025740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60db025740 20 client.16389 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32dc03620 0x55d32dc05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60db025740 4 client.16389 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d32dae8d70 con 0x55d32dc03620 2024-09-12T00:59:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60cbfff640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32dc03620 0x55d32dc05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60cbfff640 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32dc03620 0x55d32dc05a10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d182a3da6d2a9b server_cookie=9c57bc2e16adb0b in_seq=0 out_seq=0 2024-09-12T00:59:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.678+0000 7f60b9ffb640 10 client.16389 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60b9ffb640 1 -- 172.21.15.16:0/3328285572 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d32dae8d70 con 0x55d32dc03620 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60b9ffb640 10 client.16389 handle_command_reply: tid=0 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 client.16389 shutdown 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 msgr2=0x55d32db8c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f60c40b8d40 tx=0x7f60c402a480 comp rx=0 tx=0).stop 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 shutdown_connections 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d32dc03620 0x55d32dc05a10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d32db86370 0x55d32db8ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d32db7cb40 0x55d32db8c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 --2- 172.21.15.16:0/3328285572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d32db7b5d0 0x55d32db8c3f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 >> 172.21.15.16:0/3328285572 conn(0x55d32db44310 msgr2=0x55d32db764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60b9ffb640 0 client.16389 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 shutdown_connections 2024-09-12T00:59:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 1 -- 172.21.15.16:0/3328285572 wait complete. 2024-09-12T00:59:34.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60db025740 20 client.16389 trim_cache size 0 max 16384 2024-09-12T00:59:34.698 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 msgr2=0x7f60c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 0x7f60c00a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60cc01f000 tx=0x7f60cc020000 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 msgr2=0x7f60d4197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006010 tx=0x7f60c4003c80 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 shutdown_connections 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f60c00a2cf0 0x7f60c00a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.679+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60d4102ef0 0x7f60d419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.680+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60d4102570 0x7f60d41979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.680+0000 7f60d8c4c640 1 --2- 172.21.15.16:0/3573008834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60d4100e20 0x7f60d4197460 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.680+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 >> 172.21.15.16:0/3573008834 conn(0x7f60d40fc7d0 msgr2=0x7f60d40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.680+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 shutdown_connections 2024-09-12T00:59:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:34.680+0000 7f60d8c4c640 1 -- 172.21.15.16:0/3573008834 wait complete. 2024-09-12T00:59:35.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:35 smithi016 ceph-mon[30972]: pgmap v108: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:35 smithi099 ceph-mon[35654]: pgmap v108: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:35 smithi159 ceph-mon[35566]: pgmap v108: 97 pgs: 97 active+clean; 2.8 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-12T00:59:36.536 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T00:59:36.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T00:59:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 -- 172.21.15.16:0/4187448567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c102790 msgr2=0x7feb1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 --2- 172.21.15.16:0/4187448567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c102790 0x7feb1c1067e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb04009900 tx=0x7feb0402a410 comp rx=0 tx=0).stop 2024-09-12T00:59:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 -- 172.21.15.16:0/4187448567 shutdown_connections 2024-09-12T00:59:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 --2- 172.21.15.16:0/4187448567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 --2- 172.21.15.16:0/4187448567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c102790 0x7feb1c1067e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 --2- 172.21.15.16:0/4187448567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c101de0 0x7feb1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 -- 172.21.15.16:0/4187448567 >> 172.21.15.16:0/4187448567 conn(0x7feb1c0fd8c0 msgr2=0x7feb1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 -- 172.21.15.16:0/4187448567 shutdown_connections 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.282+0000 7feb20874640 1 -- 172.21.15.16:0/4187448567 wait complete. 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 Processor -- start 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 -- start start 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c102790 0x7feb1c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c111f70 con 0x7feb1c101de0 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c111df0 con 0x7feb1c106f10 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb20874640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1120f0 con 0x7feb1c102790 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52756/0 (socket says 172.21.15.16:52756) 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.283+0000 7feb1a59c640 1 -- 172.21.15.16:0/760054444 learned_addr learned my addr 172.21.15.16:0/760054444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb11d9b640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c102790 0x7feb1c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 -- 172.21.15.16:0/760054444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c102790 msgr2=0x7feb1c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c102790 0x7feb1c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 -- 172.21.15.16:0/760054444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 msgr2=0x7feb1c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 -- 172.21.15.16:0/760054444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19f640 con 0x7feb1c101de0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb1a59c640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00d950 tx=0x7feb0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb13fff640 1 -- 172.21.15.16:0/760054444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c017070 con 0x7feb1c101de0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c19f930 con 0x7feb1c101de0 2024-09-12T00:59:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb13fff640 1 -- 172.21.15.16:0/760054444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0c009c30 con 0x7feb1c101de0 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb13fff640 1 -- 172.21.15.16:0/760054444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00f6b0 con 0x7feb1c101de0 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.284+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a7210 con 0x7feb1c101de0 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb13fff640 1 -- 172.21.15.16:0/760054444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feb0c0109d0 con 0x7feb1c101de0 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 Processor -- start 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 -- start start 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c17130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f92b6f280 con 0x562f92c09b60 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562f92b16b10 con 0x562f92c0a480 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb22c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f92b6b240 con 0x562f92c14d40 2024-09-12T00:59:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.287+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52766/0 (socket says 172.21.15.16:52766) 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3184334238 learned_addr learned my addr 172.21.15.16:0/3184334238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb13fff640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 0x7feb080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c17130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3184334238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 msgr2=0x562f92c17130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c17130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3184334238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 msgr2=0x562f92c14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c14800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3184334238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f92b231f0 con 0x562f92c09b60 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c17130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 0x7feb080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb13fff640 1 -- 172.21.15.16:0/760054444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb0c0cf0b0 con 0x7feb1c101de0 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb14009fa0 tx=0x7feb1400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b434d9a5253e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb10d99640 1 -- 172.21.15.16:0/3184334238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1401a610 con 0x562f92c09b60 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb10d99640 1 -- 172.21.15.16:0/3184334238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb1401ac90 con 0x562f92c09b60 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb10d99640 1 -- 172.21.15.16:0/3184334238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14010a80 con 0x562f92c09b60 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.288+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 0x7feb080a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feb14020000 tx=0x7feb1401d000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 -- 172.21.15.16:0/3184334238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 msgr2=0x562f92c09f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb14009fa0 tx=0x7feb1400f7f0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 -- 172.21.15.16:0/3184334238 shutdown_connections 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c17130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3184334238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c09f40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 -- 172.21.15.16:0/3184334238 >> 172.21.15.16:0/3184334238 conn(0x562f92c060f0 msgr2=0x562f92c064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 -- 172.21.15.16:0/3184334238 shutdown_connections 2024-09-12T00:59:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.289+0000 7feb22c4d740 1 -- 172.21.15.16:0/3184334238 wait complete. 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 -- start start 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 Processor -- start 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c12160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f92c17910 con 0x562f92c09b60 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562f92b29f10 con 0x562f92c0a480 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f92c17a90 con 0x562f92c14d40 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb22c4d740 10 client.0 _ll_register_callbacks cb 0x562f92af2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52780/0 (socket says 172.21.15.16:52780) 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 -- 172.21.15.16:0/3048758656 learned_addr learned my addr 172.21.15.16:0/3048758656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c12160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 -- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 msgr2=0x562f92c126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 -- 172.21.15.16:0/3048758656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 msgr2=0x562f92c12160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c12160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb1a59c640 1 -- 172.21.15.16:0/3048758656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f928e06a0 con 0x562f92c09b60 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.290+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c12160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009610 tx=0x7feb0c09b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d093d02fee47ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c020040 con 0x562f92c09b60 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb0c0d10e0 con 0x562f92c09b60 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb1401ac90 con 0x562f92c09b60 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb0c09bd00 con 0x562f92c09b60 2024-09-12T00:59:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 10 client.16401 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f92c23480 con 0x562f92c09b60 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb1401ac90 con 0x562f92c09b60 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb0c084b40 con 0x562f92c09b60 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 10 client.16401 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.291+0000 7feb22c4d740 20 client.16401 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c247d0 0x562f92c26c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb22c4d740 4 client.16401 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f92b7c7f0 con 0x562f92c247d0 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c247d0 0x562f92c26c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c247d0 0x562f92c26c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d2ed2e6a5a2096 server_cookie=87afe50ebb10f86a in_seq=0 out_seq=0 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.292+0000 7feb01ffb640 10 client.16401 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb01ffb640 1 -- 172.21.15.16:0/3048758656 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f92b7c7f0 con 0x562f92c247d0 2024-09-12T00:59:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb01ffb640 10 client.16401 handle_command_reply: tid=0 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 client.16401 shutdown 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 msgr2=0x562f92c11c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009610 tx=0x7feb0c09b240 comp rx=0 tx=0).stop 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 shutdown_connections 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c247d0 0x562f92c26c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c14d40 0x562f92c126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c12160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3048758656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c09b60 0x562f92c11c20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb01ffb640 0 client.16401 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 >> 172.21.15.16:0/3048758656 conn(0x562f92c060f0 msgr2=0x562f92c0be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 shutdown_connections 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 1 -- 172.21.15.16:0/3048758656 wait complete. 2024-09-12T00:59:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.293+0000 7feb22c4d740 20 client.16401 trim_cache size 0 max 16384 2024-09-12T00:59:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 Processor -- start 2024-09-12T00:59:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 -- start start 2024-09-12T00:59:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c2c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c2c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1afaa0 con 0x562f92c14d40 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b15e0 con 0x562f92c0a480 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb22c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b0170 con 0x562f92c09b60 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52788/0 (socket says 172.21.15.16:52788) 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.311+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3355419587 learned_addr learned my addr 172.21.15.16:0/3355419587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1a59c640 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c2c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb11d9b640 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c2c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3355419587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 msgr2=0x562f92c2c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c2c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3355419587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 msgr2=0x562f92c2c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c2c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 -- 172.21.15.16:0/3355419587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f92c00680 con 0x562f92c14d40 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feb14009fd0 tx=0x7feb1400ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8818d4a20f504a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb03fff640 1 -- 172.21.15.16:0/3355419587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140094a0 con 0x562f92c14d40 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb03fff640 1 -- 172.21.15.16:0/3355419587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb14032070 con 0x562f92c14d40 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.312+0000 7feb03fff640 1 -- 172.21.15.16:0/3355419587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140122a0 con 0x562f92c14d40 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- 172.21.15.16:0/3355419587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 msgr2=0x562f92c04a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feb14009fd0 tx=0x7feb1400ffb0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- 172.21.15.16:0/3355419587 shutdown_connections 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c04a50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c2c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 --2- 172.21.15.16:0/3355419587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c2c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- 172.21.15.16:0/3355419587 >> 172.21.15.16:0/3355419587 conn(0x562f92c060f0 msgr2=0x562f92c13b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- 172.21.15.16:0/3355419587 shutdown_connections 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- 172.21.15.16:0/3355419587 wait complete. 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 -- start start 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.313+0000 7feb22c4d740 1 Processor -- start 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c18670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b0f10 con 0x562f92c14d40 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b0840 con 0x562f92c0a480 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ada80 con 0x562f92c09b60 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 10 client.0 _ll_register_callbacks cb 0x7feb0c097180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52800/0 (socket says 172.21.15.16:52800) 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 -- 172.21.15.16:0/2340365804 learned_addr learned my addr 172.21.15.16:0/2340365804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c1d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:51576/0 (socket says 172.21.15.16:51576) 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb11d9b640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c18670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 -- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 msgr2=0x562f92c1d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c1d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 -- 172.21.15.16:0/2340365804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 msgr2=0x562f92c18670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c18670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 -- 172.21.15.16:0/2340365804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f92c193e0 con 0x562f92c14d40 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb1ad9d640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feb14010600 tx=0x7feb14005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940683709f26f366 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1403a070 con 0x562f92c14d40 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7feb14012040 con 0x562f92c14d40 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.314+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb14032070 con 0x562f92c14d40 2024-09-12T00:59:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb140094a0 con 0x562f92c14d40 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 10 client.16407 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562f92c8f390 con 0x562f92c14d40 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb14032070 con 0x562f92c14d40 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb1402d440 con 0x562f92c14d40 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 10 client.16407 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 20 client.16407 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c8f9f0 0x562f92c91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 4 client.16407 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f92b76d00 con 0x562f92c8f9f0 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.315+0000 7feb1a59c640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c8f9f0 0x562f92c91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.316+0000 7feb1a59c640 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c8f9f0 0x562f92c91de0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcc15c138210f12 server_cookie=1865e4c3318485b1 in_seq=0 out_seq=0 2024-09-12T00:59:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.316+0000 7feb01ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb01ffb640 1 -- 172.21.15.16:0/2340365804 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f92b76d00 con 0x562f92c8f9f0 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb01ffb640 10 client.16407 handle_command_reply: tid=0 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 client.16407 shutdown 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 msgr2=0x562f92c18bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feb14010600 tx=0x7feb14005f70 comp rx=0 tx=0).stop 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 shutdown_connections 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562f92c8f9f0 0x562f92c91de0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f92c14d40 0x562f92c18bb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562f92c0a480 0x562f92c18670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 --2- 172.21.15.16:0/2340365804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f92c09b60 0x562f92c1d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb01ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 >> 172.21.15.16:0/2340365804 conn(0x562f92c060f0 msgr2=0x562f92c0b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 shutdown_connections 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 1 -- 172.21.15.16:0/2340365804 wait complete. 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.317+0000 7feb22c4d740 20 client.16407 trim_cache size 0 max 16384 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "404ef0f8-ec5f-4881-9f09-55e860f89658", 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T00:59:37.323 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 msgr2=0x7feb080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 0x7feb080a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feb14020000 tx=0x7feb1401d000 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 msgr2=0x7feb1c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00d950 tx=0x7feb0c00de10 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 shutdown_connections 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7feb080a2cf0 0x7feb080a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb1c106f10 0x7feb1c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb1c102790 0x7feb1c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 --2- 172.21.15.16:0/760054444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb1c101de0 0x7feb1c19a690 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 >> 172.21.15.16:0/760054444 conn(0x7feb1c0fd8c0 msgr2=0x7feb1c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:37.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 shutdown_connections 2024-09-12T00:59:37.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:37.318+0000 7feb20874640 1 -- 172.21.15.16:0/760054444 wait complete. 2024-09-12T00:59:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:37 smithi099 ceph-mon[35654]: pgmap v109: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:37 smithi159 ceph-mon[35566]: pgmap v109: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:37 smithi016 ceph-mon[30972]: pgmap v109: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:38.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T00:59:38.477 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T00:59:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:38 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T00:59:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:38 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:38 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T00:59:38.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:38 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:38 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T00:59:38.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:38 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:38 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T00:59:38.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:38 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:38 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T00:59:38.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:38 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:38 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T00:59:38.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:38 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T00:59:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 -- 172.21.15.16:0/455445075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c112b90 msgr2=0x7f099c114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 --2- 172.21.15.16:0/455445075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c112b90 0x7f099c114f80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0994009ca0 tx=0x7f099402a950 comp rx=0 tx=0).stop 2024-09-12T00:59:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 -- 172.21.15.16:0/455445075 shutdown_connections 2024-09-12T00:59:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 --2- 172.21.15.16:0/455445075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c112b90 0x7f099c114f80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:38.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 --2- 172.21.15.16:0/455445075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c074eb0 0x7f099c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 --2- 172.21.15.16:0/455445075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074590 0x7f099c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 -- 172.21.15.16:0/455445075 >> 172.21.15.16:0/455445075 conn(0x7f099c0fd8c0 msgr2=0x7f099c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 -- 172.21.15.16:0/455445075 shutdown_connections 2024-09-12T00:59:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.956+0000 7f09a2a12640 1 -- 172.21.15.16:0/455445075 wait complete. 2024-09-12T00:59:38.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 Processor -- start 2024-09-12T00:59:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 -- start start 2024-09-12T00:59:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 0x7f099c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c112b90 0x7f099c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f099c118550 con 0x7f099c074590 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1183d0 con 0x7f099c074eb0 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a2a12640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1186d0 con 0x7f099c112b90 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:38.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57992/0 (socket says 172.21.15.16:57992) 2024-09-12T00:59:38.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 -- 172.21.15.16:0/2274274446 learned_addr learned my addr 172.21.15.16:0/2274274446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:38.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c112b90 0x7f099c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:38.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f0993fff640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 0x7f099c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:38.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 -- 172.21.15.16:0/2274274446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c112b90 msgr2=0x7f099c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:38.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c112b90 0x7f099c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 -- 172.21.15.16:0/2274274446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 msgr2=0x7f099c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 0x7f099c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f099bfff640 1 -- 172.21.15.16:0/2274274446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c19efe0 con 0x7f099c074590 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.957+0000 7f0993fff640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 0x7f099c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f099bfff640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e9f0 tx=0x7f098c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:38.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f0999ffb640 1 -- 172.21.15.16:0/2274274446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00cd80 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c19f2d0 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f0999ffb640 1 -- 172.21.15.16:0/2274274446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098c010910 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f0999ffb640 1 -- 172.21.15.16:0/2274274446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0188f0 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.958+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c1ab520 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.960+0000 7f0999ffb640 1 -- 172.21.15.16:0/2274274446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f098c010ac0 con 0x7f099c074590 2024-09-12T00:59:38.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 Processor -- start 2024-09-12T00:59:38.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 -- start start 2024-09-12T00:59:38.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e01880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e0c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:38.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560701d672e0 con 0x560701e0c680 2024-09-12T00:59:39.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560701d0ece0 con 0x560701e01dc0 2024-09-12T00:59:39.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a4deb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560701d632a0 con 0x560701e014a0 2024-09-12T00:59:39.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58000/0 (socket says 172.21.15.16:58000) 2024-09-12T00:59:39.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 -- 172.21.15.16:0/3761615911 learned_addr learned my addr 172.21.15.16:0/3761615911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a0faf640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e01880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f099bfff640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e0c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0999ffb640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 0x7f09880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 -- 172.21.15.16:0/3761615911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 msgr2=0x560701e01880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e01880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 -- 172.21.15.16:0/3761615911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 msgr2=0x560701e0c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e0c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f0993fff640 1 -- 172.21.15.16:0/3761615911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560701cf3440 con 0x560701e0c680 2024-09-12T00:59:39.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a0faf640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e01880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 0x7f09880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.961+0000 7f099bfff640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.962+0000 7f0999ffb640 1 -- 172.21.15.16:0/2274274446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f098c0cf880 con 0x7f099c074590 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.962+0000 7f0993fff640 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f098400bc60 tx=0x7f098400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd83fc469242b53c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.962+0000 7f09927fc640 1 -- 172.21.15.16:0/3761615911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984015070 con 0x560701e0c680 2024-09-12T00:59:39.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.962+0000 7f09927fc640 1 -- 172.21.15.16:0/3761615911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098401a940 con 0x560701e0c680 2024-09-12T00:59:39.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.962+0000 7f09927fc640 1 -- 172.21.15.16:0/3761615911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09840118e0 con 0x560701e0c680 2024-09-12T00:59:39.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- 172.21.15.16:0/3761615911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 msgr2=0x560701e0ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 0x7f09880a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0994005e30 tx=0x7f09940095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:39.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f098400bc60 tx=0x7f098400f730 comp rx=0 tx=0).stop 2024-09-12T00:59:39.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- 172.21.15.16:0/3761615911 shutdown_connections 2024-09-12T00:59:39.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e0ea70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e0c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- 172.21.15.16:0/3761615911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e01880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- 172.21.15.16:0/3761615911 >> 172.21.15.16:0/3761615911 conn(0x560701dfda30 msgr2=0x560701dfde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- 172.21.15.16:0/3761615911 shutdown_connections 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- 172.21.15.16:0/3761615911 wait complete. 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- start start 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 Processor -- start 2024-09-12T00:59:39.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560701e0f250 con 0x560701e0c680 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560701d22050 con 0x560701e01dc0 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.963+0000 7f09a4deb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560701e0f3d0 con 0x560701e014a0 2024-09-12T00:59:39.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a4deb740 10 client.0 _ll_register_callbacks cb 0x560701d62160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f0993fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58008/0 (socket says 172.21.15.16:58008) 2024-09-12T00:59:39.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f0993fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:53308/0 (socket says 172.21.15.16:53308) 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 -- 172.21.15.16:0/4018986663 learned_addr learned my addr 172.21.15.16:0/4018986663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f099bfff640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 -- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 msgr2=0x560701e0bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 -- 172.21.15.16:0/4018986663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 msgr2=0x560701e091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e091e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 -- 172.21.15.16:0/4018986663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560701ad83c0 con 0x560701e0c680 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a0faf640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f099402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa522d954b5f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f097b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994034040 con 0x560701e0c680 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0994003d80 con 0x560701e0c680 2024-09-12T00:59:39.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.964+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098401a940 con 0x560701e0c680 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0994030910 con 0x560701e0c680 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 10 client.16419 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560701e1ab70 con 0x560701e0c680 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098401a940 con 0x560701e0c680 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f099403a430 con 0x560701e0c680 2024-09-12T00:59:39.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 10 client.16419 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 20 client.16419 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e1c080 0x560701e1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 4 client.16419 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T00:59:39.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560701d74850 con 0x560701e1c080 2024-09-12T00:59:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.965+0000 7f099bfff640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e1c080 0x560701e1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.966+0000 7f097b7fe640 10 client.16419 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.966+0000 7f099bfff640 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e1c080 0x560701e1e4d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a3496207c2acd9 server_cookie=7e9e0852f7f2ab1d in_seq=0 out_seq=0 2024-09-12T00:59:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.966+0000 7f097b7fe640 1 -- 172.21.15.16:0/4018986663 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560701d74850 con 0x560701e1c080 2024-09-12T00:59:39.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.966+0000 7f097b7fe640 10 client.16419 handle_command_reply: tid=0 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 client.16419 shutdown 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 msgr2=0x560701e09720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f099402a950 comp rx=0 tx=0).stop 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 shutdown_connections 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e1c080 0x560701e1e4d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e0c680 0x560701e09720 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e01dc0 0x560701e091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f097b7fe640 0 client.16419 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 --2- 172.21.15.16:0/4018986663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 >> 172.21.15.16:0/4018986663 conn(0x560701dfda30 msgr2=0x560701e03710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 shutdown_connections 2024-09-12T00:59:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 1 -- 172.21.15.16:0/4018986663 wait complete. 2024-09-12T00:59:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.967+0000 7f09a4deb740 20 client.16419 trim_cache size 0 max 16384 2024-09-12T00:59:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 Processor -- start 2024-09-12T00:59:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 -- start start 2024-09-12T00:59:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 0x560701e091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b3e00 con 0x560701e01dc0 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b5940 con 0x560701dfda30 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a4deb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b44d0 con 0x560701e014a0 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58010/0 (socket says 172.21.15.16:58010) 2024-09-12T00:59:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 -- 172.21.15.16:0/2824349796 learned_addr learned my addr 172.21.15.16:0/2824349796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f0993fff640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f099bfff640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 0x560701e091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 -- 172.21.15.16:0/2824349796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 msgr2=0x560701e0b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 -- 172.21.15.16:0/2824349796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 msgr2=0x560701e091a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 0x560701e091a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 -- 172.21.15.16:0/2824349796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560701e0c2a0 con 0x560701e01dc0 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f099bfff640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 0x560701e091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.985+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f099402fd90 tx=0x7f099403acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b121ac0c6ec8e3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f0991ffb640 1 -- 172.21.15.16:0/2824349796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099403a630 con 0x560701e01dc0 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f0991ffb640 1 -- 172.21.15.16:0/2824349796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994009800 con 0x560701e01dc0 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f0991ffb640 1 -- 172.21.15.16:0/2824349796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994045cb0 con 0x560701e01dc0 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 -- 172.21.15.16:0/2824349796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 msgr2=0x560701e23da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f099402fd90 tx=0x7f099403acd0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 -- 172.21.15.16:0/2824349796 shutdown_connections 2024-09-12T00:59:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701e01dc0 0x560701e23da0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e014a0 0x560701e0b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2824349796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701dfda30 0x560701e091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 -- 172.21.15.16:0/2824349796 >> 172.21.15.16:0/2824349796 conn(0x560701e09880 msgr2=0x560701e0aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 -- 172.21.15.16:0/2824349796 shutdown_connections 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.986+0000 7f09a4deb740 1 -- 172.21.15.16:0/2824349796 wait complete. 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 -- start start 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 Processor -- start 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 0x560701e12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 0x560701e127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b5270 con 0x560701dfda30 2024-09-12T00:59:39.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b4ba0 con 0x560701e014a0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1b1de0 con 0x560701e01dc0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a4deb740 10 client.0 _ll_register_callbacks cb 0x7f0994009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f099bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f099bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58018/0 (socket says 172.21.15.16:58018) 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f099bfff640 1 -- 172.21.15.16:0/2762518303 learned_addr learned my addr 172.21.15.16:0/2762518303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 0x560701e127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f0993fff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 0x560701e12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.987+0000 7f099bfff640 1 -- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 msgr2=0x560701e127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f099bfff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 0x560701e127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f099bfff640 1 -- 172.21.15.16:0/2762518303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 msgr2=0x560701e12230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f099bfff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 0x560701e12230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f099bfff640 1 -- 172.21.15.16:0/2762518303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560701e13000 con 0x560701dfda30 2024-09-12T00:59:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f09a0faf640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 0x560701e127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f0993fff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 0x560701e12230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f099bfff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f098c0cd060 tx=0x7f098c0cfb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d56f9409cc47ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f097b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0a4a40 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f098c098440 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0994009800 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f098c09b220 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f09a4deb740 10 client.16425 fetch_fsmap learned FSMap version 28 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560701e86e30 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.988+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994009800 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f098c0b1030 con 0x560701dfda30 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 10 client.16425 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T00:59:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 20 client.16425 populate_metadata read hostname 'smithi016' 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e0c680 0x560701e894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 4 client.16425 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560701d6ed60 con 0x560701e0c680 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f0993fff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e0c680 0x560701e894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f0993fff640 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e0c680 0x560701e894c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb300c52b234d631 server_cookie=a65939a03e4a4012 in_seq=0 out_seq=0 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.989+0000 7f097b7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f097b7fe640 1 -- 172.21.15.16:0/2762518303 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560701d6ed60 con 0x560701e0c680 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f097b7fe640 10 client.16425 handle_command_reply: tid=0 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 client.16425 shutdown 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 msgr2=0x560701e14e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f098c0cd060 tx=0x7f098c0cfb80 comp rx=0 tx=0).stop 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 shutdown_connections 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560701e0c680 0x560701e894c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560701e01dc0 0x560701e127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560701e014a0 0x560701e12230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 --2- 172.21.15.16:0/2762518303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560701dfda30 0x560701e14e40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 >> 172.21.15.16:0/2762518303 conn(0x560701e09880 msgr2=0x560701e0b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f097b7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 shutdown_connections 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 1 -- 172.21.15.16:0/2762518303 wait complete. 2024-09-12T00:59:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.990+0000 7f09a4deb740 20 client.16425 trim_cache size 0 max 16384 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "404ef0f8-ec5f-4881-9f09-55e860f89658": { 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "404ef0f8-ec5f-4881-9f09-55e860f89658", 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T00:59:39.025 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T00:59:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 msgr2=0x7f09880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 0x7f09880a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0994005e30 tx=0x7f09940095c0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 msgr2=0x7f099c1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T00:59:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e9f0 tx=0x7f098c00eeb0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 shutdown_connections 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09880a2cf0 0x7f09880a51b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f099c112b90 0x7f099c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099c074eb0 0x7f099c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 --2- 172.21.15.16:0/2274274446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f099c074590 0x7f099c1a46c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 >> 172.21.15.16:0/2274274446 conn(0x7f099c0fd8c0 msgr2=0x7f099c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 shutdown_connections 2024-09-12T00:59:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T00:59:38.991+0000 7f09a2a12640 1 -- 172.21.15.16:0/2274274446 wait complete. 2024-09-12T00:59:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:39 smithi099 ceph-mon[35654]: pgmap v110: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T00:59:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:39 smithi159 ceph-mon[35566]: pgmap v110: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T00:59:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:39 smithi016 ceph-mon[30972]: pgmap v110: 97 pgs: 97 active+clean; 2.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-12T00:59:39.785 INFO:tasks.cephfs.filesystem:scrub status for tag:404ef0f8-ec5f-4881-9f09-55e860f89658 - {'path': '/', 'tag': '404ef0f8-ec5f-4881-9f09-55e860f89658', 'options': 'recursive,force'} 2024-09-12T00:59:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:40 smithi099 ceph-mon[35654]: pgmap v111: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:40.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:40 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T00:59:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:40 smithi159 ceph-mon[35566]: pgmap v111: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:40.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:40 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T00:59:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:40 smithi016 ceph-mon[30972]: pgmap v111: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T00:59:40.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:40 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T00:59:42.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:42 smithi099 ceph-mon[35654]: pgmap v112: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:42.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:42 smithi159 ceph-mon[35566]: pgmap v112: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:42.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:42 smithi016 ceph-mon[30972]: pgmap v112: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:44.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:44 smithi099 ceph-mon[35654]: pgmap v113: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:44 smithi159 ceph-mon[35566]: pgmap v113: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:44 smithi016 ceph-mon[30972]: pgmap v113: 97 pgs: 97 active+clean; 3.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:46.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:46 smithi099 ceph-mon[35654]: pgmap v114: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:46.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:46 smithi159 ceph-mon[35566]: pgmap v114: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:46.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:46 smithi016 ceph-mon[30972]: pgmap v114: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:48.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:48 smithi099 ceph-mon[35654]: pgmap v115: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:48 smithi159 ceph-mon[35566]: pgmap v115: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:48 smithi016 ceph-mon[30972]: pgmap v115: 97 pgs: 97 active+clean; 3.2 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-09-12T00:59:50.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:50 smithi099 ceph-mon[35654]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-09-12T00:59:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:50 smithi159 ceph-mon[35566]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-09-12T00:59:50.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:50 smithi016 ceph-mon[30972]: pgmap v116: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-09-12T00:59:52.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:52 smithi099 ceph-mon[35654]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:52.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:52 smithi159 ceph-mon[35566]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:52 smithi016 ceph-mon[30972]: pgmap v117: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:54.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:54 smithi099 ceph-mon[35654]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:54.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:54 smithi159 ceph-mon[35566]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:54.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:54 smithi016 ceph-mon[30972]: pgmap v118: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:56.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:56 smithi099 ceph-mon[35654]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:56 smithi159 ceph-mon[35566]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:56 smithi016 ceph-mon[30972]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-09-12T00:59:58.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 00:59:58 smithi099 ceph-mon[35654]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 00:59:58 smithi159 ceph-mon[35566]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T00:59:58.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 00:59:58 smithi016 ceph-mon[30972]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-09-12T01:00:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:00 smithi099 ceph-mon[35654]: pgmap v121: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:00:00.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:00 smithi099 ceph-mon[35654]: overall HEALTH_OK 2024-09-12T01:00:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:00 smithi159 ceph-mon[35566]: pgmap v121: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:00:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:00 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T01:00:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:00 smithi016 ceph-mon[30972]: pgmap v121: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:00:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:00 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T01:00:02.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:02 smithi099 ceph-mon[35654]: pgmap v122: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:02.851 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:00:02.851 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T01:00:02.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:02.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:02 smithi159 ceph-mon[35566]: pgmap v122: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:02.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:02 smithi016 ceph-mon[30972]: pgmap v122: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:03.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 -- 172.21.15.16:0/3097621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c102ef0 msgr2=0x7f759c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:03.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 --2- 172.21.15.16:0/3097621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c102ef0 0x7f759c10da40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f7594009ca0 tx=0x7f759402a950 comp rx=0 tx=0).stop 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 -- 172.21.15.16:0/3097621401 shutdown_connections 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 --2- 172.21.15.16:0/3097621401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c102ef0 0x7f759c10da40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 --2- 172.21.15.16:0/3097621401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 0x7f759c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 --2- 172.21.15.16:0/3097621401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c100e20 0x7f759c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 -- 172.21.15.16:0/3097621401 >> 172.21.15.16:0/3097621401 conn(0x7f759c0fc7d0 msgr2=0x7f759c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 -- 172.21.15.16:0/3097621401 shutdown_connections 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.657+0000 7f75a294c640 1 -- 172.21.15.16:0/3097621401 wait complete. 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 Processor -- start 2024-09-12T01:00:03.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 -- start start 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 0x7f759c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 0x7f759c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f759c110f80 con 0x7f759c100e20 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f759c110e00 con 0x7f759c102ef0 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a294c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f759c111100 con 0x7f759c102570 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f759bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f759bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52924/0 (socket says 172.21.15.16:52924) 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f759bfff640 1 -- 172.21.15.16:0/348666974 learned_addr learned my addr 172.21.15.16:0/348666974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:03.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f7593fff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 0x7f759c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.658+0000 7f75a0ee9640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 0x7f759c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 -- 172.21.15.16:0/348666974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 msgr2=0x7f759c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 0x7f759c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 -- 172.21.15.16:0/348666974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 msgr2=0x7f759c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 0x7f759c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 -- 172.21.15.16:0/348666974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c1a6910 con 0x7f759c100e20 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f75a0ee9640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 0x7f759c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f759bfff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f758c00d950 tx=0x7f758c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c017070 con 0x7f759c100e20 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758c009c30 con 0x7f759c100e20 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c00f6b0 con 0x7f759c100e20 2024-09-12T01:00:03.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c1a6c00 con 0x7f759c100e20 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.659+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1a7140 con 0x7f759c100e20 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.660+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c101200 con 0x7f759c100e20 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.661+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f758c0109d0 con 0x7f759c100e20 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.662+0000 7f7599ffb640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.662+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f758c0cf070 con 0x7f759c100e20 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.662+0000 7f7593fff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.662+0000 7f7593fff640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 0x7f75880a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f759c0fe710 tx=0x7f758400a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.666+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f758c009de0 con 0x7f759c100e20 2024-09-12T01:00:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.881+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759c02d7c0 con 0x7f759c100e20 2024-09-12T01:00:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.882+0000 7f7599ffb640 1 -- 172.21.15.16:0/348666974 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f758c09bcc0 con 0x7f759c100e20 2024-09-12T01:00:03.882 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:03.883 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 msgr2=0x7f75880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 0x7f75880a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f759c0fe710 tx=0x7f758400a2d0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 msgr2=0x7f759c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f758c00d950 tx=0x7f758c00de10 comp rx=0 tx=0).stop 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 shutdown_connections 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f75880a2cf0 0x7f75880a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f759c102ef0 0x7f759c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f759c102570 0x7f759c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 --2- 172.21.15.16:0/348666974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759c100e20 0x7f759c0fcc80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.884+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 >> 172.21.15.16:0/348666974 conn(0x7f759c0fc7d0 msgr2=0x7f759c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.885+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 shutdown_connections 2024-09-12T01:00:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:03.885+0000 7f75a294c640 1 -- 172.21.15.16:0/348666974 wait complete. 2024-09-12T01:00:04.714 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T01:00:04.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T01:00:04.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:04 smithi099 ceph-mon[35654]: pgmap v123: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:04.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/348666974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:04.779 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:04 smithi016 ceph-mon[30972]: pgmap v123: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:04.779 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/348666974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:04.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:04 smithi159 ceph-mon[35566]: pgmap v123: 97 pgs: 97 active+clean; 3.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-09-12T01:00:04.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/348666974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:05.015 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.649+0000 7f33f3010640 1 -- 172.21.15.16:0/1515216969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10e3c0 msgr2=0x7f33ec110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.649+0000 7f33f3010640 1 --2- 172.21.15.16:0/1515216969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10e3c0 0x7f33ec110810 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009ca0 tx=0x7f33e402a950 comp rx=0 tx=0).stop 2024-09-12T01:00:05.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 -- 172.21.15.16:0/1515216969 shutdown_connections 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- 172.21.15.16:0/1515216969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec10e3c0 0x7f33ec110810 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- 172.21.15.16:0/1515216969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074f40 0x7f33ec10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- 172.21.15.16:0/1515216969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec074620 0x7f33ec074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 -- 172.21.15.16:0/1515216969 >> 172.21.15.16:0/1515216969 conn(0x7f33ec0fd8c0 msgr2=0x7f33ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 -- 172.21.15.16:0/1515216969 shutdown_connections 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 -- 172.21.15.16:0/1515216969 wait complete. 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 Processor -- start 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 -- start start 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074620 0x7f33ec19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.650+0000 7f33f3010640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec10e3c0 0x7f33ec19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f3010640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec114160 con 0x7f33ec074f40 2024-09-12T01:00:05.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f3010640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec113fe0 con 0x7f33ec074620 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f3010640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1142e0 con 0x7f33ec10e3c0 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52942/0 (socket says 172.21.15.16:52942) 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f0dac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074620 0x7f33ec19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 -- 172.21.15.16:0/1599806882 learned_addr learned my addr 172.21.15.16:0/1599806882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f15ad640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec10e3c0 0x7f33ec19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 -- 172.21.15.16:0/1599806882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec10e3c0 msgr2=0x7f33ec19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec10e3c0 0x7f33ec19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 -- 172.21.15.16:0/1599806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074620 msgr2=0x7f33ec19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074620 0x7f33ec19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 -- 172.21.15.16:0/1599806882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec19f6b0 con 0x7f33ec074f40 2024-09-12T01:00:05.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33ebfff640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f33d800d950 tx=0x7f33d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33e9ffb640 1 -- 172.21.15.16:0/1599806882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8017070 con 0x7f33ec074f40 2024-09-12T01:00:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec19f9a0 con 0x7f33ec074f40 2024-09-12T01:00:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33e9ffb640 1 -- 172.21.15.16:0/1599806882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d8009c30 con 0x7f33ec074f40 2024-09-12T01:00:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.651+0000 7f33e9ffb640 1 -- 172.21.15.16:0/1599806882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d800f6b0 con 0x7f33ec074f40 2024-09-12T01:00:05.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.652+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec1a7280 con 0x7f33ec074f40 2024-09-12T01:00:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.654+0000 7f33e9ffb640 1 -- 172.21.15.16:0/1599806882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f33d80109d0 con 0x7f33ec074f40 2024-09-12T01:00:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.654+0000 7f33f53e9740 1 Processor -- start 2024-09-12T01:00:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.654+0000 7f33f53e9740 1 -- start start 2024-09-12T01:00:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.654+0000 7f33e9ffb640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 0x7f33dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.654+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85949c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85956e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f53e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b858ae160 con 0x556b8594a170 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f53e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b85855720 con 0x556b85949850 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f53e9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b858aa120 con 0x556b85954a30 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33e9ffb640 1 -- 172.21.15.16:0/1599806882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f33d80cef00 con 0x7f33ec074f40 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f15ad640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 0x7f33dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85949c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52950/0 (socket says 172.21.15.16:52950) 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33ebfff640 1 -- 172.21.15.16:0/1570767825 learned_addr learned my addr 172.21.15.16:0/1570767825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f15ad640 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85956e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 -- 172.21.15.16:0/1570767825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 msgr2=0x556b85956e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85956e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 -- 172.21.15.16:0/1570767825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 msgr2=0x556b859544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 -- 172.21.15.16:0/1570767825 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8561fba0 con 0x556b85949850 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33ebfff640 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f15ad640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 0x7f33dc0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005e30 tx=0x7f33e402fa20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.655+0000 7f33f0dac640 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85949c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004630 tx=0x7f33e000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455aa2f96d9c654 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33d6ffd640 1 -- 172.21.15.16:0/1570767825 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0003bd0 con 0x556b85949850 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33d6ffd640 1 -- 172.21.15.16:0/1570767825 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0015070 con 0x556b85949850 2024-09-12T01:00:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33d6ffd640 1 -- 172.21.15.16:0/1570767825 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004cb0 con 0x556b85949850 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 -- 172.21.15.16:0/1570767825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 msgr2=0x556b85949c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85949c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004630 tx=0x7f33e000bb20 comp rx=0 tx=0).stop 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 -- 172.21.15.16:0/1570767825 shutdown_connections 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85956e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b859544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 --2- 172.21.15.16:0/1570767825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85949c30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 -- 172.21.15.16:0/1570767825 >> 172.21.15.16:0/1570767825 conn(0x556b85945de0 msgr2=0x556b859461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 -- 172.21.15.16:0/1570767825 shutdown_connections 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.656+0000 7f33f53e9740 1 -- 172.21.15.16:0/1570767825 wait complete. 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 -- start start 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 Processor -- start 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b859518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85952350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556b85957600 con 0x556b85949850 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b85836900 con 0x556b8594a170 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b85957780 con 0x556b85954a30 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f53e9740 10 client.0 _ll_register_callbacks cb 0x556b85834780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f0dac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b859518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:39984/0 (socket says 172.21.15.16:39984) 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33ebfff640 1 -- 172.21.15.16:0/4032132694 learned_addr learned my addr 172.21.15.16:0/4032132694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.657+0000 7f33f15ad640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85952350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 -- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 msgr2=0x556b85952350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85952350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 -- 172.21.15.16:0/4032132694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 msgr2=0x556b859518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b859518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 -- 172.21.15.16:0/4032132694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b85876c50 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33f0dac640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b859518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33ebfff640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33d8019040 tx=0x7f33d809ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973d86b533ba37a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8020070 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f33d8097b70 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e0015070 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33d8013230 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33f53e9740 10 client.25799 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.658+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b859632c0 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0015070 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33d80b4240 con 0x556b8594a170 2024-09-12T01:00:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 10 client.25799 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 10 client.25799 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 20 client.25799 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b85964720 0x556b85966bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 4 client.25799 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b858bb890 con 0x556b85964720 2024-09-12T01:00:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.659+0000 7f33f0dac640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b85964720 0x556b85966bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f0dac640 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b85964720 0x556b85966bb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ecce266ff1a0b8 server_cookie=6649ab6f44070c40 in_seq=0 out_seq=0 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33cbfff640 10 client.25799 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33cbfff640 1 -- 172.21.15.16:0/4032132694 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b858bb890 con 0x556b85964720 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33cbfff640 10 client.25799 handle_command_reply: tid=0 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 client.25799 shutdown 2024-09-12T01:00:05.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 msgr2=0x556b85951e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33d8019040 tx=0x7f33d809ae60 comp rx=0 tx=0).stop 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 shutdown_connections 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b85964720 0x556b85966bb0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b85952350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b8594a170 0x556b85951e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33cbfff640 0 client.25799 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 --2- 172.21.15.16:0/4032132694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b859518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.660+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 >> 172.21.15.16:0/4032132694 conn(0x556b85945de0 msgr2=0x556b8594bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.661+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 shutdown_connections 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.661+0000 7f33f53e9740 1 -- 172.21.15.16:0/4032132694 wait complete. 2024-09-12T01:00:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.661+0000 7f33f53e9740 20 client.25799 trim_cache size 0 max 16384 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 Processor -- start 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 -- start start 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85953840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b8596c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1af9d0 con 0x556b8594a170 2024-09-12T01:00:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b1510 con 0x556b85949850 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.678+0000 7f33f53e9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b00a0 con 0x556b85954a30 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52970/0 (socket says 172.21.15.16:52970) 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 -- 172.21.15.16:0/105049504 learned_addr learned my addr 172.21.15.16:0/105049504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33f15ad640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b8596c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33f0dac640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85953840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 -- 172.21.15.16:0/105049504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 msgr2=0x556b8596c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b8596c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 -- 172.21.15.16:0/105049504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 msgr2=0x556b85953840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85953840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 -- 172.21.15.16:0/105049504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b85949fa0 con 0x556b8594a170 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33f15ad640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b8596c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33f0dac640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85953840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33ebfff640 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33d80997b0 tx=0x7f33d8099a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123e5963adc415a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33d67fc640 1 -- 172.21.15.16:0/105049504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8020070 con 0x556b8594a170 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33d67fc640 1 -- 172.21.15.16:0/105049504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d8009de0 con 0x556b8594a170 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.679+0000 7f33d67fc640 1 -- 172.21.15.16:0/105049504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8013050 con 0x556b8594a170 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- 172.21.15.16:0/105049504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 msgr2=0x556b8596c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f33d80997b0 tx=0x7f33d8099a10 comp rx=0 tx=0).stop 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- 172.21.15.16:0/105049504 shutdown_connections 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b85954a30 0x556b8596c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b8594a170 0x556b8596c480 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 --2- 172.21.15.16:0/105049504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85949850 0x556b85953840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- 172.21.15.16:0/105049504 >> 172.21.15.16:0/105049504 conn(0x556b85945de0 msgr2=0x556b85953130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- 172.21.15.16:0/105049504 shutdown_connections 2024-09-12T01:00:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- 172.21.15.16:0/105049504 wait complete. 2024-09-12T01:00:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.680+0000 7f33f53e9740 1 -- start start 2024-09-12T01:00:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 Processor -- start 2024-09-12T01:00:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8594a170 0x556b85958530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85954a30 0x556b85958b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b0e40 con 0x556b85949850 2024-09-12T01:00:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b0770 con 0x556b85954a30 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1ad9b0 con 0x556b8594a170 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f53e9740 10 client.0 _ll_register_callbacks cb 0x7f33d809b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52978/0 (socket says 172.21.15.16:52978) 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 -- 172.21.15.16:0/3351670313 learned_addr learned my addr 172.21.15.16:0/3351670313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33ebfff640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8594a170 0x556b85958530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f15ad640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85954a30 0x556b85958b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 -- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8594a170 msgr2=0x556b85958530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8594a170 0x556b85958530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 -- 172.21.15.16:0/3351670313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85954a30 msgr2=0x556b85958b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85954a30 0x556b85958b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.681+0000 7f33f0dac640 1 -- 172.21.15.16:0/3351670313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b85959350 con 0x556b85949850 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f0dac640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f33e00039c0 tx=0x7f33e0003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d522b7e060893c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0034060 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f33e002f640 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d8009de0 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33e002f880 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f53e9740 10 client.16449 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b859cf510 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d8009de0 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33e0039900 con 0x556b85949850 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f53e9740 10 client.16449 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.682+0000 7f33f53e9740 10 client.16449 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33f53e9740 20 client.16449 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b859cf9d0 0x556b859d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33f53e9740 4 client.16449 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x556b858b5be0 con 0x556b859cf9d0 2024-09-12T01:00:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33ebfff640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b859cf9d0 0x556b859d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33ebfff640 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b859cf9d0 0x556b859d1dc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252b9bc78f52c798 server_cookie=df0f1d37b6558f7b in_seq=0 out_seq=0 2024-09-12T01:00:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.683+0000 7f33cbfff640 10 client.16449 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.695+0000 7f33cbfff640 1 -- 172.21.15.16:0/3351670313 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b858b5be0 con 0x556b859cf9d0 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.695+0000 7f33cbfff640 10 client.16449 handle_command_reply: tid=0 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.695+0000 7f33f53e9740 1 client.16449 shutdown 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 msgr2=0x556b8595d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f33e00039c0 tx=0x7f33e0003e00 comp rx=0 tx=0).stop 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 shutdown_connections 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556b859cf9d0 0x556b859d1dc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b85954a30 0x556b85958b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8594a170 0x556b85958530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 --2- 172.21.15.16:0/3351670313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556b85949850 0x556b8595d200 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 >> 172.21.15.16:0/3351670313 conn(0x556b85945de0 msgr2=0x556b85944ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33cbfff640 0 client.16449 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 shutdown_connections 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 1 -- 172.21.15.16:0/3351670313 wait complete. 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.696+0000 7f33f53e9740 20 client.16449 trim_cache size 0 max 16384 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:00:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 14, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "3fdcd3ca": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 14, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:05.699 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 msgr2=0x7f33dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 0x7f33dc0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33e4005e30 tx=0x7f33e402fa20 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 msgr2=0x7f33ec19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f33d800d950 tx=0x7f33d800de10 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 shutdown_connections 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33dc0a2cf0 0x7f33dc0a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33ec10e3c0 0x7f33ec19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec074f40 0x7f33ec19ac40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 --2- 172.21.15.16:0/1599806882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33ec074620 0x7f33ec19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 >> 172.21.15.16:0/1599806882 conn(0x7f33ec0fd8c0 msgr2=0x7f33ec0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 shutdown_connections 2024-09-12T01:00:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:05.697+0000 7f33f3010640 1 -- 172.21.15.16:0/1599806882 wait complete. 2024-09-12T01:00:06.572 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3fdcd3ca', quiesce took 3.7 seconds. Will release after: 16.45 2024-09-12T01:00:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:06 smithi099 ceph-mon[35654]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-12T01:00:06.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:06 smithi159 ceph-mon[35566]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-12T01:00:06.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:06 smithi016 ceph-mon[30972]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 34 op/s 2024-09-12T01:00:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:08 smithi099 ceph-mon[35654]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-12T01:00:08.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:08 smithi159 ceph-mon[35566]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-12T01:00:08.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:08 smithi016 ceph-mon[30972]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 44 KiB/s wr, 27 op/s 2024-09-12T01:00:09.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:00:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:10.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:10 smithi016 ceph-mon[30972]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 -- 172.21.15.16:0/2067866449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 msgr2=0x7fb7ec10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 --2- 172.21.15.16:0/2067866449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec10bb50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009900 tx=0x7fb7e002a480 comp rx=0 tx=0).stop 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 -- 172.21.15.16:0/2067866449 shutdown_connections 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 --2- 172.21.15.16:0/2067866449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec10c170 0x7fb7ec10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 --2- 172.21.15.16:0/2067866449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec10bb50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 --2- 172.21.15.16:0/2067866449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec102d40 0x7fb7ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 -- 172.21.15.16:0/2067866449 >> 172.21.15.16:0/2067866449 conn(0x7fb7ec0fc7a0 msgr2=0x7fb7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.708+0000 7fb7f1185640 1 -- 172.21.15.16:0/2067866449 shutdown_connections 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.709+0000 7fb7f1185640 1 -- 172.21.15.16:0/2067866449 wait complete. 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.709+0000 7fb7f1185640 1 Processor -- start 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.709+0000 7fb7f1185640 1 -- start start 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec102d40 0x7fb7ec1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec10c170 0x7fb7ec101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec111f70 con 0x7fb7ec103660 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec111df0 con 0x7fb7ec10c170 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7f1185640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1120f0 con 0x7fb7ec102d40 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52072/0 (socket says 172.21.15.16:52072) 2024-09-12T01:00:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 -- 172.21.15.16:0/4025393109 learned_addr learned my addr 172.21.15.16:0/4025393109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec10c170 0x7fb7ec101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec102d40 0x7fb7ec1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 -- 172.21.15.16:0/4025393109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec102d40 msgr2=0x7fb7ec1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec102d40 0x7fb7ec1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 -- 172.21.15.16:0/4025393109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec10c170 msgr2=0x7fb7ec101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec10c170 0x7fb7ec101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.710+0000 7fb7ea59c640 1 -- 172.21.15.16:0/4025393109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1a6890 con 0x7fb7ec103660 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0006010 tx=0x7fb7e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7d3fff640 1 -- 172.21.15.16:0/4025393109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e003f030 con 0x7fb7ec103660 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7d3fff640 1 -- 172.21.15.16:0/4025393109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e0004440 con 0x7fb7ec103660 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1a6b20 con 0x7fb7ec103660 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7d3fff640 1 -- 172.21.15.16:0/4025393109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0035070 con 0x7fb7ec103660 2024-09-12T01:00:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.711+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a7050 con 0x7fb7ec103660 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.713+0000 7fb7d3fff640 1 -- 172.21.15.16:0/4025393109 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb7e0042030 con 0x7fb7ec103660 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.713+0000 7fb7f355e740 1 Processor -- start 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.713+0000 7fb7f355e740 1 -- start start 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f244a0 0x556ce6f24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f24dc0 0x556ce6f2f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ce6e8a2e0 con 0x556ce6f2f680 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ce6e31ce0 con 0x556ce6f24dc0 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7f355e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ce6e862a0 con 0x556ce6f244a0 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7d3fff640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 0x7fb7d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52082/0 (socket says 172.21.15.16:52082) 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 -- 172.21.15.16:0/963599598 learned_addr learned my addr 172.21.15.16:0/963599598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 0x7fb7d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f24dc0 0x556ce6f2f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f244a0 0x556ce6f24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7d3fff640 1 -- 172.21.15.16:0/4025393109 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7e00ef5e0 con 0x7fb7ec103660 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 -- 172.21.15.16:0/963599598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f244a0 msgr2=0x556ce6f24880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f244a0 0x556ce6f24880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 -- 172.21.15.16:0/963599598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f24dc0 msgr2=0x556ce6f2f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f24dc0 0x556ce6f2f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7ea59c640 1 -- 172.21.15.16:0/963599598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce6e16440 con 0x556ce6f2f680 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.714+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 0x7fb7d40a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4005fd0 tx=0x7fb7e4009e70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00bbac0 tx=0x7fb7e003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d0792a52463a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7d17fa640 1 -- 172.21.15.16:0/963599598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0049070 con 0x556ce6f2f680 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7d17fa640 1 -- 172.21.15.16:0/963599598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e00c3260 con 0x556ce6f2f680 2024-09-12T01:00:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7d17fa640 1 -- 172.21.15.16:0/963599598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00bbb30 con 0x556ce6f2f680 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 -- 172.21.15.16:0/963599598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 msgr2=0x556ce6f31a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00bbac0 tx=0x7fb7e003b460 comp rx=0 tx=0).stop 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 -- 172.21.15.16:0/963599598 shutdown_connections 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f31a70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f24dc0 0x556ce6f2f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 --2- 172.21.15.16:0/963599598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f244a0 0x556ce6f24880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.715+0000 7fb7f355e740 1 -- 172.21.15.16:0/963599598 >> 172.21.15.16:0/963599598 conn(0x556ce6f20a30 msgr2=0x556ce6f20e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- 172.21.15.16:0/963599598 shutdown_connections 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- 172.21.15.16:0/963599598 wait complete. 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- start start 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 Processor -- start 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ce6f32010 con 0x556ce6f244a0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ce6e45050 con 0x556ce6f2f680 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ce6f32190 con 0x556ce6f24dc0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7f355e740 10 client.0 _ll_register_callbacks cb 0x556ce6e85160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52096/0 (socket says 172.21.15.16:52096) 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7ead9d640 1 -- 172.21.15.16:0/3720305724 learned_addr learned my addr 172.21.15.16:0/3720305724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.716+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 -- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 msgr2=0x556ce6f2c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 -- 172.21.15.16:0/3720305724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 msgr2=0x556ce6f2c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f2c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 -- 172.21.15.16:0/3720305724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce6bfb3c0 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f2c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc010a50 tx=0x7fb7dc010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1a6177d58405ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc01e060 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7dc00fb50 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ce6e97850 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7dc00fd50 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7f355e740 10 client.16461 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.717+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ce6f3d970 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556ce6e97850 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7dc019550 con 0x556ce6f244a0 2024-09-12T01:00:10.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 10 client.16461 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 20 client.16461 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6f3ee30 0x556ce6f41280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 4 client.16461 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ce6e91d60 con 0x556ce6f3ee30 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6f3ee30 0x556ce6f41280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6f3ee30 0x556ce6f41280 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e523d4df124d62a3 server_cookie=2a9b2f753a2545d2 in_seq=0 out_seq=0 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.718+0000 7fb7ceff5640 10 client.16461 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7ceff5640 1 -- 172.21.15.16:0/3720305724 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ce6e91d60 con 0x556ce6f3ee30 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7ceff5640 10 client.16461 handle_command_reply: tid=0 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 client.16461 shutdown 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 msgr2=0x556ce6f2efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc010a50 tx=0x7fb7dc010f10 comp rx=0 tx=0).stop 2024-09-12T01:00:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 shutdown_connections 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6f3ee30 0x556ce6f41280 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f2c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7ceff5640 0 client.16461 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 --2- 172.21.15.16:0/3720305724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f2efd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.719+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 >> 172.21.15.16:0/3720305724 conn(0x556ce6f20a30 msgr2=0x556ce6f26710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.720+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 shutdown_connections 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.720+0000 7fb7f355e740 1 -- 172.21.15.16:0/3720305724 wait complete. 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.720+0000 7fb7f355e740 20 client.16461 trim_cache size 0 max 16384 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 Processor -- start 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 -- start start 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f244a0 0x556ce6f3b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1af9c0 con 0x556ce6f2f680 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1b1500 con 0x556ce6f244a0 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7f355e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1b0090 con 0x556ce6f24dc0 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 -- 172.21.15.16:0/1960580693 learned_addr learned my addr 172.21.15.16:0/1960580693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f244a0 0x556ce6f3b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 -- 172.21.15.16:0/1960580693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 msgr2=0x556ce6f2ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 -- 172.21.15.16:0/1960580693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f244a0 msgr2=0x556ce6f3b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.737+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f244a0 0x556ce6f3b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7eb59e640 1 -- 172.21.15.16:0/1960580693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce6f2b230 con 0x556ce6f2f680 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400ff30 tx=0x7fb7e4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48eebd4f40184ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7d0ff9640 1 -- 172.21.15.16:0/1960580693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40044b0 con 0x556ce6f2f680 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7d0ff9640 1 -- 172.21.15.16:0/1960580693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4004650 con 0x556ce6f2f680 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.738+0000 7fb7d0ff9640 1 -- 172.21.15.16:0/1960580693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4020da0 con 0x556ce6f2f680 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- 172.21.15.16:0/1960580693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 msgr2=0x556ce6f46b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400ff30 tx=0x7fb7e4020000 comp rx=0 tx=0).stop 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- 172.21.15.16:0/1960580693 shutdown_connections 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f2f680 0x556ce6f46b50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f2ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- 172.21.15.16:0/1960580693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f244a0 0x556ce6f3b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- 172.21.15.16:0/1960580693 >> 172.21.15.16:0/1960580693 conn(0x556ce6f20a30 msgr2=0x556ce6f278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- 172.21.15.16:0/1960580693 shutdown_connections 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- 172.21.15.16:0/1960580693 wait complete. 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- start start 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 Processor -- start 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1b0e30 con 0x556ce6f244a0 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1b0760 con 0x556ce6f2f680 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1ad9a0 con 0x556ce6f24dc0 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.739+0000 7fb7f355e740 10 client.0 _ll_register_callbacks cb 0x7fb7dc00a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2024-09-12T01:00:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 -- 172.21.15.16:0/2006585689 learned_addr learned my addr 172.21.15.16:0/2006585689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7eb59e640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 -- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 msgr2=0x556ce6f32b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f32b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 -- 172.21.15.16:0/2006585689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 msgr2=0x556ce6f33090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f33090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 -- 172.21.15.16:0/2006585689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce6f2cb50 con 0x556ce6f244a0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ead9d640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0198f0 tx=0x7fb7dc019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8881cb11e0a3c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc031060 con 0x556ce6f244a0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7dc00fb50 con 0x556ce6f244a0 2024-09-12T01:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e4004650 con 0x556ce6f244a0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7dc01e020 con 0x556ce6f244a0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7f355e740 10 client.16467 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.740+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556ce6fa9750 con 0x556ce6f244a0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7e4004650 con 0x556ce6f244a0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7dc0048b0 con 0x556ce6f244a0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 10 client.16467 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 20 client.16467 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6fa9e70 0x556ce6fac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 4 client.16467 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ce6e0e210 con 0x556ce6fa9e70 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.741+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6fa9e70 0x556ce6fac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7ea59c640 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6fa9e70 0x556ce6fac260 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d00365b5e0f425 server_cookie=6efa0a020767a4a8 in_seq=0 out_seq=0 2024-09-12T01:00:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7ceff5640 10 client.16467 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7ceff5640 1 -- 172.21.15.16:0/2006585689 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556ce6e0e210 con 0x556ce6fa9e70 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7ceff5640 10 client.16467 handle_command_reply: tid=0 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 client.16467 shutdown 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 msgr2=0x556ce6f377d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0198f0 tx=0x7fb7dc019610 comp rx=0 tx=0).stop 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 shutdown_connections 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556ce6fa9e70 0x556ce6fac260 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ce6f2f680 0x556ce6f33090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ce6f24dc0 0x556ce6f32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 --2- 172.21.15.16:0/2006585689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ce6f244a0 0x556ce6f377d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 >> 172.21.15.16:0/2006585689 conn(0x556ce6f20a30 msgr2=0x556ce6f1f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7ceff5640 0 client.16467 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 shutdown_connections 2024-09-12T01:00:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 1 -- 172.21.15.16:0/2006585689 wait complete. 2024-09-12T01:00:10.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.742+0000 7fb7f355e740 20 client.16467 trim_cache size 0 max 16384 2024-09-12T01:00:10.758 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:10.758 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:00:10.758 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:00:10.758 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 msgr2=0x7fb7d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 0x7fb7d40a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4005fd0 tx=0x7fb7e4009e70 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 msgr2=0x7fb7ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0006010 tx=0x7fb7e0004040 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 shutdown_connections 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7d40a2d70 0x7fb7d40a5230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7ec10c170 0x7fb7ec101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ec103660 0x7fb7ec100ee0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 --2- 172.21.15.16:0/4025393109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7ec102d40 0x7fb7ec1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:10.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 >> 172.21.15.16:0/4025393109 conn(0x7fb7ec0fc7a0 msgr2=0x7fb7ec0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:10.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 shutdown_connections 2024-09-12T01:00:10.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:10.743+0000 7fb7f1185640 1 -- 172.21.15.16:0/4025393109 wait complete. 2024-09-12T01:00:10.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:10 smithi099 ceph-mon[35654]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-12T01:00:10.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:10 smithi159 ceph-mon[35566]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 52 KiB/s wr, 33 op/s 2024-09-12T01:00:11.546 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:00:11.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:12.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 -- 172.21.15.16:0/2325025729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 msgr2=0x7f961c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 --2- 172.21.15.16:0/2325025729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c1029b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f960402a480 comp rx=0 tx=0).stop 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 -- 172.21.15.16:0/2325025729 shutdown_connections 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 --2- 172.21.15.16:0/2325025729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 0x7f961c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 --2- 172.21.15.16:0/2325025729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c1029b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 --2- 172.21.15.16:0/2325025729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 -- 172.21.15.16:0/2325025729 >> 172.21.15.16:0/2325025729 conn(0x7f961c0fc7d0 msgr2=0x7f961c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 -- 172.21.15.16:0/2325025729 shutdown_connections 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.301+0000 7f962232e640 1 -- 172.21.15.16:0/2325025729 wait complete. 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 Processor -- start 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 -- start start 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:12.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 0x7f961c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f961c110f80 con 0x7f961c102570 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f961c110e00 con 0x7f961c100e20 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f962232e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f961c111100 con 0x7f961c102ef0 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52136/0 (socket says 172.21.15.16:52136) 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 -- 172.21.15.16:0/3209011661 learned_addr learned my addr 172.21.15.16:0/3209011661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f96208cb640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 0x7f961c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f961bfff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 -- 172.21.15.16:0/3209011661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 msgr2=0x7f961c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:12.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 0x7f961c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 -- 172.21.15.16:0/3209011661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 msgr2=0x7f961c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f9613fff640 1 -- 172.21.15.16:0/3209011661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1a6940 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.302+0000 7f961bfff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f9613fff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9604006010 tx=0x7f9604003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960403f070 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604031c30 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1a6bd0 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604035030 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.303+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1a7100 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.304+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c101200 con 0x7f961c102570 2024-09-12T01:00:12.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.305+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9604042070 con 0x7f961c102570 2024-09-12T01:00:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.306+0000 7f9619ffb640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 0x7f96080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.306+0000 7f961bfff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 0x7f96080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.306+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f96040f00b0 con 0x7f961c102570 2024-09-12T01:00:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.306+0000 7f961bfff640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 0x7f96080a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f960c009b40 tx=0x7f960c005e50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:12.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.309+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9604031de0 con 0x7f961c102570 2024-09-12T01:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.521+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c0fe1b0 con 0x7f961c102570 2024-09-12T01:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.522+0000 7f9619ffb640 1 -- 172.21.15.16:0/3209011661 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f961c0fe1b0 con 0x7f961c102570 2024-09-12T01:00:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:12.522 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 msgr2=0x7f96080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 0x7f96080a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f960c009b40 tx=0x7f960c005e50 comp rx=0 tx=0).stop 2024-09-12T01:00:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 msgr2=0x7f961c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:12.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9604006010 tx=0x7f9604003c10 comp rx=0 tx=0).stop 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 shutdown_connections 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96080a2d70 0x7f96080a5230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f961c102ef0 0x7f961c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c102570 0x7f961c0fd1c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 --2- 172.21.15.16:0/3209011661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f961c100e20 0x7f961c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 >> 172.21.15.16:0/3209011661 conn(0x7f961c0fc7d0 msgr2=0x7f961c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 shutdown_connections 2024-09-12T01:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:12.525+0000 7f962232e640 1 -- 172.21.15.16:0/3209011661 wait complete. 2024-09-12T01:00:12.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:12 smithi099 ceph-mon[35654]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:12.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:12 smithi159 ceph-mon[35566]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:12 smithi016 ceph-mon[30972]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:13.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:00:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:13.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:13 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3209011661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:13 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3209011661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:13.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:13 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3209011661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- 172.21.15.16:0/4217813414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0398102db0 msgr2=0x7f0398103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 --2- 172.21.15.16:0/4217813414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0398102db0 0x7f0398103190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0388009930 tx=0x7f038802a420 comp rx=0 tx=0).stop 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- 172.21.15.16:0/4217813414 shutdown_connections 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 --2- 172.21.15.16:0/4217813414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03981082e0 0x7f039810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 --2- 172.21.15.16:0/4217813414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 0x7f0398107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 --2- 172.21.15.16:0/4217813414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0398102db0 0x7f0398103190 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- 172.21.15.16:0/4217813414 >> 172.21.15.16:0/4217813414 conn(0x7f03980fc7d0 msgr2=0x7f03980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- 172.21.15.16:0/4217813414 shutdown_connections 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- 172.21.15.16:0/4217813414 wait complete. 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 Processor -- start 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.145+0000 7f039d6c3640 1 -- start start 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0398102db0 0x7f0398198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 0x7f0398198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03981120b0 con 0x7f03981082e0 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0398111f30 con 0x7f0398102db0 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f039d6c3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112230 con 0x7f0398103760 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0398102db0 0x7f0398198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 -- 172.21.15.16:0/2957486785 learned_addr learned my addr 172.21.15.16:0/2957486785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03967fc640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 0x7f0398198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 -- 172.21.15.16:0/2957486785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 msgr2=0x7f0398198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 0x7f0398198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 -- 172.21.15.16:0/2957486785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0398102db0 msgr2=0x7f0398198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0398102db0 0x7f0398198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 -- 172.21.15.16:0/2957486785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039819d520 con 0x7f03981082e0 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.146+0000 7f03977fe640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f039000cce0 tx=0x7f039000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.147+0000 7f037ffff640 1 -- 172.21.15.16:0/2957486785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039000f450 con 0x7f03981082e0 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.147+0000 7f037ffff640 1 -- 172.21.15.16:0/2957486785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039000fad0 con 0x7f03981082e0 2024-09-12T01:00:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.147+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039819d810 con 0x7f03981082e0 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.147+0000 7f037ffff640 1 -- 172.21.15.16:0/2957486785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390018c10 con 0x7f03981082e0 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.147+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a50f0 con 0x7f03981082e0 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.149+0000 7f037ffff640 1 -- 172.21.15.16:0/2957486785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f039000f5f0 con 0x7f03981082e0 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.149+0000 7f039fa9c740 1 Processor -- start 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.149+0000 7f039fa9c740 1 -- start start 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559eb76122f0 con 0x559eb76ac5d0 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eb75b9cf0 con 0x559eb76b7370 2024-09-12T01:00:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f039fa9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559eb760e2b0 con 0x559eb76adb40 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f037ffff640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52150/0 (socket says 172.21.15.16:52150) 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 -- 172.21.15.16:0/4069561234 learned_addr learned my addr 172.21.15.16:0/4069561234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f03977fe640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f03967fc640 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f037ffff640 1 -- 172.21.15.16:0/2957486785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f03900cfab0 con 0x7f03981082e0 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f03967fc640 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 -- 172.21.15.16:0/4069561234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 msgr2=0x559eb76ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 -- 172.21.15.16:0/4069561234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 msgr2=0x559eb76b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.150+0000 7f0396ffd640 1 -- 172.21.15.16:0/4069561234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb759e3a0 con 0x559eb76ac5d0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f03977fe640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 0x7f03800a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f039009bf30 tx=0x7f039006f010 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f0396ffd640 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0388009930 tx=0x7f0388004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf24c3d090d75eac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f037d7fa640 1 -- 172.21.15.16:0/4069561234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388035030 con 0x559eb76ac5d0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f037d7fa640 1 -- 172.21.15.16:0/4069561234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388040070 con 0x559eb76ac5d0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f037d7fa640 1 -- 172.21.15.16:0/4069561234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388031ba0 con 0x559eb76ac5d0 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f039fa9c740 1 -- 172.21.15.16:0/4069561234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 msgr2=0x559eb76aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.151+0000 7f039fa9c740 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0388009930 tx=0x7f0388004200 comp rx=0 tx=0).stop 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- 172.21.15.16:0/4069561234 shutdown_connections 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76aca10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- 172.21.15.16:0/4069561234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- 172.21.15.16:0/4069561234 >> 172.21.15.16:0/4069561234 conn(0x559eb7675310 msgr2=0x559eb76a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- 172.21.15.16:0/4069561234 shutdown_connections 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- 172.21.15.16:0/4069561234 wait complete. 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- start start 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 Processor -- start 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76ac5d0 0x559eb76b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 0x559eb76b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559eb76ba010 con 0x559eb76adb40 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eb75cd060 con 0x559eb76ac5d0 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559eb76ba190 con 0x559eb76b7370 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f039fa9c740 10 client.0 _ll_register_callbacks cb 0x559eb73208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f0396ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52166/0 (socket says 172.21.15.16:52166) 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.152+0000 7f0396ffd640 1 -- 172.21.15.16:0/2977627223 learned_addr learned my addr 172.21.15.16:0/2977627223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f03967fc640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76ac5d0 0x559eb76b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f03967fc640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 0x559eb76b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 -- 172.21.15.16:0/2977627223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 msgr2=0x559eb76b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 0x559eb76b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 -- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76ac5d0 msgr2=0x559eb76b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76ac5d0 0x559eb76b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 -- 172.21.15.16:0/2977627223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb73833c0 con 0x559eb76adb40 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f03967fc640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 0x559eb76b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:14.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f0396ffd640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0388006010 tx=0x7f0388031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7054a7a9ca46309c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f037aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388052060 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f03880037f0 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0388040070 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0388035070 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f039fa9c740 10 client.16485 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.153+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eb76c5ae0 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0388040070 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0388057620 con 0x559eb76adb40 2024-09-12T01:00:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 10 client.16485 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 10 client.16485 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 20 client.16485 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb76c6e30 0x559eb76c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 4 client.16485 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb761f860 con 0x559eb76c6e30 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.154+0000 7f03967fc640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb76c6e30 0x559eb76c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.155+0000 7f037aff5640 10 client.16485 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.155+0000 7f03967fc640 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb76c6e30 0x559eb76c9220 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66b327f35d3c96d4 server_cookie=ce71875cc518da7c in_seq=0 out_seq=0 2024-09-12T01:00:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.155+0000 7f037aff5640 1 -- 172.21.15.16:0/2977627223 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eb761f860 con 0x559eb76c6e30 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.155+0000 7f037aff5640 10 client.16485 handle_command_reply: tid=0 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 client.16485 shutdown 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 msgr2=0x559eb76b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0388006010 tx=0x7f0388031b50 comp rx=0 tx=0).stop 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 shutdown_connections 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb76c6e30 0x559eb76c9220 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76b7370 0x559eb76b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76adb40 0x559eb76b4030 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 --2- 172.21.15.16:0/2977627223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76ac5d0 0x559eb76b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f037aff5640 0 client.16485 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 >> 172.21.15.16:0/2977627223 conn(0x559eb7675310 msgr2=0x559eb76ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 shutdown_connections 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 1 -- 172.21.15.16:0/2977627223 wait complete. 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.156+0000 7f039fa9c740 20 client.16485 trim_cache size 0 max 16384 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 Processor -- start 2024-09-12T01:00:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 -- start start 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76ac5d0 0x559eb76b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 0x559eb76af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ad780 con 0x559eb76b7370 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03981af2c0 con 0x559eb76adb40 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f039fa9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ade50 con 0x559eb76ac5d0 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52172/0 (socket says 172.21.15.16:52172) 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 -- 172.21.15.16:0/1359663261 learned_addr learned my addr 172.21.15.16:0/1359663261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76ac5d0 0x559eb76b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f0396ffd640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 0x559eb76af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 -- 172.21.15.16:0/1359663261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76ac5d0 msgr2=0x559eb76b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76ac5d0 0x559eb76b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 -- 172.21.15.16:0/1359663261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 msgr2=0x559eb76af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 0x559eb76af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 -- 172.21.15.16:0/1359663261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb76b4c20 con 0x559eb76b7370 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f0396ffd640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 0x559eb76af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.169+0000 7f03967fc640 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x559eb76b9aa0 tx=0x7f038c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3bf777f8f8a15c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f037cff9640 1 -- 172.21.15.16:0/1359663261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c026070 con 0x559eb76b7370 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f037cff9640 1 -- 172.21.15.16:0/1359663261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038c024070 con 0x559eb76b7370 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f037cff9640 1 -- 172.21.15.16:0/1359663261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c003850 con 0x559eb76b7370 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 -- 172.21.15.16:0/1359663261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 msgr2=0x559eb76af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x559eb76b9aa0 tx=0x7f038c004730 comp rx=0 tx=0).stop 2024-09-12T01:00:14.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 -- 172.21.15.16:0/1359663261 shutdown_connections 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76b7370 0x559eb76af660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76adb40 0x559eb76af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1359663261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76ac5d0 0x559eb76b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 -- 172.21.15.16:0/1359663261 >> 172.21.15.16:0/1359663261 conn(0x559eb7675310 msgr2=0x559eb76b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 -- 172.21.15.16:0/1359663261 shutdown_connections 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.170+0000 7f039fa9c740 1 -- 172.21.15.16:0/1359663261 wait complete. 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 -- start start 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 Processor -- start 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03981aebf0 con 0x559eb76ac5d0 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ae520 con 0x559eb76b7370 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03981ab760 con 0x559eb76adb40 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f039fa9c740 10 client.0 _ll_register_callbacks cb 0x7f038c00ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52188/0 (socket says 172.21.15.16:52188) 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 -- 172.21.15.16:0/1870506853 learned_addr learned my addr 172.21.15.16:0/1870506853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f0396ffd640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 -- 172.21.15.16:0/1870506853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 msgr2=0x559eb76ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 -- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 msgr2=0x559eb76cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.171+0000 7f03967fc640 1 -- 172.21.15.16:0/1870506853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb76a9960 con 0x559eb76ac5d0 2024-09-12T01:00:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f03967fc640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f038c00eb60 tx=0x7f038c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21bdd0dbe2fda87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c026040 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f038c01f4d0 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c024070 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f038c004450 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 10 client.16491 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eb7731950 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038c024070 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f038c029070 con 0x559eb76ac5d0 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 10 client.16491 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 20 client.16491 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb7731f70 0x559eb7734390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 4 client.16491 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.172+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eb7619d70 con 0x559eb7731f70 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f0396ffd640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb7731f70 0x559eb7734390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f0396ffd640 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb7731f70 0x559eb7734390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a73ce85082976e2 server_cookie=f7e1263e5daad9f7 in_seq=0 out_seq=0 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f037aff5640 10 client.16491 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f037aff5640 1 -- 172.21.15.16:0/1870506853 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eb7619d70 con 0x559eb7731f70 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f037aff5640 10 client.16491 handle_command_reply: tid=0 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 client.16491 shutdown 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 msgr2=0x559eb76ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f038c00eb60 tx=0x7f038c00ef20 comp rx=0 tx=0).stop 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 shutdown_connections 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559eb7731f70 0x559eb7734390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eb76b7370 0x559eb76cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559eb76adb40 0x559eb76ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 --2- 172.21.15.16:0/1870506853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eb76ac5d0 0x559eb76ca140 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 >> 172.21.15.16:0/1870506853 conn(0x559eb7675310 msgr2=0x559eb76b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.173+0000 7f037aff5640 0 client.16491 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:14.199 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 shutdown_connections 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039fa9c740 1 -- 172.21.15.16:0/1870506853 wait complete. 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039fa9c740 20 client.16491 trim_cache size 0 max 16384 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 msgr2=0x7f03800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 0x7f03800a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f039009bf30 tx=0x7f039006f010 comp rx=0 tx=0).stop 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 msgr2=0x7f039819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f039000cce0 tx=0x7f039000a620 comp rx=0 tx=0).stop 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 shutdown_connections 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03800a2cf0 0x7f03800a51b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03981082e0 0x7f039819ce40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0398103760 0x7f0398198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 --2- 172.21.15.16:0/2957486785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0398102db0 0x7f0398198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 >> 172.21.15.16:0/2957486785 conn(0x7f03980fc7d0 msgr2=0x7f0398106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 shutdown_connections 2024-09-12T01:00:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:14.174+0000 7f039d6c3640 1 -- 172.21.15.16:0/2957486785 wait complete. 2024-09-12T01:00:14.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:14 smithi016 ceph-mon[30972]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:14.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:14 smithi099 ceph-mon[35654]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:14.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:14 smithi159 ceph-mon[35566]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 26 op/s 2024-09-12T01:00:15.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:00:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 -- 172.21.15.16:0/2384965340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90106f20 msgr2=0x7f2d9010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 --2- 172.21.15.16:0/2384965340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90106f20 0x7f2d9010eaa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009900 tx=0x7f2d8802a410 comp rx=0 tx=0).stop 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 -- 172.21.15.16:0/2384965340 shutdown_connections 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 --2- 172.21.15.16:0/2384965340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90106f20 0x7f2d9010eaa0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 --2- 172.21.15.16:0/2384965340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d901027a0 0x7f2d901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 --2- 172.21.15.16:0/2384965340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d90101df0 0x7f2d901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 -- 172.21.15.16:0/2384965340 >> 172.21.15.16:0/2384965340 conn(0x7f2d900fd8f0 msgr2=0x7f2d900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 -- 172.21.15.16:0/2384965340 shutdown_connections 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.754+0000 7f2d973d6640 1 -- 172.21.15.16:0/2384965340 wait complete. 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 Processor -- start 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 -- start start 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 0x7f2d90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 0x7f2d90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90111fa0 con 0x7f2d90101df0 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90111e20 con 0x7f2d90106f20 2024-09-12T01:00:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d973d6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90112120 con 0x7f2d901027a0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52194/0 (socket says 172.21.15.16:52194) 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 -- 172.21.15.16:0/1237406830 learned_addr learned my addr 172.21.15.16:0/1237406830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95973640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 0x7f2d90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d94971640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 0x7f2d90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 -- 172.21.15.16:0/1237406830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 msgr2=0x7f2d90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 0x7f2d90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 -- 172.21.15.16:0/1237406830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 msgr2=0x7f2d90074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 0x7f2d90074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95172640 1 -- 172.21.15.16:0/1237406830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a68c0 con 0x7f2d90101df0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d94971640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 0x7f2d90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.755+0000 7f2d95973640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 0x7f2d90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d95172640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ba50 tx=0x7f2d8c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d827fc640 1 -- 172.21.15.16:0/1237406830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c019070 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d827fc640 1 -- 172.21.15.16:0/1237406830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c00cbe0 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a6bb0 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d827fc640 1 -- 172.21.15.16:0/1237406830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0076e0 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.756+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901a70a0 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.758+0000 7f2d827fc640 1 -- 172.21.15.16:0/1237406830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d8c01c070 con 0x7f2d90101df0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.758+0000 7f2d997af740 1 Processor -- start 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.758+0000 7f2d997af740 1 -- start start 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559768ec12f0 con 0x559768f5cb40 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559768e68cf0 con 0x559768f5b5d0 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d997af740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559768ebd2b0 con 0x559768f66370 2024-09-12T01:00:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52202/0 (socket says 172.21.15.16:52202) 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 -- 172.21.15.16:0/3430826779 learned_addr learned my addr 172.21.15.16:0/3430826779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d827fc640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 0x7f2d7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d94971640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 0x7f2d7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95172640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d827fc640 1 -- 172.21.15.16:0/1237406830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2d8c0cea20 con 0x7f2d90101df0 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 -- 172.21.15.16:0/3430826779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 msgr2=0x559768f68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 -- 172.21.15.16:0/3430826779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 msgr2=0x559768f5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f5ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95973640 1 -- 172.21.15.16:0/3430826779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559768e4d3a0 con 0x559768f5cb40 2024-09-12T01:00:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.759+0000 7f2d95172640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.760+0000 7f2d95973640 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2d88031550 tx=0x7f2d880318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab51db0a68ef4df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.760+0000 7f2d7b7fe640 1 -- 172.21.15.16:0/3430826779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88035070 con 0x559768f5cb40 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.760+0000 7f2d95973640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 0x7f2d7c0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005940 tx=0x7f2d88047000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.760+0000 7f2d7b7fe640 1 -- 172.21.15.16:0/3430826779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d88004460 con 0x559768f5cb40 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.760+0000 7f2d7b7fe640 1 -- 172.21.15.16:0/3430826779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004d60 con 0x559768f5cb40 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- 172.21.15.16:0/3430826779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 msgr2=0x559768f5b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2d88031550 tx=0x7f2d880318f0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- 172.21.15.16:0/3430826779 shutdown_connections 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 --2- 172.21.15.16:0/3430826779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f5b090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- 172.21.15.16:0/3430826779 >> 172.21.15.16:0/3430826779 conn(0x559768f24310 msgr2=0x559768f57c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- 172.21.15.16:0/3430826779 shutdown_connections 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- 172.21.15.16:0/3430826779 wait complete. 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 -- start start 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.761+0000 7f2d997af740 1 Processor -- start 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5b5d0 0x559768f65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f66370 0x559768f63800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559768f68fb0 con 0x559768f5cb40 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559768e7c060 con 0x559768f66370 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559768f69130 con 0x559768f5b5d0 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d997af740 10 client.0 _ll_register_callbacks cb 0x559768bcf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52208/0 (socket says 172.21.15.16:52208) 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 -- 172.21.15.16:0/3167167061 learned_addr learned my addr 172.21.15.16:0/3167167061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d95172640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5b5d0 0x559768f65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d95973640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f66370 0x559768f63800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 -- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5b5d0 msgr2=0x559768f65dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5b5d0 0x559768f65dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 -- 172.21.15.16:0/3167167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f66370 msgr2=0x559768f63800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f66370 0x559768f63800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 -- 172.21.15.16:0/3167167061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559768c323c0 con 0x559768f5cb40 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d94971640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400bf70 tx=0x7f2d84012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7177e9df29b508 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.762+0000 7f2d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840035f0 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2d84003790 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d88004460 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d84003990 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 10 client.16503 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559768f74b50 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d88004460 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d84019070 con 0x559768f5cb40 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 10 client.16503 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 10 client.16503 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 20 client.16503 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768f760b0 0x559768f784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 4 client.16503 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.763+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559768ece860 con 0x559768f760b0 2024-09-12T01:00:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.764+0000 7f2d95172640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768f760b0 0x559768f784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.764+0000 7f2d95172640 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768f760b0 0x559768f784d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a68ce0ba3bcc4bf server_cookie=a5413ca826ea448 in_seq=0 out_seq=0 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.764+0000 7f2d78ff9640 10 client.16503 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d78ff9640 1 -- 172.21.15.16:0/3167167061 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559768ece860 con 0x559768f760b0 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d78ff9640 10 client.16503 handle_command_reply: tid=0 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 client.16503 shutdown 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 msgr2=0x559768f63070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400bf70 tx=0x7f2d84012820 comp rx=0 tx=0).stop 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 shutdown_connections 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f66370 0x559768f63800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768f760b0 0x559768f784d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5cb40 0x559768f63070 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 --2- 172.21.15.16:0/3167167061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5b5d0 0x559768f65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 >> 172.21.15.16:0/3167167061 conn(0x559768f24310 msgr2=0x559768f5d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d78ff9640 0 client.16503 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 shutdown_connections 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 1 -- 172.21.15.16:0/3167167061 wait complete. 2024-09-12T01:00:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.765+0000 7f2d997af740 20 client.16503 trim_cache size 0 max 16384 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.778+0000 7f2d997af740 1 Processor -- start 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 -- start start 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5cb40 0x559768f7dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f7e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901af9b0 con 0x559768f5b5d0 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901b14f0 con 0x559768f5cb40 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d997af740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901b0080 con 0x559768f66370 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52214/0 (socket says 172.21.15.16:52214) 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 -- 172.21.15.16:0/2803723492 learned_addr learned my addr 172.21.15.16:0/2803723492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95973640 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f7e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d94971640 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5cb40 0x559768f7dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 -- 172.21.15.16:0/2803723492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 msgr2=0x559768f7e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f7e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 -- 172.21.15.16:0/2803723492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5cb40 msgr2=0x559768f7dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5cb40 0x559768f7dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 -- 172.21.15.16:0/2803723492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559768f68ad0 con 0x559768f5b5d0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.779+0000 7f2d95172640 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c097480 tx=0x7f2d8c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff462b18b286f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d7affd640 1 -- 172.21.15.16:0/2803723492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c09f3d0 con 0x559768f5b5d0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d7affd640 1 -- 172.21.15.16:0/2803723492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c0a3c90 con 0x559768f5b5d0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d7affd640 1 -- 172.21.15.16:0/2803723492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00f160 con 0x559768f5b5d0 2024-09-12T01:00:15.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 -- 172.21.15.16:0/2803723492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 msgr2=0x559768f63040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c097480 tx=0x7f2d8c0bb040 comp rx=0 tx=0).stop 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 -- 172.21.15.16:0/2803723492 shutdown_connections 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f66370 0x559768f7e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5cb40 0x559768f7dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.780+0000 7f2d997af740 1 --2- 172.21.15.16:0/2803723492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f5b5d0 0x559768f63040 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- 172.21.15.16:0/2803723492 >> 172.21.15.16:0/2803723492 conn(0x559768f24310 msgr2=0x559768f62930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- 172.21.15.16:0/2803723492 shutdown_connections 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- 172.21.15.16:0/2803723492 wait complete. 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- start start 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 Processor -- start 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f6e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5cb40 0x559768f6ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901b0e20 con 0x559768f66370 2024-09-12T01:00:15.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901b0750 con 0x559768f5b5d0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901ad990 con 0x559768f5cb40 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.781+0000 7f2d997af740 10 client.0 _ll_register_callbacks cb 0x7f2d8400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52216/0 (socket says 172.21.15.16:52216) 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95973640 1 -- 172.21.15.16:0/4262350751 learned_addr learned my addr 172.21.15.16:0/4262350751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f6e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d94971640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5cb40 0x559768f6ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 -- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5cb40 msgr2=0x559768f6ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5cb40 0x559768f6ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 -- 172.21.15.16:0/4262350751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 msgr2=0x559768f69a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 -- 172.21.15.16:0/4262350751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559768f6a280 con 0x559768f5b5d0 2024-09-12T01:00:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95973640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d95172640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f6e6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ba20 tx=0x7f2d8c0ced20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7547d751a9f4b29a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00cd90 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c0a3c90 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.782+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f2d8c097690 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d8c0cc3c0 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 10 client.25850 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559768fe0940 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8c0a3c90 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d8c095b90 con 0x559768f5b5d0 2024-09-12T01:00:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 10 client.25850 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 10 client.25850 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 20 client.25850 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768fe0fb0 0x559768fe33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 4 client.25850 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559768ec8d70 con 0x559768fe0fb0 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.783+0000 7f2d94971640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768fe0fb0 0x559768fe33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d78ff9640 10 client.25850 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d94971640 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768fe0fb0 0x559768fe33a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f613ea0cd28a1d4f server_cookie=c7c0c119608c004 in_seq=0 out_seq=0 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d78ff9640 1 -- 172.21.15.16:0/4262350751 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559768ec8d70 con 0x559768fe0fb0 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d78ff9640 10 client.25850 handle_command_reply: tid=0 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 client.25850 shutdown 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 msgr2=0x559768f6e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f6e6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ba20 tx=0x7f2d8c0ced20 comp rx=0 tx=0).stop 2024-09-12T01:00:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 shutdown_connections 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559768fe0fb0 0x559768fe33a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559768f66370 0x559768f69a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559768f5cb40 0x559768f6ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 --2- 172.21.15.16:0/4262350751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559768f5b5d0 0x559768f6e6c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d78ff9640 0 client.25850 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 >> 172.21.15.16:0/4262350751 conn(0x559768f24310 msgr2=0x559768f61940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 shutdown_connections 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.784+0000 7f2d997af740 1 -- 172.21.15.16:0/4262350751 wait complete. 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d997af740 20 client.25850 trim_cache size 0 max 16384 2024-09-12T01:00:15.793 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 msgr2=0x7f2d7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 0x7f2d7c0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005940 tx=0x7f2d88047000 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 msgr2=0x7f2d900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00ba50 tx=0x7f2d8c00bf10 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 shutdown_connections 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d7c0a2cf0 0x7f2d7c0a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d90106f20 0x7f2d90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d901027a0 0x7f2d90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 --2- 172.21.15.16:0/1237406830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d90101df0 0x7f2d900744f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 >> 172.21.15.16:0/1237406830 conn(0x7f2d900fd8f0 msgr2=0x7f2d900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 shutdown_connections 2024-09-12T01:00:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:15.785+0000 7f2d973d6640 1 -- 172.21.15.16:0/1237406830 wait complete. 2024-09-12T01:00:16.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:00:16.773 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:16 smithi099 ceph-mon[35654]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-09-12T01:00:16.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:16 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:00:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:16 smithi159 ceph-mon[35566]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-09-12T01:00:16.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:16 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:00:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:16 smithi016 ceph-mon[30972]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-09-12T01:00:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:16 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:00:16.960 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:17.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 -- 172.21.15.16:0/1990143818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48101de0 msgr2=0x7f0a481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 --2- 172.21.15.16:0/1990143818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48101de0 0x7f0a481021c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009900 tx=0x7f0a4402a410 comp rx=0 tx=0).stop 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 -- 172.21.15.16:0/1990143818 shutdown_connections 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 --2- 172.21.15.16:0/1990143818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48106f10 0x7f0a4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 --2- 172.21.15.16:0/1990143818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48102790 0x7f0a481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 --2- 172.21.15.16:0/1990143818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48101de0 0x7f0a481021c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 -- 172.21.15.16:0/1990143818 >> 172.21.15.16:0/1990143818 conn(0x7f0a480fd8c0 msgr2=0x7f0a480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 -- 172.21.15.16:0/1990143818 shutdown_connections 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.450+0000 7f0a50445640 1 -- 172.21.15.16:0/1990143818 wait complete. 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 Processor -- start 2024-09-12T01:00:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 -- start start 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48101de0 0x7f0a4819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48102790 0x7f0a4819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48075a70 con 0x7f0a48106f10 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a480758f0 con 0x7f0a48102790 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a50445640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48075bf0 con 0x7f0a48101de0 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a4e9e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a4e9e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52234/0 (socket says 172.21.15.16:52234) 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a4e9e2640 1 -- 172.21.15.16:0/3178842170 learned_addr learned my addr 172.21.15.16:0/3178842170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.451+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48101de0 0x7f0a4819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48102790 0x7f0a4819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 -- 172.21.15.16:0/3178842170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48101de0 msgr2=0x7f0a4819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48101de0 0x7f0a4819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 -- 172.21.15.16:0/3178842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48102790 msgr2=0x7f0a4819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48102790 0x7f0a4819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 -- 172.21.15.16:0/3178842170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4819f690 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a4e9e2640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000eaa0 tx=0x7f0a4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a3f7fe640 1 -- 172.21.15.16:0/3178842170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000cdb0 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4819f980 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a3f7fe640 1 -- 172.21.15.16:0/3178842170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a40005020 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a3f7fe640 1 -- 172.21.15.16:0/3178842170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a400116b0 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.452+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a481a7260 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a3f7fe640 1 -- 172.21.15.16:0/3178842170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0a40005560 con 0x7f0a48106f10 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 Processor -- start 2024-09-12T01:00:17.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 -- start start 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fd8370 0x564e79fda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e79f332f0 con 0x564e79fcd5d0 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e79edacf0 con 0x564e79fd8370 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a5281e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e79f2f2b0 con 0x564e79fceb40 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a4e1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a3f7fe640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 0x7f0a340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a4e1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52240/0 (socket says 172.21.15.16:52240) 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.455+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/3121011833 learned_addr learned my addr 172.21.15.16:0/3121011833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e9e2640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 0x7f0a340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fd8370 0x564e79fda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/3121011833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 msgr2=0x564e79fcd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fcd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/3121011833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fd8370 msgr2=0x564e79fda760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fd8370 0x564e79fda760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/3121011833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e79ebf3a0 con 0x564e79fcd5d0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fcd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a3f7fe640 1 -- 172.21.15.16:0/3178842170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0a400cf5c0 con 0x7f0a48106f10 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009900 tx=0x7f0a44031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce514f6bbf285bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a4e9e2640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 0x7f0a340a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a4009b860 tx=0x7f0a40080f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a3cdea640 1 -- 172.21.15.16:0/3121011833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44040070 con 0x564e79fcd5d0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a3cdea640 1 -- 172.21.15.16:0/3121011833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44005020 con 0x564e79fcd5d0 2024-09-12T01:00:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.456+0000 7f0a3cdea640 1 -- 172.21.15.16:0/3121011833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44035040 con 0x564e79fcd5d0 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 -- 172.21.15.16:0/3121011833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 msgr2=0x564e79fcda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009900 tx=0x7f0a44031630 comp rx=0 tx=0).stop 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 -- 172.21.15.16:0/3121011833 shutdown_connections 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fd8370 0x564e79fda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fcda10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 --2- 172.21.15.16:0/3121011833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fcd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 -- 172.21.15.16:0/3121011833 >> 172.21.15.16:0/3121011833 conn(0x564e79f96310 msgr2=0x564e79fc9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 -- 172.21.15.16:0/3121011833 shutdown_connections 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.457+0000 7f0a5281e740 1 -- 172.21.15.16:0/3121011833 wait complete. 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 -- start start 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 Processor -- start 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fd7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fd5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e79fdb010 con 0x564e79fd8370 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e79eee060 con 0x564e79fcd5d0 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e79fdb190 con 0x564e79fceb40 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a5281e740 10 client.0 _ll_register_callbacks cb 0x564e79c418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52246/0 (socket says 172.21.15.16:52246) 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2822644784 learned_addr learned my addr 172.21.15.16:0/2822644784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fd5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fd7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2822644784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 msgr2=0x564e79fd5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fd5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 msgr2=0x564e79fd7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fd7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.458+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2822644784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e79ca43c0 con 0x564e79fd8370 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0a38012e70 tx=0x7f0a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6032d8f1d3b7f53a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800dc50 con 0x564e79fd8370 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0a3800ddf0 con 0x564e79fd8370 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a44005020 con 0x564e79fd8370 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a38010640 con 0x564e79fd8370 2024-09-12T01:00:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 10 client.16521 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e79fe6b40 con 0x564e79fd8370 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44005020 con 0x564e79fd8370 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a380288c0 con 0x564e79fd8370 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 10 client.16521 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 10 client.16521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 20 client.16521 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e79fe7f50 0x564e79fea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 4 client.16521 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.459+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e79f40860 con 0x564e79fe7f50 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.460+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e79fe7f50 0x564e79fea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.460+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e79fe7f50 0x564e79fea340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4fbb2b33d369b49 server_cookie=3321d36016f1f17f in_seq=0 out_seq=0 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.460+0000 7f0a31ffb640 10 client.16521 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:17.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a31ffb640 1 -- 172.21.15.16:0/2822644784 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e79f40860 con 0x564e79fe7f50 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a31ffb640 10 client.16521 handle_command_reply: tid=0 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 client.16521 shutdown 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 msgr2=0x564e79fd5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0a38012e70 tx=0x7f0a380046c0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 shutdown_connections 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e79fe7f50 0x564e79fea340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fd8370 0x564e79fd5640 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fceb40 0x564e79fd5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2822644784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fd7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a31ffb640 0 client.16521 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 >> 172.21.15.16:0/2822644784 conn(0x564e79f96310 msgr2=0x564e79fcf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 shutdown_connections 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 1 -- 172.21.15.16:0/2822644784 wait complete. 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.461+0000 7f0a5281e740 20 client.16521 trim_cache size 0 max 16384 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 Processor -- start 2024-09-12T01:00:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 -- start start 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 0x564e79fd6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481afa30 con 0x564e79fcd5d0 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b1570 con 0x564e79fceb40 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a5281e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b0100 con 0x564e79fd8370 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a4d9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a4d9e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52254/0 (socket says 172.21.15.16:52254) 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.474+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2510939673 learned_addr learned my addr 172.21.15.16:0/2510939673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 0x564e79fd6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2510939673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 msgr2=0x564e79fd0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2510939673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 msgr2=0x564e79fd6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 0x564e79fd6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 -- 172.21.15.16:0/2510939673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e79fd4000 con 0x564e79fcd5d0 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 0x564e79fd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0a38012860 tx=0x7f0a38012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a62254b9987a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a33fff640 1 -- 172.21.15.16:0/2510939673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800f420 con 0x564e79fcd5d0 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a33fff640 1 -- 172.21.15.16:0/2510939673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a38005520 con 0x564e79fcd5d0 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.475+0000 7f0a33fff640 1 -- 172.21.15.16:0/2510939673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38010a40 con 0x564e79fcd5d0 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- 172.21.15.16:0/2510939673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 msgr2=0x564e79fd48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0a38012860 tx=0x7f0a38012890 comp rx=0 tx=0).stop 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- 172.21.15.16:0/2510939673 shutdown_connections 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fceb40 0x564e79fd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- 172.21.15.16:0/2510939673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fcd5d0 0x564e79fd48f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- 172.21.15.16:0/2510939673 >> 172.21.15.16:0/2510939673 conn(0x564e79f96310 msgr2=0x564e79fd6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- 172.21.15.16:0/2510939673 shutdown_connections 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- 172.21.15.16:0/2510939673 wait complete. 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- start start 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 Processor -- start 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fe0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b0ea0 con 0x564e79fceb40 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481b07d0 con 0x564e79fcd5d0 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481ada10 con 0x564e79fd8370 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a5281e740 10 client.0 _ll_register_callbacks cb 0x7f0a3800a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.476+0000 7f0a4e1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52270/0 (socket says 172.21.15.16:52270) 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/157632073 learned_addr learned my addr 172.21.15.16:0/157632073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fe0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/157632073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 msgr2=0x564e79fd1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 msgr2=0x564e79fe0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fe0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 -- 172.21.15.16:0/157632073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e79fd24f0 con 0x564e79fceb40 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fe0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a4e1e1640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a440092a0 tx=0x7f0a44031a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff55604b7ef8505e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44040070 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0a44004450 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a38005520 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a4405d050 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a5281e740 10 client.16527 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.477+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e7a052870 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a38005520 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a440035f0 con 0x564e79fceb40 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 10 client.16527 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 10 client.16527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:00:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 20 client.16527 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e7a052bd0 0x564e7a054ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 4 client.16527 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e79f3ad70 con 0x564e7a052bd0 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e7a052bd0 0x564e7a054ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a4d9e0640 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e7a052bd0 0x564e7a054ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17a3dbfbaeefe4bf server_cookie=72ab1727324f6423 in_seq=0 out_seq=0 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.478+0000 7f0a31ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a31ffb640 1 -- 172.21.15.16:0/157632073 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e79f3ad70 con 0x564e7a052bd0 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a31ffb640 10 client.16527 handle_command_reply: tid=0 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 client.16527 shutdown 2024-09-12T01:00:17.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 msgr2=0x564e79fd1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0a440092a0 tx=0x7f0a44031a70 comp rx=0 tx=0).stop 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 shutdown_connections 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564e7a052bd0 0x564e7a054ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e79fd8370 0x564e79fd1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a31ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e79fceb40 0x564e79fd1a40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 --2- 172.21.15.16:0/157632073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e79fcd5d0 0x564e79fe0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 >> 172.21.15.16:0/157632073 conn(0x564e79f96310 msgr2=0x564e79fe32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 shutdown_connections 2024-09-12T01:00:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 1 -- 172.21.15.16:0/157632073 wait complete. 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.479+0000 7f0a5281e740 20 client.16527 trim_cache size 0 max 16384 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 msgr2=0x7f0a340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 0x7f0a340a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a4009b860 tx=0x7f0a40080f90 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 msgr2=0x7f0a4819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000eaa0 tx=0x7f0a4000ef60 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 shutdown_connections 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a340a2cf0 0x7f0a340a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a48106f10 0x7f0a4819efb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a48102790 0x7f0a4819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 --2- 172.21.15.16:0/3178842170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a48101de0 0x7f0a4819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 >> 172.21.15.16:0/3178842170 conn(0x7f0a480fd8c0 msgr2=0x7f0a480fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 shutdown_connections 2024-09-12T01:00:17.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:17.480+0000 7f0a50445640 1 -- 172.21.15.16:0/3178842170 wait complete. 2024-09-12T01:00:17.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:17 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:00:17.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:17 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:00:17.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:17 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:00:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:17 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:00:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:17 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:00:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:17 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:00:17.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:17 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:00:17.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:17 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:00:17.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:17 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:00:18.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:00:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:19.008 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:18 smithi016 ceph-mon[30972]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-12T01:00:19.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- 172.21.15.16:0/2160830146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518102790 msgr2=0x7f15181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2160830146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518102790 0x7f15181067e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f150402a410 comp rx=0 tx=0).stop 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- 172.21.15.16:0/2160830146 shutdown_connections 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2160830146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 0x7f151810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2160830146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518102790 0x7f15181067e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2160830146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518101de0 0x7f15181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- 172.21.15.16:0/2160830146 >> 172.21.15.16:0/2160830146 conn(0x7f15180fd8c0 msgr2=0x7f15180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- 172.21.15.16:0/2160830146 shutdown_connections 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- 172.21.15.16:0/2160830146 wait complete. 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 Processor -- start 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 -- start start 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.093+0000 7f151ea8c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 0x7f1518198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151ea8c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 0x7f151819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151ea8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111f70 con 0x7f1518101de0 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151ea8c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1518111df0 con 0x7f1518106f10 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151ea8c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15181120f0 con 0x7f1518102790 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34576/0 (socket says 172.21.15.16:34576) 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 -- 172.21.15.16:0/2587182773 learned_addr learned my addr 172.21.15.16:0/2587182773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151d029640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 0x7f151819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f1517fff640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 0x7f1518198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 -- 172.21.15.16:0/2587182773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 msgr2=0x7f1518198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 0x7f1518198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 -- 172.21.15.16:0/2587182773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 msgr2=0x7f151819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 0x7f151819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 -- 172.21.15.16:0/2587182773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151819d380 con 0x7f1518101de0 2024-09-12T01:00:19.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f1517fff640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 0x7f1518198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151c828640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c970 tx=0x7f150c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f1515ffb640 1 -- 172.21.15.16:0/2587182773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c007bb0 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f1515ffb640 1 -- 172.21.15.16:0/2587182773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c007d50 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151819d670 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.094+0000 7f1515ffb640 1 -- 172.21.15.16:0/2587182773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00f450 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.095+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15181a4f50 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.097+0000 7f1515ffb640 1 -- 172.21.15.16:0/2587182773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f150c005020 con 0x7f1518101de0 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.097+0000 7f1520e65740 1 Processor -- start 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.097+0000 7f1520e65740 1 -- start start 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627805090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627816b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e62776b2f0 con 0x55e6278055d0 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e627712cf0 con 0x55e627814780 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1520e65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e6277672b0 con 0x55e627806b40 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34590/0 (socket says 172.21.15.16:34590) 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151c828640 1 -- 172.21.15.16:0/2528325092 learned_addr learned my addr 172.21.15.16:0/2528325092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627816b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151d029640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627805090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1515ffb640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 0x7f15080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 -- 172.21.15.16:0/2528325092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 msgr2=0x55e627805090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627805090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 -- 172.21.15.16:0/2528325092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 msgr2=0x55e627805a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f1517fff640 1 -- 172.21.15.16:0/2528325092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6276f73a0 con 0x55e627814780 2024-09-12T01:00:19.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151c828640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.098+0000 7f151d029640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 0x7f15080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f1515ffb640 1 -- 172.21.15.16:0/2587182773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f150c0cf200 con 0x7f1518101de0 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f1517fff640 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627816b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1504009230 tx=0x7f15040316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0a290b57f28854 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f151d029640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 0x7f15080a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1510007fc0 tx=0x7f1510004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f1502ffd640 1 -- 172.21.15.16:0/2528325092 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150403f070 con 0x55e627814780 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f1502ffd640 1 -- 172.21.15.16:0/2528325092 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15040035f0 con 0x55e627814780 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.099+0000 7f1502ffd640 1 -- 172.21.15.16:0/2528325092 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504035070 con 0x55e627814780 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- 172.21.15.16:0/2528325092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 msgr2=0x55e627816b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627816b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1504009230 tx=0x7f15040316e0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- 172.21.15.16:0/2528325092 shutdown_connections 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627816b70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627805a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 --2- 172.21.15.16:0/2528325092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627805090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- 172.21.15.16:0/2528325092 >> 172.21.15.16:0/2528325092 conn(0x55e6277ce310 msgr2=0x55e627801c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- 172.21.15.16:0/2528325092 shutdown_connections 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- 172.21.15.16:0/2528325092 wait complete. 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 -- start start 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.100+0000 7f1520e65740 1 Processor -- start 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627811440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627811a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e627817420 con 0x55e6278055d0 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e627726060 con 0x55e627814780 2024-09-12T01:00:19.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e6278175a0 con 0x55e627806b40 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1520e65740 10 client.0 _ll_register_callbacks cb 0x55e6274798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34594/0 (socket says 172.21.15.16:34594) 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 -- 172.21.15.16:0/3335471670 learned_addr learned my addr 172.21.15.16:0/3335471670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151d029640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627811a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1517fff640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627811440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 -- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 msgr2=0x55e627811440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627811440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 -- 172.21.15.16:0/3335471670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 msgr2=0x55e627811a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627811a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 -- 172.21.15.16:0/3335471670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6274dc3c0 con 0x55e6278055d0 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f1517fff640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627811440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:19.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151d029640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627811a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f151c828640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f150c0058c0 tx=0x7f150c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21db6d6912c64d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.101+0000 7f14f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0cf030 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f150c097dc0 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15040035f0 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f150c01d040 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 10 client.16539 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e627822f50 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15040035f0 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f150c005310 con 0x55e6278055d0 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 10 client.16539 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 10 client.16539 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:00:19.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 20 client.16539 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e627824360 0x55e6278267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 4 client.16539 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.102+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e627778860 con 0x55e627824360 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.103+0000 7f1517fff640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e627824360 0x55e6278267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.103+0000 7f1517fff640 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e627824360 0x55e6278267f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8af410930f0b229 server_cookie=67c5f7afccfa19eb in_seq=0 out_seq=0 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.103+0000 7f14f7fff640 10 client.16539 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f14f7fff640 1 -- 172.21.15.16:0/3335471670 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e627778860 con 0x55e627824360 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f14f7fff640 10 client.16539 handle_command_reply: tid=0 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 client.16539 shutdown 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 msgr2=0x55e627814190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f150c0058c0 tx=0x7f150c00cf60 comp rx=0 tx=0).stop 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 shutdown_connections 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e627824360 0x55e6278267f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e627811a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e627811440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 --2- 172.21.15.16:0/3335471670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e627814190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f14f7fff640 0 client.16539 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 >> 172.21.15.16:0/3335471670 conn(0x55e6277ce310 msgr2=0x55e627807960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 shutdown_connections 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 1 -- 172.21.15.16:0/3335471670 wait complete. 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.104+0000 7f1520e65740 20 client.16539 trim_cache size 0 max 16384 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 Processor -- start 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 -- start start 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e6278133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e62782c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15181ab4f0 con 0x55e6278055d0 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15181ad030 con 0x55e627814780 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.117+0000 7f1520e65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15181abbc0 con 0x55e627806b40 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34610/0 (socket says 172.21.15.16:34610) 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 -- 172.21.15.16:0/1178272131 learned_addr learned my addr 172.21.15.16:0/1178272131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151d029640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e62782c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f1517fff640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e6278133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 -- 172.21.15.16:0/1178272131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 msgr2=0x55e6278133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e6278133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 -- 172.21.15.16:0/1178272131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 msgr2=0x55e62782c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e62782c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 -- 172.21.15.16:0/1178272131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6278103f0 con 0x55e6278055d0 2024-09-12T01:00:19.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f1517fff640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e6278133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:19.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f151c828640 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f150c097bb0 tx=0x7f150c086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb43a96843a9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f15027fc640 1 -- 172.21.15.16:0/1178272131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c09b0b0 con 0x55e6278055d0 2024-09-12T01:00:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f15027fc640 1 -- 172.21.15.16:0/1178272131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c09be50 con 0x55e6278055d0 2024-09-12T01:00:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.118+0000 7f15027fc640 1 -- 172.21.15.16:0/1178272131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c09aba0 con 0x55e6278055d0 2024-09-12T01:00:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- 172.21.15.16:0/1178272131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 msgr2=0x55e6278206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f150c097bb0 tx=0x7f150c086050 comp rx=0 tx=0).stop 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- 172.21.15.16:0/1178272131 shutdown_connections 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e627814780 0x55e62782c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627806b40 0x55e6278133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 --2- 172.21.15.16:0/1178272131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6278055d0 0x55e6278206c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- 172.21.15.16:0/1178272131 >> 172.21.15.16:0/1178272131 conn(0x55e6277ce310 msgr2=0x55e627812ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- 172.21.15.16:0/1178272131 shutdown_connections 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- 172.21.15.16:0/1178272131 wait complete. 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.119+0000 7f1520e65740 1 -- start start 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 Processor -- start 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6278055d0 0x55e62780e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627814780 0x55e62780f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15181ac960 con 0x55e627806b40 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15181ac290 con 0x55e6278055d0 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15181a94d0 con 0x55e627814780 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1520e65740 10 client.0 _ll_register_callbacks cb 0x7f150c005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34624/0 (socket says 172.21.15.16:34624) 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 -- 172.21.15.16:0/1286791298 learned_addr learned my addr 172.21.15.16:0/1286791298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f151c828640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6278055d0 0x55e62780e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f151d029640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627814780 0x55e62780f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 -- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627814780 msgr2=0x55e62780f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627814780 0x55e62780f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 -- 172.21.15.16:0/1286791298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6278055d0 msgr2=0x55e62780e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.120+0000 7f1517fff640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6278055d0 0x55e62780e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1517fff640 1 -- 172.21.15.16:0/1286791298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62788e330 con 0x55e627806b40 2024-09-12T01:00:19.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1517fff640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f1504009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86cb3ae2f61aa6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150403f040 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c09be50 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f1504005040 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f150405e050 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1520e65740 10 client.16545 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e62788f5d0 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f150c09be50 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f150404c070 con 0x55e627806b40 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1520e65740 10 client.16545 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.121+0000 7f1520e65740 10 client.16545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f1520e65740 20 client.16545 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e62788f910 0x55e627891d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:19.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f1520e65740 4 client.16545 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e627772d70 con 0x55e62788f910 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f151c828640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e62788f910 0x55e627891d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f151c828640 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e62788f910 0x55e627891d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb7fe4c7a50e4ded server_cookie=17e1f0d1858b2b35 in_seq=0 out_seq=0 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f14f7fff640 10 client.16545 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f14f7fff640 1 -- 172.21.15.16:0/1286791298 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e627772d70 con 0x55e62788f910 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f14f7fff640 10 client.16545 handle_command_reply: tid=0 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.122+0000 7f1520e65740 1 client.16545 shutdown 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 msgr2=0x55e62780eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f1504009260 comp rx=0 tx=0).stop 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 shutdown_connections 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e627814780 0x55e62780f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e627806b40 0x55e62780eb80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55e62788f910 0x55e627891d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 --2- 172.21.15.16:0/1286791298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e6278055d0 0x55e62780e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f14f7fff640 0 client.16545 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 >> 172.21.15.16:0/1286791298 conn(0x55e6277ce310 msgr2=0x55e6278134c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 shutdown_connections 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 1 -- 172.21.15.16:0/1286791298 wait complete. 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f1520e65740 20 client.16545 trim_cache size 0 max 16384 2024-09-12T01:00:19.137 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 msgr2=0x7f15080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 0x7f15080a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1510007fc0 tx=0x7f1510004a40 comp rx=0 tx=0).stop 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 msgr2=0x7f15181983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.123+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c970 tx=0x7f150c00ce30 comp rx=0 tx=0).stop 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 shutdown_connections 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f15080a2cf0 0x7f15080a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1518106f10 0x7f151819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1518102790 0x7f1518198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 --2- 172.21.15.16:0/2587182773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1518101de0 0x7f15181983d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 >> 172.21.15.16:0/2587182773 conn(0x7f15180fd8c0 msgr2=0x7f15180fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 shutdown_connections 2024-09-12T01:00:19.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:19.124+0000 7f151ea8c640 1 -- 172.21.15.16:0/2587182773 wait complete. 2024-09-12T01:00:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:18 smithi099 ceph-mon[35654]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-12T01:00:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:18 smithi159 ceph-mon[35566]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 9 op/s 2024-09-12T01:00:19.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:00:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:20.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 -- 172.21.15.16:0/1409696158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102790 msgr2=0x7fd0601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 --2- 172.21.15.16:0/1409696158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102790 0x7fd0601067e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd054009900 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 -- 172.21.15.16:0/1409696158 shutdown_connections 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 --2- 172.21.15.16:0/1409696158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060106f10 0x7fd06010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 --2- 172.21.15.16:0/1409696158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060102790 0x7fd0601067e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 --2- 172.21.15.16:0/1409696158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 0x7fd0601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 -- 172.21.15.16:0/1409696158 >> 172.21.15.16:0/1409696158 conn(0x7fd0600fd8c0 msgr2=0x7fd0600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.712+0000 7fd066472640 1 -- 172.21.15.16:0/1409696158 shutdown_connections 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.713+0000 7fd066472640 1 -- 172.21.15.16:0/1409696158 wait complete. 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.713+0000 7fd066472640 1 Processor -- start 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.713+0000 7fd066472640 1 -- start start 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 0x7fd060198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060102790 0x7fd060198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd060111f90 con 0x7fd060106f10 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd060111e10 con 0x7fd060102790 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd060112110 con 0x7fd060101de0 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34636/0 (socket says 172.21.15.16:34636) 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 -- 172.21.15.16:0/801984667 learned_addr learned my addr 172.21.15.16:0/801984667 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060102790 0x7fd060198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd05ffff640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 0x7fd060198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 -- 172.21.15.16:0/801984667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 msgr2=0x7fd060198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 0x7fd060198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 -- 172.21.15.16:0/801984667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060102790 msgr2=0x7fd060198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060102790 0x7fd060198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 -- 172.21.15.16:0/801984667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06019d3c0 con 0x7fd060106f10 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd064a0f640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd05800cce0 tx=0x7fd05800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd05d7fa640 1 -- 172.21.15.16:0/801984667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800f450 con 0x7fd060106f10 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd05d7fa640 1 -- 172.21.15.16:0/801984667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd05800fad0 con 0x7fd060106f10 2024-09-12T01:00:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06019d6b0 con 0x7fd060106f10 2024-09-12T01:00:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd05d7fa640 1 -- 172.21.15.16:0/801984667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058018c10 con 0x7fd060106f10 2024-09-12T01:00:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.714+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0601a4f90 con 0x7fd060106f10 2024-09-12T01:00:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05d7fa640 1 -- 172.21.15.16:0/801984667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd05800f5f0 con 0x7fd060106f10 2024-09-12T01:00:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 Processor -- start 2024-09-12T01:00:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 -- start start 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d15370 0x55d1a3d17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3c702f0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3c17cf0 con 0x55d1a3d15370 2024-09-12T01:00:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd06884b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3c6c2b0 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34646/0 (socket says 172.21.15.16:34646) 2024-09-12T01:00:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05f7fe640 1 -- 172.21.15.16:0/2084165287 learned_addr learned my addr 172.21.15.16:0/2084165287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05d7fa640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 0x7fd0480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.717+0000 7fd05ffff640 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 -- 172.21.15.16:0/2084165287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 msgr2=0x55d1a3d0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd064a0f640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 0x7fd0480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 -- 172.21.15.16:0/2084165287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d15370 msgr2=0x55d1a3d17760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d15370 0x55d1a3d17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 -- 172.21.15.16:0/2084165287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a3bfc3a0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05ffff640 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05d7fa640 1 -- 172.21.15.16:0/801984667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd0580cfb10 con 0x7fd060106f10 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ec0 tx=0x7fd0540316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b2d4cba0980651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd0467fc640 1 -- 172.21.15.16:0/2084165287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054035040 con 0x55d1a3d0bb40 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd0467fc640 1 -- 172.21.15.16:0/2084165287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd054005020 con 0x55d1a3d0bb40 2024-09-12T01:00:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.718+0000 7fd0467fc640 1 -- 172.21.15.16:0/2084165287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054004b90 con 0x55d1a3d0bb40 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- 172.21.15.16:0/2084165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 msgr2=0x55d1a3d0a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd054005ec0 tx=0x7fd0540316f0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd064a0f640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 0x7fd0480a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd0580cd340 tx=0x7fd058067080 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- 172.21.15.16:0/2084165287 shutdown_connections 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d15370 0x55d1a3d17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 --2- 172.21.15.16:0/2084165287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d0a090 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- 172.21.15.16:0/2084165287 >> 172.21.15.16:0/2084165287 conn(0x55d1a3cd3310 msgr2=0x55d1a3d06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- 172.21.15.16:0/2084165287 shutdown_connections 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- 172.21.15.16:0/2084165287 wait complete. 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 -- start start 2024-09-12T01:00:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.719+0000 7fd06884b740 1 Processor -- start 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3d18010 con 0x55d1a3d15370 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3c2b060 con 0x55d1a3d0bb40 2024-09-12T01:00:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1a3d18190 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 10 client.0 _ll_register_callbacks cb 0x55d1a397e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34658/0 (socket says 172.21.15.16:34658) 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 -- 172.21.15.16:0/1460564303 learned_addr learned my addr 172.21.15.16:0/1460564303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05ffff640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 -- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 msgr2=0x55d1a3d14d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d14d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 -- 172.21.15.16:0/1460564303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 msgr2=0x55d1a3d11fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d11fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 -- 172.21.15.16:0/1460564303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a39e13c0 con 0x55d1a3d15370 2024-09-12T01:00:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05ffff640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd054009340 tx=0x7fd0540045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a455a96fd679be7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd03f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054035070 con 0x55d1a3d15370 2024-09-12T01:00:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd054004910 con 0x55d1a3d15370 2024-09-12T01:00:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.720+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd054005020 con 0x55d1a3d15370 2024-09-12T01:00:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd05405a050 con 0x55d1a3d15370 2024-09-12T01:00:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 10 client.16557 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1a3d23a70 con 0x55d1a3d15370 2024-09-12T01:00:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd054005020 con 0x55d1a3d15370 2024-09-12T01:00:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd054052020 con 0x55d1a3d15370 2024-09-12T01:00:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 10 client.16557 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 20 client.16557 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d24dc0 0x55d1a3d271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 4 client.16557 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a3c7d860 con 0x55d1a3d24dc0 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.721+0000 7fd05ffff640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d24dc0 0x55d1a3d271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.722+0000 7fd03f7fe640 10 client.16557 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.722+0000 7fd05ffff640 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d24dc0 0x55d1a3d271b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=984cb2954590877b server_cookie=7dbfcaa77e261426 in_seq=0 out_seq=0 2024-09-12T01:00:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.722+0000 7fd03f7fe640 1 -- 172.21.15.16:0/1460564303 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1a3c7d860 con 0x55d1a3d24dc0 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.722+0000 7fd03f7fe640 10 client.16557 handle_command_reply: tid=0 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.722+0000 7fd06884b740 1 client.16557 shutdown 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 msgr2=0x55d1a3d125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd054009340 tx=0x7fd0540045b0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 shutdown_connections 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d24dc0 0x55d1a3d271b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d15370 0x55d1a3d125d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d11fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd03f7fe640 0 client.16557 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 --2- 172.21.15.16:0/1460564303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d14d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 >> 172.21.15.16:0/1460564303 conn(0x55d1a3cd3310 msgr2=0x55d1a3d0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 shutdown_connections 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 1 -- 172.21.15.16:0/1460564303 wait complete. 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.723+0000 7fd06884b740 20 client.16557 trim_cache size 0 max 16384 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 Processor -- start 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 -- start start 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601ad670 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601af1b0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.736+0000 7fd06884b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601add40 con 0x55d1a3d15370 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34662/0 (socket says 172.21.15.16:34662) 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 -- 172.21.15.16:0/4245258485 learned_addr learned my addr 172.21.15.16:0/4245258485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05ffff640 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 -- 172.21.15.16:0/4245258485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 msgr2=0x55d1a3d2a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 -- 172.21.15.16:0/4245258485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 msgr2=0x55d1a3d2a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d2a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 -- 172.21.15.16:0/4245258485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a3d0d120 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd054031bf0 tx=0x7fd05402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e9472fa0de457e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd045ffb640 1 -- 172.21.15.16:0/4245258485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054049030 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd045ffb640 1 -- 172.21.15.16:0/4245258485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0540316f0 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.737+0000 7fd045ffb640 1 -- 172.21.15.16:0/4245258485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054035070 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- 172.21.15.16:0/4245258485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 msgr2=0x55d1a3d09bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd054031bf0 tx=0x7fd05402a410 comp rx=0 tx=0).stop 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- 172.21.15.16:0/4245258485 shutdown_connections 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d2a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 --2- 172.21.15.16:0/4245258485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d09bd0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- 172.21.15.16:0/4245258485 >> 172.21.15.16:0/4245258485 conn(0x55d1a3cd3310 msgr2=0x55d1a3d108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- 172.21.15.16:0/4245258485 shutdown_connections 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- 172.21.15.16:0/4245258485 wait complete. 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.738+0000 7fd06884b740 1 -- start start 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 Processor -- start 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601aeae0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601ae410 con 0x55d1a3d0a5d0 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0601ab650 con 0x55d1a3d15370 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd06884b740 10 client.0 _ll_register_callbacks cb 0x7fd054031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34678/0 (socket says 172.21.15.16:34678) 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 -- 172.21.15.16:0/3499850798 learned_addr learned my addr 172.21.15.16:0/3499850798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 -- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 msgr2=0x55d1a3d28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 -- 172.21.15.16:0/3499850798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0a5d0 msgr2=0x55d1a3d2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.739+0000 7fd05ffff640 1 -- 172.21.15.16:0/3499850798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a3d28b80 con 0x55d1a3d0bb40 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd05ffff640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd05000ba90 tx=0x7fd0500157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858cb6dd1c42476a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500233f0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd0500223a0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0540316f0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0500059c0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd06884b740 10 client.16563 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1a3d8f870 con 0x55d1a3d0bb40 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0540316f0 con 0x55d1a3d0bb40 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd050006020 con 0x55d1a3d0bb40 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.740+0000 7fd06884b740 10 client.16563 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 20 client.16563 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d8ffd0 0x55d1a3d923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 4 client.16563 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1a3c77d70 con 0x55d1a3d8ffd0 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d8ffd0 0x55d1a3d923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd05f7fe640 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d8ffd0 0x55d1a3d923f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e602bea3645c422f server_cookie=f2474c15848ec95e in_seq=0 out_seq=0 2024-09-12T01:00:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd03f7fe640 10 client.16563 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd03f7fe640 1 -- 172.21.15.16:0/3499850798 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1a3c77d70 con 0x55d1a3d8ffd0 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd03f7fe640 10 client.16563 handle_command_reply: tid=0 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.741+0000 7fd06884b740 1 client.16563 shutdown 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 msgr2=0x55d1a3d27d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd05000ba90 tx=0x7fd0500157f0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 shutdown_connections 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d1a3d8ffd0 0x55d1a3d923f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1a3d15370 0x55d1a3d28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1a3d0bb40 0x55d1a3d27d60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd03f7fe640 0 client.16563 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 --2- 172.21.15.16:0/3499850798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1a3d0a5d0 0x55d1a3d2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 >> 172.21.15.16:0/3499850798 conn(0x55d1a3cd3310 msgr2=0x55d1a3d0c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 shutdown_connections 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 1 -- 172.21.15.16:0/3499850798 wait complete. 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd06884b740 20 client.16563 trim_cache size 0 max 16384 2024-09-12T01:00:20.773 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 msgr2=0x7fd0480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 0x7fd0480a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd0580cd340 tx=0x7fd058067080 comp rx=0 tx=0).stop 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 msgr2=0x7fd06019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.742+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd05800cce0 tx=0x7fd05800a620 comp rx=0 tx=0).stop 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 shutdown_connections 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0480a2cf0 0x7fd0480a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd060106f10 0x7fd06019cce0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd060102790 0x7fd060198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 --2- 172.21.15.16:0/801984667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd060101de0 0x7fd060198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 >> 172.21.15.16:0/801984667 conn(0x7fd0600fd8c0 msgr2=0x7fd0600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 shutdown_connections 2024-09-12T01:00:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:20.743+0000 7fd066472640 1 -- 172.21.15.16:0/801984667 wait complete. 2024-09-12T01:00:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:20 smithi099 ceph-mon[35654]: pgmap v131: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-09-12T01:00:21.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:20 smithi159 ceph-mon[35566]: pgmap v131: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-09-12T01:00:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:20 smithi016 ceph-mon[30972]: pgmap v131: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 14 op/s 2024-09-12T01:00:22.616 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:00:22.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:00:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:23.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:23.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:22 smithi016 ceph-mon[30972]: pgmap v132: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:22 smithi099 ceph-mon[35654]: pgmap v132: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:22 smithi159 ceph-mon[35566]: pgmap v132: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.406+0000 7fe96b37a640 1 -- 172.21.15.16:0/3812776852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964074eb0 msgr2=0x7fe964112660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.406+0000 7fe96b37a640 1 --2- 172.21.15.16:0/3812776852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964074eb0 0x7fe964112660 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe958009900 tx=0x7fe95802a480 comp rx=0 tx=0).stop 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- 172.21.15.16:0/3812776852 shutdown_connections 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- 172.21.15.16:0/3812776852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964112ba0 0x7fe964114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- 172.21.15.16:0/3812776852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964074eb0 0x7fe964112660 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- 172.21.15.16:0/3812776852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 0x7fe964074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- 172.21.15.16:0/3812776852 >> 172.21.15.16:0/3812776852 conn(0x7fe9640fd8f0 msgr2=0x7fe9640ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- 172.21.15.16:0/3812776852 shutdown_connections 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- 172.21.15.16:0/3812776852 wait complete. 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 Processor -- start 2024-09-12T01:00:23.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- start start 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 0x7fe9641a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 0x7fe9641a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.407+0000 7fe96b37a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe964118550 con 0x7fe964112ba0 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe96b37a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641183d0 con 0x7fe964074eb0 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe96b37a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641186d0 con 0x7fe964074590 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34690/0 (socket says 172.21.15.16:34690) 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 -- 172.21.15.16:0/134715679 learned_addr learned my addr 172.21.15.16:0/134715679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe968915640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 0x7fe9641a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969116640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 0x7fe9641a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 -- 172.21.15.16:0/134715679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 msgr2=0x7fe9641a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 0x7fe9641a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 -- 172.21.15.16:0/134715679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 msgr2=0x7fe9641a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 0x7fe9641a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 -- 172.21.15.16:0/134715679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96419f010 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe968915640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 0x7fe9641a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe969917640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9d0 tx=0x7fe95400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe9627fc640 1 -- 172.21.15.16:0/134715679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95400cd80 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe9627fc640 1 -- 172.21.15.16:0/134715679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe954005020 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe96419f300 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe9627fc640 1 -- 172.21.15.16:0/134715679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540116d0 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.408+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9641ab5f0 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.410+0000 7fe9627fc640 1 -- 172.21.15.16:0/134715679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe9540051d0 con 0x7fe964112ba0 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 Processor -- start 2024-09-12T01:00:23.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 -- start start 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56543a506280 con 0x56543a5a0b60 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56543a4adb10 con 0x56543a5abd40 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe96d753740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56543a502240 con 0x56543a5a1480 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe9627fc640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 0x7fe94c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34696/0 (socket says 172.21.15.16:34696) 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.411+0000 7fe968915640 1 -- 172.21.15.16:0/4104621378 learned_addr learned my addr 172.21.15.16:0/4104621378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe969116640 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe9627fc640 1 -- 172.21.15.16:0/134715679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe9540cf9a0 con 0x7fe964112ba0 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe969917640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 0x7fe94c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 -- 172.21.15.16:0/4104621378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 msgr2=0x56543a5ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 -- 172.21.15.16:0/4104621378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 msgr2=0x56543a5ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 -- 172.21.15.16:0/4104621378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543a4ba1f0 con 0x56543a5a0b60 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe968915640 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x56543a59e470 tx=0x7fe958031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb38421a481a2ced server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe969917640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 0x7fe94c0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe9540cd1d0 tx=0x7fe95400c7f0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe94b7fe640 1 -- 172.21.15.16:0/4104621378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958035050 con 0x56543a5a0b60 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe94b7fe640 1 -- 172.21.15.16:0/4104621378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958031950 con 0x56543a5a0b60 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.412+0000 7fe94b7fe640 1 -- 172.21.15.16:0/4104621378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9580048e0 con 0x56543a5a0b60 2024-09-12T01:00:23.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- 172.21.15.16:0/4104621378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 msgr2=0x56543a5a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x56543a59e470 tx=0x7fe958031480 comp rx=0 tx=0).stop 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- 172.21.15.16:0/4104621378 shutdown_connections 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 --2- 172.21.15.16:0/4104621378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a0b60 0x56543a5a0f40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- 172.21.15.16:0/4104621378 >> 172.21.15.16:0/4104621378 conn(0x56543a59d0f0 msgr2=0x56543a59d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- 172.21.15.16:0/4104621378 shutdown_connections 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- 172.21.15.16:0/4104621378 wait complete. 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.413+0000 7fe96d753740 1 -- start start 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 Processor -- start 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56543a5ae910 con 0x56543a5a1480 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56543a4c0f10 con 0x56543a5abd40 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56543a5aea90 con 0x56543a5a0b60 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe96d753740 10 client.0 _ll_register_callbacks cb 0x56543a489a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe969116640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe969116640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34712/0 (socket says 172.21.15.16:34712) 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe969116640 1 -- 172.21.15.16:0/1551827775 learned_addr learned my addr 172.21.15.16:0/1551827775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe968915640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe968915640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.414+0000 7fe969116640 1 -- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 msgr2=0x56543a5a8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe969116640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe969116640 1 -- 172.21.15.16:0/1551827775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 msgr2=0x56543a5a9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe969116640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5a9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe969116640 1 -- 172.21.15.16:0/1551827775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543a2776a0 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe968915640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe969116640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe95c011770 tx=0x7fe95c011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e190f76c628782ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c01e060 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe95c0193f0 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958031950 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe95c00fe70 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 10 client.16575 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56543a5ba5f0 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe95c019ad0 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe95c018030 con 0x56543a5a1480 2024-09-12T01:00:23.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 10 client.16575 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 20 client.16575 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.415+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a5bbbf0 0x56543a5bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.416+0000 7fe96d753740 4 client.16575 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.416+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe958031950 con 0x56543a5bbbf0 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.416+0000 7fe968915640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a5bbbf0 0x56543a5bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.416+0000 7fe968915640 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a5bbbf0 0x56543a5bdfe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8062bb03a5afc6 server_cookie=c3efc90bd734594b in_seq=0 out_seq=0 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.416+0000 7fe948ff9640 10 client.16575 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe948ff9640 1 -- 172.21.15.16:0/1551827775 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe958031950 con 0x56543a5bbbf0 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe948ff9640 10 client.16575 handle_command_reply: tid=0 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 client.16575 shutdown 2024-09-12T01:00:23.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 msgr2=0x56543a5a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe95c011770 tx=0x7fe95c011c30 comp rx=0 tx=0).stop 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 shutdown_connections 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a5bbbf0 0x56543a5bdfe0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5abd40 0x56543a5a9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5a1480 0x56543a5a91d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 --2- 172.21.15.16:0/1551827775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe948ff9640 0 client.16575 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 >> 172.21.15.16:0/1551827775 conn(0x56543a59d0f0 msgr2=0x56543a5a2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 shutdown_connections 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.417+0000 7fe96d753740 1 -- 172.21.15.16:0/1551827775 wait complete. 2024-09-12T01:00:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.418+0000 7fe96d753740 20 client.16575 trim_cache size 0 max 16384 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 Processor -- start 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 -- start start 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 0x56543a5aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b3d90 con 0x56543a5abd40 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b58d0 con 0x56543a5a1480 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.435+0000 7fe96d753740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b4460 con 0x56543a5a0b60 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34726/0 (socket says 172.21.15.16:34726) 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 -- 172.21.15.16:0/1306567683 learned_addr learned my addr 172.21.15.16:0/1306567683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe969116640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 0x56543a5aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 -- 172.21.15.16:0/1306567683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 msgr2=0x56543a5a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 -- 172.21.15.16:0/1306567683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 msgr2=0x56543a5aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 0x56543a5aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 -- 172.21.15.16:0/1306567683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543a5a4870 con 0x56543a5abd40 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe969116640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 0x56543a5aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe968915640 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe9580055e0 tx=0x7fe95803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81212b844cdd7027 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe94affd640 1 -- 172.21.15.16:0/1306567683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958035040 con 0x56543a5abd40 2024-09-12T01:00:23.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe94affd640 1 -- 172.21.15.16:0/1306567683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958031b80 con 0x56543a5abd40 2024-09-12T01:00:23.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.436+0000 7fe94affd640 1 -- 172.21.15.16:0/1306567683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958003a70 con 0x56543a5abd40 2024-09-12T01:00:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- 172.21.15.16:0/1306567683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 msgr2=0x56543a5c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe9580055e0 tx=0x7fe95803f040 comp rx=0 tx=0).stop 2024-09-12T01:00:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- 172.21.15.16:0/1306567683 shutdown_connections 2024-09-12T01:00:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a5c37f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a1480 0x56543a5aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 --2- 172.21.15.16:0/1306567683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a0b60 0x56543a5a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- 172.21.15.16:0/1306567683 >> 172.21.15.16:0/1306567683 conn(0x56543a59d0f0 msgr2=0x56543a5aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- 172.21.15.16:0/1306567683 shutdown_connections 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- 172.21.15.16:0/1306567683 wait complete. 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.437+0000 7fe96d753740 1 -- start start 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 Processor -- start 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a0b60 0x56543a5b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b5200 con 0x56543a5abd40 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b4b30 con 0x56543a5a0b60 2024-09-12T01:00:23.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641b1d70 con 0x56543a5a1480 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe96d753740 10 client.0 _ll_register_callbacks cb 0x7fe95c00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe968915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34732/0 (socket says 172.21.15.16:34732) 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe968915640 1 -- 172.21.15.16:0/1131251041 learned_addr learned my addr 172.21.15.16:0/1131251041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe969116640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.438+0000 7fe968915640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a0b60 0x56543a5b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 -- 172.21.15.16:0/1131251041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a0b60 msgr2=0x56543a5b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a0b60 0x56543a5b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 -- 172.21.15.16:0/1131251041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 msgr2=0x56543a6257c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 -- 172.21.15.16:0/1131251041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543a625d00 con 0x56543a5a1480 2024-09-12T01:00:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe968915640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe969116640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5b05e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010010 tx=0x7fe95c0236b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d404c21d38b89eff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00f790 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe958031b80 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe95c011020 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe95c01e030 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe96d753740 10 client.26011 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.439+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56543a626f20 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe958031b80 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe95c018050 con 0x56543a5a1480 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 10 client.26011 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 10 client.26011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 20 client.26011 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a627690 0x56543a629a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 4 client.26011 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56543a5137f0 con 0x56543a627690 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.440+0000 7fe968915640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a627690 0x56543a629a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.441+0000 7fe948ff9640 10 client.26011 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.441+0000 7fe968915640 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a627690 0x56543a629a80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510262da23477ab5 server_cookie=af0ae3602b829a7d in_seq=0 out_seq=0 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe948ff9640 1 -- 172.21.15.16:0/1131251041 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56543a5137f0 con 0x56543a627690 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe948ff9640 10 client.26011 handle_command_reply: tid=0 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 client.26011 shutdown 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 msgr2=0x56543a5b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5b05e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe95c010010 tx=0x7fe95c0236b0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 shutdown_connections 2024-09-12T01:00:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56543a627690 0x56543a629a80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56543a5abd40 0x56543a6257c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56543a5a1480 0x56543a5b05e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 --2- 172.21.15.16:0/1131251041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56543a5a0b60 0x56543a5b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 >> 172.21.15.16:0/1131251041 conn(0x56543a59d0f0 msgr2=0x56543a59c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe948ff9640 0 client.26011 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 shutdown_connections 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 1 -- 172.21.15.16:0/1131251041 wait complete. 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.442+0000 7fe96d753740 20 client.26011 trim_cache size 0 max 16384 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:00:23.446 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "ccf757f5-4c6a-48f6-8930-a671238bb605", 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 msgr2=0x7fe94c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 0x7fe94c0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe9540cd1d0 tx=0x7fe95400c7f0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 msgr2=0x7fe96419e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9d0 tx=0x7fe95400ee90 comp rx=0 tx=0).stop 2024-09-12T01:00:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 shutdown_connections 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe94c0a2d70 0x7fe94c0a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe964112ba0 0x7fe96419e8e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe964074eb0 0x7fe9641a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 --2- 172.21.15.16:0/134715679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe964074590 0x7fe9641a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 >> 172.21.15.16:0/134715679 conn(0x7fe9640fd8f0 msgr2=0x7fe9640ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 shutdown_connections 2024-09-12T01:00:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:23.443+0000 7fe96b37a640 1 -- 172.21.15.16:0/134715679 wait complete. 2024-09-12T01:00:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:23 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:00:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:23 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:23 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:00:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:23 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:23 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:00:24.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:23 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:23 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:00:24.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:23 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:23 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:00:24.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:23 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:23 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:00:24.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:23 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:24.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 -- 172.21.15.16:0/363480460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 msgr2=0x7fb7c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/363480460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c4101200 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009ca0 tx=0x7fb7bc02a950 comp rx=0 tx=0).stop 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 -- 172.21.15.16:0/363480460 shutdown_connections 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/363480460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102ef0 0x7fb7c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/363480460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102570 0x7fb7c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/363480460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c4101200 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:24.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 -- 172.21.15.16:0/363480460 >> 172.21.15.16:0/363480460 conn(0x7fb7c40fc7d0 msgr2=0x7fb7c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 -- 172.21.15.16:0/363480460 shutdown_connections 2024-09-12T01:00:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.873+0000 7fb7cad6e640 1 -- 172.21.15.16:0/363480460 wait complete. 2024-09-12T01:00:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 Processor -- start 2024-09-12T01:00:24.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 -- start start 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 0x7fb7c40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102ef0 0x7fb7c40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4110fd0 con 0x7fb7c4100e20 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4110e50 con 0x7fb7c4102ef0 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7cad6e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4111150 con 0x7fb7c4102570 2024-09-12T01:00:24.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7c8b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7c8b0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34748/0 (socket says 172.21.15.16:34748) 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7c8b0a640 1 -- 172.21.15.16:0/2295151404 learned_addr learned my addr 172.21.15.16:0/2295151404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.874+0000 7fb7c3fff640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 0x7fb7c40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c930b640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102ef0 0x7fb7c40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 -- 172.21.15.16:0/2295151404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 msgr2=0x7fb7c40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 0x7fb7c40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 -- 172.21.15.16:0/2295151404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102ef0 msgr2=0x7fb7c40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102ef0 0x7fb7c40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 -- 172.21.15.16:0/2295151404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c41a69f0 con 0x7fb7c4100e20 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c3fff640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 0x7fb7c40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:24.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c8b0a640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005e30 tx=0x7fb7bc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc034040 con 0x7fb7c4100e20 2024-09-12T01:00:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7bc02fe10 con 0x7fb7c4100e20 2024-09-12T01:00:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc045480 con 0x7fb7c4100e20 2024-09-12T01:00:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.875+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41a6c80 con 0x7fb7c4100e20 2024-09-12T01:00:24.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.876+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a7110 con 0x7fb7c4100e20 2024-09-12T01:00:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.877+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c4101200 con 0x7fb7c4100e20 2024-09-12T01:00:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.878+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb7bc037020 con 0x7fb7c4100e20 2024-09-12T01:00:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.879+0000 7fb7c1ffb640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 0x7fb7b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.879+0000 7fb7c3fff640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 0x7fb7b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.879+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7bc0f0210 con 0x7fb7c4100e20 2024-09-12T01:00:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.879+0000 7fb7c3fff640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 0x7fb7b00a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40fe710 tx=0x7fb7ac00b410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:24.886+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7bc0bcf70 con 0x7fb7c4100e20 2024-09-12T01:00:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:25.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.103+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7c410a0a0 con 0x7fb7c4100e20 2024-09-12T01:00:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.104+0000 7fb7c1ffb640 1 -- 172.21.15.16:0/2295151404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fb7bc0098b0 con 0x7fb7c4100e20 2024-09-12T01:00:25.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:25.104 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:25.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 msgr2=0x7fb7b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 0x7fb7b00a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40fe710 tx=0x7fb7ac00b410 comp rx=0 tx=0).stop 2024-09-12T01:00:25.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 msgr2=0x7fb7c40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005e30 tx=0x7fb7bc003f90 comp rx=0 tx=0).stop 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 shutdown_connections 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7b00a2cf0 0x7fb7b00a51b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7c4102ef0 0x7fb7c40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7c4102570 0x7fb7c40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 --2- 172.21.15.16:0/2295151404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c4100e20 0x7fb7c40fcc80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 >> 172.21.15.16:0/2295151404 conn(0x7fb7c40fc7d0 msgr2=0x7fb7c406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 shutdown_connections 2024-09-12T01:00:25.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.106+0000 7fb7cad6e640 1 -- 172.21.15.16:0/2295151404 wait complete. 2024-09-12T01:00:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:24 smithi016 ceph-mon[30972]: pgmap v133: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:24 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:00:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:24 smithi099 ceph-mon[35654]: pgmap v133: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:24 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:00:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:24 smithi159 ceph-mon[35566]: pgmap v133: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-12T01:00:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:24 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:00:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 -- 172.21.15.16:0/3762320570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8101420 msgr2=0x7f7ff810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 --2- 172.21.15.16:0/3762320570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8101420 0x7f7ff810d950 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009cd0 tx=0x7f7ff002a8f0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 -- 172.21.15.16:0/3762320570 shutdown_connections 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 --2- 172.21.15.16:0/3762320570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8101420 0x7f7ff810d950 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 --2- 172.21.15.16:0/3762320570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 --2- 172.21.15.16:0/3762320570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8106a80 0x7f7ff8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 -- 172.21.15.16:0/3762320570 >> 172.21.15.16:0/3762320570 conn(0x7f7ff80fc7a0 msgr2=0x7f7ff80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 -- 172.21.15.16:0/3762320570 shutdown_connections 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.612+0000 7f7fffa89640 1 -- 172.21.15.16:0/3762320570 wait complete. 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 Processor -- start 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 -- start start 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8101420 0x7f7ff8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8110e80 con 0x7f7ff8106a80 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8110d00 con 0x7f7ff8100a80 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7fffa89640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8111000 con 0x7f7ff8101420 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34766/0 (socket says 172.21.15.16:34766) 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 -- 172.21.15.16:0/347401071 learned_addr learned my addr 172.21.15.16:0/347401071 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffd024640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8101420 0x7f7ff8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffd825640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 -- 172.21.15.16:0/347401071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8101420 msgr2=0x7f7ff8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8101420 0x7f7ff8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 -- 172.21.15.16:0/347401071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 msgr2=0x7f7ff8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.613+0000 7f7ffe026640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7ffe026640 1 -- 172.21.15.16:0/347401071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff81a46f0 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7ffd825640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7ffe026640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009ca0 tx=0x7f7ff00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7feeffd640 1 -- 172.21.15.16:0/347401071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0004040 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7feeffd640 1 -- 172.21.15.16:0/347401071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0040070 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff81a48c0 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7feeffd640 1 -- 172.21.15.16:0/347401071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0045cd0 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.614+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff81a4df0 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.616+0000 7f7feeffd640 1 -- 172.21.15.16:0/347401071 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7ff00041e0 con 0x7f7ff8106a80 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 Processor -- start 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 -- start start 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef57184a0 0x55bef5718880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 0x55bef5725a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bef567e2e0 con 0x55bef5718dc0 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bef5625ce0 con 0x55bef57184a0 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f8001e62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bef567a2a0 con 0x55bef5723680 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffd825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7feeffd640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 0x7f7fe40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffd825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34776/0 (socket says 172.21.15.16:34776) 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffd825640 1 -- 172.21.15.16:0/3160417426 learned_addr learned my addr 172.21.15.16:0/3160417426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 0x55bef5725a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffe026640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 0x7f7fe40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.617+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef57184a0 0x55bef5718880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 -- 172.21.15.16:0/3160417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 msgr2=0x55bef5725a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 0x55bef5725a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 -- 172.21.15.16:0/3160417426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef57184a0 msgr2=0x55bef5718880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef57184a0 0x55bef5718880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 -- 172.21.15.16:0/3160417426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef560a440 con 0x55bef5718dc0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7feeffd640 1 -- 172.21.15.16:0/347401071 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7ff00f0790 con 0x7f7ff8106a80 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 0x55bef5725a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4008be0 tx=0x7f7ff400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873db4262b07db40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7fe3fff640 1 -- 172.21.15.16:0/3160417426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4014070 con 0x55bef5718dc0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7ffe026640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 0x7f7fe40a5230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00ee510 tx=0x7f7ff008bd20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7fe3fff640 1 -- 172.21.15.16:0/3160417426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff40047a0 con 0x55bef5718dc0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.618+0000 7f7fe3fff640 1 -- 172.21.15.16:0/3160417426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4010650 con 0x55bef5718dc0 2024-09-12T01:00:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- 172.21.15.16:0/3160417426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 msgr2=0x55bef5723140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4008be0 tx=0x7f7ff400e720 comp rx=0 tx=0).stop 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- 172.21.15.16:0/3160417426 shutdown_connections 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5723680 0x55bef5725a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5723140 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 --2- 172.21.15.16:0/3160417426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef57184a0 0x55bef5718880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- 172.21.15.16:0/3160417426 >> 172.21.15.16:0/3160417426 conn(0x55bef5714a30 msgr2=0x55bef5714e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- 172.21.15.16:0/3160417426 shutdown_connections 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- 172.21.15.16:0/3160417426 wait complete. 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 -- start start 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.619+0000 7f8001e62740 1 Processor -- start 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 0x55bef5722f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5720760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bef5726250 con 0x55bef5718dc0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bef5639050 con 0x55bef5723680 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bef57263d0 con 0x55bef57184a0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f8001e62740 10 client.0 _ll_register_callbacks cb 0x55bef5679160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34782/0 (socket says 172.21.15.16:34782) 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 -- 172.21.15.16:0/115457909 learned_addr learned my addr 172.21.15.16:0/115457909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd024640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5720760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd024640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 0x55bef5722f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 -- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 msgr2=0x55bef5722f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 0x55bef5722f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 -- 172.21.15.16:0/115457909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 msgr2=0x55bef5720760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5720760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 -- 172.21.15.16:0/115457909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef53ef3c0 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd024640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 0x55bef5722f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7ffd825640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40057c0 tx=0x7f7ff4004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5435d4df637f4415 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.620+0000 7f7fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4014030 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff40047a0 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7ff4005540 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ff4004050 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 10 client.16599 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bef5731c20 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff40047a0 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7ff4010430 con 0x55bef5718dc0 2024-09-12T01:00:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 10 client.16599 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 20 client.16599 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef5733030 0x55bef5735420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 4 client.16599 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.621+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef568b850 con 0x55bef5733030 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.622+0000 7f7ffd024640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef5733030 0x55bef5735420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.622+0000 7f7ffd024640 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef5733030 0x55bef5735420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c2c84676cb5672 server_cookie=c1c1483a69d7e5ff in_seq=0 out_seq=0 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.622+0000 7f7fe17fa640 10 client.16599 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f7fe17fa640 1 -- 172.21.15.16:0/115457909 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bef568b850 con 0x55bef5733030 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f7fe17fa640 10 client.16599 handle_command_reply: tid=0 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 client.16599 shutdown 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 msgr2=0x55bef5720220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40057c0 tx=0x7f7ff4004ee0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 shutdown_connections 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef5733030 0x55bef5735420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5720760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef5718dc0 0x55bef5720220 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 --2- 172.21.15.16:0/115457909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef57184a0 0x55bef5722f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f7fe17fa640 0 client.16599 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 >> 172.21.15.16:0/115457909 conn(0x55bef5714a30 msgr2=0x55bef5713b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 shutdown_connections 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 1 -- 172.21.15.16:0/115457909 wait complete. 2024-09-12T01:00:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.623+0000 7f8001e62740 20 client.16599 trim_cache size 0 max 16384 2024-09-12T01:00:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 Processor -- start 2024-09-12T01:00:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 -- start start 2024-09-12T01:00:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef573abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef573b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81ad6c0 con 0x55bef57184a0 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81af200 con 0x55bef5723680 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f8001e62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81add90 con 0x55bef5718dc0 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34790/0 (socket says 172.21.15.16:34790) 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 -- 172.21.15.16:0/902824672 learned_addr learned my addr 172.21.15.16:0/902824672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd825640 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef573abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 -- 172.21.15.16:0/902824672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 msgr2=0x55bef573abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef573abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 -- 172.21.15.16:0/902824672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 msgr2=0x55bef573b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef573b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.641+0000 7f7ffd024640 1 -- 172.21.15.16:0/902824672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef5719a10 con 0x55bef57184a0 2024-09-12T01:00:25.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f7ffd024640 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bef5735b80 tx=0x7f7fe80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513469284af7791a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f7fe37fe640 1 -- 172.21.15.16:0/902824672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8005020 con 0x55bef57184a0 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f7fe37fe640 1 -- 172.21.15.16:0/902824672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe80047a0 con 0x55bef57184a0 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f7fe37fe640 1 -- 172.21.15.16:0/902824672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe801e620 con 0x55bef57184a0 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 -- 172.21.15.16:0/902824672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 msgr2=0x55bef572f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bef5735b80 tx=0x7f7fe80128e0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 -- 172.21.15.16:0/902824672 shutdown_connections 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef573b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef573abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 --2- 172.21.15.16:0/902824672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef572f4b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.642+0000 7f8001e62740 1 -- 172.21.15.16:0/902824672 >> 172.21.15.16:0/902824672 conn(0x55bef5714a30 msgr2=0x55bef57222a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.643+0000 7f8001e62740 1 -- 172.21.15.16:0/902824672 shutdown_connections 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.643+0000 7f8001e62740 1 -- 172.21.15.16:0/902824672 wait complete. 2024-09-12T01:00:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.643+0000 7f8001e62740 1 -- start start 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.643+0000 7f8001e62740 1 Processor -- start 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.643+0000 7f8001e62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef5728b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5729080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81aeb30 con 0x55bef57184a0 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81ae460 con 0x55bef5723680 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81ab6a0 con 0x55bef5718dc0 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 10 client.0 _ll_register_callbacks cb 0x7f7ff4008a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34800/0 (socket says 172.21.15.16:34800) 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 -- 172.21.15.16:0/3422467633 learned_addr learned my addr 172.21.15.16:0/3422467633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef5728b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 -- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 msgr2=0x55bef5728b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef5728b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 -- 172.21.15.16:0/3422467633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 msgr2=0x55bef5729080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5729080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 -- 172.21.15.16:0/3422467633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef579be00 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7ffd024640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8004770 tx=0x7f7fe8005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1472a76494ceb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8004b60 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7fe8004d00 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.644+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe80047a0 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fe801e3e0 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 10 client.16605 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bef579d110 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe80047a0 con 0x55bef57184a0 2024-09-12T01:00:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fe8010730 con 0x55bef57184a0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 10 client.16605 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 20 client.16605 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef579d7d0 0x55bef579fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 4 client.16605 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bef5685d60 con 0x55bef579d7d0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.645+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef579d7d0 0x55bef579fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f7ffd825640 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef579d7d0 0x55bef579fbc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0fa02fd17e41e1 server_cookie=b0d08f8b55c0a362 in_seq=0 out_seq=0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f7fe17fa640 10 client.16605 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f7fe17fa640 1 -- 172.21.15.16:0/3422467633 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bef5685d60 con 0x55bef579d7d0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f7fe17fa640 10 client.16605 handle_command_reply: tid=0 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 client.16605 shutdown 2024-09-12T01:00:25.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 msgr2=0x55bef5728600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8004770 tx=0x7f7fe8005320 comp rx=0 tx=0).stop 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 shutdown_connections 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bef579d7d0 0x55bef579fbc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bef5723680 0x55bef5729080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bef5718dc0 0x55bef5728b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 --2- 172.21.15.16:0/3422467633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bef57184a0 0x55bef5728600 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f7fe17fa640 0 client.16605 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 >> 172.21.15.16:0/3422467633 conn(0x55bef5714a30 msgr2=0x55bef5713350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 shutdown_connections 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 1 -- 172.21.15.16:0/3422467633 wait complete. 2024-09-12T01:00:25.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.646+0000 7f8001e62740 20 client.16605 trim_cache size 0 max 16384 2024-09-12T01:00:25.657 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:00:25.657 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:00:25.657 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 msgr2=0x7f7fe40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 0x7f7fe40a5230 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00ee510 tx=0x7f7ff008bd20 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 msgr2=0x7f7ff81a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009ca0 tx=0x7f7ff00039f0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 shutdown_connections 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fe40a2d70 0x7f7fe40a5230 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8106a80 0x7f7ff81a41b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ff8101420 0x7f7ff8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 --2- 172.21.15.16:0/347401071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ff8100a80 0x7f7ff8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 >> 172.21.15.16:0/347401071 conn(0x7f7ff80fc7a0 msgr2=0x7f7ff80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 shutdown_connections 2024-09-12T01:00:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:25.647+0000 7f7fffa89640 1 -- 172.21.15.16:0/347401071 wait complete. 2024-09-12T01:00:26.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2295151404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:26.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2295151404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:26.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2295151404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:26.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:00:26.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:26.997 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 3fdcd3ca --release --await' 2024-09-12T01:00:26.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 3fdcd3ca --release --await 2024-09-12T01:00:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:27.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:26 smithi016 ceph-mon[30972]: pgmap v134: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-12T01:00:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:26 smithi099 ceph-mon[35654]: pgmap v134: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-12T01:00:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:26 smithi159 ceph-mon[35566]: pgmap v134: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 -- 172.21.15.16:0/2035459243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 msgr2=0x7f591010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 --2- 172.21.15.16:0/2035459243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591010eaa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5908009ca0 tx=0x7f590802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 -- 172.21.15.16:0/2035459243 shutdown_connections 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 --2- 172.21.15.16:0/2035459243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591010eaa0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 --2- 172.21.15.16:0/2035459243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 0x7f59101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 --2- 172.21.15.16:0/2035459243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 0x7f59101021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 -- 172.21.15.16:0/2035459243 >> 172.21.15.16:0/2035459243 conn(0x7f59100fd8b0 msgr2=0x7f59100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 -- 172.21.15.16:0/2035459243 shutdown_connections 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.589+0000 7f5916860640 1 -- 172.21.15.16:0/2035459243 wait complete. 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.590+0000 7f5916860640 1 Processor -- start 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.590+0000 7f5916860640 1 -- start start 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.590+0000 7f5916860640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 0x7f59101984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 0x7f59101989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5910111f80 con 0x7f5910106f20 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5910111e00 con 0x7f5910101df0 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5910112100 con 0x7f59101027a0 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34822/0 (socket says 172.21.15.16:34822) 2024-09-12T01:00:27.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 -- 172.21.15.16:0/3793887807 learned_addr learned my addr 172.21.15.16:0/3793887807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f590ffff640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 0x7f59101984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f590f7fe640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 0x7f59101989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 -- 172.21.15.16:0/3793887807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 msgr2=0x7f59101989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 0x7f59101989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 -- 172.21.15.16:0/3793887807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 msgr2=0x7f59101984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 0x7f59101984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 -- 172.21.15.16:0/3793887807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591019d450 con 0x7f5910106f20 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5914dfd640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5908031a20 tx=0x7f5908031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908003c30 con 0x7f5910106f20 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5908003dd0 con 0x7f5910106f20 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.591+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591019d6e0 con 0x7f5910106f20 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.592+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590802fcf0 con 0x7f5910106f20 2024-09-12T01:00:27.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.592+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59101a5020 con 0x7f5910106f20 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.593+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59101021d0 con 0x7f5910106f20 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.594+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5908004050 con 0x7f5910106f20 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.594+0000 7f590d7fa640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 0x7f58fc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.595+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f59080f0580 con 0x7f5910106f20 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.595+0000 7f590ffff640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 0x7f58fc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.595+0000 7f590ffff640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 0x7f58fc0a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58f800bbe0 tx=0x7f58f800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.599+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59080bd490 con 0x7f5910106f20 2024-09-12T01:00:27.648 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.841+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f591010a160 con 0x7f5910106f20 2024-09-12T01:00:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.842+0000 7f590d7fa640 1 -- 172.21.15.16:0/3793887807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f590802fe90 con 0x7f5910106f20 2024-09-12T01:00:27.842 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:27.842 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 msgr2=0x7f58fc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 0x7f58fc0a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58f800bbe0 tx=0x7f58f800b390 comp rx=0 tx=0).stop 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 msgr2=0x7f591019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5908031a20 tx=0x7f5908031a50 comp rx=0 tx=0).stop 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 shutdown_connections 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58fc0a2d70 0x7f58fc0a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5910106f20 0x7f591019cd70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59101027a0 0x7f59101989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 --2- 172.21.15.16:0/3793887807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5910101df0 0x7f59101984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 >> 172.21.15.16:0/3793887807 conn(0x7f59100fd8b0 msgr2=0x7f59100febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 shutdown_connections 2024-09-12T01:00:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:27.844+0000 7f5916860640 1 -- 172.21.15.16:0/3793887807 wait complete. 2024-09-12T01:00:28.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:27 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3793887807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:28.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:27 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3793887807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:28.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:27 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3793887807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 -- 172.21.15.16:0/712481611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd53810c180 msgr2=0x7fd53810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/712481611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd53810c180 0x7fd53810e600 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd530009ca0 tx=0x7fd53002a950 comp rx=0 tx=0).stop 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 -- 172.21.15.16:0/712481611 shutdown_connections 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/712481611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd53810c180 0x7fd53810e600 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/712481611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5381036c0 0x7fd53810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/712481611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 0x7fd538103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.395+0000 7fd53d4d4640 1 -- 172.21.15.16:0/712481611 >> 172.21.15.16:0/712481611 conn(0x7fd5380fc7a0 msgr2=0x7fd5380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.396+0000 7fd53d4d4640 1 -- 172.21.15.16:0/712481611 shutdown_connections 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.396+0000 7fd53d4d4640 1 -- 172.21.15.16:0/712481611 wait complete. 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.396+0000 7fd53d4d4640 1 Processor -- start 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.396+0000 7fd53d4d4640 1 -- start start 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.396+0000 7fd53d4d4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 0x7fd538198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd53810c180 0x7fd53819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd538111fc0 con 0x7fd5381036c0 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd538111e40 con 0x7fd53810c180 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd538112140 con 0x7fd538102da0 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34838/0 (socket says 172.21.15.16:34838) 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 -- 172.21.15.16:0/3707589054 learned_addr learned my addr 172.21.15.16:0/3707589054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd536ffd640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 0x7fd538198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5377fe640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd53810c180 0x7fd53819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 -- 172.21.15.16:0/3707589054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 msgr2=0x7fd538198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 0x7fd538198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 -- 172.21.15.16:0/3707589054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd53810c180 msgr2=0x7fd53819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd53810c180 0x7fd53819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 -- 172.21.15.16:0/3707589054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53819d510 con 0x7fd5381036c0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd5367fc640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00b4a0 tx=0x7fd52c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd51ffff640 1 -- 172.21.15.16:0/3707589054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c014070 con 0x7fd5381036c0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd51ffff640 1 -- 172.21.15.16:0/3707589054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd52c010af0 con 0x7fd5381036c0 2024-09-12T01:00:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53819d800 con 0x7fd5381036c0 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd51ffff640 1 -- 172.21.15.16:0/3707589054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c017d10 con 0x7fd5381036c0 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.397+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381a50e0 con 0x7fd5381036c0 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd51ffff640 1 -- 172.21.15.16:0/3707589054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd52c017070 con 0x7fd5381036c0 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 Processor -- start 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 -- start start 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a342110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a344a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b72a298620 con 0x55b72a332f40 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b72a23fe50 con 0x55b72a333860 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b72a2945e0 con 0x55b72a342650 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34844/0 (socket says 172.21.15.16:34844) 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd5377fe640 1 -- 172.21.15.16:0/580672886 learned_addr learned my addr 172.21.15.16:0/580672886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.400+0000 7fd536ffd640 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a342110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5367fc640 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a344a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 -- 172.21.15.16:0/580672886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 msgr2=0x55b72a344a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a344a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 -- 172.21.15.16:0/580672886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 msgr2=0x55b72a342110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a342110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 -- 172.21.15.16:0/580672886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72a230d40 con 0x55b72a332f40 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd51ffff640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd5377fe640 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd530035000 tx=0x7fd5300098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4230183e3e500669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd536ffd640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd51ffff640 1 -- 172.21.15.16:0/3707589054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd52c0ce630 con 0x7fd5381036c0 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd51d7fa640 1 -- 172.21.15.16:0/580672886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530048070 con 0x55b72a332f40 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd51d7fa640 1 -- 172.21.15.16:0/580672886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53002fa80 con 0x55b72a332f40 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.401+0000 7fd51d7fa640 1 -- 172.21.15.16:0/580672886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530034040 con 0x55b72a332f40 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- 172.21.15.16:0/580672886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 msgr2=0x55b72a333320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd530035000 tx=0x7fd5300098d0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd536ffd640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 0x7fd5200a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55b72a335c50 tx=0x7fd52800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- 172.21.15.16:0/580672886 shutdown_connections 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a344a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a342110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/580672886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a333320 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- 172.21.15.16:0/580672886 >> 172.21.15.16:0/580672886 conn(0x55b72a32f5f0 msgr2=0x55b72a32f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- 172.21.15.16:0/580672886 shutdown_connections 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- 172.21.15.16:0/580672886 wait complete. 2024-09-12T01:00:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 -- start start 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.402+0000 7fd53f8ad740 1 Processor -- start 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 0x55b72a33f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a333860 0x55b72a33fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b72a345220 con 0x55b72a342650 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b72a220f10 con 0x55b72a332f40 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b72a3453a0 con 0x55b72a333860 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd53f8ad740 10 client.0 _ll_register_callbacks cb 0x55b72a244db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45606/0 (socket says 172.21.15.16:45606) 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 -- 172.21.15.16:0/1182269798 learned_addr learned my addr 172.21.15.16:0/1182269798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 0x55b72a33f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5367fc640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a333860 0x55b72a33fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 -- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a333860 msgr2=0x55b72a33fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a333860 0x55b72a33fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 -- 172.21.15.16:0/1182269798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 msgr2=0x55b72a33f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 0x55b72a33f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 -- 172.21.15.16:0/1182269798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b729fa7580 con 0x55b72a342650 2024-09-12T01:00:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 0x55b72a33f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd5377fe640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd530005cc0 tx=0x7fd53002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3782afdbec2302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.403+0000 7fd51aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530048030 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd53002fa80 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd530040050 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd530034040 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 10 client.16623 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b72a071960 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53002fa80 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd530030670 con 0x55b72a342650 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 10 client.16623 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 10 client.16623 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 20 client.16623 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a352310 0x55b72a354700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 4 client.16623 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72a2a5b90 con 0x55b72a352310 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.404+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a352310 0x55b72a354700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.405+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a352310 0x55b72a354700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e0973a2e51bb19 server_cookie=366cbf3fe14dc3a0 in_seq=0 out_seq=0 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.405+0000 7fd51aff5640 10 client.16623 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.405+0000 7fd51aff5640 1 -- 172.21.15.16:0/1182269798 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b72a2a5b90 con 0x55b72a352310 2024-09-12T01:00:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.405+0000 7fd51aff5640 10 client.16623 handle_command_reply: tid=0 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 client.16623 shutdown 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 msgr2=0x55b72a340040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd530005cc0 tx=0x7fd53002f650 comp rx=0 tx=0).stop 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 shutdown_connections 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a352310 0x55b72a354700 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a342650 0x55b72a340040 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a333860 0x55b72a33fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1182269798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a332f40 0x55b72a33f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 >> 172.21.15.16:0/1182269798 conn(0x55b72a32f5f0 msgr2=0x55b72a335350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd51aff5640 0 client.16623 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 shutdown_connections 2024-09-12T01:00:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1182269798 wait complete. 2024-09-12T01:00:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.406+0000 7fd53f8ad740 20 client.16623 trim_cache size 0 max 16384 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 Processor -- start 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 -- start start 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a335160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a359fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381ab560 con 0x55b72a332f40 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381ad0a0 con 0x55b72a333860 2024-09-12T01:00:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381abc30 con 0x55b72a342650 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45616/0 (socket says 172.21.15.16:45616) 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.423+0000 7fd536ffd640 1 -- 172.21.15.16:0/274955671 learned_addr learned my addr 172.21.15.16:0/274955671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd5377fe640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a359fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd5367fc640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a335160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 -- 172.21.15.16:0/274955671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 msgr2=0x55b72a359fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a359fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 -- 172.21.15.16:0/274955671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 msgr2=0x55b72a335160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a335160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 -- 172.21.15.16:0/274955671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72a333690 con 0x55b72a332f40 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd5367fc640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a335160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd536ffd640 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55b72a354f70 tx=0x7fd528016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f7447e6987da89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd51cff9640 1 -- 172.21.15.16:0/274955671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52801ee40 con 0x55b72a332f40 2024-09-12T01:00:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd51cff9640 1 -- 172.21.15.16:0/274955671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd528003f20 con 0x55b72a332f40 2024-09-12T01:00:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.424+0000 7fd51cff9640 1 -- 172.21.15.16:0/274955671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52802a680 con 0x55b72a332f40 2024-09-12T01:00:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- 172.21.15.16:0/274955671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 msgr2=0x55b72a33de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55b72a354f70 tx=0x7fd528016000 comp rx=0 tx=0).stop 2024-09-12T01:00:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- 172.21.15.16:0/274955671 shutdown_connections 2024-09-12T01:00:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a359fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a335160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/274955671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a33de90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- 172.21.15.16:0/274955671 >> 172.21.15.16:0/274955671 conn(0x55b72a32f5f0 msgr2=0x55b72a334a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- 172.21.15.16:0/274955671 shutdown_connections 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- 172.21.15.16:0/274955671 wait complete. 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 -- start start 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.425+0000 7fd53f8ad740 1 Processor -- start 2024-09-12T01:00:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a346910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a346e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381ac9d0 con 0x55b72a332f40 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381ac300 con 0x55b72a333860 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381a9560 con 0x55b72a342650 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 10 client.0 _ll_register_callbacks cb 0x7fd530009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45628/0 (socket says 172.21.15.16:45628) 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 -- 172.21.15.16:0/1186350222 learned_addr learned my addr 172.21.15.16:0/1186350222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd5367fc640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a346910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd5377fe640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a346e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 -- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 msgr2=0x55b72a346e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a346e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 -- 172.21.15.16:0/1186350222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 msgr2=0x55b72a346910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a346910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 -- 172.21.15.16:0/1186350222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72a3bbff0 con 0x55b72a332f40 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd5367fc640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a346910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd536ffd640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd528018040 tx=0x7fd52800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e77824e5677b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd51aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52802a420 con 0x55b72a332f40 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd52802a5c0 con 0x55b72a332f40 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.426+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd528003f20 con 0x55b72a332f40 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd52802a7c0 con 0x55b72a332f40 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 10 client.16629 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b72a3bd280 con 0x55b72a332f40 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd528003f20 con 0x55b72a332f40 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd52803dc40 con 0x55b72a332f40 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 10 client.16629 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 10 client.16629 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 20 client.16629 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a3bd910 0x55b72a3bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 4 client.16629 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3fdcd3ca", "release": true, "await": true}] 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3fdcd3ca", "release": true, "await": true}) -- 0x55b72a2a00a0 con 0x55b72a3bd910 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.427+0000 7fd5367fc640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a3bd910 0x55b72a3bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.428+0000 7fd51aff5640 10 client.16629 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.428+0000 7fd5367fc640 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a3bd910 0x55b72a3bfd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c180f09aea83e6d server_cookie=7344727e5861928c in_seq=0 out_seq=0 2024-09-12T01:00:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd51aff5640 1 -- 172.21.15.16:0/1186350222 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b72a2a00a0 con 0x55b72a3bd910 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd51aff5640 10 client.16629 handle_command_reply: tid=0 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 client.16629 shutdown 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 msgr2=0x55b72a3463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd528018040 tx=0x7fd52800bf20 comp rx=0 tx=0).stop 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 shutdown_connections 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b72a3bd910 0x55b72a3bfd30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b72a342650 0x55b72a346e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b72a333860 0x55b72a346910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd51aff5640 0 client.16629 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 --2- 172.21.15.16:0/1186350222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b72a332f40 0x55b72a3463d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 >> 172.21.15.16:0/1186350222 conn(0x55b72a32f5f0 msgr2=0x55b72a32e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 shutdown_connections 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 1 -- 172.21.15.16:0/1186350222 wait complete. 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.431+0000 7fd53f8ad740 20 client.16629 trim_cache size 0 max 16384 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 16, 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:00:28.435 INFO:teuthology.orchestra.run.smithi016.stdout: "3fdcd3ca": { 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 16, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:00:28.436 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:28.437 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 msgr2=0x7fd5200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 0x7fd5200a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55b72a335c50 tx=0x7fd52800b390 comp rx=0 tx=0).stop 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 msgr2=0x7fd538198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00b4a0 tx=0x7fd52c00b960 comp rx=0 tx=0).stop 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 shutdown_connections 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5200a2cf0 0x7fd5200a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd53810c180 0x7fd53819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5381036c0 0x7fd538198aa0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 --2- 172.21.15.16:0/3707589054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd538102da0 0x7fd538198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 >> 172.21.15.16:0/3707589054 conn(0x7fd5380fc7a0 msgr2=0x7fd5380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 shutdown_connections 2024-09-12T01:00:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:28.432+0000 7fd53d4d4640 1 -- 172.21.15.16:0/3707589054 wait complete. 2024-09-12T01:00:29.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:00:29.252 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:28 smithi016 ceph-mon[30972]: pgmap v135: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T01:00:29.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:28 smithi099 ceph-mon[35654]: pgmap v135: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T01:00:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:28 smithi159 ceph-mon[35566]: pgmap v135: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T01:00:29.844 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3fdcd3ca', total seconds elapsed: 27.0 2024-09-12T01:00:29.844 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T01:00:29.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 -- 172.21.15.16:0/1556675184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c10c2f0 msgr2=0x7f049c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 --2- 172.21.15.16:0/1556675184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c10c2f0 0x7f049c10e790 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0494009ca0 tx=0x7f049402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 -- 172.21.15.16:0/1556675184 shutdown_connections 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 --2- 172.21.15.16:0/1556675184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c10c2f0 0x7f049c10e790 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 --2- 172.21.15.16:0/1556675184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c1037a0 0x7f049c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 --2- 172.21.15.16:0/1556675184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c102df0 0x7f049c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 -- 172.21.15.16:0/1556675184 >> 172.21.15.16:0/1556675184 conn(0x7f049c0fc7d0 msgr2=0x7f049c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 -- 172.21.15.16:0/1556675184 shutdown_connections 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.453+0000 7f04a129e640 1 -- 172.21.15.16:0/1556675184 wait complete. 2024-09-12T01:00:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 Processor -- start 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 -- start start 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c102df0 0x7f049c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c10c2f0 0x7f049c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1120f0 con 0x7f049c1037a0 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f049c111f70 con 0x7f049c102df0 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04a129e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f049c112270 con 0x7f049c10c2f0 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45656/0 (socket says 172.21.15.16:45656) 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f04927fc640 1 -- 172.21.15.16:0/1390869516 learned_addr learned my addr 172.21.15.16:0/1390869516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.454+0000 7f049affd640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c102df0 0x7f049c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f049b7fe640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c10c2f0 0x7f049c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 -- 172.21.15.16:0/1390869516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c10c2f0 msgr2=0x7f049c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c10c2f0 0x7f049c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 -- 172.21.15.16:0/1390869516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c102df0 msgr2=0x7f049c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c102df0 0x7f049c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 -- 172.21.15.16:0/1390869516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c19d500 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04927fc640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f0498ff9640 1 -- 172.21.15.16:0/1390869516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048400ccc0 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f0498ff9640 1 -- 172.21.15.16:0/1390869516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0484010910 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c19d7f0 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f0498ff9640 1 -- 172.21.15.16:0/1390869516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840188a0 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.455+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c1a50d0 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.457+0000 7f0498ff9640 1 -- 172.21.15.16:0/1390869516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0484010ac0 con 0x7f049c1037a0 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 Processor -- start 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 -- start start 2024-09-12T01:00:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 0x55f5a3ae8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 0x55f5a3ae8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f0498ff9640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 0x7f04880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3a4e2f0 con 0x55f5a3ae85d0 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f5a39f5cf0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.458+0000 7f04a3677740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3a4a2b0 con 0x55f5a3af3370 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049affd640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 0x7f04880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 0x55f5a3ae8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 0x55f5a3ae8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:58444/0 (socket says 172.21.15.16:58444) 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049b7fe640 1 -- 172.21.15.16:0/1342972215 learned_addr learned my addr 172.21.15.16:0/1342972215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f0498ff9640 1 -- 172.21.15.16:0/1390869516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f04840cf770 con 0x7f049c1037a0 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049b7fe640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 0x55f5a3ae8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049affd640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 0x7f04880a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f048c00abe0 tx=0x7f048c00a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 -- 172.21.15.16:0/1342972215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 msgr2=0x55f5a3af5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 -- 172.21.15.16:0/1342972215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 msgr2=0x55f5a3ae8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 0x55f5a3ae8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 -- 172.21.15.16:0/1342972215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a39da3a0 con 0x55f5a3ae85d0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f049b7fe640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.459+0000 7f04927fc640 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 0x55f5a3ae8a10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f04840ccef0 tx=0x7f0484009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39aa5b66d82bd9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04917fa640 1 -- 172.21.15.16:0/1342972215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484099e00 con 0x55f5a3ae85d0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04917fa640 1 -- 172.21.15.16:0/1342972215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048409c3c0 con 0x55f5a3ae85d0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04917fa640 1 -- 172.21.15.16:0/1342972215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484099310 con 0x55f5a3ae85d0 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 -- 172.21.15.16:0/1342972215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 msgr2=0x55f5a3ae8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 0x55f5a3ae8a10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f04840ccef0 tx=0x7f0484009740 comp rx=0 tx=0).stop 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 -- 172.21.15.16:0/1342972215 shutdown_connections 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae85d0 0x55f5a3ae8a10 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 --2- 172.21.15.16:0/1342972215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae9b40 0x55f5a3ae8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 -- 172.21.15.16:0/1342972215 >> 172.21.15.16:0/1342972215 conn(0x55f5a3ab1310 msgr2=0x55f5a3ae4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 -- 172.21.15.16:0/1342972215 shutdown_connections 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.460+0000 7f04a3677740 1 -- 172.21.15.16:0/1342972215 wait complete. 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 -- start start 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 Processor -- start 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3af5fb0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3a09060 con 0x55f5a3ae85d0 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.461+0000 7f04a3677740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3af6130 con 0x55f5a3af3370 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04a3677740 10 client.0 _ll_register_callbacks cb 0x55f5a375c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45678/0 (socket says 172.21.15.16:45678) 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 -- 172.21.15.16:0/1461191912 learned_addr learned my addr 172.21.15.16:0/1461191912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f049affd640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f049b7fe640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 -- 172.21.15.16:0/1461191912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 msgr2=0x55f5a3af0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 -- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 msgr2=0x55f5a3af2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 -- 172.21.15.16:0/1461191912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a37bf3c0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f049affd640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04927fc640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f048409b9c0 tx=0x7f04840096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60207470e4f8e4dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04827fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840994f0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0484099d60 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f048409c3c0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.462+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04840a9050 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 10 client.16641 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5a3b01a00 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048409c3c0 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f04840a4b80 con 0x55f5a3ae9b40 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 10 client.16641 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 10 client.16641 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 20 client.16641 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b02d50 0x55f5a3b05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 4 client.16641 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a3a5b860 con 0x55f5a3b02d50 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.463+0000 7f049affd640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b02d50 0x55f5a3b05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f049affd640 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b02d50 0x55f5a3b05140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=952c0c2628f5598a server_cookie=b9c3ee67f5a4edcd in_seq=0 out_seq=0 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04827fc640 10 client.16641 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04827fc640 1 -- 172.21.15.16:0/1461191912 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5a3a5b860 con 0x55f5a3b02d50 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04827fc640 10 client.16641 handle_command_reply: tid=0 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04a3677740 1 client.16641 shutdown 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 msgr2=0x55f5a3aeffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.464+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f048409b9c0 tx=0x7f04840096a0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 shutdown_connections 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b02d50 0x55f5a3b05140 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3af3370 0x55f5a3af0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3ae9b40 0x55f5a3aeffd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04827fc640 0 client.16641 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 --2- 172.21.15.16:0/1461191912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 >> 172.21.15.16:0/1461191912 conn(0x55f5a3ab1310 msgr2=0x55f5a3aea990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 shutdown_connections 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 1 -- 172.21.15.16:0/1461191912 wait complete. 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.465+0000 7f04a3677740 20 client.16641 trim_cache size 0 max 16384 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 Processor -- start 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 -- start start 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.494 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b08160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1ad790 con 0x55f5a3af3370 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1af2d0 con 0x55f5a3ae85d0 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.478+0000 7f04a3677740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1ade60 con 0x55f5a3ae9b40 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45688/0 (socket says 172.21.15.16:45688) 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 -- 172.21.15.16:0/308975864 learned_addr learned my addr 172.21.15.16:0/308975864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:30.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f04927fc640 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b08160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049affd640 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 -- 172.21.15.16:0/308975864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 msgr2=0x55f5a3b08160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b08160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 -- 172.21.15.16:0/308975864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 msgr2=0x55f5a3af1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 -- 172.21.15.16:0/308975864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a3aee520 con 0x55f5a3af3370 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f049b7fe640 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0494009ca0 tx=0x7f049402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5628b2d53b485131 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f0490ff9640 1 -- 172.21.15.16:0/308975864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494031650 con 0x55f5a3af3370 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f0490ff9640 1 -- 172.21.15.16:0/308975864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494036040 con 0x55f5a3af3370 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.479+0000 7f0490ff9640 1 -- 172.21.15.16:0/308975864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494030820 con 0x55f5a3af3370 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 -- 172.21.15.16:0/308975864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 msgr2=0x55f5a3b086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0494009ca0 tx=0x7f049402f470 comp rx=0 tx=0).stop 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 -- 172.21.15.16:0/308975864 shutdown_connections 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b086a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b08160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 --2- 172.21.15.16:0/308975864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3af1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 -- 172.21.15.16:0/308975864 >> 172.21.15.16:0/308975864 conn(0x55f5a3ab1310 msgr2=0x55f5a3af1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 -- 172.21.15.16:0/308975864 shutdown_connections 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.480+0000 7f04a3677740 1 -- 172.21.15.16:0/308975864 wait complete. 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 -- start start 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 Processor -- start 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3b08890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1aec00 con 0x55f5a3af3370 2024-09-12T01:00:30.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1ae530 con 0x55f5a3ae85d0 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1ab770 con 0x55f5a3ae9b40 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04a3677740 10 client.0 _ll_register_callbacks cb 0x7f048409b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45696/0 (socket says 172.21.15.16:45696) 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 -- 172.21.15.16:0/3158890676 learned_addr learned my addr 172.21.15.16:0/3158890676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049affd640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3b08890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04927fc640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 -- 172.21.15.16:0/3158890676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 msgr2=0x55f5a3b05be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b05be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 -- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 msgr2=0x55f5a3b08890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3b08890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f049b7fe640 1 -- 172.21.15.16:0/3158890676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a3b06920 con 0x55f5a3af3370 2024-09-12T01:00:30.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.481+0000 7f04927fc640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f049b7fe640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f049402ade0 tx=0x7f0494003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ee0901bb6bac3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494004d50 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0494036040 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f0494031450 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0494031650 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04a3677740 10 client.16647 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f5a3b6d660 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494036040 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.482+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0494004040 con 0x55f5a3af3370 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 10 client.16647 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 10 client.16647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:00:30.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 20 client.16647 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b6dd20 0x55f5a3b70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 4 client.16647 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5a3a55d70 con 0x55f5a3b6dd20 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f049affd640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b6dd20 0x55f5a3b70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04827fc640 10 client.16647 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f049affd640 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b6dd20 0x55f5a3b70110 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5858b4beee26f49 server_cookie=d6d72d503666b7a2 in_seq=0 out_seq=0 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04827fc640 1 -- 172.21.15.16:0/3158890676 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5a3a55d70 con 0x55f5a3b6dd20 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04827fc640 10 client.16647 handle_command_reply: tid=0 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.483+0000 7f04a3677740 1 client.16647 shutdown 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 msgr2=0x55f5a3b06120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f049402ade0 tx=0x7f0494003e80 comp rx=0 tx=0).stop 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 shutdown_connections 2024-09-12T01:00:30.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f5a3b6dd20 0x55f5a3b70110 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a3af3370 0x55f5a3b06120 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5a3ae9b40 0x55f5a3b05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 --2- 172.21.15.16:0/3158890676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f5a3ae85d0 0x55f5a3b08890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04827fc640 0 client.16647 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 >> 172.21.15.16:0/3158890676 conn(0x55f5a3ab1310 msgr2=0x55f5a3ae3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 shutdown_connections 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 1 -- 172.21.15.16:0/3158890676 wait complete. 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a3677740 20 client.16647 trim_cache size 0 max 16384 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 msgr2=0x7f04880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 0x7f04880a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f048c00abe0 tx=0x7f048c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 msgr2=0x7f049c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.484+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400edf0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 shutdown_connections 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04880a2cf0 0x7f04880a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f049c10c2f0 0x7f049c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f049c1037a0 0x7f049c198a90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 --2- 172.21.15.16:0/1390869516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f049c102df0 0x7f049c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 >> 172.21.15.16:0/1390869516 conn(0x7f049c0fc7d0 msgr2=0x7f049c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 shutdown_connections 2024-09-12T01:00:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:30.485+0000 7f04a129e640 1 -- 172.21.15.16:0/1390869516 wait complete. 2024-09-12T01:00:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:30 smithi016 ceph-mon[30972]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-12T01:00:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:30 smithi099 ceph-mon[35654]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-12T01:00:31.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:00:31.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:30 smithi159 ceph-mon[35566]: pgmap v136: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-09-12T01:00:31.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 -- 172.21.15.16:0/1166941867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 msgr2=0x7f959810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:31.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 --2- 172.21.15.16:0/1166941867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810d950 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9590009cd0 tx=0x7f959002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 -- 172.21.15.16:0/1166941867 shutdown_connections 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 --2- 172.21.15.16:0/1166941867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810d950 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 --2- 172.21.15.16:0/1166941867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 0x7f9598100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 --2- 172.21.15.16:0/1166941867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 0x7f9598106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 -- 172.21.15.16:0/1166941867 >> 172.21.15.16:0/1166941867 conn(0x7f95980fc7a0 msgr2=0x7f95980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 -- 172.21.15.16:0/1166941867 shutdown_connections 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.910+0000 7f959eb31640 1 -- 172.21.15.16:0/1166941867 wait complete. 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 Processor -- start 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 -- start start 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 0x7f959810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 0x7f959810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:31.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111fa0 con 0x7f9598101420 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111e20 con 0x7f9598100a80 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959eb31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9598112120 con 0x7f9598106a80 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f9597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f9597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45708/0 (socket says 172.21.15.16:45708) 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f9597fff640 1 -- 172.21.15.16:0/2331630530 learned_addr learned my addr 172.21.15.16:0/2331630530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959d0ce640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 0x7f959810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.911+0000 7f959c8cd640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 0x7f959810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 -- 172.21.15.16:0/2331630530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 msgr2=0x7f959810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 0x7f959810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 -- 172.21.15.16:0/2331630530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 msgr2=0x7f959810e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 0x7f959810e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:31.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 -- 172.21.15.16:0/2331630530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95981a68e0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9597fff640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f958000d950 tx=0x7f958000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580017070 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9580009c30 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958000f6b0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981a6bd0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.912+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a70c0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.913+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9598100ee0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.914+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95800109d0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.915+0000 7f9595ffb640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.915+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f95800cefb0 con 0x7f9598101420 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.915+0000 7f959c8cd640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.915+0000 7f959c8cd640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 0x7f95840a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9588005fd0 tx=0x7f958800b510 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:31.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:31.919+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9580009de0 con 0x7f9598101420 2024-09-12T01:00:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.139+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f959802d7c0 con 0x7f9598101420 2024-09-12T01:00:32.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.140+0000 7f9595ffb640 1 -- 172.21.15.16:0/2331630530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f958009bc00 con 0x7f9598101420 2024-09-12T01:00:32.140 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:32.140 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:32.140 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:32.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 msgr2=0x7f95840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 0x7f95840a5230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9588005fd0 tx=0x7f958800b510 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 msgr2=0x7f959810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f958000d950 tx=0x7f958000de10 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 shutdown_connections 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598106a80 0x7f959810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9598101420 0x7f959810e920 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 --2- 172.21.15.16:0/2331630530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9598100a80 0x7f959810e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 >> 172.21.15.16:0/2331630530 conn(0x7f95980fc7a0 msgr2=0x7f95980fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 shutdown_connections 2024-09-12T01:00:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:32.142+0000 7f959eb31640 1 -- 172.21.15.16:0/2331630530 wait complete. 2024-09-12T01:00:33.049 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T01:00:33.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T01:00:33.213 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:32 smithi016 ceph-mon[30972]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:33.213 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2331630530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:32 smithi099 ceph-mon[35654]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:32 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2331630530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:33.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.300+0000 7f955cb26640 1 -- 172.21.15.16:0/413349374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558100e20 msgr2=0x7f9558101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.300+0000 7f955cb26640 1 --2- 172.21.15.16:0/413349374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558100e20 0x7f9558101200 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9548009900 tx=0x7f954802a480 comp rx=0 tx=0).stop 2024-09-12T01:00:33.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 -- 172.21.15.16:0/413349374 shutdown_connections 2024-09-12T01:00:33.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 --2- 172.21.15.16:0/413349374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102ef0 0x7f955810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 --2- 172.21.15.16:0/413349374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558102570 0x7f95581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 --2- 172.21.15.16:0/413349374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558100e20 0x7f9558101200 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 -- 172.21.15.16:0/413349374 >> 172.21.15.16:0/413349374 conn(0x7f95580fc7d0 msgr2=0x7f95580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 -- 172.21.15.16:0/413349374 shutdown_connections 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 -- 172.21.15.16:0/413349374 wait complete. 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 Processor -- start 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.301+0000 7f955cb26640 1 -- start start 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 0x7f9558199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 0x7f95581998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9558110ff0 con 0x7f9558102ef0 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9558110e70 con 0x7f9558100e20 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955cb26640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9558111170 con 0x7f9558102570 2024-09-12T01:00:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45722/0 (socket says 172.21.15.16:45722) 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 -- 172.21.15.16:0/2501345937 learned_addr learned my addr 172.21.15.16:0/2501345937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955659c640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 0x7f9558199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9555d9b640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 0x7f95581998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 -- 172.21.15.16:0/2501345937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 msgr2=0x7f95581998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 0x7f95581998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 -- 172.21.15.16:0/2501345937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 msgr2=0x7f9558199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 0x7f9558199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 -- 172.21.15.16:0/2501345937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95581a4950 con 0x7f9558102ef0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f955659c640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 0x7f9558199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9555d9b640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 0x7f95581998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f9556d9d640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f955000b970 tx=0x7f955000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.302+0000 7f954f7fe640 1 -- 172.21.15.16:0/2501345937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550019070 con 0x7f9558102ef0 2024-09-12T01:00:33.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.303+0000 7f954f7fe640 1 -- 172.21.15.16:0/2501345937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955000cb40 con 0x7f9558102ef0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.303+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95581a4b20 con 0x7f9558102ef0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.303+0000 7f954f7fe640 1 -- 172.21.15.16:0/2501345937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550007650 con 0x7f9558102ef0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.303+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95581a4fe0 con 0x7f9558102ef0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.305+0000 7f954f7fe640 1 -- 172.21.15.16:0/2501345937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f955001c070 con 0x7f9558102ef0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.305+0000 7f955eeff740 1 Processor -- start 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.305+0000 7f955eeff740 1 -- start start 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49250090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49261b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc491b62f0 con 0x55dc492505d0 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc4915dcf0 con 0x55dc49251b40 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955eeff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dc491b22b0 con 0x55dc4925f780 2024-09-12T01:00:33.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 -- 172.21.15.16:0/1500067505 learned_addr learned my addr 172.21.15.16:0/1500067505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f954f7fe640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 0x7f95440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f9555d9b640 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49250090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f954f7fe640 1 -- 172.21.15.16:0/2501345937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f95500ce9b0 con 0x7f9558102ef0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 -- 172.21.15.16:0/1500067505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 msgr2=0x55dc49261b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f9556d9d640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 0x7f95440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49261b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 -- 172.21.15.16:0/1500067505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 msgr2=0x55dc49250090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49250090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.306+0000 7f955659c640 1 -- 172.21.15.16:0/1500067505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc491423a0 con 0x55dc492505d0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.307+0000 7f955659c640 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f95480314e0 tx=0x7f9548004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b51c7afbee7ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.307+0000 7f9556d9d640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 0x7f95440a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95500979e0 tx=0x7f9550014d00 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.307+0000 7f954cff9640 1 -- 172.21.15.16:0/1500067505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954803f070 con 0x55dc492505d0 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.307+0000 7f954cff9640 1 -- 172.21.15.16:0/1500067505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9548003cd0 con 0x55dc492505d0 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.307+0000 7f954cff9640 1 -- 172.21.15.16:0/1500067505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548035070 con 0x55dc492505d0 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- 172.21.15.16:0/1500067505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 msgr2=0x55dc49250a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f95480314e0 tx=0x7f9548004390 comp rx=0 tx=0).stop 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- 172.21.15.16:0/1500067505 shutdown_connections 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49261b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49250a10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 --2- 172.21.15.16:0/1500067505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49250090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- 172.21.15.16:0/1500067505 >> 172.21.15.16:0/1500067505 conn(0x55dc49219310 msgr2=0x55dc4924cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- 172.21.15.16:0/1500067505 shutdown_connections 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- 172.21.15.16:0/1500067505 wait complete. 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 -- start start 2024-09-12T01:00:33.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.308+0000 7f955eeff740 1 Processor -- start 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 0x55dc4925c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc4925f780 0x55dc4925c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dc49262420 con 0x55dc492505d0 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dc49171060 con 0x55dc4925f780 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dc492625a0 con 0x55dc49251b40 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 10 client.0 _ll_register_callbacks cb 0x55dc48ec48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45732/0 (socket says 172.21.15.16:45732) 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 -- 172.21.15.16:0/1828260441 learned_addr learned my addr 172.21.15.16:0/1828260441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955659c640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 0x55dc4925c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 -- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 msgr2=0x55dc4925c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 0x55dc4925c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 -- 172.21.15.16:0/1828260441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc4925f780 msgr2=0x55dc4925c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc4925f780 0x55dc4925c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 -- 172.21.15.16:0/1828260441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc48f273c0 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955659c640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 0x55dc4925c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f9555d9b640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f954000bee0 tx=0x7f9540012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c78c09a65ab5af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954000eae0 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f954000ec80 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.309+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9548003cd0 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f954001b470 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 10 client.16665 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc4926df20 con 0x55dc492505d0 2024-09-12T01:00:33.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9548003cd0 con 0x55dc492505d0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f954000f780 con 0x55dc492505d0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 10 client.16665 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 10 client.16665 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 20 client.16665 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc4926f3e0 0x55dc492717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 4 client.16665 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc491c3860 con 0x55dc4926f3e0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.310+0000 7f955659c640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc4926f3e0 0x55dc492717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.311+0000 7f953dffb640 10 client.16665 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.311+0000 7f955659c640 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc4926f3e0 0x55dc492717f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5eb5bbc75fa0c5ea server_cookie=578f4682208047fd in_seq=0 out_seq=0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f953dffb640 1 -- 172.21.15.16:0/1828260441 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc491c3860 con 0x55dc4926f3e0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f953dffb640 10 client.16665 handle_command_reply: tid=0 2024-09-12T01:00:33.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 client.16665 shutdown 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 msgr2=0x55dc4925f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f954000bee0 tx=0x7f9540012e10 comp rx=0 tx=0).stop 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 shutdown_connections 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc4925f780 0x55dc4925c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc49251b40 0x55dc4925c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc4926f3e0 0x55dc492717f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 --2- 172.21.15.16:0/1828260441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc4925f150 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f953dffb640 0 client.16665 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 >> 172.21.15.16:0/1828260441 conn(0x55dc49219310 msgr2=0x55dc49252930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 shutdown_connections 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 1 -- 172.21.15.16:0/1828260441 wait complete. 2024-09-12T01:00:33.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.312+0000 7f955eeff740 20 client.16665 trim_cache size 0 max 16384 2024-09-12T01:00:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 Processor -- start 2024-09-12T01:00:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 -- start start 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 0x55dc4926b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 0x55dc492770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95581ab5e0 con 0x55dc4925f780 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95581ad120 con 0x55dc49251b40 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955eeff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95581abcb0 con 0x55dc492505d0 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 0x55dc492770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.325+0000 7f955659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:58538/0 (socket says 172.21.15.16:58538) 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 -- 172.21.15.16:0/572476174 learned_addr learned my addr 172.21.15.16:0/572476174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 0x55dc492770c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45734/0 (socket says 172.21.15.16:45734) 2024-09-12T01:00:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f9555d9b640 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 0x55dc4926b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 -- 172.21.15.16:0/572476174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 msgr2=0x55dc4926b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 0x55dc4926b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 -- 172.21.15.16:0/572476174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 msgr2=0x55dc492770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 0x55dc492770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 -- 172.21.15.16:0/572476174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc49251300 con 0x55dc49251b40 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f9555d9b640 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 0x55dc4926b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f955659c640 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9548009900 tx=0x7f95480317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88757a1b0f737452 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f953ffff640 1 -- 172.21.15.16:0/572476174 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548052070 con 0x55dc49251b40 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f953ffff640 1 -- 172.21.15.16:0/572476174 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9548031d90 con 0x55dc49251b40 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.326+0000 7f953ffff640 1 -- 172.21.15.16:0/572476174 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954803f070 con 0x55dc49251b40 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- 172.21.15.16:0/572476174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 msgr2=0x55dc49252820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9548009900 tx=0x7f95480317c0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- 172.21.15.16:0/572476174 shutdown_connections 2024-09-12T01:00:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc4925f780 0x55dc492770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49252820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 --2- 172.21.15.16:0/572476174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc492505d0 0x55dc4926b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- 172.21.15.16:0/572476174 >> 172.21.15.16:0/572476174 conn(0x55dc49219310 msgr2=0x55dc49252110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- 172.21.15.16:0/572476174 shutdown_connections 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- 172.21.15.16:0/572476174 wait complete. 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.327+0000 7f955eeff740 1 -- start start 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 Processor -- start 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49267da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49264830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95581aca50 con 0x55dc492505d0 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95581ac380 con 0x55dc49251b40 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95581a95c0 con 0x55dc4925f780 2024-09-12T01:00:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955eeff740 10 client.0 _ll_register_callbacks cb 0x7f954000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45742/0 (socket says 172.21.15.16:45742) 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 -- 172.21.15.16:0/2520074349 learned_addr learned my addr 172.21.15.16:0/2520074349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 -- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 msgr2=0x55dc49264830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49264830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f955659c640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49267da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 -- 172.21.15.16:0/2520074349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 msgr2=0x55dc49267da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49267da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 -- 172.21.15.16:0/2520074349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc49264d70 con 0x55dc492505d0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f9555d9b640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f954000fbd0 tx=0x7f954001bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c817585648f6e897 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.328+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540028040 con 0x55dc492505d0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f954000f6e0 con 0x55dc492505d0 2024-09-12T01:00:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9548031d90 con 0x55dc492505d0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f954000f900 con 0x55dc492505d0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 10 client.16671 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc492da840 con 0x55dc492505d0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9548031d90 con 0x55dc492505d0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f954002b070 con 0x55dc492505d0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 10 client.16671 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 10 client.16671 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 20 client.16671 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc492daf60 0x55dc492dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 4 client.16671 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.329+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc491bdd70 con 0x55dc492daf60 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955659c640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc492daf60 0x55dc492dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955659c640 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc492daf60 0x55dc492dd350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc8a6da65622b437 server_cookie=a6588fd5e736cdae in_seq=0 out_seq=0 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f953dffb640 10 client.16671 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f953dffb640 1 -- 172.21.15.16:0/2520074349 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc491bdd70 con 0x55dc492daf60 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f953dffb640 10 client.16671 handle_command_reply: tid=0 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 client.16671 shutdown 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 msgr2=0x55dc49267860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f954000fbd0 tx=0x7f954001bd30 comp rx=0 tx=0).stop 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 shutdown_connections 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dc4925f780 0x55dc49264830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dc49251b40 0x55dc49267da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dc492daf60 0x55dc492dd350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.330+0000 7f955eeff740 1 --2- 172.21.15.16:0/2520074349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dc492505d0 0x55dc49267860 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f953dffb640 0 client.16671 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 >> 172.21.15.16:0/2520074349 conn(0x55dc49219310 msgr2=0x55dc49252930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 shutdown_connections 2024-09-12T01:00:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955eeff740 1 -- 172.21.15.16:0/2520074349 wait complete. 2024-09-12T01:00:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955eeff740 20 client.16671 trim_cache size 0 max 16384 2024-09-12T01:00:33.342 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 msgr2=0x7f95440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 0x7f95440a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95500979e0 tx=0x7f9550014d00 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 msgr2=0x7f95581a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f955000b970 tx=0x7f955000be30 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 shutdown_connections 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95440a2cf0 0x7f95440a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9558102ef0 0x7f95581a4410 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9558102570 0x7f95581998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 --2- 172.21.15.16:0/2501345937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9558100e20 0x7f9558199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.331+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 >> 172.21.15.16:0/2501345937 conn(0x7f95580fc7d0 msgr2=0x7f95580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.332+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 shutdown_connections 2024-09-12T01:00:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:33.332+0000 7f955cb26640 1 -- 172.21.15.16:0/2501345937 wait complete. 2024-09-12T01:00:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:32 smithi159 ceph-mon[35566]: pgmap v137: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:32 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2331630530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:34.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:00:34.156 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:34.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 -- 172.21.15.16:0/2644202799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0100a80 msgr2=0x7f83d0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 --2- 172.21.15.16:0/2644202799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0100a80 0x7f83d0100ee0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009930 tx=0x7f83c002a420 comp rx=0 tx=0).stop 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 -- 172.21.15.16:0/2644202799 shutdown_connections 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 --2- 172.21.15.16:0/2644202799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0101420 0x7f83d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 --2- 172.21.15.16:0/2644202799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0100a80 0x7f83d0100ee0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 --2- 172.21.15.16:0/2644202799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 0x7f83d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 -- 172.21.15.16:0/2644202799 >> 172.21.15.16:0/2644202799 conn(0x7f83d00fc7a0 msgr2=0x7f83d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 -- 172.21.15.16:0/2644202799 shutdown_connections 2024-09-12T01:00:34.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.676+0000 7f83d6be0640 1 -- 172.21.15.16:0/2644202799 wait complete. 2024-09-12T01:00:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 Processor -- start 2024-09-12T01:00:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 -- start start 2024-09-12T01:00:34.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 0x7f83d01973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 0x7f83d019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110ef0 con 0x7f83d0101420 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0110d70 con 0x7f83d0100a80 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d6be0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0111070 con 0x7f83d0106a80 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45750/0 (socket says 172.21.15.16:45750) 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83c7fff640 1 -- 172.21.15.16:0/1236403280 learned_addr learned my addr 172.21.15.16:0/1236403280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d517d640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 0x7f83d019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.677+0000 7f83d497c640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 0x7f83d01973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 -- 172.21.15.16:0/1236403280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 msgr2=0x7f83d019bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 0x7f83d019bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 -- 172.21.15.16:0/1236403280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 msgr2=0x7f83d01973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 0x7f83d01973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 -- 172.21.15.16:0/1236403280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d019c380 con 0x7f83d0101420 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83d497c640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 0x7f83d01973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f83c002a940 tx=0x7f83c0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c5ffb640 1 -- 172.21.15.16:0/1236403280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c003f070 con 0x7f83d0101420 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d019c610 con 0x7f83d0101420 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c5ffb640 1 -- 172.21.15.16:0/1236403280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c0031c90 con 0x7f83d0101420 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83c5ffb640 1 -- 172.21.15.16:0/1236403280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0035030 con 0x7f83d0101420 2024-09-12T01:00:34.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.678+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d006a790 con 0x7f83d0101420 2024-09-12T01:00:34.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.681+0000 7f83c5ffb640 1 -- 172.21.15.16:0/1236403280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83c0042070 con 0x7f83d0101420 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 Processor -- start 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 -- start start 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83c5ffb640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 0x7f83b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d517d640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 0x7f83b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d1722dc0 0x5586d17216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d17350d0 0x5586d17374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586d1686160 con 0x5586d1721c10 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586d162d720 con 0x5586d17350d0 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586d1682120 con 0x5586d1722dc0 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83c5ffb640 1 -- 172.21.15.16:0/1236403280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83c00efbf0 con 0x7f83d0101420 2024-09-12T01:00:34.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45754/0 (socket says 172.21.15.16:45754) 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83c7fff640 1 -- 172.21.15.16:0/1682463177 learned_addr learned my addr 172.21.15.16:0/1682463177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d497c640 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d1722dc0 0x5586d17216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.682+0000 7f83d517d640 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d17350d0 0x5586d17374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83d517d640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 0x7f83b80a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009f70 tx=0x7f83c8009650 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 -- 172.21.15.16:0/1682463177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d1722dc0 msgr2=0x5586d17216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d1722dc0 0x5586d17216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 -- 172.21.15.16:0/1682463177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d17350d0 msgr2=0x5586d17374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d17350d0 0x5586d17374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 -- 172.21.15.16:0/1682463177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d13f7ba0 con 0x5586d1721c10 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83c7fff640 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f83c00c4820 tx=0x7f83c00bbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceac1a6fcf6dbc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83b6ffd640 1 -- 172.21.15.16:0/1682463177 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0049070 con 0x5586d1721c10 2024-09-12T01:00:34.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83b6ffd640 1 -- 172.21.15.16:0/1682463177 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c00bc0e0 con 0x5586d1721c10 2024-09-12T01:00:34.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.683+0000 7f83b6ffd640 1 -- 172.21.15.16:0/1682463177 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00b8750 con 0x5586d1721c10 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 -- 172.21.15.16:0/1682463177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 msgr2=0x5586d1722070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f83c00c4820 tx=0x7f83c00bbfe0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 -- 172.21.15.16:0/1682463177 shutdown_connections 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d17350d0 0x5586d17374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d1722070 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/1682463177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d1722dc0 0x5586d17216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 -- 172.21.15.16:0/1682463177 >> 172.21.15.16:0/1682463177 conn(0x5586d171dde0 msgr2=0x5586d171e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 -- 172.21.15.16:0/1682463177 shutdown_connections 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.684+0000 7f83d8fb9740 1 -- 172.21.15.16:0/1682463177 wait complete. 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 -- start start 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 Processor -- start 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d17254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 0x5586d1725a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1725f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586d1737bf0 con 0x5586d1721c10 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586d160e900 con 0x5586d1722dc0 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586d1737d70 con 0x5586d17350d0 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d8fb9740 10 client.0 _ll_register_callbacks cb 0x5586d160c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:34.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d497c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d17254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 0x5586d1725a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.685+0000 7f83d497c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d17254e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45758/0 (socket says 172.21.15.16:45758) 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83d497c640 1 -- 172.21.15.16:0/2423694769 learned_addr learned my addr 172.21.15.16:0/2423694769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 -- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 msgr2=0x5586d1725f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1725f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 -- 172.21.15.16:0/2423694769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 msgr2=0x5586d17254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d17254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 -- 172.21.15.16:0/2423694769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d164ec50 con 0x5586d1722dc0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 0x5586d1725a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83c00b8ab0 tx=0x7f83c0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ad3fc10db79961 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83affff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00d4070 con 0x5586d1722dc0 2024-09-12T01:00:34.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83c00b8750 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c00bc0e0 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83c0049030 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83d8fb9740 10 client.25979 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.686+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586d1737bf0 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c00bc0e0 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83c003b5f0 con 0x5586d1722dc0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 10 client.25979 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 10 client.25979 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 20 client.25979 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d1744ae0 0x5586d1746f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 4 client.25979 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d1693890 con 0x5586d1744ae0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.687+0000 7f83d497c640 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d1744ae0 0x5586d1746f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.688+0000 7f83d497c640 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d1744ae0 0x5586d1746f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c19127bd3b62080 server_cookie=834378b8f849e2aa in_seq=0 out_seq=0 2024-09-12T01:00:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.688+0000 7f83affff640 10 client.25979 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:34.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.688+0000 7f83affff640 1 -- 172.21.15.16:0/2423694769 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586d1693890 con 0x5586d1744ae0 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.688+0000 7f83affff640 10 client.25979 handle_command_reply: tid=0 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.688+0000 7f83d8fb9740 1 client.25979 shutdown 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 msgr2=0x5586d1725a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 0x5586d1725a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83c00b8ab0 tx=0x7f83c0005c00 comp rx=0 tx=0).stop 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 shutdown_connections 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d1744ae0 0x5586d1746f60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1725f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1722dc0 0x5586d1725a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2423694769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1721c10 0x5586d17254e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83affff640 0 client.25979 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 >> 172.21.15.16:0/2423694769 conn(0x5586d171dde0 msgr2=0x5586d1723b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 shutdown_connections 2024-09-12T01:00:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2423694769 wait complete. 2024-09-12T01:00:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.689+0000 7f83d8fb9740 20 client.25979 trim_cache size 0 max 16384 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 Processor -- start 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 -- start start 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d1727810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1724820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ab630 con 0x5586d1722dc0 2024-09-12T01:00:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ad170 con 0x5586d1721c10 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01abd00 con 0x5586d17350d0 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45774/0 (socket says 172.21.15.16:45774) 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83c7fff640 1 -- 172.21.15.16:0/3314478614 learned_addr learned my addr 172.21.15.16:0/3314478614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d497c640 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d1727810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.708+0000 7f83d517d640 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1724820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 -- 172.21.15.16:0/3314478614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 msgr2=0x5586d1724820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1724820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 -- 172.21.15.16:0/3314478614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 msgr2=0x5586d1727810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d1727810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 -- 172.21.15.16:0/3314478614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d17193f0 con 0x5586d1722dc0 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83c7fff640 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004650 tx=0x7f83c00bbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746d9cd73ef35c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83b67fc640 1 -- 172.21.15.16:0/3314478614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00d4070 con 0x5586d1722dc0 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83b67fc640 1 -- 172.21.15.16:0/3314478614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c00a1050 con 0x5586d1722dc0 2024-09-12T01:00:34.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.709+0000 7f83b67fc640 1 -- 172.21.15.16:0/3314478614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00b8750 con 0x5586d1722dc0 2024-09-12T01:00:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 -- 172.21.15.16:0/3314478614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 msgr2=0x5586d17242e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004650 tx=0x7f83c00bbfe0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 -- 172.21.15.16:0/3314478614 shutdown_connections 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d1724820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d17242e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/3314478614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d1727810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 -- 172.21.15.16:0/3314478614 >> 172.21.15.16:0/3314478614 conn(0x5586d171dde0 msgr2=0x5586d1725cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 -- 172.21.15.16:0/3314478614 shutdown_connections 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.710+0000 7f83d8fb9740 1 -- 172.21.15.16:0/3314478614 wait complete. 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.711+0000 7f83d8fb9740 1 -- start start 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.711+0000 7f83d8fb9740 1 Processor -- start 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.711+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d173d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.711+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.711+0000 7f83d8fb9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d173a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01acaa0 con 0x5586d1722dc0 2024-09-12T01:00:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83c7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45778/0 (socket says 172.21.15.16:45778) 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83c7fff640 1 -- 172.21.15.16:0/2137121171 learned_addr learned my addr 172.21.15.16:0/2137121171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d8fb9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01ac3d0 con 0x5586d1721c10 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d173d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83d01a9610 con 0x5586d17350d0 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d8fb9740 10 client.0 _ll_register_callbacks cb 0x7f83c8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d517d640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d173a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 -- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 msgr2=0x5586d173a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d173a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 -- 172.21.15.16:0/2137121171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 msgr2=0x5586d1739f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 -- 172.21.15.16:0/2137121171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d173aac0 con 0x5586d1721c10 2024-09-12T01:00:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d517d640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d173a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83d497c640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d173d510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83c00c0400 tx=0x7f83cc01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bbc3817b559935 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.712+0000 7f83affff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0035f0 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f83cc003790 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c00a1050 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83cc003990 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 10 client.25985 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586d17b0650 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c00a1050 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f83cc02e450 con 0x5586d1721c10 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 10 client.25985 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 10 client.25985 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 20 client.25985 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d17b0de0 0x5586d17b3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 4 client.25985 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.713+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5586d168dbe0 con 0x5586d17b0de0 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.714+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d17b0de0 0x5586d17b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.714+0000 7f83c7fff640 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d17b0de0 0x5586d17b3200 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e6adbe47977342 server_cookie=bb3e74089570f2dc in_seq=0 out_seq=0 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.714+0000 7f83affff640 10 client.25985 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.726+0000 7f83affff640 1 -- 172.21.15.16:0/2137121171 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586d168dbe0 con 0x5586d17b0de0 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.726+0000 7f83affff640 10 client.25985 handle_command_reply: tid=0 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.726+0000 7f83d8fb9740 1 client.25985 shutdown 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 msgr2=0x5586d173d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d173d510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83c00c0400 tx=0x7f83cc01c8e0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 shutdown_connections 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586d17b0de0 0x5586d17b3200 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586d17350d0 0x5586d173a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586d1722dc0 0x5586d1739f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83affff640 0 client.25985 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 --2- 172.21.15.16:0/2137121171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586d1721c10 0x5586d173d510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 >> 172.21.15.16:0/2137121171 conn(0x5586d171dde0 msgr2=0x5586d173fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 shutdown_connections 2024-09-12T01:00:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 1 -- 172.21.15.16:0/2137121171 wait complete. 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.727+0000 7f83d8fb9740 20 client.25985 trim_cache size 0 max 16384 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 18, 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "abb9bace": { 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 18, 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:00:34.728 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:34.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 msgr2=0x7f83b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 0x7f83b80a5230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009f70 tx=0x7f83c8009650 comp rx=0 tx=0).stop 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 msgr2=0x7f83d0197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f83c002a940 tx=0x7f83c0003b90 comp rx=0 tx=0).stop 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 shutdown_connections 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83b80a2d70 0x7f83b80a5230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83d0106a80 0x7f83d019bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83d0101420 0x7f83d0197910 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 --2- 172.21.15.16:0/1236403280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83d0100a80 0x7f83d01973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 >> 172.21.15.16:0/1236403280 conn(0x7f83d00fc7a0 msgr2=0x7f83d00fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 shutdown_connections 2024-09-12T01:00:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:34.728+0000 7f83d6be0640 1 -- 172.21.15.16:0/1236403280 wait complete. 2024-09-12T01:00:35.217 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:34 smithi016 ceph-mon[30972]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:35.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:34 smithi099 ceph-mon[35654]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:34 smithi159 ceph-mon[35566]: pgmap v138: 97 pgs: 97 active+clean; 3.8 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:00:35.895 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'abb9bace', quiesce took 6.1 seconds. Will release after: 14.049999999999999 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 -- 172.21.15.16:0/525047962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 msgr2=0x7ff5a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 --2- 172.21.15.16:0/525047962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a010da40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ca0 tx=0x7ff59802a950 comp rx=0 tx=0).stop 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 -- 172.21.15.16:0/525047962 shutdown_connections 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 --2- 172.21.15.16:0/525047962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a010da40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 --2- 172.21.15.16:0/525047962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0102570 0x7ff5a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 --2- 172.21.15.16:0/525047962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0100e20 0x7ff5a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 -- 172.21.15.16:0/525047962 >> 172.21.15.16:0/525047962 conn(0x7ff5a00fc7d0 msgr2=0x7ff5a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 -- 172.21.15.16:0/525047962 shutdown_connections 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.202+0000 7ff5a5337640 1 -- 172.21.15.16:0/525047962 wait complete. 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 Processor -- start 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 -- start start 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0100e20 0x7ff5a0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0102570 0x7ff5a0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0110f80 con 0x7ff5a0102ef0 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0110e00 con 0x7ff5a0102570 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff5a5337640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a0111100 con 0x7ff5a0100e20 2024-09-12T01:00:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff59f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff59f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45798/0 (socket says 172.21.15.16:45798) 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.203+0000 7ff59f7fe640 1 -- 172.21.15.16:0/4115396986 learned_addr learned my addr 172.21.15.16:0/4115396986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59effd640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0100e20 0x7ff5a0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff597fff640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0102570 0x7ff5a0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 -- 172.21.15.16:0/4115396986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0100e20 msgr2=0x7ff5a0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0100e20 0x7ff5a0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 -- 172.21.15.16:0/4115396986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0102570 msgr2=0x7ff5a0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0102570 0x7ff5a0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 -- 172.21.15.16:0/4115396986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a019d4b0 con 0x7ff5a0102ef0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59f7fe640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff59802ae70 tx=0x7ff59802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59cff9640 1 -- 172.21.15.16:0/4115396986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598034040 con 0x7ff5a0102ef0 2024-09-12T01:00:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a019d740 con 0x7ff5a0102ef0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59cff9640 1 -- 172.21.15.16:0/4115396986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff598030920 con 0x7ff5a0102ef0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff59cff9640 1 -- 172.21.15.16:0/4115396986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598045480 con 0x7ff5a0102ef0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.204+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a01a5080 con 0x7ff5a0102ef0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.206+0000 7ff59cff9640 1 -- 172.21.15.16:0/4115396986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff598037020 con 0x7ff5a0102ef0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 Processor -- start 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 -- start start 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf131a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633bf0972f0 con 0x5633bf132b40 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633bf03ecf0 con 0x5633bf1315d0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff5a7710740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5633bf0932b0 con 0x5633bf140780 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff59cff9640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 0x7ff58c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45810/0 (socket says 172.21.15.16:45810) 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.207+0000 7ff597fff640 1 -- 172.21.15.16:0/4277201304 learned_addr learned my addr 172.21.15.16:0/4277201304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff59f7fe640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 0x7ff58c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff59effd640 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf131a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff59cff9640 1 -- 172.21.15.16:0/4115396986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff5980f0120 con 0x7ff5a0102ef0 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 -- 172.21.15.16:0/4277201304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 msgr2=0x5633bf142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 -- 172.21.15.16:0/4277201304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 msgr2=0x5633bf131a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf131a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 -- 172.21.15.16:0/4277201304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bf0233a0 con 0x5633bf132b40 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff597fff640 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff588004650 tx=0x7ff5880041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d74b5ff3ce4e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff59f7fe640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 0x7ff58c0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5980bc710 tx=0x7ff5980095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff5957fa640 1 -- 172.21.15.16:0/4277201304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588005020 con 0x5633bf132b40 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff5957fa640 1 -- 172.21.15.16:0/4277201304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5880056a0 con 0x5633bf132b40 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.208+0000 7ff5957fa640 1 -- 172.21.15.16:0/4277201304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588010660 con 0x5633bf132b40 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- 172.21.15.16:0/4277201304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 msgr2=0x5633bf131090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff588004650 tx=0x7ff5880041d0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- 172.21.15.16:0/4277201304 shutdown_connections 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf131a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 --2- 172.21.15.16:0/4277201304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf131090 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- 172.21.15.16:0/4277201304 >> 172.21.15.16:0/4277201304 conn(0x5633bf0fa310 msgr2=0x5633bf12dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- 172.21.15.16:0/4277201304 shutdown_connections 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- 172.21.15.16:0/4277201304 wait complete. 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.209+0000 7ff5a7710740 1 -- start start 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 Processor -- start 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf13d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633bf143420 con 0x5633bf132b40 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633bf052060 con 0x5633bf1315d0 2024-09-12T01:00:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5633bf1435a0 con 0x5633bf140780 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff5a7710740 10 client.0 _ll_register_callbacks cb 0x5633beda58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45818/0 (socket says 172.21.15.16:45818) 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 -- 172.21.15.16:0/2962485859 learned_addr learned my addr 172.21.15.16:0/2962485859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff597fff640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf13d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff597fff640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 -- 172.21.15.16:0/2962485859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 msgr2=0x5633bf13d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf13d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 -- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 msgr2=0x5633bf1400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf1400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff59effd640 1 -- 172.21.15.16:0/2962485859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bee083c0 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.210+0000 7ff597fff640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf1400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff59effd640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff5900114c0 tx=0x7ff590011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4c9b61abaa7d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590004020 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff5900041c0 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5880056a0 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5900043c0 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 10 client.16701 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633bf14ee30 con 0x5633bf132b40 2024-09-12T01:00:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5880056a0 con 0x5633bf132b40 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff590003b50 con 0x5633bf132b40 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 10 client.16701 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 10 client.16701 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 20 client.16701 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf152760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 4 client.16701 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.211+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633bf0a4860 con 0x5633bf150340 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.212+0000 7ff597fff640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.212+0000 7ff597fff640 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf152760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfe2b30f992c16b5 server_cookie=7a795bc2010a91f3 in_seq=0 out_seq=0 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.212+0000 7ff5867fc640 10 client.16701 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5867fc640 1 -- 172.21.15.16:0/2962485859 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633bf0a4860 con 0x5633bf150340 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5867fc640 10 client.16701 handle_command_reply: tid=0 2024-09-12T01:00:36.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 client.16701 shutdown 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 msgr2=0x5633bf13d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff5900114c0 tx=0x7ff590011980 comp rx=0 tx=0).stop 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 shutdown_connections 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf152760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5867fc640 0 client.16701 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf13d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf132b40 0x5633bf13d3a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2962485859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf1400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 >> 172.21.15.16:0/2962485859 conn(0x5633bf0fa310 msgr2=0x5633bf133930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 shutdown_connections 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 1 -- 172.21.15.16:0/2962485859 wait complete. 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.213+0000 7ff5a7710740 20 client.16701 trim_cache size 0 max 16384 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.226+0000 7ff5a7710740 1 Processor -- start 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.226+0000 7ff5a7710740 1 -- start start 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf132b40 0x5633bf14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01ab590 con 0x5633bf1315d0 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01ad0d0 con 0x5633bf132b40 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff5a7710740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01abc60 con 0x5633bf140780 2024-09-12T01:00:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45824/0 (socket says 172.21.15.16:45824) 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 -- 172.21.15.16:0/879287753 learned_addr learned my addr 172.21.15.16:0/879287753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff59effd640 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf132b40 0x5633bf14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 -- 172.21.15.16:0/879287753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 msgr2=0x5633bf158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 -- 172.21.15.16:0/879287753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf132b40 msgr2=0x5633bf14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf132b40 0x5633bf14c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 -- 172.21.15.16:0/879287753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bf13dc70 con 0x5633bf1315d0 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.227+0000 7ff597fff640 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff588003f50 tx=0x7ff588004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139b16238728e686 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff594ff9640 1 -- 172.21.15.16:0/879287753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588032070 con 0x5633bf1315d0 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff594ff9640 1 -- 172.21.15.16:0/879287753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588004850 con 0x5633bf1315d0 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff594ff9640 1 -- 172.21.15.16:0/879287753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880106c0 con 0x5633bf1315d0 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 -- 172.21.15.16:0/879287753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 msgr2=0x5633bf13d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff588003f50 tx=0x7ff588004040 comp rx=0 tx=0).stop 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 -- 172.21.15.16:0/879287753 shutdown_connections 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf140780 0x5633bf158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf132b40 0x5633bf14c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 --2- 172.21.15.16:0/879287753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf1315d0 0x5633bf13d370 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.228+0000 7ff5a7710740 1 -- 172.21.15.16:0/879287753 >> 172.21.15.16:0/879287753 conn(0x5633bf0fa310 msgr2=0x5633bf13cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- 172.21.15.16:0/879287753 shutdown_connections 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- 172.21.15.16:0/879287753 wait complete. 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- start start 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 Processor -- start 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf148a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf132b40 0x5633bf148f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01aca00 con 0x5633bf140780 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01ac330 con 0x5633bf1315d0 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a01a9570 con 0x5633bf132b40 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff5a7710740 10 client.0 _ll_register_callbacks cb 0x7ff59000bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45832/0 (socket says 172.21.15.16:45832) 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff597fff640 1 -- 172.21.15.16:0/2425980463 learned_addr learned my addr 172.21.15.16:0/2425980463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.229+0000 7ff59effd640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf132b40 0x5633bf148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 -- 172.21.15.16:0/2425980463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf132b40 msgr2=0x5633bf148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf132b40 0x5633bf148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 -- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 msgr2=0x5633bf148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 -- 172.21.15.16:0/2425980463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bf13a3e0 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff597fff640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff588005750 tx=0x7ff58800bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db26edb8c4e9d3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5867fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588032050 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff588004f10 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff588004850 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff588005110 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5a7710740 10 client.16707 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633bf1baae0 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.230+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588004850 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff58803b450 con 0x5633bf140780 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 10 client.16707 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 10 client.16707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 20 client.16707 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf1bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 4 client.16707 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633bf09ed70 con 0x5633bf150340 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff59effd640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf1bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff59effd640 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf1bcf70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c10004b7cc6e9a00 server_cookie=408913464c7dad07 in_seq=0 out_seq=0 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5867fc640 10 client.16707 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5867fc640 1 -- 172.21.15.16:0/2425980463 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633bf09ed70 con 0x5633bf150340 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5867fc640 10 client.16707 handle_command_reply: tid=0 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.231+0000 7ff5a7710740 1 client.16707 shutdown 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 msgr2=0x5633bf139bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff588005750 tx=0x7ff58800bfb0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 shutdown_connections 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5633bf150340 0x5633bf1bcf70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633bf140780 0x5633bf139bd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633bf132b40 0x5633bf148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 --2- 172.21.15.16:0/2425980463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633bf1315d0 0x5633bf148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 >> 172.21.15.16:0/2425980463 conn(0x5633bf0fa310 msgr2=0x5633bf13bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5867fc640 0 client.16707 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 shutdown_connections 2024-09-12T01:00:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 1 -- 172.21.15.16:0/2425980463 wait complete. 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a7710740 20 client.16707 trim_cache size 0 max 16384 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 msgr2=0x7ff58c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 0x7ff58c0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5980bc710 tx=0x7ff5980095c0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 msgr2=0x7ff5a019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff59802ae70 tx=0x7ff59802f5d0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 shutdown_connections 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff58c0a2cf0 0x7ff58c0a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5a0102ef0 0x7ff5a019cdd0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5a0102570 0x7ff5a0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 --2- 172.21.15.16:0/4115396986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a0100e20 0x7ff5a0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.232+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 >> 172.21.15.16:0/4115396986 conn(0x7ff5a00fc7d0 msgr2=0x7ff5a006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.233+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 shutdown_connections 2024-09-12T01:00:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:36.233+0000 7ff5a5337640 1 -- 172.21.15.16:0/4115396986 wait complete. 2024-09-12T01:00:36.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:00:37.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:36 smithi016 ceph-mon[30972]: pgmap v139: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T01:00:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:36 smithi099 ceph-mon[35654]: pgmap v139: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T01:00:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:36 smithi159 ceph-mon[35566]: pgmap v139: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-09-12T01:00:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 -- 172.21.15.16:0/4268738918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14100a80 msgr2=0x7fda14100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 --2- 172.21.15.16:0/4268738918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14100a80 0x7fda14100ee0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fda04009930 tx=0x7fda0402a420 comp rx=0 tx=0).stop 2024-09-12T01:00:37.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 -- 172.21.15.16:0/4268738918 shutdown_connections 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 --2- 172.21.15.16:0/4268738918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 0x7fda1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 --2- 172.21.15.16:0/4268738918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14100a80 0x7fda14100ee0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 --2- 172.21.15.16:0/4268738918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14106a80 0x7fda14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 -- 172.21.15.16:0/4268738918 >> 172.21.15.16:0/4268738918 conn(0x7fda140fc7a0 msgr2=0x7fda140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 -- 172.21.15.16:0/4268738918 shutdown_connections 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.678+0000 7fda1bf63640 1 -- 172.21.15.16:0/4268738918 wait complete. 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 Processor -- start 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 -- start start 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14100a80 0x7fda1410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 0x7fda1410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda14111fa0 con 0x7fda14106a80 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda14111e20 con 0x7fda14100a80 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1bf63640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda14112120 con 0x7fda14101420 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45858/0 (socket says 172.21.15.16:45858) 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 -- 172.21.15.16:0/933304249 learned_addr learned my addr 172.21.15.16:0/933304249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda194fe640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 0x7fda1410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda19cff640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14100a80 0x7fda1410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 -- 172.21.15.16:0/933304249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 msgr2=0x7fda1410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 0x7fda1410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 -- 172.21.15.16:0/933304249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14100a80 msgr2=0x7fda1410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.679+0000 7fda1a500640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14100a80 0x7fda1410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda1a500640 1 -- 172.21.15.16:0/933304249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda141a6940 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda1a500640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00eaa0 tx=0x7fda0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda0affd640 1 -- 172.21.15.16:0/933304249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cdb0 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda0affd640 1 -- 172.21.15.16:0/933304249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0c005020 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda141a6c30 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda0affd640 1 -- 172.21.15.16:0/933304249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0116b0 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.680+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141a70f0 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.683+0000 7fda1e33c740 1 Processor -- start 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.683+0000 7fda0affd640 1 -- 172.21.15.16:0/933304249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda0c005560 con 0x7fda14106a80 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.683+0000 7fda1e33c740 1 -- start start 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.683+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aaaf9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaaf95d0 0x5624aaaf9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1e33c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624aaa5f2f0 con 0x5624aab04370 2024-09-12T01:00:37.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1e33c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624aaa06cf0 con 0x5624aaaf95d0 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1e33c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5624aaa5b2b0 con 0x5624aaafab40 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45866/0 (socket says 172.21.15.16:45866) 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 -- 172.21.15.16:0/4120268131 learned_addr learned my addr 172.21.15.16:0/4120268131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda19cff640 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaaf95d0 0x5624aaaf9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda0affd640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 0x7fda000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aaaf9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 -- 172.21.15.16:0/4120268131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 msgr2=0x5624aaaf9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aaaf9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 -- 172.21.15.16:0/4120268131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaaf95d0 msgr2=0x5624aaaf9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaaf95d0 0x5624aaaf9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 -- 172.21.15.16:0/4120268131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aa9eb3a0 con 0x5624aab04370 2024-09-12T01:00:37.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda1a500640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 0x7fda000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda0affd640 1 -- 172.21.15.16:0/933304249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fda0c0cf670 con 0x7fda14106a80 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.684+0000 7fda194fe640 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda04009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443e20d284c280f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fd9fffff640 1 -- 172.21.15.16:0/4120268131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04035070 con 0x5624aab04370 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fd9fffff640 1 -- 172.21.15.16:0/4120268131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004030 con 0x5624aab04370 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fd9fffff640 1 -- 172.21.15.16:0/4120268131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040047a0 con 0x5624aab04370 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 -- 172.21.15.16:0/4120268131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 msgr2=0x5624aab06760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda04009560 comp rx=0 tx=0).stop 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1a500640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 0x7fda000a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0a4300 tx=0x7fda0c0667f0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 -- 172.21.15.16:0/4120268131 shutdown_connections 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab06760 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaaf95d0 0x5624aaaf9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 --2- 172.21.15.16:0/4120268131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aaaf9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 -- 172.21.15.16:0/4120268131 >> 172.21.15.16:0/4120268131 conn(0x5624aaac2310 msgr2=0x5624aaaf5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 -- 172.21.15.16:0/4120268131 shutdown_connections 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.685+0000 7fda1e33c740 1 -- 172.21.15.16:0/4120268131 wait complete. 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 -- start start 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 Processor -- start 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaafab40 0x5624aab01030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624aab07010 con 0x5624aab04370 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624aaa1a060 con 0x5624aaafab40 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5624aab07190 con 0x5624aaaf95d0 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda1e33c740 10 client.0 _ll_register_callbacks cb 0x5624aa76d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45878/0 (socket says 172.21.15.16:45878) 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.686+0000 7fda194fe640 1 -- 172.21.15.16:0/831863031 learned_addr learned my addr 172.21.15.16:0/831863031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda19cff640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaafab40 0x5624aab01030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 -- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 msgr2=0x5624aab03d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab03d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 -- 172.21.15.16:0/831863031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaafab40 msgr2=0x5624aab01030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaafab40 0x5624aab01030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 -- 172.21.15.16:0/831863031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aa7d03c0 con 0x5624aab04370 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda194fe640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fda0402a940 tx=0x7fda04005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6ad9ba30507de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fd9fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04035040 con 0x5624aab04370 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fda040043f0 con 0x5624aab04370 2024-09-12T01:00:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04004030 con 0x5624aab04370 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda040316b0 con 0x5624aab04370 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda1e33c740 10 client.16719 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624aab12b10 con 0x5624aab04370 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.687+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04004030 con 0x5624aab04370 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda04003710 con 0x5624aab04370 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 10 client.16719 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 10 client.16719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 20 client.16719 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab13e60 0x5624aab16250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 4 client.16719 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624aaa6c860 con 0x5624aab13e60 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda19cff640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab13e60 0x5624aab16250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fda19cff640 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab13e60 0x5624aab16250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3309ed3bd544feff server_cookie=847bf3e1b9a3371f in_seq=0 out_seq=0 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.688+0000 7fd9fd7fa640 10 client.16719 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/831863031 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624aaa6c860 con 0x5624aab13e60 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fd9fd7fa640 10 client.16719 handle_command_reply: tid=0 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 client.16719 shutdown 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 msgr2=0x5624aab01640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fda0402a940 tx=0x7fda04005240 comp rx=0 tx=0).stop 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 shutdown_connections 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aab04370 0x5624aab01640 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.689+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab13e60 0x5624aab16250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aaafab40 0x5624aab01030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 1 --2- 172.21.15.16:0/831863031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab03d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 >> 172.21.15.16:0/831863031 conn(0x5624aaac2310 msgr2=0x5624aaafb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fd9fd7fa640 0 client.16719 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 shutdown_connections 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 1 -- 172.21.15.16:0/831863031 wait complete. 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.690+0000 7fda1e33c740 20 client.16719 trim_cache size 0 max 16384 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 Processor -- start 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 -- start start 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab10460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda141af9e0 con 0x5624aaafab40 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b1520 con 0x5624aab04370 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda1e33c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b00b0 con 0x5624aaaf95d0 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45886/0 (socket says 172.21.15.16:45886) 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 -- 172.21.15.16:0/3780524739 learned_addr learned my addr 172.21.15.16:0/3780524739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda194fe640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab10460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda194fe640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 -- 172.21.15.16:0/3780524739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 msgr2=0x5624aab10460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab10460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 -- 172.21.15.16:0/3780524739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 msgr2=0x5624aab008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 -- 172.21.15.16:0/3780524739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aab00020 con 0x5624aaafab40 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda194fe640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.703+0000 7fda19cff640 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5624aab02670 tx=0x7fda100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4b93edf89d3de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/3780524739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10004030 con 0x5624aaafab40 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/3780524739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda100041d0 con 0x5624aaafab40 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fd9ff7fe640 1 -- 172.21.15.16:0/3780524739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda100116a0 con 0x5624aaafab40 2024-09-12T01:00:37.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 -- 172.21.15.16:0/3780524739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 msgr2=0x5624aaaf5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5624aab02670 tx=0x7fda100046c0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 -- 172.21.15.16:0/3780524739 shutdown_connections 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaafab40 0x5624aaaf5e10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 --2- 172.21.15.16:0/3780524739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaaf95d0 0x5624aab10460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:37.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 -- 172.21.15.16:0/3780524739 >> 172.21.15.16:0/3780524739 conn(0x5624aaac2310 msgr2=0x5624aaaf5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 -- 172.21.15.16:0/3780524739 shutdown_connections 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.704+0000 7fda1e33c740 1 -- 172.21.15.16:0/3780524739 wait complete. 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 -- start start 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 Processor -- start 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aab16af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab17030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b0e50 con 0x5624aaaf95d0 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda141b0780 con 0x5624aab04370 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda141ad9c0 con 0x5624aaafab40 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda1e33c740 10 client.0 _ll_register_callbacks cb 0x7fda1000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda194fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45888/0 (socket says 172.21.15.16:45888) 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda194fe640 1 -- 172.21.15.16:0/931033636 learned_addr learned my addr 172.21.15.16:0/931033636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.705+0000 7fda19cff640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aab16af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab17030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 -- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 msgr2=0x5624aab16af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aab16af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 -- 172.21.15.16:0/931033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 msgr2=0x5624aab17030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab17030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 -- 172.21.15.16:0/931033636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aab040f0 con 0x5624aaaf95d0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda194fe640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fda04003ab0 tx=0x7fda04005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2f9d5f7950783 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fd9fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0405d040 con 0x5624aaaf95d0 2024-09-12T01:00:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fda040035f0 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda100041d0 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda0406a050 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda1e33c740 10 client.16725 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624aab7e310 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.706+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda100041d0 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fda04004c50 con 0x5624aaaf95d0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 10 client.16725 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 10 client.16725 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 20 client.16725 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab7e980 0x5624aab80da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 4 client.16725 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624aaa66d70 con 0x5624aab7e980 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda19cff640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab7e980 0x5624aab80da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fd9fd7fa640 10 client.16725 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.707+0000 7fda19cff640 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab7e980 0x5624aab80da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f155d382ae0eb4e8 server_cookie=71bdac23efebeb52 in_seq=0 out_seq=0 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fd9fd7fa640 1 -- 172.21.15.16:0/931033636 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624aaa66d70 con 0x5624aab7e980 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fd9fd7fa640 10 client.16725 handle_command_reply: tid=0 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 client.16725 shutdown 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 msgr2=0x5624aab197a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fda04003ab0 tx=0x7fda04005240 comp rx=0 tx=0).stop 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 shutdown_connections 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5624aab7e980 0x5624aab80da0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aab04370 0x5624aab17030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5624aaafab40 0x5624aab16af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fd9fd7fa640 0 client.16725 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 --2- 172.21.15.16:0/931033636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624aaaf95d0 0x5624aab197a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 >> 172.21.15.16:0/931033636 conn(0x5624aaac2310 msgr2=0x5624aaaf56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 shutdown_connections 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 1 -- 172.21.15.16:0/931033636 wait complete. 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1e33c740 20 client.16725 trim_cache size 0 max 16384 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 msgr2=0x7fda000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 0x7fda000a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0a4300 tx=0x7fda0c0667f0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.708+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 msgr2=0x7fda1410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00eaa0 tx=0x7fda0c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 shutdown_connections 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda000a2cf0 0x7fda000a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda14106a80 0x7fda1410ee60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda14101420 0x7fda1410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 --2- 172.21.15.16:0/933304249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda14100a80 0x7fda1410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:37.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 >> 172.21.15.16:0/933304249 conn(0x7fda140fc7a0 msgr2=0x7fda140fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 shutdown_connections 2024-09-12T01:00:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:37.709+0000 7fda1bf63640 1 -- 172.21.15.16:0/933304249 wait complete. 2024-09-12T01:00:38.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:00:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:39.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:38 smithi016 ceph-mon[30972]: pgmap v140: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T01:00:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:38 smithi099 ceph-mon[35654]: pgmap v140: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T01:00:39.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 -- 172.21.15.16:0/2531221302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c106f50 msgr2=0x7fd31c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 --2- 172.21.15.16:0/2531221302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c106f50 0x7fd31c10eaf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd314009930 tx=0x7fd31402a420 comp rx=0 tx=0).stop 2024-09-12T01:00:39.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 -- 172.21.15.16:0/2531221302 shutdown_connections 2024-09-12T01:00:39.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 --2- 172.21.15.16:0/2531221302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c106f50 0x7fd31c10eaf0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 --2- 172.21.15.16:0/2531221302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c100d00 0x7fd31c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 --2- 172.21.15.16:0/2531221302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100350 0x7fd31c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 -- 172.21.15.16:0/2531221302 >> 172.21.15.16:0/2531221302 conn(0x7fd31c077210 msgr2=0x7fd31c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 -- 172.21.15.16:0/2531221302 shutdown_connections 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.277+0000 7fd3231e2640 1 -- 172.21.15.16:0/2531221302 wait complete. 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 Processor -- start 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 -- start start 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100d00 0x7fd31c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 0x7fd31c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c112000 con 0x7fd31c100350 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c111e80 con 0x7fd31c100d00 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd3231e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c112180 con 0x7fd31c106f50 2024-09-12T01:00:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52158/0 (socket says 172.21.15.16:52158) 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd320f7e640 1 -- 172.21.15.16:0/600951001 learned_addr learned my addr 172.21.15.16:0/600951001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.278+0000 7fd32177f640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 0x7fd31c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd31bfff640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100d00 0x7fd31c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 -- 172.21.15.16:0/600951001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 msgr2=0x7fd31c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 0x7fd31c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 -- 172.21.15.16:0/600951001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100d00 msgr2=0x7fd31c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100d00 0x7fd31c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 -- 172.21.15.16:0/600951001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c19d470 con 0x7fd31c100350 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd32177f640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 0x7fd31c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd320f7e640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd319ffb640 1 -- 172.21.15.16:0/600951001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304017030 con 0x7fd31c100350 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd319ffb640 1 -- 172.21.15.16:0/600951001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd304004020 con 0x7fd31c100350 2024-09-12T01:00:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c19d760 con 0x7fd31c100350 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd319ffb640 1 -- 172.21.15.16:0/600951001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3040047a0 con 0x7fd31c100350 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.279+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1a5040 con 0x7fd31c100350 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.281+0000 7fd319ffb640 1 -- 172.21.15.16:0/600951001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd3040041d0 con 0x7fd31c100350 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 Processor -- start 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 -- start start 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd319ffb640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 0x7fd3080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456260090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x56345626d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5634561c62f0 con 0x5634562605d0 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56345616dcf0 con 0x563456261b40 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd3255bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5634561c22b0 con 0x56345626b370 2024-09-12T01:00:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd32177f640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 0x7fd3080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd319ffb640 1 -- 172.21.15.16:0/600951001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd3040cef20 con 0x7fd31c100350 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd31bfff640 1 -- 172.21.15.16:0/2995709769 learned_addr learned my addr 172.21.15.16:0/2995709769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd320f7e640 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456260090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.282+0000 7fd32177f640 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x56345626d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 -- 172.21.15.16:0/2995709769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 msgr2=0x56345626d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x56345626d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 -- 172.21.15.16:0/2995709769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 msgr2=0x563456260090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456260090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 -- 172.21.15.16:0/2995709769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634561523a0 con 0x5634562605d0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd32177f640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 0x7fd3080a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd314005ab0 tx=0x7fd314031690 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd30c007db0 tx=0x7fd30c00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6b678ca10b31be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd312ffd640 1 -- 172.21.15.16:0/2995709769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c015020 con 0x5634562605d0 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd312ffd640 1 -- 172.21.15.16:0/2995709769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c011910 con 0x5634562605d0 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.283+0000 7fd312ffd640 1 -- 172.21.15.16:0/2995709769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c010980 con 0x5634562605d0 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- 172.21.15.16:0/2995709769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 msgr2=0x563456260a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd30c007db0 tx=0x7fd30c00c560 comp rx=0 tx=0).stop 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- 172.21.15.16:0/2995709769 shutdown_connections 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x56345626d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456260a10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2995709769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456260090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- 172.21.15.16:0/2995709769 >> 172.21.15.16:0/2995709769 conn(0x563456229310 msgr2=0x56345625cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- 172.21.15.16:0/2995709769 shutdown_connections 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- 172.21.15.16:0/2995709769 wait complete. 2024-09-12T01:00:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 -- start start 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.284+0000 7fd3255bb740 1 Processor -- start 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456267f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x5634562684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56345626e070 con 0x5634562605d0 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563456181060 con 0x563456261b40 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56345626e1f0 con 0x56345626b370 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd3255bb740 10 client.0 _ll_register_callbacks cb 0x563455ed48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52176/0 (socket says 172.21.15.16:52176) 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 -- 172.21.15.16:0/1496458773 learned_addr learned my addr 172.21.15.16:0/1496458773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd32177f640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x5634562684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd31bfff640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456267f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 -- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 msgr2=0x5634562684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x5634562684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 -- 172.21.15.16:0/1496458773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 msgr2=0x563456267f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456267f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 -- 172.21.15.16:0/1496458773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563455f373c0 con 0x5634562605d0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.285+0000 7fd320f7e640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd304005c60 tx=0x7fd30409aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebb3a77910d7196 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.286+0000 7fd2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.286+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30409bb10 con 0x5634562605d0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.286+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd3040cc8c0 con 0x5634562605d0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.286+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c011910 con 0x5634562605d0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3040ccac0 con 0x5634562605d0 2024-09-12T01:00:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 10 client.16737 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634562799f0 con 0x5634562605d0 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c011910 con 0x5634562605d0 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd3040b4200 con 0x5634562605d0 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 10 client.16737 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 20 client.16737 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56345627ae00 0x56345627d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 4 client.16737 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634561d3860 con 0x56345627ae00 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.287+0000 7fd31bfff640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56345627ae00 0x56345627d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.288+0000 7fd31bfff640 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56345627ae00 0x56345627d230 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562c7499a885e182 server_cookie=50afa230b2c50ce1 in_seq=0 out_seq=0 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.288+0000 7fd2fbfff640 10 client.16737 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd2fbfff640 1 -- 172.21.15.16:0/1496458773 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634561d3860 con 0x56345627ae00 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd2fbfff640 10 client.16737 handle_command_reply: tid=0 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 client.16737 shutdown 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 msgr2=0x56345626acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd304005c60 tx=0x7fd30409aea0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 shutdown_connections 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56345627ae00 0x56345627d230 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56345626b370 0x5634562684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563456261b40 0x563456267f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd2fbfff640 0 client.16737 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1496458773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x56345626acc0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 >> 172.21.15.16:0/1496458773 conn(0x563456229310 msgr2=0x563456262960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 shutdown_connections 2024-09-12T01:00:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 1 -- 172.21.15.16:0/1496458773 wait complete. 2024-09-12T01:00:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.289+0000 7fd3255bb740 20 client.16737 trim_cache size 0 max 16384 2024-09-12T01:00:39.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 Processor -- start 2024-09-12T01:00:39.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 -- start start 2024-09-12T01:00:39.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634562605d0 0x563456277310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456261300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1ad870 con 0x56345626b370 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1af3b0 con 0x5634562605d0 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.302+0000 7fd3255bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1adf40 con 0x563456261b40 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52190/0 (socket says 172.21.15.16:52190) 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 -- 172.21.15.16:0/1705344114 learned_addr learned my addr 172.21.15.16:0/1705344114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd320f7e640 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634562605d0 0x563456277310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd31bfff640 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456261300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 -- 172.21.15.16:0/1705344114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 msgr2=0x563456261300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456261300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 -- 172.21.15.16:0/1705344114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634562605d0 msgr2=0x563456277310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634562605d0 0x563456277310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 -- 172.21.15.16:0/1705344114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56345627d860 con 0x56345626b370 2024-09-12T01:00:39.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd32177f640 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd314009930 tx=0x7fd314005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab897934eeec5cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd3127fc640 1 -- 172.21.15.16:0/1705344114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314035050 con 0x56345626b370 2024-09-12T01:00:39.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd3127fc640 1 -- 172.21.15.16:0/1705344114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140046c0 con 0x56345626b370 2024-09-12T01:00:39.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.303+0000 7fd3127fc640 1 -- 172.21.15.16:0/1705344114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314003850 con 0x56345626b370 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- 172.21.15.16:0/1705344114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 msgr2=0x56345625c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd314009930 tx=0x7fd314005a20 comp rx=0 tx=0).stop 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- 172.21.15.16:0/1705344114 shutdown_connections 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56345626b370 0x56345625c700 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456261300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 --2- 172.21.15.16:0/1705344114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5634562605d0 0x563456277310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- 172.21.15.16:0/1705344114 >> 172.21.15.16:0/1705344114 conn(0x563456229310 msgr2=0x563456257a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- 172.21.15.16:0/1705344114 shutdown_connections 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- 172.21.15.16:0/1705344114 wait complete. 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 -- start start 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.304+0000 7fd3255bb740 1 Processor -- start 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456264b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 0x5634562650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1aece0 con 0x5634562605d0 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1ae610 con 0x56345626b370 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c1ab850 con 0x563456261b40 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd3255bb740 10 client.0 _ll_register_callbacks cb 0x7fd3040976d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52196/0 (socket says 172.21.15.16:52196) 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 -- 172.21.15.16:0/2711571012 learned_addr learned my addr 172.21.15.16:0/2711571012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456264b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd32177f640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 0x5634562650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 -- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 msgr2=0x563456264b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456264b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 -- 172.21.15.16:0/2711571012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 msgr2=0x5634562650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 0x5634562650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd320f7e640 1 -- 172.21.15.16:0/2711571012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563456265770 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.305+0000 7fd32177f640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 0x5634562650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd320f7e640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd304097de0 tx=0x7fd30400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5939236e0a1cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304014070 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd30409aa60 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3140046c0 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3040ccad0 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 10 client.16743 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634562e5960 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140046c0 con 0x5634562605d0 2024-09-12T01:00:39.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd3040662e0 con 0x5634562605d0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 10 client.16743 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 20 client.16743 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634562e60b0 0x5634562e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 4 client.16743 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.306+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634561cdd70 con 0x5634562e60b0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634562e60b0 0x5634562e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd31bfff640 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634562e60b0 0x5634562e84d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42413f618a4c813e server_cookie=4eb6c836494ad90 in_seq=0 out_seq=0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd2fbfff640 10 client.16743 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd2fbfff640 1 -- 172.21.15.16:0/2711571012 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634561cdd70 con 0x5634562e60b0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd2fbfff640 10 client.16743 handle_command_reply: tid=0 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd3255bb740 1 client.16743 shutdown 2024-09-12T01:00:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 msgr2=0x563456273a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd304097de0 tx=0x7fd30400b530 comp rx=0 tx=0).stop 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 shutdown_connections 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5634562e60b0 0x5634562e84d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.307+0000 7fd2fbfff640 0 client.16743 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56345626b370 0x5634562650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563456261b40 0x563456264b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 --2- 172.21.15.16:0/2711571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634562605d0 0x563456273a50 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 >> 172.21.15.16:0/2711571012 conn(0x563456229310 msgr2=0x56345625c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 shutdown_connections 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 1 -- 172.21.15.16:0/2711571012 wait complete. 2024-09-12T01:00:39.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3255bb740 20 client.16743 trim_cache size 0 max 16384 2024-09-12T01:00:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:00:39.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 msgr2=0x7fd3080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 0x7fd3080a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd314005ab0 tx=0x7fd314031690 comp rx=0 tx=0).stop 2024-09-12T01:00:39.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 msgr2=0x7fd31c1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d950 tx=0x7fd30400de10 comp rx=0 tx=0).stop 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 shutdown_connections 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd3080a2cf0 0x7fd3080a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd31c106f50 0x7fd31c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd31c100d00 0x7fd31c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 --2- 172.21.15.16:0/600951001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31c100350 0x7fd31c1984c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.308+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 >> 172.21.15.16:0/600951001 conn(0x7fd31c077210 msgr2=0x7fd31c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.309+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 shutdown_connections 2024-09-12T01:00:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:39.309+0000 7fd3231e2640 1 -- 172.21.15.16:0/600951001 wait complete. 2024-09-12T01:00:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:38 smithi159 ceph-mon[35566]: pgmap v140: 97 pgs: 97 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T01:00:41.155 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:00:41.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:00:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:40 smithi099 ceph-mon[35654]: pgmap v141: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:00:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:40 smithi159 ceph-mon[35566]: pgmap v141: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:00:41.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:40 smithi016 ceph-mon[30972]: pgmap v141: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:00:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- 172.21.15.16:0/616342405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50106a80 msgr2=0x7f6c50106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 --2- 172.21.15.16:0/616342405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50106a80 0x7f6c50106e60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009930 tx=0x7f6c4002a420 comp rx=0 tx=0).stop 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- 172.21.15.16:0/616342405 shutdown_connections 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 --2- 172.21.15.16:0/616342405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50101420 0x7f6c5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 --2- 172.21.15.16:0/616342405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50100a80 0x7f6c50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 --2- 172.21.15.16:0/616342405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50106a80 0x7f6c50106e60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- 172.21.15.16:0/616342405 >> 172.21.15.16:0/616342405 conn(0x7f6c500fc7a0 msgr2=0x7f6c500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- 172.21.15.16:0/616342405 shutdown_connections 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- 172.21.15.16:0/616342405 wait complete. 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 Processor -- start 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.958+0000 7f6c563af640 1 -- start start 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50101420 0x7f6c5019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50106a80 0x7f6c5019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50110e40 con 0x7f6c50100a80 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50110cc0 con 0x7f6c50106a80 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c563af640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50110fc0 con 0x7f6c50101420 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52216/0 (socket says 172.21.15.16:52216) 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2578758741 learned_addr learned my addr 172.21.15.16:0/2578758741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c5494c640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50106a80 0x7f6c5019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c47fff640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50101420 0x7f6c5019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2578758741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50101420 msgr2=0x7f6c5019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50101420 0x7f6c5019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2578758741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50106a80 msgr2=0x7f6c5019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50106a80 0x7f6c5019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2578758741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5019f630 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009230 tx=0x7f6c40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4dffb640 1 -- 172.21.15.16:0/2578758741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4003f030 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.959+0000 7f6c4dffb640 1 -- 172.21.15.16:0/2578758741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40003ad0 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.960+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5019f8c0 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.960+0000 7f6c4dffb640 1 -- 172.21.15.16:0/2578758741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40035070 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.960+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c501a7200 con 0x7f6c50100a80 2024-09-12T01:00:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.962+0000 7f6c4dffb640 1 -- 172.21.15.16:0/2578758741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c40042030 con 0x7f6c50100a80 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.962+0000 7f6c58788740 1 Processor -- start 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.962+0000 7f6c58788740 1 -- start start 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a47b60 0x565147a47f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a48480 0x565147a52800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651479ad280 con 0x565147a52d40 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565147954b10 con 0x565147a48480 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c58788740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651479a9240 con 0x565147a47b60 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52224/0 (socket says 172.21.15.16:52224) 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 -- 172.21.15.16:0/3764715261 learned_addr learned my addr 172.21.15.16:0/3764715261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a48480 0x565147a52800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c5494c640 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a47b60 0x565147a47f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c4dffb640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 0x7f6c3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 -- 172.21.15.16:0/3764715261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a47b60 msgr2=0x565147a47f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a47b60 0x565147a47f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 -- 172.21.15.16:0/3764715261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a48480 msgr2=0x565147a52800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a48480 0x565147a52800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 -- 172.21.15.16:0/3764715261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651479611f0 con 0x565147a52d40 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c5494c640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 0x7f6c3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c4dffb640 1 -- 172.21.15.16:0/2578758741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6c400ef930 con 0x7f6c50100a80 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.963+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800cbc0 tx=0x7f6c3800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301fd26231dbaafd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c467fc640 1 -- 172.21.15.16:0/3764715261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38010630 con 0x565147a52d40 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c467fc640 1 -- 172.21.15.16:0/3764715261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38010cb0 con 0x565147a52d40 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c467fc640 1 -- 172.21.15.16:0/3764715261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3801ac90 con 0x565147a52d40 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c5494c640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 0x7f6c3c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005fd0 tx=0x7f6c48004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 -- 172.21.15.16:0/3764715261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 msgr2=0x565147a55130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800cbc0 tx=0x7f6c3800e720 comp rx=0 tx=0).stop 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 -- 172.21.15.16:0/3764715261 shutdown_connections 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a55130 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a48480 0x565147a52800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 --2- 172.21.15.16:0/3764715261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a47b60 0x565147a47f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 -- 172.21.15.16:0/3764715261 >> 172.21.15.16:0/3764715261 conn(0x565147a440f0 msgr2=0x565147a444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 -- 172.21.15.16:0/3764715261 shutdown_connections 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.964+0000 7f6c58788740 1 -- 172.21.15.16:0/3764715261 wait complete. 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 -- start start 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 Processor -- start 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a4fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a52d40 0x565147a506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565147a55910 con 0x565147a48480 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565147967f10 con 0x565147a47b60 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565147a55a90 con 0x565147a52d40 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c58788740 10 client.0 _ll_register_callbacks cb 0x565147930a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52238/0 (socket says 172.21.15.16:52238) 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.965+0000 7f6c47fff640 1 -- 172.21.15.16:0/3732417467 learned_addr learned my addr 172.21.15.16:0/3732417467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a4fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c5494c640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a52d40 0x565147a506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 -- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a52d40 msgr2=0x565147a506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a52d40 0x565147a506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 -- 172.21.15.16:0/3732417467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 msgr2=0x565147a4fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a4fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 -- 172.21.15.16:0/3732417467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514771e6a0 con 0x565147a48480 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c38005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee0351767fd3046 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38010e60 con 0x565147a48480 2024-09-12T01:00:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c38010cb0 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.966+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6c38005240 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c3801a410 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 10 client.16755 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565147a61540 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38010cb0 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c380046c0 con 0x565147a48480 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 10 client.16755 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 20 client.16755 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147a64d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 4 client.16755 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651479ba7f0 con 0x565147a62950 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.967+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147a64d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.968+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147a64d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f394d9e996f852 server_cookie=fe2f760c43245bfb in_seq=0 out_seq=0 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.968+0000 7f6c2f7fe640 10 client.16755 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.968+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/3732417467 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651479ba7f0 con 0x565147a62950 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.968+0000 7f6c2f7fe640 10 client.16755 handle_command_reply: tid=0 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.968+0000 7f6c58788740 1 client.16755 shutdown 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 msgr2=0x565147a50190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c38005d40 comp rx=0 tx=0).stop 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 shutdown_connections 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147a64d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c2f7fe640 0 client.16755 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a52d40 0x565147a506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a48480 0x565147a50190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 --2- 172.21.15.16:0/3732417467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a4fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 >> 172.21.15.16:0/3732417467 conn(0x565147a440f0 msgr2=0x565147a49dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 shutdown_connections 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 1 -- 172.21.15.16:0/3732417467 wait complete. 2024-09-12T01:00:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.969+0000 7f6c58788740 20 client.16755 trim_cache size 0 max 16384 2024-09-12T01:00:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.986+0000 7f6c58788740 1 Processor -- start 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.986+0000 7f6c58788740 1 -- start start 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a6a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501afb00 con 0x565147a52d40 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501b1640 con 0x565147a47b60 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c58788740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501b01d0 con 0x565147a48480 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 -- 172.21.15.16:0/3070187671 learned_addr learned my addr 172.21.15.16:0/3070187671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c47fff640 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a6a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 -- 172.21.15.16:0/3070187671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 msgr2=0x565147a6a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a6a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 -- 172.21.15.16:0/3070187671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 msgr2=0x565147a6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 -- 172.21.15.16:0/3070187671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565147a55490 con 0x565147a52d40 2024-09-12T01:00:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.987+0000 7f6c5494c640 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009ea0 tx=0x7f6c48016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fb6ae298601a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c45ffb640 1 -- 172.21.15.16:0/3070187671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48012c80 con 0x565147a52d40 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c45ffb640 1 -- 172.21.15.16:0/3070187671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c48012e20 con 0x565147a52d40 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c45ffb640 1 -- 172.21.15.16:0/3070187671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48010960 con 0x565147a52d40 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 -- 172.21.15.16:0/3070187671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 msgr2=0x565147a4ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009ea0 tx=0x7f6c48016000 comp rx=0 tx=0).stop 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 -- 172.21.15.16:0/3070187671 shutdown_connections 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a52d40 0x565147a4ceb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a6a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 --2- 172.21.15.16:0/3070187671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a47b60 0x565147a6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 -- 172.21.15.16:0/3070187671 >> 172.21.15.16:0/3070187671 conn(0x565147a440f0 msgr2=0x565147a49dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 -- 172.21.15.16:0/3070187671 shutdown_connections 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.988+0000 7f6c58788740 1 -- 172.21.15.16:0/3070187671 wait complete. 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 -- start start 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 Processor -- start 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a52d40 0x565147acc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501b0f70 con 0x565147a47b60 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501b08a0 con 0x565147a52d40 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c501adae0 con 0x565147a48480 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c58788740 10 client.0 _ll_register_callbacks cb 0x7f6c38014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52260/0 (socket says 172.21.15.16:52260) 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.989+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2315755551 learned_addr learned my addr 172.21.15.16:0/2315755551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c47fff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c5494c640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a52d40 0x565147acc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 msgr2=0x565147a57350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a57350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2315755551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a52d40 msgr2=0x565147acc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a52d40 0x565147acc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 -- 172.21.15.16:0/2315755551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565147a57be0 con 0x565147a47b60 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c47fff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c4ffff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c400045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd5df8927370e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40049040 con 0x565147a47b60 2024-09-12T01:00:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6c40003c80 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c48012e20 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6c400bc580 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c58788740 10 client.16761 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565147acd960 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.990+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c48012e20 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6c400b84f0 con 0x565147a47b60 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 10 client.16761 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 20 client.16761 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147ad0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 4 client.16761 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5651479b4d00 con 0x565147a62950 2024-09-12T01:00:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c47fff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147ad0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c47fff640 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147ad0090 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d32cd969000580 server_cookie=ea5ec46cbdec0f67 in_seq=0 out_seq=0 2024-09-12T01:00:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.991+0000 7f6c2f7fe640 10 client.16761 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c2f7fe640 1 -- 172.21.15.16:0/2315755551 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5651479b4d00 con 0x565147a62950 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c2f7fe640 10 client.16761 handle_command_reply: tid=0 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 client.16761 shutdown 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 msgr2=0x565147a56e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c400045f0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 shutdown_connections 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x565147a62950 0x565147ad0090 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565147a52d40 0x565147acc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565147a48480 0x565147a57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 --2- 172.21.15.16:0/2315755551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565147a47b60 0x565147a56e10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.992+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 >> 172.21.15.16:0/2315755551 conn(0x565147a440f0 msgr2=0x565147a512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c2f7fe640 0 client.16761 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 shutdown_connections 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c58788740 1 -- 172.21.15.16:0/2315755551 wait complete. 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c58788740 20 client.16761 trim_cache size 0 max 16384 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "aee1f9a8-f616-4e8e-b77e-b6afb2cf6479", 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:00:41.996 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 msgr2=0x7f6c3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 0x7f6c3c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c48005fd0 tx=0x7f6c48004a40 comp rx=0 tx=0).stop 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 msgr2=0x7f6c5019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009230 tx=0x7f6c40004270 comp rx=0 tx=0).stop 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 shutdown_connections 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c3c0a2cf0 0x7f6c3c0a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c50106a80 0x7f6c5019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c50101420 0x7f6c5019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 --2- 172.21.15.16:0/2578758741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c50100a80 0x7f6c5019a680 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.993+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 >> 172.21.15.16:0/2578758741 conn(0x7f6c500fc7a0 msgr2=0x7f6c5006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.994+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 shutdown_connections 2024-09-12T01:00:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:41.994+0000 7f6c563af640 1 -- 172.21.15.16:0/2578758741 wait complete. 2024-09-12T01:00:42.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:00:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:42 smithi016 ceph-mon[30972]: pgmap v142: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:42 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:42 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:42 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:00:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:42 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:42 smithi099 ceph-mon[35654]: pgmap v142: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:42 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:00:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:42 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:42 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:00:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:42 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:42 smithi159 ceph-mon[35566]: pgmap v142: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:42 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:00:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:42 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:42 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:00:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:42 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 -- 172.21.15.16:0/2654772237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 msgr2=0x7fe80c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 --2- 172.21.15.16:0/2654772237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c1021c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009900 tx=0x7fe7fc02a410 comp rx=0 tx=0).stop 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 -- 172.21.15.16:0/2654772237 shutdown_connections 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 --2- 172.21.15.16:0/2654772237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c106f10 0x7fe80c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 --2- 172.21.15.16:0/2654772237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c102790 0x7fe80c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 --2- 172.21.15.16:0/2654772237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c1021c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 -- 172.21.15.16:0/2654772237 >> 172.21.15.16:0/2654772237 conn(0x7fe80c0fd8c0 msgr2=0x7fe80c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 -- 172.21.15.16:0/2654772237 shutdown_connections 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.630+0000 7fe810d15640 1 -- 172.21.15.16:0/2654772237 wait complete. 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 Processor -- start 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 -- start start 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c102790 0x7fe80c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 0x7fe80c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c111fb0 con 0x7fe80c101de0 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c111e30 con 0x7fe80c106f10 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe810d15640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c112130 con 0x7fe80c102790 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52272/0 (socket says 172.21.15.16:52272) 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.631+0000 7fe80a59c640 1 -- 172.21.15.16:0/2488362915 learned_addr learned my addr 172.21.15.16:0/2488362915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe803fff640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c102790 0x7fe80c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 0x7fe80c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 -- 172.21.15.16:0/2488362915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c102790 msgr2=0x7fe80c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c102790 0x7fe80c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 -- 172.21.15.16:0/2488362915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 msgr2=0x7fe80c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 0x7fe80c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 -- 172.21.15.16:0/2488362915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1a69d0 con 0x7fe80c101de0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 0x7fe80c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe80a59c640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006010 tx=0x7fe7fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe8037fe640 1 -- 172.21.15.16:0/2488362915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc03f030 con 0x7fe80c101de0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe8037fe640 1 -- 172.21.15.16:0/2488362915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc003ce0 con 0x7fe80c101de0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1a6c60 con 0x7fe80c101de0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe8037fe640 1 -- 172.21.15.16:0/2488362915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc035070 con 0x7fe80c101de0 2024-09-12T01:00:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.632+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1a7190 con 0x7fe80c101de0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.635+0000 7fe8037fe640 1 -- 172.21.15.16:0/2488362915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe7fc042030 con 0x7fe80c101de0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.635+0000 7fe8130ee740 1 Processor -- start 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 -- start start 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8037fe640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 0x7fe7f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x56331716b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x56331716da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633170c62e0 con 0x5633171604a0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56331706dce0 con 0x563317160dc0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8130ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5633170c22a0 con 0x56331716b680 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 0x7fe7f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52284/0 (socket says 172.21.15.16:52284) 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe80a59c640 1 -- 172.21.15.16:0/1097496062 learned_addr learned my addr 172.21.15.16:0/1097496062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe8037fe640 1 -- 172.21.15.16:0/2488362915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe7fc0ef830 con 0x7fe80c101de0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x56331716da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.636+0000 7fe803fff640 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x56331716b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 -- 172.21.15.16:0/1097496062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 msgr2=0x56331716da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x56331716da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 -- 172.21.15.16:0/1097496062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 msgr2=0x56331716b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x56331716b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 -- 172.21.15.16:0/1097496062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563317052440 con 0x5633171604a0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 0x7fe7f80a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe804009fd0 tx=0x7fe804009d60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe80a59c640 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0b86f0 tx=0x7fe7fc049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5d2b86b4377a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe800ff9640 1 -- 172.21.15.16:0/1097496062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc049070 con 0x5633171604a0 2024-09-12T01:00:43.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe800ff9640 1 -- 172.21.15.16:0/1097496062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc03b510 con 0x5633171604a0 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.637+0000 7fe800ff9640 1 -- 172.21.15.16:0/1097496062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0bc420 con 0x5633171604a0 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- 172.21.15.16:0/1097496062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 msgr2=0x563317160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0b86f0 tx=0x7fe7fc049bd0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- 172.21.15.16:0/1097496062 shutdown_connections 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x56331716da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x56331716b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- 172.21.15.16:0/1097496062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317160880 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- 172.21.15.16:0/1097496062 >> 172.21.15.16:0/1097496062 conn(0x56331715ca30 msgr2=0x56331715ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- 172.21.15.16:0/1097496062 shutdown_connections 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- 172.21.15.16:0/1097496062 wait complete. 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 -- start start 2024-09-12T01:00:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 Processor -- start 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 0x56331716b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.638+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 0x5633171682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56331716e010 con 0x56331716b680 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563317081050 con 0x5633171604a0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56331716e190 con 0x563317160dc0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 10 client.0 _ll_register_callbacks cb 0x5633170c1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52294/0 (socket says 172.21.15.16:52294) 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3821844108 learned_addr learned my addr 172.21.15.16:0/3821844108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 0x56331716b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe803fff640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 0x5633171682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 msgr2=0x5633171682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 0x5633171682e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3821844108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 msgr2=0x56331716b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 0x56331716b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3821844108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563316e373c0 con 0x56331716b680 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe803fff640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 0x5633171682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 0x56331716b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe804014340 tx=0x7fe804004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732d439cf4b0a0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804012d80 con 0x56331716b680 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633170d3850 con 0x56331716b680 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe804004b20 con 0x56331716b680 2024-09-12T01:00:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.639+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe804004d20 con 0x56331716b680 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 10 client.16773 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563317179990 con 0x56331716b680 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5633170d3850 con 0x56331716b680 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe804032450 con 0x56331716b680 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 10 client.16773 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 20 client.16773 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56331717adf0 0x56331717d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 4 client.16773 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633170cdd60 con 0x56331717adf0 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.640+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56331717adf0 0x56331717d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.641+0000 7fe7f1ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.641+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56331717adf0 0x56331717d290 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f57c05616e5c0 server_cookie=74e67feebe1a083e in_seq=0 out_seq=0 2024-09-12T01:00:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.641+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3821844108 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633170cdd60 con 0x56331717adf0 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.641+0000 7fe7f1ffb640 10 client.16773 handle_command_reply: tid=0 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 client.16773 shutdown 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 msgr2=0x563317168880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe804014340 tx=0x7fe804004ab0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 shutdown_connections 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56331717adf0 0x56331717d290 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x563317168880 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563317160dc0 0x5633171682e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3821844108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633171604a0 0x56331716b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe7f1ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 >> 172.21.15.16:0/3821844108 conn(0x56331715ca30 msgr2=0x56331715bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 shutdown_connections 2024-09-12T01:00:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 1 -- 172.21.15.16:0/3821844108 wait complete. 2024-09-12T01:00:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.642+0000 7fe8130ee740 20 client.16773 trim_cache size 0 max 16384 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 Processor -- start 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 -- start start 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 0x56331716a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317182b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1afa30 con 0x56331716b680 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1b1570 con 0x563317160dc0 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe8130ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1b0100 con 0x5633171604a0 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe80ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52302/0 (socket says 172.21.15.16:52302) 2024-09-12T01:00:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.659+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3253103182 learned_addr learned my addr 172.21.15.16:0/3253103182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3253103182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 msgr2=0x56331716a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 0x56331716a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe803fff640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317182b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 0x56331716a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3253103182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 msgr2=0x563317182b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317182b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 -- 172.21.15.16:0/3253103182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633171602d0 con 0x56331716b680 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 0x56331716a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe804000f80 tx=0x7fe804014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf27b523932e99ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe7f3fff640 1 -- 172.21.15.16:0/3253103182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804014430 con 0x56331716b680 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe7f3fff640 1 -- 172.21.15.16:0/3253103182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe804032450 con 0x56331716b680 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.660+0000 7fe7f3fff640 1 -- 172.21.15.16:0/3253103182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8040298e0 con 0x56331716b680 2024-09-12T01:00:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- 172.21.15.16:0/3253103182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 msgr2=0x5633171830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe804000f80 tx=0x7fe804014a00 comp rx=0 tx=0).stop 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- 172.21.15.16:0/3253103182 shutdown_connections 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56331716b680 0x5633171830a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317182b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3253103182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633171604a0 0x56331716a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- 172.21.15.16:0/3253103182 >> 172.21.15.16:0/3253103182 conn(0x56331715ca30 msgr2=0x563317169b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- 172.21.15.16:0/3253103182 shutdown_connections 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- 172.21.15.16:0/3253103182 wait complete. 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 -- start start 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.661+0000 7fe8130ee740 1 Processor -- start 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317170bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x563317171100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1b0ea0 con 0x5633171604a0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1b07d0 con 0x563317160dc0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1ada10 con 0x56331716b680 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 10 client.0 _ll_register_callbacks cb 0x7fe8040148b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52310/0 (socket says 172.21.15.16:52310) 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 -- 172.21.15.16:0/3558679494 learned_addr learned my addr 172.21.15.16:0/3558679494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe803fff640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317170bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x563317171100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 -- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 msgr2=0x563317171100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x563317171100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 -- 172.21.15.16:0/3558679494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 msgr2=0x563317170bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317170bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 -- 172.21.15.16:0/3558679494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563317171740 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80ad9d640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x563317171100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe80a59c640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0b8180 tx=0x7fe7fc03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a22eca7ed4e6f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe7f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc049040 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe7fc0ba020 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.662+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe804032450 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7fc0bb680 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 10 client.16779 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633171e5910 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe804032450 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7fc0ed1d0 con 0x5633171604a0 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 10 client.16779 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 20 client.16779 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5633171e5e70 0x5633171e8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 4 client.16779 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56331704a210 con 0x5633171e5e70 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.663+0000 7fe803fff640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5633171e5e70 0x5633171e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe803fff640 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5633171e5e70 0x5633171e8290 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d662e7d895571a6 server_cookie=934fd50d6f1be10d in_seq=0 out_seq=0 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe7f1ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe7f1ffb640 1 -- 172.21.15.16:0/3558679494 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56331704a210 con 0x5633171e5e70 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe7f1ffb640 10 client.16779 handle_command_reply: tid=0 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 client.16779 shutdown 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 msgr2=0x563317173870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0b8180 tx=0x7fe7fc03b460 comp rx=0 tx=0).stop 2024-09-12T01:00:43.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 shutdown_connections 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5633171e5e70 0x5633171e8290 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56331716b680 0x563317171100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563317160dc0 0x563317170bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe7f1ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 --2- 172.21.15.16:0/3558679494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633171604a0 0x563317173870 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 >> 172.21.15.16:0/3558679494 conn(0x56331715ca30 msgr2=0x56331716a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 shutdown_connections 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 1 -- 172.21.15.16:0/3558679494 wait complete. 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.664+0000 7fe8130ee740 20 client.16779 trim_cache size 0 max 16384 2024-09-12T01:00:43.675 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "aee1f9a8-f616-4e8e-b77e-b6afb2cf6479": { 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "aee1f9a8-f616-4e8e-b77e-b6afb2cf6479", 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:43.676 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 msgr2=0x7fe7f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 0x7fe7f80a5230 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe804009fd0 tx=0x7fe804009d60 comp rx=0 tx=0).stop 2024-09-12T01:00:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 msgr2=0x7fe80c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006010 tx=0x7fe7fc0042f0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 shutdown_connections 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7f80a2d70 0x7fe7f80a5230 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe80c106f10 0x7fe80c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe80c102790 0x7fe80c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 --2- 172.21.15.16:0/2488362915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe80c101de0 0x7fe80c0744f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 >> 172.21.15.16:0/2488362915 conn(0x7fe80c0fd8c0 msgr2=0x7fe80c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 shutdown_connections 2024-09-12T01:00:43.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:43.665+0000 7fe810d15640 1 -- 172.21.15.16:0/2488362915 wait complete. 2024-09-12T01:00:44.468 INFO:tasks.cephfs.filesystem:scrub status for tag:aee1f9a8-f616-4e8e-b77e-b6afb2cf6479 - {'path': '/', 'tag': 'aee1f9a8-f616-4e8e-b77e-b6afb2cf6479', 'options': 'recursive,force'} 2024-09-12T01:00:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:44 smithi099 ceph-mon[35654]: pgmap v143: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:44 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:00:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:44 smithi159 ceph-mon[35566]: pgmap v143: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:44 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:00:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:44 smithi016 ceph-mon[30972]: pgmap v143: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-12T01:00:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:44 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:00:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:46 smithi099 ceph-mon[35654]: pgmap v144: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T01:00:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:46 smithi159 ceph-mon[35566]: pgmap v144: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T01:00:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:46 smithi016 ceph-mon[30972]: pgmap v144: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-12T01:00:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:48 smithi099 ceph-mon[35654]: pgmap v145: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T01:00:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:48 smithi159 ceph-mon[35566]: pgmap v145: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T01:00:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:48 smithi016 ceph-mon[30972]: pgmap v145: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T01:00:49.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:00:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 -- 172.21.15.16:0/1358014991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 msgr2=0x7ffa00110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 --2- 172.21.15.16:0/1358014991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa00110830 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009ca0 tx=0x7ff9f802a950 comp rx=0 tx=0).stop 2024-09-12T01:00:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 -- 172.21.15.16:0/1358014991 shutdown_connections 2024-09-12T01:00:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 --2- 172.21.15.16:0/1358014991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa00110830 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 --2- 172.21.15.16:0/1358014991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074e30 0x7ffa0010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 --2- 172.21.15.16:0/1358014991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074510 0x7ffa000748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 -- 172.21.15.16:0/1358014991 >> 172.21.15.16:0/1358014991 conn(0x7ffa000fd8c0 msgr2=0x7ffa000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 -- 172.21.15.16:0/1358014991 shutdown_connections 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.725+0000 7ffa073ea640 1 -- 172.21.15.16:0/1358014991 wait complete. 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 Processor -- start 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 -- start start 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074510 0x7ffa0019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074e30 0x7ffa0019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00114160 con 0x7ffa0010e440 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa00113fe0 con 0x7ffa00074510 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa073ea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa001142e0 con 0x7ffa00074e30 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53828/0 (socket says 172.21.15.16:53828) 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 -- 172.21.15.16:0/3287803679 learned_addr learned my addr 172.21.15.16:0/3287803679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05186640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074510 0x7ffa0019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa04985640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074e30 0x7ffa0019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 -- 172.21.15.16:0/3287803679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074e30 msgr2=0x7ffa0019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074e30 0x7ffa0019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 -- 172.21.15.16:0/3287803679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074510 msgr2=0x7ffa0019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074510 0x7ffa0019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.726+0000 7ffa05987640 1 -- 172.21.15.16:0/3287803679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0019f5f0 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ffa05987640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004af0 tx=0x7ff9f802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8034040 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9f8030920 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0019f880 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8004780 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.727+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa001a71c0 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.728+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa000748f0 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.729+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff9f8037020 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.730+0000 7ff9f67fc640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 0x7ff9ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.730+0000 7ffa05186640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 0x7ff9ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.730+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7ff9f80f0080 con 0x7ffa0010e440 2024-09-12T01:00:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.730+0000 7ffa05186640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 0x7ff9ec0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009bd0 tx=0x7ff9fc005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.733+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9f80bcde0 con 0x7ffa0010e440 2024-09-12T01:00:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.946+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa001095c0 con 0x7ffa0010e440 2024-09-12T01:00:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.946+0000 7ff9f67fc640 1 -- 172.21.15.16:0/3287803679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7ff9f80bc6e0 con 0x7ffa0010e440 2024-09-12T01:00:50.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:00:50.946 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:00:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:00:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 msgr2=0x7ff9ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 0x7ff9ec0a5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009bd0 tx=0x7ff9fc005f90 comp rx=0 tx=0).stop 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 msgr2=0x7ffa0019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8004af0 tx=0x7ff9f802f5b0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 shutdown_connections 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9ec0a2d70 0x7ff9ec0a5230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0010e440 0x7ffa0019ef10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00074e30 0x7ffa0019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 --2- 172.21.15.16:0/3287803679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa00074510 0x7ffa0019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 >> 172.21.15.16:0/3287803679 conn(0x7ffa000fd8c0 msgr2=0x7ffa000fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 shutdown_connections 2024-09-12T01:00:50.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:50.949+0000 7ffa073ea640 1 -- 172.21.15.16:0/3287803679 wait complete. 2024-09-12T01:00:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:50 smithi099 ceph-mon[35654]: pgmap v146: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T01:00:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:50 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3287803679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:50 smithi159 ceph-mon[35566]: pgmap v146: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T01:00:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:50 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3287803679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:51.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:50 smithi016 ceph-mon[30972]: pgmap v146: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T01:00:51.373 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:50 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3287803679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:00:51.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id abb9bace --release --await' 2024-09-12T01:00:51.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id abb9bace --release --await 2024-09-12T01:00:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 -- 172.21.15.16:0/4198702423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c1055b0 msgr2=0x7f6e8c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/4198702423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c10d130 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400aa10 tx=0x7f6e8402b720 comp rx=0 tx=0).stop 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 -- 172.21.15.16:0/4198702423 shutdown_connections 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/4198702423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c10d130 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/4198702423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c100e30 0x7f6e8c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/4198702423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100480 0x7f6e8c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 -- 172.21.15.16:0/4198702423 >> 172.21.15.16:0/4198702423 conn(0x7f6e8c077210 msgr2=0x7f6e8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 -- 172.21.15.16:0/4198702423 shutdown_connections 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.531+0000 7f6e91fbb640 1 -- 172.21.15.16:0/4198702423 wait complete. 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 Processor -- start 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 -- start start 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100e30 0x7f6e8c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c10fde0 con 0x7f6e8c100480 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c10fc60 con 0x7f6e8c1055b0 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e91fbb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c10ff60 con 0x7f6e8c100e30 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53842/0 (socket says 172.21.15.16:53842) 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/187129831 learned_addr learned my addr 172.21.15.16:0/187129831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.532+0000 7f6e8affd640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100e30 0x7f6e8c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/187129831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100e30 msgr2=0x7f6e8c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100e30 0x7f6e8c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/187129831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c1055b0 msgr2=0x7f6e8c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/187129831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c1a6860 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00cce0 tx=0x7f6e7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e88ff9640 1 -- 172.21.15.16:0/187129831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c00f650 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e88ff9640 1 -- 172.21.15.16:0/187129831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e7c00fcd0 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c1a6b50 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e88ff9640 1 -- 172.21.15.16:0/187129831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c018c70 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.533+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1a7090 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e88ff9640 1 -- 172.21.15.16:0/187129831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e7c00f7f0 con 0x7f6e8c100480 2024-09-12T01:00:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 Processor -- start 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 -- start start 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d690440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f2d5f5620 con 0x560f2d69b240 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f2d59ce50 con 0x560f2d690980 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e94394740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f2d5f15e0 con 0x560f2d690060 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e88ff9640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 0x7f6e740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53846/0 (socket says 172.21.15.16:53846) 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e8affd640 1 -- 172.21.15.16:0/3130429913 learned_addr learned my addr 172.21.15.16:0/3130429913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.536+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d690440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 0x7f6e740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e88ff9640 1 -- 172.21.15.16:0/187129831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6e7c0cf810 con 0x7f6e8c100480 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 -- 172.21.15.16:0/3130429913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 msgr2=0x560f2d690440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d690440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 -- 172.21.15.16:0/3130429913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 msgr2=0x560f2d69ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 -- 172.21.15.16:0/3130429913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2d58dd40 con 0x560f2d69b240 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8affd640 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bc30 tx=0x7f6e800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da45ac433f4dc665 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e71ffb640 1 -- 172.21.15.16:0/3130429913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8001c480 con 0x560f2d69b240 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 0x7f6e740a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e84030660 tx=0x7f6e840046c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e71ffb640 1 -- 172.21.15.16:0/3130429913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8001cb00 con 0x560f2d69b240 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.537+0000 7f6e71ffb640 1 -- 172.21.15.16:0/3130429913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011ba0 con 0x560f2d69b240 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- 172.21.15.16:0/3130429913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 msgr2=0x560f2d69d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000bc30 tx=0x7f6e800107a0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- 172.21.15.16:0/3130429913 shutdown_connections 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d69d630 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 --2- 172.21.15.16:0/3130429913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d690440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- 172.21.15.16:0/3130429913 >> 172.21.15.16:0/3130429913 conn(0x560f2d68c5f0 msgr2=0x560f2d68c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- 172.21.15.16:0/3130429913 shutdown_connections 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- 172.21.15.16:0/3130429913 wait complete. 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 -- start start 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.538+0000 7f6e94394740 1 Processor -- start 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d6980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d698620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f2d69de10 con 0x560f2d69b240 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f2d57df10 con 0x560f2d690980 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f2d69df90 con 0x560f2d690060 2024-09-12T01:00:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e94394740 10 client.0 _ll_register_callbacks cb 0x560f2d5a1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53856/0 (socket says 172.21.15.16:53856) 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 -- 172.21.15.16:0/951035309 learned_addr learned my addr 172.21.15.16:0/951035309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8affd640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d698620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d6980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 -- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 msgr2=0x560f2d6980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d6980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 -- 172.21.15.16:0/951035309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 msgr2=0x560f2d698620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d698620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 -- 172.21.15.16:0/951035309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2d304580 con 0x560f2d69b240 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6e84034000 tx=0x7f6e8400a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e06c04fe25e50a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.539+0000 7f6e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8404d030 con 0x560f2d69b240 2024-09-12T01:00:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6e84043070 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e8001cb00 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e84057050 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 10 client.16797 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f2d3ce960 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8001cb00 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e84048070 con 0x560f2d69b240 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 10 client.16797 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 10 client.16797 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 20 client.16797 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d6aad90 0x560f2d6ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 4 client.16797 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.540+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f2d602b90 con 0x560f2d6aad90 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.541+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d6aad90 0x560f2d6ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.541+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d6aad90 0x560f2d6ad230 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b337b639539af7 server_cookie=20494a4556316b5c in_seq=0 out_seq=0 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.541+0000 7f6e6affd640 10 client.16797 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e6affd640 1 -- 172.21.15.16:0/951035309 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f2d602b90 con 0x560f2d6aad90 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e6affd640 10 client.16797 handle_command_reply: tid=0 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 client.16797 shutdown 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 msgr2=0x560f2d698b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6e84034000 tx=0x7f6e8400a370 comp rx=0 tx=0).stop 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 shutdown_connections 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d6aad90 0x560f2d6ad230 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d69b240 0x560f2d698b60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d698620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 --2- 172.21.15.16:0/951035309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d6980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 >> 172.21.15.16:0/951035309 conn(0x560f2d68c5f0 msgr2=0x560f2d6922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e6affd640 0 client.16797 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 shutdown_connections 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 1 -- 172.21.15.16:0/951035309 wait complete. 2024-09-12T01:00:52.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.542+0000 7f6e94394740 20 client.16797 trim_cache size 0 max 16384 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 Processor -- start 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 -- start start 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d68b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d69b240 0x560f2d6b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1af9a0 con 0x560f2d690980 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1b14e0 con 0x560f2d69b240 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e94394740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1b0070 con 0x560f2d690060 2024-09-12T01:00:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53872/0 (socket says 172.21.15.16:53872) 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e8affd640 1 -- 172.21.15.16:0/1164292734 learned_addr learned my addr 172.21.15.16:0/1164292734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.559+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d68b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d69b240 0x560f2d6b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 -- 172.21.15.16:0/1164292734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 msgr2=0x560f2d68b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d68b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 -- 172.21.15.16:0/1164292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d69b240 msgr2=0x560f2d6b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d69b240 0x560f2d6b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 -- 172.21.15.16:0/1164292734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f2d6907b0 con 0x560f2d690980 2024-09-12T01:00:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x560f2d68d970 tx=0x7f6e800116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f467152b6738d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e717fa640 1 -- 172.21.15.16:0/1164292734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000bd20 con 0x560f2d690980 2024-09-12T01:00:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e717fa640 1 -- 172.21.15.16:0/1164292734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8002c070 con 0x560f2d690980 2024-09-12T01:00:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.560+0000 7f6e717fa640 1 -- 172.21.15.16:0/1164292734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011730 con 0x560f2d690980 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- 172.21.15.16:0/1164292734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 msgr2=0x560f2d6b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x560f2d68d970 tx=0x7f6e800116c0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- 172.21.15.16:0/1164292734 shutdown_connections 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d69b240 0x560f2d6b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690980 0x560f2d6b2b50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 --2- 172.21.15.16:0/1164292734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d690060 0x560f2d68b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- 172.21.15.16:0/1164292734 >> 172.21.15.16:0/1164292734 conn(0x560f2d68c5f0 msgr2=0x560f2d68af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- 172.21.15.16:0/1164292734 shutdown_connections 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- 172.21.15.16:0/1164292734 wait complete. 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 -- start start 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.561+0000 7f6e94394740 1 Processor -- start 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 0x560f2d69fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1b0e10 con 0x560f2d690060 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1b0740 con 0x560f2d690980 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1ad980 con 0x560f2d69b240 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e94394740 10 client.0 _ll_register_callbacks cb 0x7f6e8400a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:00:52.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53884/0 (socket says 172.21.15.16:53884) 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/1509403818 learned_addr learned my addr 172.21.15.16:0/1509403818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 msgr2=0x560f2d69fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 0x560f2d69fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 0x560f2d69fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/1509403818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 msgr2=0x560f2d69f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 -- 172.21.15.16:0/1509403818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2d714e10 con 0x560f2d690060 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8bfff640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 0x560f2d69fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:00:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e8b7fe640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0981c0 tx=0x7f6e7c06dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef9a2b4d26e67fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.562+0000 7f6e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c06a070 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6e7c0983d0 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e8002c070 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e7c09c460 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 10 client.16803 fetch_fsmap learned FSMap version 28 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f2d7161b0 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8002c070 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e7c09bfc0 con 0x560f2d690060 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 10 client.16803 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:00:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 10 client.16803 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 20 client.16803 populate_metadata read hostname 'smithi016' 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d716830 0x560f2d718c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 4 client.16803 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "abb9bace", "release": true, "await": true}] 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "abb9bace", "release": true, "await": true}) -- 0x560f2d5fd0a0 con 0x560f2d716830 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.563+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d716830 0x560f2d718c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.564+0000 7f6e8affd640 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d716830 0x560f2d718c40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a650e309f79276ae server_cookie=e9e93b51f6c9ddc6 in_seq=0 out_seq=0 2024-09-12T01:00:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.564+0000 7f6e6affd640 10 client.16803 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e6affd640 1 -- 172.21.15.16:0/1509403818 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f2d5fd0a0 con 0x560f2d716830 2024-09-12T01:00:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e6affd640 10 client.16803 handle_command_reply: tid=0 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 client.16803 shutdown 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 msgr2=0x560f2d69f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0981c0 tx=0x7f6e7c06dd20 comp rx=0 tx=0).stop 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 shutdown_connections 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f2d716830 0x560f2d718c40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e6affd640 0 client.16803 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f2d69b240 0x560f2d69fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f2d690980 0x560f2d69f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 --2- 172.21.15.16:0/1509403818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2d690060 0x560f2d69f280 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 >> 172.21.15.16:0/1509403818 conn(0x560f2d68c5f0 msgr2=0x560f2d69a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.567+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 shutdown_connections 2024-09-12T01:00:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e94394740 1 -- 172.21.15.16:0/1509403818 wait complete. 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e94394740 20 client.16803 trim_cache size 0 max 16384 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 20, 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "abb9bace": { 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 20, 2024-09-12T01:00:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.2, 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:00:52.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:00:52.573 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 msgr2=0x7f6e740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 0x7f6e740a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e84030660 tx=0x7f6e840046c0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 msgr2=0x7f6e8c0fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00cce0 tx=0x7f6e7c00a620 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 shutdown_connections 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e740a2cf0 0x7f6e740a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e8c1055b0 0x7f6e8c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e8c100e30 0x7f6e8c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 --2- 172.21.15.16:0/187129831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e8c100480 0x7f6e8c0fc690 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 >> 172.21.15.16:0/187129831 conn(0x7f6e8c077210 msgr2=0x7f6e8c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:00:52.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 shutdown_connections 2024-09-12T01:00:52.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:00:52.568+0000 7f6e91fbb640 1 -- 172.21.15.16:0/187129831 wait complete. 2024-09-12T01:00:53.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:52 smithi099 ceph-mon[35654]: pgmap v147: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:53.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:52 smithi159 ceph-mon[35566]: pgmap v147: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:53.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:52 smithi016 ceph-mon[30972]: pgmap v147: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:53.480 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'abb9bace', total seconds elapsed: 23.6 2024-09-12T01:00:53.572 INFO:tasks.workunit.client.0.smithi016.stdout:64/600: open fd = 3 2024-09-12T01:00:53.572 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.572 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.572 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.572 INFO:tasks.workunit.client.0.smithi016.stdout:65/600: open fd = 3 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:66/600: open fd = 3 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:67/600: open fd = 3 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.573 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:68/600: open fd = 3 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:69/600: open fd = 3 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:70/600: open fd = 3 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.574 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:71/600: open fd = 3 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:72/600: open fd = 3 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:73/600: open fd = 3 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.575 INFO:tasks.workunit.client.0.smithi016.stdout:74/600: open fd = 3 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:75/600: open fd = 3 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:76/600: open fd = 3 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.576 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:77/600: open fd = 3 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:78/600: open fd = 3 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:79/600: open fd = 3 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.577 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:80/600: open fd = 3 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:81/600: open fd = 3 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:82/600: open fd = 3 2024-09-12T01:00:53.578 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:83/600: open fd = 3 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:84/600: open fd = 3 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:85/600: open fd = 3 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:86/600: open fd = 3 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:87/600: open fd = 3 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:88/600: open fd = 3 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:89/600: open fd = 3 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:90/600: open fd = 3 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:91/600: open fd = 3 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:92/600: open fd = 3 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:93/600: open fd = 3 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:94/600: open fd = 3 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:95/600: open fd = 3 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:96/600: open fd = 3 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:97/600: open fd = 3 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:98/600: open fd = 3 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:99/600: open fd = 3 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:100/600: open fd = 3 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:101/600: open fd = 3 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:102/600: open fd = 3 2024-09-12T01:00:53.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:103/600: open fd = 3 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:104/600: open fd = 3 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:105/600: open fd = 3 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:106/600: open fd = 3 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:107/600: open fd = 3 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:108/600: open fd = 3 2024-09-12T01:00:53.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:109/600: open fd = 3 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:110/600: open fd = 3 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.588 INFO:tasks.workunit.client.0.smithi016.stdout:111/600: open fd = 3 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:112/600: open fd = 3 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:113/600: open fd = 3 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.589 INFO:tasks.workunit.client.0.smithi016.stdout:114/600: open fd = 3 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:115/600: open fd = 3 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:116/600: open fd = 3 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:117/600: open fd = 3 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:118/600: open fd = 3 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:119/600: open fd = 3 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:120/600: open fd = 3 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:121/600: open fd = 3 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:122/600: open fd = 3 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:123/600: open fd = 3 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:124/600: open fd = 3 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:125/600: open fd = 3 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:53.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:00:53.594 INFO:tasks.workunit.client.0.smithi016.stdout:126/600: open fd = 3 2024-09-12T01:00:53.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:00:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:54 smithi099 ceph-mon[35654]: pgmap v148: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:54 smithi159 ceph-mon[35566]: pgmap v148: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:54 smithi016 ceph-mon[30972]: pgmap v148: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-12T01:00:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:56 smithi099 ceph-mon[35654]: pgmap v149: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-09-12T01:00:57.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:56 smithi159 ceph-mon[35566]: pgmap v149: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-09-12T01:00:57.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:56 smithi016 ceph-mon[30972]: pgmap v149: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 13 op/s 2024-09-12T01:00:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:00:58 smithi099 ceph-mon[35654]: pgmap v150: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T01:00:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:00:58 smithi159 ceph-mon[35566]: pgmap v150: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T01:00:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:00:58 smithi016 ceph-mon[30972]: pgmap v150: 97 pgs: 97 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-09-12T01:01:01.265 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:00 smithi159 ceph-mon[35566]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T01:01:01.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:00 smithi099 ceph-mon[35654]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T01:01:01.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:00 smithi016 ceph-mon[30972]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-09-12T01:01:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:02 smithi099 ceph-mon[35654]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:02 smithi159 ceph-mon[35566]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:02 smithi016 ceph-mon[30972]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:05.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:05 smithi099 ceph-mon[35654]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:05 smithi159 ceph-mon[35566]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:05 smithi016 ceph-mon[30972]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:01:07.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:07 smithi099 ceph-mon[35654]: pgmap v154: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-12T01:01:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:07 smithi159 ceph-mon[35566]: pgmap v154: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-12T01:01:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:07 smithi016 ceph-mon[30972]: pgmap v154: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-12T01:01:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:09 smithi099 ceph-mon[35654]: pgmap v155: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-09-12T01:01:09.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:09 smithi159 ceph-mon[35566]: pgmap v155: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-09-12T01:01:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:09 smithi016 ceph-mon[30972]: pgmap v155: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-09-12T01:01:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:11 smithi099 ceph-mon[35654]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-12T01:01:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:11 smithi159 ceph-mon[35566]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-12T01:01:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:11 smithi016 ceph-mon[30972]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-12T01:01:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:13 smithi099 ceph-mon[35654]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:13 smithi159 ceph-mon[35566]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:13 smithi016 ceph-mon[30972]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:14.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:01:14.745 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:15 smithi016 ceph-mon[30972]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:15 smithi099 ceph-mon[35654]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 -- 172.21.15.16:0/3345073529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8108290 msgr2=0x7f9bf810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 --2- 172.21.15.16:0/3345073529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8108290 0x7f9bf810ec30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009cd0 tx=0x7f9bf002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 -- 172.21.15.16:0/3345073529 shutdown_connections 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 --2- 172.21.15.16:0/3345073529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8108290 0x7f9bf810ec30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 --2- 172.21.15.16:0/3345073529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8103740 0x7f9bf8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 --2- 172.21.15.16:0/3345073529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 0x7f9bf8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.238+0000 7f9c006cd640 1 -- 172.21.15.16:0/3345073529 >> 172.21.15.16:0/3345073529 conn(0x7f9bf80fc7d0 msgr2=0x7f9bf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.239+0000 7f9c006cd640 1 -- 172.21.15.16:0/3345073529 shutdown_connections 2024-09-12T01:01:15.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.239+0000 7f9c006cd640 1 -- 172.21.15.16:0/3345073529 wait complete. 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.239+0000 7f9c006cd640 1 Processor -- start 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.239+0000 7f9c006cd640 1 -- start start 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.239+0000 7f9c006cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 0x7f9bf8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 0x7f9bf819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8112090 con 0x7f9bf8103740 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8111f10 con 0x7f9bf8108290 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8112210 con 0x7f9bf8102d90 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44362/0 (socket says 172.21.15.16:44362) 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 -- 172.21.15.16:0/2962036786 learned_addr learned my addr 172.21.15.16:0/2962036786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:15.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 0x7f9bf8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 0x7f9bf819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 -- 172.21.15.16:0/2962036786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 msgr2=0x7f9bf8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 0x7f9bf8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 -- 172.21.15.16:0/2962036786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 msgr2=0x7f9bf819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 0x7f9bf819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 -- 172.21.15.16:0/2962036786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf819d520 con 0x7f9bf8103740 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 0x7f9bf819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9be8005fd0 tx=0x7f9be800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2962036786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8017070 con 0x7f9bf8103740 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.240+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf819d810 con 0x7f9bf8103740 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.241+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2962036786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9be8009c30 con 0x7f9bf8103740 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.241+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2962036786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8010460 con 0x7f9bf8103740 2024-09-12T01:01:15.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.241+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf81a50f0 con 0x7f9bf8103740 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.243+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2962036786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9be8003e70 con 0x7f9bf8103740 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.243+0000 7f9c02aa6740 1 Processor -- start 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 -- start start 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e909373a0 0x557e90937780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 0x557e90948e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e9089d2e0 con 0x557e90937cc0 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e90844ce0 con 0x557e90946a90 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e908992a0 con 0x557e909373a0 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44372/0 (socket says 172.21.15.16:44372) 2024-09-12T01:01:15.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 -- 172.21.15.16:0/2573843656 learned_addr learned my addr 172.21.15.16:0/2573843656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e909373a0 0x557e90937780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 0x557e90948e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 -- 172.21.15.16:0/2573843656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e909373a0 msgr2=0x557e90937780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e909373a0 0x557e90937780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 -- 172.21.15.16:0/2573843656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 msgr2=0x557e90948e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 0x557e90948e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfe469640 1 -- 172.21.15.16:0/2573843656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e90829440 con 0x557e90937cc0 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 0x557e90948e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.244+0000 7f9bef7fe640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 0x7f9be40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009c90 tx=0x7f9bf40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70de2a9d5008a60c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9bef7fe640 1 -- 172.21.15.16:0/2962036786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f9be80cf0b0 con 0x7f9bf8103740 2024-09-12T01:01:15.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 0x7f9be40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9becff9640 1 -- 172.21.15.16:0/2573843656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4012450 con 0x557e90937cc0 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9becff9640 1 -- 172.21.15.16:0/2573843656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4012ad0 con 0x557e90937cc0 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.245+0000 7f9becff9640 1 -- 172.21.15.16:0/2573843656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400fbd0 con 0x557e90937cc0 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2573843656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 msgr2=0x557e90946550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009c90 tx=0x7f9bf40117b0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 0x7f9be40a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005e30 tx=0x7f9bf0009800 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2573843656 shutdown_connections 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90946a90 0x557e90948e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90937cc0 0x557e90946550 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2573843656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e909373a0 0x557e90937780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2573843656 >> 172.21.15.16:0/2573843656 conn(0x557e90933a30 msgr2=0x557e90933e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2573843656 shutdown_connections 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2573843656 wait complete. 2024-09-12T01:01:15.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 -- start start 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.246+0000 7f9c02aa6740 1 Processor -- start 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e90946350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90943600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557e90949710 con 0x557e90946a90 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557e90858050 con 0x557e909373a0 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e90949890 con 0x557e90937cc0 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9c02aa6740 10 client.0 _ll_register_callbacks cb 0x557e90898160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44378/0 (socket says 172.21.15.16:44378) 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 -- 172.21.15.16:0/3306385860 learned_addr learned my addr 172.21.15.16:0/3306385860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfe469640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e90946350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90943600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 -- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 msgr2=0x557e90943600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90943600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 -- 172.21.15.16:0/3306385860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 msgr2=0x557e90946350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e90946350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 -- 172.21.15.16:0/3306385860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9060e3c0 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009990 tx=0x7f9bf00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18785e4e2b0dc8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.247+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00041b0 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f9bf0004350 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf4012ad0 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0004550 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 10 client.16815 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e90955010 con 0x557e90946a90 2024-09-12T01:01:15.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4012ad0 con 0x557e90946a90 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bf0030070 con 0x557e90946a90 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 10 client.16815 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 20 client.16815 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e90956420 0x557e909588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 4 client.16815 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.248+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e908aa850 con 0x557e90956420 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.249+0000 7f9bfe469640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e90956420 0x557e909588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.249+0000 7f9bfe469640 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e90956420 0x557e909588a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365431a649bf48a8 server_cookie=1839f117271f11cc in_seq=0 out_seq=0 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.249+0000 7f9be1ffb640 10 client.16815 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9be1ffb640 1 -- 172.21.15.16:0/3306385860 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e908aa850 con 0x557e90956420 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9be1ffb640 10 client.16815 handle_command_reply: tid=0 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 client.16815 shutdown 2024-09-12T01:01:15.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 msgr2=0x557e90943b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009990 tx=0x7f9bf00095c0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 shutdown_connections 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e90956420 0x557e909588a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e90943b40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90943600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/3306385860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e90946350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9be1ffb640 0 client.16815 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 >> 172.21.15.16:0/3306385860 conn(0x557e90933a30 msgr2=0x557e90939760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 shutdown_connections 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 1 -- 172.21.15.16:0/3306385860 wait complete. 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.250+0000 7f9c02aa6740 20 client.16815 trim_cache size 0 max 16384 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.267+0000 7f9c02aa6740 1 Processor -- start 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.267+0000 7f9c02aa6740 1 -- start start 2024-09-12T01:01:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e909529b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90932a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81ab570 con 0x557e90946a90 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81ad0b0 con 0x557e909373a0 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81abc40 con 0x557e90937cc0 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44390/0 (socket says 172.21.15.16:44390) 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 -- 172.21.15.16:0/1549385061 learned_addr learned my addr 172.21.15.16:0/1549385061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfe469640 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e909529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90932a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 -- 172.21.15.16:0/1549385061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 msgr2=0x557e90932a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90932a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 -- 172.21.15.16:0/1549385061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 msgr2=0x557e909529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e909529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 -- 172.21.15.16:0/1549385061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e90933860 con 0x557e90946a90 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.268+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0050450 tx=0x7f9bf0031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8e97cd42e7d2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.269+0000 7f9be3fff640 1 -- 172.21.15.16:0/1549385061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf005d070 con 0x557e90946a90 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.269+0000 7f9be3fff640 1 -- 172.21.15.16:0/1549385061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf0036040 con 0x557e90946a90 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.269+0000 7f9be3fff640 1 -- 172.21.15.16:0/1549385061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00305a0 con 0x557e90946a90 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- 172.21.15.16:0/1549385061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 msgr2=0x557e9095e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0050450 tx=0x7f9bf0031490 comp rx=0 tx=0).stop 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- 172.21.15.16:0/1549385061 shutdown_connections 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e90946a90 0x557e9095e1c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90937cc0 0x557e90932a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/1549385061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e909373a0 0x557e909529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- 172.21.15.16:0/1549385061 >> 172.21.15.16:0/1549385061 conn(0x557e90933a30 msgr2=0x557e90932350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- 172.21.15.16:0/1549385061 shutdown_connections 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- 172.21.15.16:0/1549385061 wait complete. 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 -- start start 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.270+0000 7f9c02aa6740 1 Processor -- start 2024-09-12T01:01:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 0x557e9094b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90946a90 0x557e9094b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81ac9e0 con 0x557e909373a0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81ac310 con 0x557e90937cc0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81a9550 con 0x557e90946a90 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9c02aa6740 10 client.0 _ll_register_callbacks cb 0x7f9bf4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44400/0 (socket says 172.21.15.16:44400) 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 -- 172.21.15.16:0/2229847335 learned_addr learned my addr 172.21.15.16:0/2229847335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 0x557e9094b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfec6a640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90946a90 0x557e9094b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 -- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90946a90 msgr2=0x557e9094b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90946a90 0x557e9094b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 -- 172.21.15.16:0/2229847335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 msgr2=0x557e9094b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 0x557e9094b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 -- 172.21.15.16:0/2229847335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e909c0390 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 0x557e9094b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9bfe469640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x557e9093a090 tx=0x7f9bf4009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2833c1b21b14bffd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.271+0000 7f9be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400fab0 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f9bf4020040 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf0036040 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf403c050 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 10 client.16821 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e909c1620 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf0036040 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9bf4003710 con 0x557e909373a0 2024-09-12T01:01:15.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 10 client.16821 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 20 client.16821 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e909c1af0 0x557e909c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 4 client.16821 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.272+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e908a4d60 con 0x557e909c1af0 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e909c1af0 0x557e909c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9be1ffb640 10 client.16821 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9bfdc68640 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e909c1af0 0x557e909c3ee0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34cc1aca34b6177e server_cookie=8b44c729f3ce6a8a in_seq=0 out_seq=0 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9be1ffb640 1 -- 172.21.15.16:0/2229847335 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e908a4d60 con 0x557e909c1af0 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9be1ffb640 10 client.16821 handle_command_reply: tid=0 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 client.16821 shutdown 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 msgr2=0x557e9094aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x557e9093a090 tx=0x7f9bf4009be0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 shutdown_connections 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x557e909c1af0 0x557e909c3ee0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e90946a90 0x557e9094b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557e90937cc0 0x557e9094b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 --2- 172.21.15.16:0/2229847335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557e909373a0 0x557e9094aad0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 >> 172.21.15.16:0/2229847335 conn(0x557e90933a30 msgr2=0x557e90938ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9be1ffb640 0 client.16821 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.273+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 shutdown_connections 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c02aa6740 1 -- 172.21.15.16:0/2229847335 wait complete. 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c02aa6740 20 client.16821 trim_cache size 0 max 16384 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 msgr2=0x7f9be40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 0x7f9be40a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005e30 tx=0x7f9bf0009800 comp rx=0 tx=0).stop 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 msgr2=0x7f9bf8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9be8005fd0 tx=0x7f9be800dde0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 shutdown_connections 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9be40a2cf0 0x7f9be40a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf8108290 0x7f9bf819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8103740 0x7f9bf8198ab0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 --2- 172.21.15.16:0/2962036786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bf8102d90 0x7f9bf8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 >> 172.21.15.16:0/2962036786 conn(0x7f9bf80fc7d0 msgr2=0x7f9bf8105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 shutdown_connections 2024-09-12T01:01:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:15.274+0000 7f9c006cd640 1 -- 172.21.15.16:0/2962036786 wait complete. 2024-09-12T01:01:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:15 smithi159 ceph-mon[35566]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 27 op/s 2024-09-12T01:01:16.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:01:16.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:01:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 -- 172.21.15.16:0/3630510531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8100e20 msgr2=0x7fe4a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/3630510531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8100e20 0x7fe4a8101200 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe498009900 tx=0x7fe49802a480 comp rx=0 tx=0).stop 2024-09-12T01:01:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 -- 172.21.15.16:0/3630510531 shutdown_connections 2024-09-12T01:01:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/3630510531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/3630510531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8102570 0x7fe4a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/3630510531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8100e20 0x7fe4a8101200 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 -- 172.21.15.16:0/3630510531 >> 172.21.15.16:0/3630510531 conn(0x7fe4a80fc7d0 msgr2=0x7fe4a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 -- 172.21.15.16:0/3630510531 shutdown_connections 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 -- 172.21.15.16:0/3630510531 wait complete. 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.870+0000 7fe4adcc3640 1 Processor -- start 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 -- start start 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8100e20 0x7fe4a8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8110f80 con 0x7fe4a8102570 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8110e00 con 0x7fe4a8102ef0 2024-09-12T01:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4adcc3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8111100 con 0x7fe4a8100e20 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44424/0 (socket says 172.21.15.16:44424) 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 -- 172.21.15.16:0/471327376 learned_addr learned my addr 172.21.15.16:0/471327376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a77fe640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8100e20 0x7fe4a8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a7fff640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 -- 172.21.15.16:0/471327376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8100e20 msgr2=0x7fe4a8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8100e20 0x7fe4a8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 -- 172.21.15.16:0/471327376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 msgr2=0x7fe4a819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a6ffd640 1 -- 172.21.15.16:0/471327376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a819d4d0 con 0x7fe4a8102570 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.871+0000 7fe4a7fff640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:16.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4a6ffd640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00d770 tx=0x7fe49c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c017070 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c005500 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a819d7c0 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004e00 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.872+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81a50a0 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.873+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a8101200 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.874+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe49c004fa0 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.875+0000 7fe4a4ff9640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 0x7fe4900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.875+0000 7fe4a77fe640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 0x7fe4900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.875+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe49c0ceea0 con 0x7fe4a8102570 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.875+0000 7fe4a77fe640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 0x7fe4900a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe498009230 tx=0x7fe4980058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:16.879+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe49c09baf0 con 0x7fe4a8102570 2024-09-12T01:01:17.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.125+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4a81a5270 con 0x7fe4a8102570 2024-09-12T01:01:17.125 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:17 smithi016 ceph-mon[30972]: pgmap v159: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T01:01:17.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.127+0000 7fe4a4ff9640 1 -- 172.21.15.16:0/471327376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fe49c09b3f0 con 0x7fe4a8102570 2024-09-12T01:01:17.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:01:17.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:01:17.127 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 msgr2=0x7fe4900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 0x7fe4900a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe498009230 tx=0x7fe4980058d0 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 msgr2=0x7fe4a8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00d770 tx=0x7fe49c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 shutdown_connections 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4900a2cf0 0x7fe4900a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4a8102ef0 0x7fe4a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4a8102570 0x7fe4a8198a60 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 --2- 172.21.15.16:0/471327376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4a8100e20 0x7fe4a8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 >> 172.21.15.16:0/471327376 conn(0x7fe4a80fc7d0 msgr2=0x7fe4a806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 shutdown_connections 2024-09-12T01:01:17.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:17.129+0000 7fe4adcc3640 1 -- 172.21.15.16:0/471327376 wait complete. 2024-09-12T01:01:17.273 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:17 smithi099 ceph-mon[35654]: pgmap v159: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T01:01:17.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:17 smithi159 ceph-mon[35566]: pgmap v159: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 67 KiB/s wr, 33 op/s 2024-09-12T01:01:17.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:01:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:18 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:18 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/471327376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:18 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:01:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:18 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:01:18.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:18 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:01:18.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:18 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:18 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/471327376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:18 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:01:18.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:18 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:01:18.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:18 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:01:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:18 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:18 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/471327376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:18.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:18 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:01:18.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:18 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:01:18.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:18 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1898406689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 msgr2=0x7fe76810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1898406689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76810ea90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe760009ca0 tx=0x7fe76002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1898406689 shutdown_connections 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1898406689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76810ea90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1898406689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe7681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1898406689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 0x7fe7681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1898406689 >> 172.21.15.16:0/1898406689 conn(0x7fe7680fd8c0 msgr2=0x7fe7680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1898406689 shutdown_connections 2024-09-12T01:01:18.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.703+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1898406689 wait complete. 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 Processor -- start 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 -- start start 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 0x7fe768198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe768198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111fb0 con 0x7fe768106f10 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111e30 con 0x7fe768102790 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76f3fa640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe768112130 con 0x7fe768101de0 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42602/0 (socket says 172.21.15.16:42602) 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 -- 172.21.15.16:0/1309565568 learned_addr learned my addr 172.21.15.16:0/1309565568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d196640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 0x7fe768198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 -- 172.21.15.16:0/1309565568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 msgr2=0x7fe768198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 0x7fe768198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76d997640 1 -- 172.21.15.16:0/1309565568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 msgr2=0x7fe768198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.704+0000 7fe76c995640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe768198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76d997640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe768198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76d997640 1 -- 172.21.15.16:0/1309565568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76819d4e0 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76c995640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe768198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76d997640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe760031a20 tx=0x7fe760031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe75e7fc640 1 -- 172.21.15.16:0/1309565568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760003c30 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe75e7fc640 1 -- 172.21.15.16:0/1309565568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe760003dd0 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76819d770 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe75e7fc640 1 -- 172.21.15.16:0/1309565568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76002fcf0 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.705+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a50b0 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.707+0000 7fe75e7fc640 1 -- 172.21.15.16:0/1309565568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe760004050 con 0x7fe768106f10 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 Processor -- start 2024-09-12T01:01:18.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 -- start start 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec3b40 0x55ad3dec2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 0x55ad3decf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad3de282f0 con 0x55ad3dec25d0 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ddcfcf0 con 0x55ad3decd370 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe7717d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad3de242b0 con 0x55ad3dec3b40 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42606/0 (socket says 172.21.15.16:42606) 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 -- 172.21.15.16:0/1312922422 learned_addr learned my addr 172.21.15.16:0/1312922422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe75e7fc640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 0x7fe7540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76c995640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec3b40 0x55ad3dec2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d997640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 0x7fe7540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76c995640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 0x55ad3decf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 -- 172.21.15.16:0/1312922422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec3b40 msgr2=0x55ad3dec2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec3b40 0x55ad3dec2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 -- 172.21.15.16:0/1312922422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 msgr2=0x55ad3decf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 0x55ad3decf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe75e7fc640 1 -- 172.21.15.16:0/1309565568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe7600f0760 con 0x7fe768106f10 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76d196640 1 -- 172.21.15.16:0/1312922422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3ddb43a0 con 0x55ad3dec25d0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.708+0000 7fe76c995640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 0x55ad3decf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.709+0000 7fe76d196640 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe76400bcf0 tx=0x7fe76400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a8a2593bd52ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.709+0000 7fe76d997640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 0x7fe7540a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7600b92b0 tx=0x7fe7600b90f0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.709+0000 7fe7537fe640 1 -- 172.21.15.16:0/1312922422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764015070 con 0x55ad3dec25d0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.709+0000 7fe7537fe640 1 -- 172.21.15.16:0/1312922422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764011920 con 0x55ad3dec25d0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.709+0000 7fe7537fe640 1 -- 172.21.15.16:0/1312922422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764010930 con 0x55ad3dec25d0 2024-09-12T01:01:18.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- 172.21.15.16:0/1312922422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 msgr2=0x55ad3dec2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe76400bcf0 tx=0x7fe76400f730 comp rx=0 tx=0).stop 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- 172.21.15.16:0/1312922422 shutdown_connections 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3decd370 0x55ad3decf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3dec2a10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1312922422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec3b40 0x55ad3dec2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- 172.21.15.16:0/1312922422 >> 172.21.15.16:0/1312922422 conn(0x55ad3de8b310 msgr2=0x55ad3debec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- 172.21.15.16:0/1312922422 shutdown_connections 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- 172.21.15.16:0/1312922422 wait complete. 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 -- start start 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.710+0000 7fe7717d3740 1 Processor -- start 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec25d0 0x55ad3deccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3deca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ded0070 con 0x55ad3decd370 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ad3dde3060 con 0x55ad3dec3b40 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ded01f0 con 0x55ad3dec25d0 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe7717d3740 10 client.0 _ll_register_callbacks cb 0x55ad3db368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42616/0 (socket says 172.21.15.16:42616) 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 -- 172.21.15.16:0/1814612129 learned_addr learned my addr 172.21.15.16:0/1814612129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec25d0 0x55ad3deccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76d196640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3deca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 -- 172.21.15.16:0/1814612129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec25d0 msgr2=0x55ad3deccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec25d0 0x55ad3deccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 -- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 msgr2=0x55ad3deca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3deca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 -- 172.21.15.16:0/1814612129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3db993c0 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe76c995640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c4e0 tx=0x7fe7580047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9a518bbc59a93e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.711+0000 7fe750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.712+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75800f040 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.712+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe758016630 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.712+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe764011920 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe758018e90 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 10 client.16839 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad3dedbb40 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764011920 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe758015070 con 0x55ad3decd370 2024-09-12T01:01:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 10 client.16839 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 10 client.16839 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 20 client.16839 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3dedd000 0x55ad3dedf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 4 client.16839 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad3de35860 con 0x55ad3dedd000 2024-09-12T01:01:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.713+0000 7fe76d196640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3dedd000 0x55ad3dedf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.714+0000 7fe750ff9640 10 client.16839 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.714+0000 7fe76d196640 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3dedd000 0x55ad3dedf410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a00ee61de415a3e server_cookie=894f0b36a39879a4 in_seq=0 out_seq=0 2024-09-12T01:01:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe750ff9640 1 -- 172.21.15.16:0/1814612129 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad3de35860 con 0x55ad3dedd000 2024-09-12T01:01:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe750ff9640 10 client.16839 handle_command_reply: tid=0 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 client.16839 shutdown 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 msgr2=0x55ad3deca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe75800c4e0 tx=0x7fe7580047d0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 shutdown_connections 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3dedd000 0x55ad3dedf410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3decd370 0x55ad3deca640 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3deca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1814612129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3dec25d0 0x55ad3deccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 >> 172.21.15.16:0/1814612129 conn(0x55ad3de8b310 msgr2=0x55ad3dec4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe750ff9640 0 client.16839 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 shutdown_connections 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 1 -- 172.21.15.16:0/1814612129 wait complete. 2024-09-12T01:01:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.715+0000 7fe7717d3740 20 client.16839 trim_cache size 0 max 16384 2024-09-12T01:01:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 Processor -- start 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 -- start start 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec25d0 0x55ad3debee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ad770 con 0x55ad3dec3b40 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681af2b0 con 0x55ad3dec25d0 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.728+0000 7fe7717d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ade40 con 0x55ad3decd370 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42618/0 (socket says 172.21.15.16:42618) 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 -- 172.21.15.16:0/1747161972 learned_addr learned my addr 172.21.15.16:0/1747161972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76c995640 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec25d0 0x55ad3debee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76c995640 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 -- 172.21.15.16:0/1747161972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 msgr2=0x55ad3dec5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 -- 172.21.15.16:0/1747161972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec25d0 msgr2=0x55ad3debee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec25d0 0x55ad3debee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 -- 172.21.15.16:0/1747161972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3dec85a0 con 0x55ad3dec3b40 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe76d196640 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55ad3debfac0 tx=0x7fe764010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a39db3a6fea1407 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe752ffd640 1 -- 172.21.15.16:0/1747161972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764015040 con 0x55ad3dec3b40 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe752ffd640 1 -- 172.21.15.16:0/1747161972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76402b070 con 0x55ad3dec3b40 2024-09-12T01:01:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.729+0000 7fe752ffd640 1 -- 172.21.15.16:0/1747161972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764004e00 con 0x55ad3dec3b40 2024-09-12T01:01:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- 172.21.15.16:0/1747161972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 msgr2=0x55ad3ded9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x55ad3debfac0 tx=0x7fe764010470 comp rx=0 tx=0).stop 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- 172.21.15.16:0/1747161972 shutdown_connections 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec3b40 0x55ad3ded9310 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 --2- 172.21.15.16:0/1747161972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec25d0 0x55ad3debee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- 172.21.15.16:0/1747161972 >> 172.21.15.16:0/1747161972 conn(0x55ad3de8b310 msgr2=0x55ad3debe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- 172.21.15.16:0/1747161972 shutdown_connections 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- 172.21.15.16:0/1747161972 wait complete. 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 -- start start 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.730+0000 7fe7717d3740 1 Processor -- start 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3ded5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681aebe0 con 0x55ad3dec25d0 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ae510 con 0x55ad3dec3b40 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ab750 con 0x55ad3decd370 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe7717d3740 10 client.0 _ll_register_callbacks cb 0x7fe75800cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42624/0 (socket says 172.21.15.16:42624) 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 -- 172.21.15.16:0/432707470 learned_addr learned my addr 172.21.15.16:0/432707470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76d196640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3ded5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 -- 172.21.15.16:0/432707470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 msgr2=0x55ad3dec6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 -- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 msgr2=0x55ad3ded5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3ded5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 -- 172.21.15.16:0/432707470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3dec7060 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe76c995640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe75801c010 tx=0x7fe75800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6163fb9250a41a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.731+0000 7fe750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758006ba0 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe7580065f0 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76402b070 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe75803e050 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 10 client.16845 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad3df47730 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76402b070 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe758015040 con 0x55ad3dec25d0 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 10 client.16845 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 10 client.16845 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:01:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 20 client.16845 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3df47e20 0x55ad3df4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 4 client.16845 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.732+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad3de2fd70 con 0x55ad3df47e20 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe76d196640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3df47e20 0x55ad3df4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe76d196640 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3df47e20 0x55ad3df4a240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c166d3a3861f7e1 server_cookie=2dc1cd01e5edfad0 in_seq=0 out_seq=0 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe750ff9640 10 client.16845 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe750ff9640 1 -- 172.21.15.16:0/432707470 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad3de2fd70 con 0x55ad3df47e20 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe750ff9640 10 client.16845 handle_command_reply: tid=0 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 client.16845 shutdown 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 msgr2=0x55ad3ded5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe75801c010 tx=0x7fe75800c8b0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 shutdown_connections 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55ad3df47e20 0x55ad3df4a240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ad3decd370 0x55ad3dec6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ad3dec3b40 0x55ad3ded5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 --2- 172.21.15.16:0/432707470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad3dec25d0 0x55ad3ded5660 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 >> 172.21.15.16:0/432707470 conn(0x55ad3de8b310 msgr2=0x55ad3decc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.733+0000 7fe750ff9640 0 client.16845 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 shutdown_connections 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe7717d3740 1 -- 172.21.15.16:0/432707470 wait complete. 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe7717d3740 20 client.16845 trim_cache size 0 max 16384 2024-09-12T01:01:18.738 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 msgr2=0x7fe7540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 0x7fe7540a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7600b92b0 tx=0x7fe7600b90f0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 msgr2=0x7fe76819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe760031a20 tx=0x7fe760031a50 comp rx=0 tx=0).stop 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 shutdown_connections 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe7540a2cf0 0x7fe7540a51b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe768106f10 0x7fe76819ce00 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe768102790 0x7fe768198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 --2- 172.21.15.16:0/1309565568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe768101de0 0x7fe768198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 >> 172.21.15.16:0/1309565568 conn(0x7fe7680fd8c0 msgr2=0x7fe7680fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 shutdown_connections 2024-09-12T01:01:18.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:18.734+0000 7fe76f3fa640 1 -- 172.21.15.16:0/1309565568 wait complete. 2024-09-12T01:01:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:19 smithi099 ceph-mon[35654]: pgmap v160: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:01:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:19 smithi159 ceph-mon[35566]: pgmap v160: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:01:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:19 smithi016 ceph-mon[30972]: pgmap v160: 97 pgs: 97 active+clean; 4.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:01:19.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:01:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:20.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 -- 172.21.15.16:0/1636260116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118106a80 msgr2=0x7fe118106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 --2- 172.21.15.16:0/1636260116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118106a80 0x7fe118106e60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe114009930 tx=0x7fe11402a420 comp rx=0 tx=0).stop 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 -- 172.21.15.16:0/1636260116 shutdown_connections 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 --2- 172.21.15.16:0/1636260116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118101420 0x7fe11810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 --2- 172.21.15.16:0/1636260116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118100a80 0x7fe118100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 --2- 172.21.15.16:0/1636260116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118106a80 0x7fe118106e60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 -- 172.21.15.16:0/1636260116 >> 172.21.15.16:0/1636260116 conn(0x7fe1180fc7a0 msgr2=0x7fe1180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 -- 172.21.15.16:0/1636260116 shutdown_connections 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.440+0000 7fe11f10b640 1 -- 172.21.15.16:0/1636260116 wait complete. 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 Processor -- start 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 -- start start 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118100a80 0x7fe118199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 0x7fe1181a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe118110e40 con 0x7fe118101420 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe118110cc0 con 0x7fe118106a80 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11f10b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe118110fc0 con 0x7fe118100a80 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42648/0 (socket says 172.21.15.16:42648) 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe10ffff640 1 -- 172.21.15.16:0/2060864658 learned_addr learned my addr 172.21.15.16:0/2060864658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11cea7640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118100a80 0x7fe118199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.441+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 0x7fe1181a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 -- 172.21.15.16:0/2060864658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118100a80 msgr2=0x7fe118199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118100a80 0x7fe118199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 -- 172.21.15.16:0/2060864658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 msgr2=0x7fe1181a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 0x7fe1181a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 -- 172.21.15.16:0/2060864658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1181a4810 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 0x7fe1181a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c920 tx=0x7fe10800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10dffb640 1 -- 172.21.15.16:0/2060864658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108007c70 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10dffb640 1 -- 172.21.15.16:0/2060864658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe108007e10 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1181a49e0 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe10dffb640 1 -- 172.21.15.16:0/2060864658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108018890 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.442+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1181a4f20 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.445+0000 7fe10dffb640 1 -- 172.21.15.16:0/2060864658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe108018a70 con 0x7fe118101420 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.445+0000 7fe1214e4740 1 Processor -- start 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.445+0000 7fe1214e4740 1 -- start start 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4eb40 0x55b206b4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b206ab32f0 con 0x55b206b4d5d0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b206a5acf0 con 0x55b206b4eb40 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe1214e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b206aaf2b0 con 0x55b206b58370 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42664/0 (socket says 172.21.15.16:42664) 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 -- 172.21.15.16:0/1429118182 learned_addr learned my addr 172.21.15.16:0/1429118182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4eb40 0x55b206b4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe10ffff640 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe10dffb640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 0x7fe1000a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 -- 172.21.15.16:0/1429118182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 msgr2=0x55b206b5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 -- 172.21.15.16:0/1429118182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4eb40 msgr2=0x55b206b4d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4eb40 0x55b206b4d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 -- 172.21.15.16:0/1429118182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b206a3f3a0 con 0x55b206b4d5d0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 0x7fe1000a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe10dffb640 1 -- 172.21.15.16:0/2060864658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe1080cf570 con 0x7fe118101420 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe11cea7640 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe114009930 tx=0x7fe114031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b457e2451a53cce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.446+0000 7fe0feffd640 1 -- 172.21.15.16:0/1429118182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114040070 con 0x55b206b4d5d0 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.447+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 0x7fe1000a5230 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe110005fd0 tx=0x7fe110004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.447+0000 7fe0feffd640 1 -- 172.21.15.16:0/1429118182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114005020 con 0x55b206b4d5d0 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.447+0000 7fe0feffd640 1 -- 172.21.15.16:0/1429118182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114035040 con 0x55b206b4d5d0 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- 172.21.15.16:0/1429118182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 msgr2=0x55b206b4da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe114009930 tx=0x7fe114031630 comp rx=0 tx=0).stop 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- 172.21.15.16:0/1429118182 shutdown_connections 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4d5d0 0x55b206b4da10 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1429118182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4eb40 0x55b206b4d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- 172.21.15.16:0/1429118182 >> 172.21.15.16:0/1429118182 conn(0x55b206b16310 msgr2=0x55b206b49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- 172.21.15.16:0/1429118182 shutdown_connections 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- 172.21.15.16:0/1429118182 wait complete. 2024-09-12T01:01:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 -- start start 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.448+0000 7fe1214e4740 1 Processor -- start 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b206b5b010 con 0x55b206b4eb40 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b206a6e060 con 0x55b206b4d5d0 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b206b5b190 con 0x55b206b58370 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 10 client.0 _ll_register_callbacks cb 0x55b2067c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42666/0 (socket says 172.21.15.16:42666) 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 -- 172.21.15.16:0/2261797620 learned_addr learned my addr 172.21.15.16:0/2261797620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe11cea7640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 -- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 msgr2=0x55b206b553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 -- 172.21.15.16:0/2261797620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 msgr2=0x55b206b57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 -- 172.21.15.16:0/2261797620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2068243c0 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe10ffff640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe108098430 tx=0x7fe10809b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d559507c626801 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe0f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080a46d0 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe10800fb30 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.449+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe114005020 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe10809ba60 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 10 client.16857 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b206b66a70 con 0x55b206b4eb40 2024-09-12T01:01:20.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114005020 con 0x55b206b4eb40 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe10809bcc0 con 0x55b206b4eb40 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 10 client.16857 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 10 client.16857 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 20 client.16857 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206b67f30 0x55b206b6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 4 client.16857 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b206ac0860 con 0x55b206b67f30 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.450+0000 7fe11cea7640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206b67f30 0x55b206b6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.451+0000 7fe0f7fff640 10 client.16857 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.451+0000 7fe11cea7640 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206b67f30 0x55b206b6a340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a622f18067b14840 server_cookie=943f86e85c685c51 in_seq=0 out_seq=0 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe0f7fff640 1 -- 172.21.15.16:0/2261797620 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b206ac0860 con 0x55b206b67f30 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe0f7fff640 10 client.16857 handle_command_reply: tid=0 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 client.16857 shutdown 2024-09-12T01:01:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 msgr2=0x55b206b54ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe108098430 tx=0x7fe10809b480 comp rx=0 tx=0).stop 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 shutdown_connections 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206b67f30 0x55b206b6a340 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b54ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 --2- 172.21.15.16:0/2261797620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 >> 172.21.15.16:0/2261797620 conn(0x55b206b16310 msgr2=0x55b206b4f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe0f7fff640 0 client.16857 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 shutdown_connections 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 1 -- 172.21.15.16:0/2261797620 wait complete. 2024-09-12T01:01:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.453+0000 7fe1214e4740 20 client.16857 trim_cache size 0 max 16384 2024-09-12T01:01:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 Processor -- start 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 -- start start 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 0x55b206b64310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b58370 0x55b206b59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181ad790 con 0x55b206b4eb40 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181af2d0 con 0x55b206b58370 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.466+0000 7fe1214e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181ade60 con 0x55b206b4d5d0 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42678/0 (socket says 172.21.15.16:42678) 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 -- 172.21.15.16:0/3279819203 learned_addr learned my addr 172.21.15.16:0/3279819203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe11cea7640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 0x55b206b64310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b58370 0x55b206b59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 -- 172.21.15.16:0/3279819203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 msgr2=0x55b206b64310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 0x55b206b64310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 -- 172.21.15.16:0/3279819203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b58370 msgr2=0x55b206b59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b58370 0x55b206b59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 -- 172.21.15.16:0/3279819203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b206b50120 con 0x55b206b4eb40 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe11cea7640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 0x55b206b64310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe10ffff640 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe10809b760 tx=0x7fe108099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65860340ec68e42c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe0fe7fc640 1 -- 172.21.15.16:0/3279819203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10800fb30 con 0x55b206b4eb40 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe0fe7fc640 1 -- 172.21.15.16:0/3279819203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe108018d60 con 0x55b206b4eb40 2024-09-12T01:01:20.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.467+0000 7fe0fe7fc640 1 -- 172.21.15.16:0/3279819203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080ccf10 con 0x55b206b4eb40 2024-09-12T01:01:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- 172.21.15.16:0/3279819203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 msgr2=0x55b206b5a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe10809b760 tx=0x7fe108099f70 comp rx=0 tx=0).stop 2024-09-12T01:01:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- 172.21.15.16:0/3279819203 shutdown_connections 2024-09-12T01:01:20.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b58370 0x55b206b59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b5a370 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 --2- 172.21.15.16:0/3279819203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b4d5d0 0x55b206b64310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- 172.21.15.16:0/3279819203 >> 172.21.15.16:0/3279819203 conn(0x55b206b16310 msgr2=0x55b206b54e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- 172.21.15.16:0/3279819203 shutdown_connections 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- 172.21.15.16:0/3279819203 wait complete. 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.468+0000 7fe1214e4740 1 -- start start 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 Processor -- start 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b51880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181aec00 con 0x55b206b4eb40 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181ae530 con 0x55b206b4d5d0 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1181ab770 con 0x55b206b58370 2024-09-12T01:01:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe1214e4740 10 client.0 _ll_register_callbacks cb 0x7fe10809b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42694/0 (socket says 172.21.15.16:42694) 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe11cea7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 -- 172.21.15.16:0/1196166052 learned_addr learned my addr 172.21.15.16:0/1196166052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe11d6a8640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b51880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 -- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 msgr2=0x55b206b51880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b51880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 -- 172.21.15.16:0/1196166052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 msgr2=0x55b206b60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.469+0000 7fe10ffff640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b60720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe10ffff640 1 -- 172.21.15.16:0/1196166052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b206b52080 con 0x55b206b4eb40 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe10ffff640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe1080a4890 tx=0x7fe1080ba560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb49c83c64cda44a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10807e070 con 0x55b206b4eb40 2024-09-12T01:01:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe10800fb30 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe108018d60 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1080a4240 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 10 client.16863 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b206b60000 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe108018d60 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe10809a920 con 0x55b206b4eb40 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 10 client.16863 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 10 client.16863 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 20 client.16863 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206bd6ff0 0x55b206bd93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 4 client.16863 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.470+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b206b5ab40 con 0x55b206bd6ff0 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe11cea7640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206bd6ff0 0x55b206bd93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe11cea7640 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206bd6ff0 0x55b206bd93e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98730eeef1e90601 server_cookie=7b384e3dee521e8f in_seq=0 out_seq=0 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe0f7fff640 10 client.16863 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe0f7fff640 1 -- 172.21.15.16:0/1196166052 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b206b5ab40 con 0x55b206bd6ff0 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe0f7fff640 10 client.16863 handle_command_reply: tid=0 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 client.16863 shutdown 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 msgr2=0x55b206b60c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe1080a4890 tx=0x7fe1080ba560 comp rx=0 tx=0).stop 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 shutdown_connections 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b206b58370 0x55b206b51880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55b206bd6ff0 0x55b206bd93e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b206b4eb40 0x55b206b60c60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 --2- 172.21.15.16:0/1196166052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b206b4d5d0 0x55b206b60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 >> 172.21.15.16:0/1196166052 conn(0x55b206b16310 msgr2=0x55b206b56700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.471+0000 7fe0f7fff640 0 client.16863 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 shutdown_connections 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe1214e4740 1 -- 172.21.15.16:0/1196166052 wait complete. 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe1214e4740 20 client.16863 trim_cache size 0 max 16384 2024-09-12T01:01:20.476 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 msgr2=0x7fe1000a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 0x7fe1000a5230 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe110005fd0 tx=0x7fe110004a40 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 msgr2=0x7fe118199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.472+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c920 tx=0x7fe10800cde0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 shutdown_connections 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe1000a2d70 0x7fe1000a5230 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe118106a80 0x7fe1181a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe118101420 0x7fe118199790 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 --2- 172.21.15.16:0/2060864658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe118100a80 0x7fe118199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 >> 172.21.15.16:0/2060864658 conn(0x7fe1180fc7a0 msgr2=0x7fe1180fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 shutdown_connections 2024-09-12T01:01:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:20.473+0000 7fe11f10b640 1 -- 172.21.15.16:0/2060864658 wait complete. 2024-09-12T01:01:21.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:01:21.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:21 smithi159 ceph-mon[35566]: pgmap v161: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:21 smithi016 ceph-mon[30972]: pgmap v161: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:21 smithi099 ceph-mon[35654]: pgmap v161: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 -- 172.21.15.16:0/1726702636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101036e0 msgr2=0x7f4b10107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 --2- 172.21.15.16:0/1726702636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101036e0 0x7f4b10107b90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009930 tx=0x7f4b0002a420 comp rx=0 tx=0).stop 2024-09-12T01:01:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 -- 172.21.15.16:0/1726702636 shutdown_connections 2024-09-12T01:01:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 --2- 172.21.15.16:0/1726702636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b101082a0 0x7f4b1010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 --2- 172.21.15.16:0/1726702636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101036e0 0x7f4b10107b90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 --2- 172.21.15.16:0/1726702636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b10102d30 0x7f4b10103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 -- 172.21.15.16:0/1726702636 >> 172.21.15.16:0/1726702636 conn(0x7f4b100fc7d0 msgr2=0x7f4b100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 -- 172.21.15.16:0/1726702636 shutdown_connections 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.940+0000 7f4b17b56640 1 -- 172.21.15.16:0/1726702636 wait complete. 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 Processor -- start 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 -- start start 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b10102d30 0x7f4b10198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 0x7f4b10198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101120a0 con 0x7f4b101082a0 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10111f20 con 0x7f4b10102d30 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b17b56640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10112220 con 0x7f4b101036e0 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42712/0 (socket says 172.21.15.16:42712) 2024-09-12T01:01:21.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 -- 172.21.15.16:0/736537965 learned_addr learned my addr 172.21.15.16:0/736537965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b158f2640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b10102d30 0x7f4b10198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b150f1640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 0x7f4b10198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 -- 172.21.15.16:0/736537965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 msgr2=0x7f4b10198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 0x7f4b10198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 -- 172.21.15.16:0/736537965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b10102d30 msgr2=0x7f4b10198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.941+0000 7f4b160f3640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b10102d30 0x7f4b10198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b160f3640 1 -- 172.21.15.16:0/736537965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1019d530 con 0x7f4b101082a0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b150f1640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 0x7f4b10198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b160f3640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800d750 tx=0x7f4b0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b06ffd640 1 -- 172.21.15.16:0/736537965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08017070 con 0x7f4b101082a0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b06ffd640 1 -- 172.21.15.16:0/736537965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b08005500 con 0x7f4b101082a0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1019d820 con 0x7f4b101082a0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b06ffd640 1 -- 172.21.15.16:0/736537965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08004d90 con 0x7f4b101082a0 2024-09-12T01:01:21.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.942+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101a5100 con 0x7f4b101082a0 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.944+0000 7f4b06ffd640 1 -- 172.21.15.16:0/736537965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4b08004f30 con 0x7f4b101082a0 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 Processor -- start 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 -- start start 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ba630b2f0 con 0x564ba63a55d0 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ba62b2cf0 con 0x564ba63a6b40 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ba63072b0 con 0x564ba63b0370 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42726/0 (socket says 172.21.15.16:42726) 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.945+0000 7f4b158f2640 1 -- 172.21.15.16:0/2296701594 learned_addr learned my addr 172.21.15.16:0/2296701594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b150f1640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 0x7f4afc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b150f1640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 -- 172.21.15.16:0/2296701594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 msgr2=0x564ba63b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 -- 172.21.15.16:0/2296701594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 msgr2=0x564ba63a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 -- 172.21.15.16:0/2296701594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba62973a0 con 0x564ba63a55d0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b150f1640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b160f3640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 0x7f4afc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b06ffd640 1 -- 172.21.15.16:0/736537965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4b080ced90 con 0x7f4b101082a0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4b158f2640 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00af40 tx=0x7f4b0c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab94f883d43e1f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4afbfff640 1 -- 172.21.15.16:0/2296701594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c01d070 con 0x564ba63a55d0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4afbfff640 1 -- 172.21.15.16:0/2296701594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c010de0 con 0x564ba63a55d0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.946+0000 7f4afbfff640 1 -- 172.21.15.16:0/2296701594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c016340 con 0x564ba63a55d0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- 172.21.15.16:0/2296701594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 msgr2=0x564ba63a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00af40 tx=0x7f4b0c00f850 comp rx=0 tx=0).stop 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b160f3640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 0x7f4afc0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b080cc510 tx=0x7f4b08009bd0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- 172.21.15.16:0/2296701594 shutdown_connections 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63a5a10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/2296701594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- 172.21.15.16:0/2296701594 >> 172.21.15.16:0/2296701594 conn(0x564ba636e310 msgr2=0x564ba63a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- 172.21.15.16:0/2296701594 shutdown_connections 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- 172.21.15.16:0/2296701594 wait complete. 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.947+0000 7f4b19f2f740 1 -- start start 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 Processor -- start 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 0x564ba63afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63b0370 0x564ba63ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ba63b3070 con 0x564ba63a6b40 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564ba62c6060 con 0x564ba63b0370 2024-09-12T01:01:21.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ba63b31f0 con 0x564ba63a55d0 2024-09-12T01:01:21.981 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b19f2f740 10 client.0 _ll_register_callbacks cb 0x564ba60198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42742/0 (socket says 172.21.15.16:42742) 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 -- 172.21.15.16:0/3406202329 learned_addr learned my addr 172.21.15.16:0/3406202329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b150f1640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 0x564ba63afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 -- 172.21.15.16:0/3406202329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 msgr2=0x564ba63afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 0x564ba63afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 -- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63b0370 msgr2=0x564ba63ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63b0370 0x564ba63ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b158f2640 1 -- 172.21.15.16:0/3406202329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba607c3c0 con 0x564ba63a6b40 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.948+0000 7f4b150f1640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 0x564ba63afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b158f2640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c018b70 tx=0x7f4b0c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c7c0d971701ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c026020 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0c010de0 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4b0c010700 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c031050 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 10 client.16875 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ba63bea60 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0c010de0 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b0c005820 con 0x564ba63a6b40 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 10 client.16875 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 10 client.16875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 20 client.16875 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba63bfec0 0x564ba63c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 4 client.16875 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.949+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ba6318860 con 0x564ba63bfec0 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.950+0000 7f4b150f1640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba63bfec0 0x564ba63c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.950+0000 7f4b150f1640 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba63bfec0 0x564ba63c22b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7a62ca59ee461e8 server_cookie=758e25c9b86f0b38 in_seq=0 out_seq=0 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.950+0000 7f4af97fa640 10 client.16875 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4af97fa640 1 -- 172.21.15.16:0/3406202329 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ba6318860 con 0x564ba63bfec0 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4af97fa640 10 client.16875 handle_command_reply: tid=0 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 client.16875 shutdown 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 msgr2=0x564ba63ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c018b70 tx=0x7f4b0c00a330 comp rx=0 tx=0).stop 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 shutdown_connections 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba63bfec0 0x564ba63c22b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63b0370 0x564ba63ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63ad000 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/3406202329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63a55d0 0x564ba63afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4af97fa640 0 client.16875 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 >> 172.21.15.16:0/3406202329 conn(0x564ba636e310 msgr2=0x564ba63a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 shutdown_connections 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 1 -- 172.21.15.16:0/3406202329 wait complete. 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.951+0000 7f4b19f2f740 20 client.16875 trim_cache size 0 max 16384 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.964+0000 7f4b19f2f740 1 Processor -- start 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.964+0000 7f4b19f2f740 1 -- start start 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ab640 con 0x564ba63a55d0 2024-09-12T01:01:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ad180 con 0x564ba63a6b40 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101abd10 con 0x564ba63b0370 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42750/0 (socket says 172.21.15.16:42750) 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 -- 172.21.15.16:0/4244029671 learned_addr learned my addr 172.21.15.16:0/4244029671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b158f2640 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 -- 172.21.15.16:0/4244029671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 msgr2=0x564ba63c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 -- 172.21.15.16:0/4244029671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 msgr2=0x564ba63a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 -- 172.21.15.16:0/4244029671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba63a0310 con 0x564ba63a55d0 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4b150f1640 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009900 tx=0x7f4b00000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0a6adf9bb44396 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4afb7fe640 1 -- 172.21.15.16:0/4244029671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00035040 con 0x564ba63a55d0 2024-09-12T01:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4afb7fe640 1 -- 172.21.15.16:0/4244029671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0004d070 con 0x564ba63a55d0 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.965+0000 7f4afb7fe640 1 -- 172.21.15.16:0/4244029671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003850 con 0x564ba63a55d0 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- 172.21.15.16:0/4244029671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 msgr2=0x564ba63bc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009900 tx=0x7f4b00000bc0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- 172.21.15.16:0/4244029671 shutdown_connections 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a6b40 0x564ba63a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/4244029671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a55d0 0x564ba63bc310 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- 172.21.15.16:0/4244029671 >> 172.21.15.16:0/4244029671 conn(0x564ba636e310 msgr2=0x564ba63a0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- 172.21.15.16:0/4244029671 shutdown_connections 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- 172.21.15.16:0/4244029671 wait complete. 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.966+0000 7f4b19f2f740 1 -- start start 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 Processor -- start 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 0x564ba63b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101acab0 con 0x564ba63a6b40 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ac3e0 con 0x564ba63a55d0 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101a9620 con 0x564ba63b0370 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b19f2f740 10 client.0 _ll_register_callbacks cb 0x7f4b0c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42762/0 (socket says 172.21.15.16:42762) 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 -- 172.21.15.16:0/1919790262 learned_addr learned my addr 172.21.15.16:0/1919790262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b150f1640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 0x564ba63b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 -- 172.21.15.16:0/1919790262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 msgr2=0x564ba63aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 -- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 msgr2=0x564ba63b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 0x564ba63b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b158f2640 1 -- 172.21.15.16:0/1919790262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba63a8120 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.967+0000 7f4b150f1640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 0x564ba63b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b158f2640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00a5d0 tx=0x7f4b0c018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81367e9a3ccd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c026040 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4b0c02d940 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0004d070 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c01d070 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 10 client.16881 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564ba642a6b0 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0004d070 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b0c004140 con 0x564ba63a6b40 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 10 client.16881 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 10 client.16881 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 20 client.16881 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba642ae50 0x564ba642d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 4 client.16881 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.968+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ba6312d70 con 0x564ba642ae50 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b150f1640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba642ae50 0x564ba642d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4af97fa640 10 client.16881 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b150f1640 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba642ae50 0x564ba642d270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e555de203b8052a0 server_cookie=74e63ca79b98ff12 in_seq=0 out_seq=0 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4af97fa640 1 -- 172.21.15.16:0/1919790262 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ba6312d70 con 0x564ba642ae50 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4af97fa640 10 client.16881 handle_command_reply: tid=0 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 client.16881 shutdown 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 msgr2=0x564ba63a9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00a5d0 tx=0x7f4b0c018a40 comp rx=0 tx=0).stop 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 shutdown_connections 2024-09-12T01:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564ba642ae50 0x564ba642d270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ba63b0370 0x564ba63aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ba63a6b40 0x564ba63a9ce0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 --2- 172.21.15.16:0/1919790262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564ba63a55d0 0x564ba63b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.969+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 >> 172.21.15.16:0/1919790262 conn(0x564ba636e310 msgr2=0x564ba63abd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4af97fa640 0 client.16881 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 shutdown_connections 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b19f2f740 1 -- 172.21.15.16:0/1919790262 wait complete. 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b19f2f740 20 client.16881 trim_cache size 0 max 16384 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 msgr2=0x7f4afc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 0x7f4afc0a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4b080cc510 tx=0x7f4b08009bd0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 msgr2=0x7f4b1019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800d750 tx=0x7f4b0800dc10 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 shutdown_connections 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4afc0a2d70 0x7f4afc0a5230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b101082a0 0x7f4b1019ce50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b101036e0 0x7f4b10198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 --2- 172.21.15.16:0/736537965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b10102d30 0x7f4b10198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 >> 172.21.15.16:0/736537965 conn(0x7f4b100fc7d0 msgr2=0x7f4b10105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 shutdown_connections 2024-09-12T01:01:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:21.970+0000 7f4b17b56640 1 -- 172.21.15.16:0/736537965 wait complete. 2024-09-12T01:01:22.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:01:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:23.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:23 smithi016 ceph-mon[30972]: pgmap v162: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:23 smithi159 ceph-mon[35566]: pgmap v162: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:23 smithi099 ceph-mon[35654]: pgmap v162: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/2555468374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0103740 msgr2=0x7fe5b0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/2555468374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0103740 0x7fe5b0107b80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe598009930 tx=0x7fe59802a440 comp rx=0 tx=0).stop 2024-09-12T01:01:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/2555468374 shutdown_connections 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/2555468374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0108290 0x7fe5b010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/2555468374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0103740 0x7fe5b0107b80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/2555468374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 0x7fe5b0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/2555468374 >> 172.21.15.16:0/2555468374 conn(0x7fe5b00fc7d0 msgr2=0x7fe5b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/2555468374 shutdown_connections 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/2555468374 wait complete. 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 Processor -- start 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.542+0000 7fe5b4dd9640 1 -- start start 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 0x7fe5b01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0103740 0x7fe5b0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01120d0 con 0x7fe5b0108290 2024-09-12T01:01:23.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0111f50 con 0x7fe5b0102d90 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5b4dd9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0112250 con 0x7fe5b0103740 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42782/0 (socket says 172.21.15.16:42782) 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 -- 172.21.15.16:0/3305910322 learned_addr learned my addr 172.21.15.16:0/3305910322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 0x7fe5b01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0103740 0x7fe5b0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 -- 172.21.15.16:0/3305910322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0103740 msgr2=0x7fe5b0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0103740 0x7fe5b0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 -- 172.21.15.16:0/3305910322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 msgr2=0x7fe5b01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 0x7fe5b01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 -- 172.21.15.16:0/3305910322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01a6af0 con 0x7fe5b0108290 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.543+0000 7fe5aed9d640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800b990 tx=0x7fe5a800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.544+0000 7fe5a77fe640 1 -- 172.21.15.16:0/3305910322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8019070 con 0x7fe5b0108290 2024-09-12T01:01:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.544+0000 7fe5a77fe640 1 -- 172.21.15.16:0/3305910322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a800cba0 con 0x7fe5b0108290 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.544+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01a6de0 con 0x7fe5b0108290 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.544+0000 7fe5a77fe640 1 -- 172.21.15.16:0/3305910322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8007650 con 0x7fe5b0108290 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.544+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a7240 con 0x7fe5b0108290 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.546+0000 7fe5a77fe640 1 -- 172.21.15.16:0/3305910322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5a801c070 con 0x7fe5b0108290 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.546+0000 7fe5b71b2740 1 Processor -- start 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 -- start start 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 0x5610ee6a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6ae370 0x5610ee6b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610ee6092f0 con 0x5610ee6a4b40 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5610ee5b0cf0 con 0x5610ee6a35d0 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5610ee6052b0 con 0x5610ee6ae370 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42798/0 (socket says 172.21.15.16:42798) 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a77fe640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 -- 172.21.15.16:0/1267658576 learned_addr learned my addr 172.21.15.16:0/1267658576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 0x5610ee6a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6ae370 0x5610ee6b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5aed9d640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a77fe640 1 -- 172.21.15.16:0/3305910322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe5a80ce8e0 con 0x7fe5b0108290 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 -- 172.21.15.16:0/1267658576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6ae370 msgr2=0x5610ee6b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6ae370 0x5610ee6b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 -- 172.21.15.16:0/1267658576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 msgr2=0x5610ee6a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 0x5610ee6a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5a7fff640 1 -- 172.21.15.16:0/1267658576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ee5953a0 con 0x5610ee6a4b40 2024-09-12T01:01:23.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.547+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 0x5610ee6a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe598031550 tx=0x7fe598005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72647fb2fe674053 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5aed9d640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80cc220 tx=0x7fe5a806cfb0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5a4ff9640 1 -- 172.21.15.16:0/1267658576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035050 con 0x5610ee6a4b40 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5a4ff9640 1 -- 172.21.15.16:0/1267658576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598031730 con 0x5610ee6a4b40 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5a4ff9640 1 -- 172.21.15.16:0/1267658576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980046c0 con 0x5610ee6a4b40 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5b71b2740 1 -- 172.21.15.16:0/1267658576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 msgr2=0x5610ee6a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.548+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe598031550 tx=0x7fe598005be0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- 172.21.15.16:0/1267658576 shutdown_connections 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6ae370 0x5610ee6b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6a35d0 0x5610ee6a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/1267658576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a3090 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- 172.21.15.16:0/1267658576 >> 172.21.15.16:0/1267658576 conn(0x5610ee66c310 msgr2=0x5610ee69fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- 172.21.15.16:0/1267658576 shutdown_connections 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- 172.21.15.16:0/1267658576 wait complete. 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- start start 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 Processor -- start 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610ee6b1010 con 0x5610ee6a4b40 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5610ee5c4060 con 0x5610ee6ae370 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5610ee6b1190 con 0x5610ee6a35d0 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5b71b2740 10 client.0 _ll_register_callbacks cb 0x5610ee3178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.549+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42804/0 (socket says 172.21.15.16:42804) 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 -- 172.21.15.16:0/3092630873 learned_addr learned my addr 172.21.15.16:0/3092630873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 -- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 msgr2=0x5610ee6add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 -- 172.21.15.16:0/3092630873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 msgr2=0x5610ee6ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 -- 172.21.15.16:0/3092630873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ee37a3c0 con 0x5610ee6a4b40 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000baa0 tx=0x7fe5a000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adea7cab43ac57d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001b640 con 0x5610ee6a4b40 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe5a0011440 con 0x5610ee6a4b40 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe598031730 con 0x5610ee6a4b40 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5a001b7e0 con 0x5610ee6a4b40 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5b71b2740 10 client.16893 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:23.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.550+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610ee6bcaf0 con 0x5610ee6a4b40 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598031730 con 0x5610ee6a4b40 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a0019020 con 0x5610ee6a4b40 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 10 client.16893 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 10 client.16893 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 20 client.16893 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee6bdf50 0x5610ee6c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 4 client.16893 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ee616860 con 0x5610ee6bdf50 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.551+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee6bdf50 0x5610ee6c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.552+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee6bdf50 0x5610ee6c0370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae2c38c479d7b626 server_cookie=eee7c27ca787b4ae in_seq=0 out_seq=0 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.552+0000 7fe595ffb640 10 client.16893 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.552+0000 7fe595ffb640 1 -- 172.21.15.16:0/3092630873 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610ee616860 con 0x5610ee6bdf50 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.552+0000 7fe595ffb640 10 client.16893 handle_command_reply: tid=0 2024-09-12T01:01:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 client.16893 shutdown 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 msgr2=0x5610ee6aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000baa0 tx=0x7fe5a000bf60 comp rx=0 tx=0).stop 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 shutdown_connections 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee6bdf50 0x5610ee6c0370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6aafd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe595ffb640 0 client.16893 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/3092630873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 >> 172.21.15.16:0/3092630873 conn(0x5610ee66c310 msgr2=0x5610ee6a5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 shutdown_connections 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 1 -- 172.21.15.16:0/3092630873 wait complete. 2024-09-12T01:01:23.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.553+0000 7fe5b71b2740 20 client.16893 trim_cache size 0 max 16384 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 Processor -- start 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 -- start start 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01af940 con 0x5610ee6a4b40 2024-09-12T01:01:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01b1480 con 0x5610ee6ae370 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01b0010 con 0x5610ee6a35d0 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42810/0 (socket says 172.21.15.16:42810) 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 -- 172.21.15.16:0/4153737253 learned_addr learned my addr 172.21.15.16:0/4153737253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 -- 172.21.15.16:0/4153737253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 msgr2=0x5610ee6ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6ba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 -- 172.21.15.16:0/4153737253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 msgr2=0x5610ee6c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.566+0000 7fe5ae59c640 1 -- 172.21.15.16:0/4153737253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ee69fde0 con 0x5610ee6a4b40 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004000 tx=0x7fe5a0004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478b3c9b6e44676f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe597fff640 1 -- 172.21.15.16:0/4153737253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001b440 con 0x5610ee6a4b40 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe597fff640 1 -- 172.21.15.16:0/4153737253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a001b5e0 con 0x5610ee6a4b40 2024-09-12T01:01:23.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe597fff640 1 -- 172.21.15.16:0/4153737253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00117b0 con 0x5610ee6a4b40 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 -- 172.21.15.16:0/4153737253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 msgr2=0x5610ee6a9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0004000 tx=0x7fe5a0004230 comp rx=0 tx=0).stop 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 -- 172.21.15.16:0/4153737253 shutdown_connections 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6a9fb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/4153737253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.567+0000 7fe5b71b2740 1 -- 172.21.15.16:0/4153737253 >> 172.21.15.16:0/4153737253 conn(0x5610ee66c310 msgr2=0x5610ee6a98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- 172.21.15.16:0/4153737253 shutdown_connections 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- 172.21.15.16:0/4153737253 wait complete. 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- start start 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 Processor -- start 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01b0db0 con 0x5610ee6a4b40 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01b06e0 con 0x5610ee6ae370 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01ad940 con 0x5610ee6a35d0 2024-09-12T01:01:23.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5b71b2740 10 client.0 _ll_register_callbacks cb 0x7fe5a0004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42816/0 (socket says 172.21.15.16:42816) 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5ae59c640 1 -- 172.21.15.16:0/2217271327 learned_addr learned my addr 172.21.15.16:0/2217271327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.568+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 -- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 msgr2=0x5610ee6b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 -- 172.21.15.16:0/2217271327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 msgr2=0x5610ee6b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 -- 172.21.15.16:0/2217271327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ee6b2660 con 0x5610ee6a4b40 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:23.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5ae59c640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002d000 tx=0x7fe5a00052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c018d08139e19dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0005650 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fe5a001b980 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a001b5e0 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5a0040050 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5b71b2740 10 client.16899 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610ee728900 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.569+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a001b5e0 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a0019040 con 0x5610ee6a4b40 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 10 client.16899 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 10 client.16899 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 20 client.16899 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee729010 0x5610ee72b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:23.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 4 client.16899 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610ee610d70 con 0x5610ee729010 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.570+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee729010 0x5610ee72b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe595ffb640 10 client.16899 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5a7fff640 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee729010 0x5610ee72b430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=307f70053b29a57d server_cookie=3895dc080aea6fa7 in_seq=0 out_seq=0 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe595ffb640 1 -- 172.21.15.16:0/2217271327 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610ee610d70 con 0x5610ee729010 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe595ffb640 10 client.16899 handle_command_reply: tid=0 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 client.16899 shutdown 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 msgr2=0x5610ee6b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002d000 tx=0x7fe5a00052b0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 shutdown_connections 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5610ee729010 0x5610ee72b430 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe595ffb640 0 client.16899 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610ee6ae370 0x5610ee6b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610ee6a4b40 0x5610ee6b1be0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 --2- 172.21.15.16:0/2217271327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610ee6a35d0 0x5610ee6b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 >> 172.21.15.16:0/2217271327 conn(0x5610ee66c310 msgr2=0x5610ee69e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 shutdown_connections 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 1 -- 172.21.15.16:0/2217271327 wait complete. 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.571+0000 7fe5b71b2740 20 client.16899 trim_cache size 0 max 16384 2024-09-12T01:01:23.581 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 msgr2=0x7fe59c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80cc220 tx=0x7fe5a806cfb0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 msgr2=0x7fe5b0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800b990 tx=0x7fe5a800be50 comp rx=0 tx=0).stop 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 shutdown_connections 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe59c0a2cf0 0x7fe59c0a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5b0108290 0x7fe5b0101470 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b0103740 0x7fe5b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 --2- 172.21.15.16:0/3305910322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0102d90 0x7fe5b01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 >> 172.21.15.16:0/3305910322 conn(0x7fe5b00fc7d0 msgr2=0x7fe5b0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 shutdown_connections 2024-09-12T01:01:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:23.572+0000 7fe5b4dd9640 1 -- 172.21.15.16:0/3305910322 wait complete. 2024-09-12T01:01:24.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:01:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:25 smithi016 ceph-mon[30972]: pgmap v163: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 -- 172.21.15.16:0/2080368961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341010c170 msgr2=0x7f341010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 --2- 172.21.15.16:0/2080368961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341010c170 0x7f341010e610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3408009ca0 tx=0x7f340802a950 comp rx=0 tx=0).stop 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 -- 172.21.15.16:0/2080368961 shutdown_connections 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 --2- 172.21.15.16:0/2080368961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f341010c170 0x7f341010e610 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 --2- 172.21.15.16:0/2080368961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 0x7f341010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 --2- 172.21.15.16:0/2080368961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3410102d40 0x7f3410103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.235+0000 7f34167c2640 1 -- 172.21.15.16:0/2080368961 >> 172.21.15.16:0/2080368961 conn(0x7f34100fc7a0 msgr2=0x7f34100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.236+0000 7f34167c2640 1 -- 172.21.15.16:0/2080368961 shutdown_connections 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.236+0000 7f34167c2640 1 -- 172.21.15.16:0/2080368961 wait complete. 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.236+0000 7f34167c2640 1 Processor -- start 2024-09-12T01:01:25.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.236+0000 7f34167c2640 1 -- start start 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 0x7f3410100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 0x7f3410101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3410112030 con 0x7f3410102d40 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3410111eb0 con 0x7f341010c170 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f34167c2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34101121b0 con 0x7f3410103660 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42836/0 (socket says 172.21.15.16:42836) 2024-09-12T01:01:25.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 -- 172.21.15.16:0/3328043190 learned_addr learned my addr 172.21.15.16:0/3328043190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 0x7f3410100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 0x7f3410101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 -- 172.21.15.16:0/3328043190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 msgr2=0x7f3410100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 0x7f3410100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 -- 172.21.15.16:0/3328043190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 msgr2=0x7f3410101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 0x7f3410101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f340ffff640 1 -- 172.21.15.16:0/3328043190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34101a69f0 con 0x7f3410102d40 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.237+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 0x7f3410101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f340ffff640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f340000b4a0 tx=0x7f340000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f340d7fa640 1 -- 172.21.15.16:0/3328043190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400014070 con 0x7f3410102d40 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34101a6ce0 con 0x7f3410102d40 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f340d7fa640 1 -- 172.21.15.16:0/3328043190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400010af0 con 0x7f3410102d40 2024-09-12T01:01:25.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f340d7fa640 1 -- 172.21.15.16:0/3328043190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400017d10 con 0x7f3410102d40 2024-09-12T01:01:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.238+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34101a7210 con 0x7f3410102d40 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f340d7fa640 1 -- 172.21.15.16:0/3328043190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3400017070 con 0x7f3410102d40 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 Processor -- start 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 -- start start 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf25d0 0x5632cacf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632cac582f0 con 0x5632cacfd370 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632cabffcf0 con 0x5632cacf25d0 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f3418b9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5632cac542b0 con 0x5632cacf3b40 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42838/0 (socket says 172.21.15.16:42838) 2024-09-12T01:01:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.241+0000 7f340f7fe640 1 -- 172.21.15.16:0/3261992768 learned_addr learned my addr 172.21.15.16:0/3261992768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340ffff640 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf25d0 0x5632cacf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340d7fa640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 0x7f33f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 0x7f33f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 -- 172.21.15.16:0/3261992768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 msgr2=0x5632cacf2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 -- 172.21.15.16:0/3261992768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf25d0 msgr2=0x5632cacf2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf25d0 0x5632cacf2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 -- 172.21.15.16:0/3261992768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cabe43a0 con 0x5632cacfd370 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340d7fa640 1 -- 172.21.15.16:0/3328043190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f34000ce730 con 0x7f3410102d40 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3404007740 tx=0x7f3404007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377a5ef127f5c284 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f33f67fc640 1 -- 172.21.15.16:0/3261992768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404007e50 con 0x5632cacfd370 2024-09-12T01:01:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 0x7f33f80a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3408009ca0 tx=0x7f340802a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f33f67fc640 1 -- 172.21.15.16:0/3261992768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3404018070 con 0x5632cacfd370 2024-09-12T01:01:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.242+0000 7f33f67fc640 1 -- 172.21.15.16:0/3261992768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040123e0 con 0x5632cacfd370 2024-09-12T01:01:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 -- 172.21.15.16:0/3261992768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 msgr2=0x5632cacff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3404007740 tx=0x7f3404007c00 comp rx=0 tx=0).stop 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 -- 172.21.15.16:0/3261992768 shutdown_connections 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacfd370 0x5632cacff760 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf25d0 0x5632cacf2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3261992768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 -- 172.21.15.16:0/3261992768 >> 172.21.15.16:0/3261992768 conn(0x5632cacbb310 msgr2=0x5632caceec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 -- 172.21.15.16:0/3261992768 shutdown_connections 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.243+0000 7f3418b9b740 1 -- 172.21.15.16:0/3261992768 wait complete. 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 -- start start 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 Processor -- start 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 0x5632cacfcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cacfa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632cad00070 con 0x5632cacf3b40 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632cac13060 con 0x5632cacfd370 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5632cad001f0 con 0x5632cacf25d0 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f3418b9b740 10 client.0 _ll_register_callbacks cb 0x5632ca9668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42848/0 (socket says 172.21.15.16:42848) 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.244+0000 7f340f7fe640 1 -- 172.21.15.16:0/3092563258 learned_addr learned my addr 172.21.15.16:0/3092563258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340ffff640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 0x5632cacfcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cacfa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 -- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 msgr2=0x5632cacfcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 0x5632cacfcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 -- 172.21.15.16:0/3092563258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 msgr2=0x5632cacfa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cacfa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 -- 172.21.15.16:0/3092563258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ca9c93c0 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340ffff640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 0x5632cacfcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3404004700 tx=0x7f3404013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98496bc41760da68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f33ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340400dd60 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f3404024420 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3404018070 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3404007e50 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f3418b9b740 10 client.16911 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.245+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632cad0bad0 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3404018070 con 0x5632cacf3b40 2024-09-12T01:01:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f340401c6c0 con 0x5632cacf3b40 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 10 client.16911 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 20 client.16911 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad0cf30 0x5632cad0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 4 client.16911 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632cac65860 con 0x5632cad0cf30 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.246+0000 7f340ffff640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad0cf30 0x5632cad0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.247+0000 7f340ffff640 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad0cf30 0x5632cad0f320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d1d8bf7e935b3b server_cookie=9de15554f1cf970f in_seq=0 out_seq=0 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.247+0000 7f33ef7fe640 10 client.16911 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3092563258 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632cac65860 con 0x5632cad0cf30 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f33ef7fe640 10 client.16911 handle_command_reply: tid=0 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 client.16911 shutdown 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 msgr2=0x5632cacf9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3404004700 tx=0x7f3404013830 comp rx=0 tx=0).stop 2024-09-12T01:01:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 shutdown_connections 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad0cf30 0x5632cad0f320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cacfa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf3b40 0x5632cacf9fe0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3092563258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf25d0 0x5632cacfcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 >> 172.21.15.16:0/3092563258 conn(0x5632cacbb310 msgr2=0x5632cacf4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f33ef7fe640 0 client.16911 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 shutdown_connections 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 1 -- 172.21.15.16:0/3092563258 wait complete. 2024-09-12T01:01:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.248+0000 7f3418b9b740 20 client.16911 trim_cache size 0 max 16384 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 Processor -- start 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 -- start start 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cad149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34101afac0 con 0x5632cacf25d0 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b1600 con 0x5632cacfd370 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f3418b9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b0190 con 0x5632cacf3b40 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.261+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42854/0 (socket says 172.21.15.16:42854) 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 -- 172.21.15.16:0/3035593651 learned_addr learned my addr 172.21.15.16:0/3035593651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cad149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 -- 172.21.15.16:0/3035593651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 msgr2=0x5632cacf98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 -- 172.21.15.16:0/3035593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 msgr2=0x5632cad149b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cad149b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 -- 172.21.15.16:0/3035593651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632cacfa940 con 0x5632cacf25d0 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cad149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f340ffff640 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3400017fd0 tx=0x7f34000d0c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412aac5bac89ee64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f33f5ffb640 1 -- 172.21.15.16:0/3035593651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340009f140 con 0x5632cacf25d0 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f33f5ffb640 1 -- 172.21.15.16:0/3035593651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34000a3a00 con 0x5632cacf25d0 2024-09-12T01:01:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.262+0000 7f33f5ffb640 1 -- 172.21.15.16:0/3035593651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400097400 con 0x5632cacf25d0 2024-09-12T01:01:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- 172.21.15.16:0/3035593651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 msgr2=0x5632cad09310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3400017fd0 tx=0x7f34000d0c40 comp rx=0 tx=0).stop 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- 172.21.15.16:0/3035593651 shutdown_connections 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacfd370 0x5632cad149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacf3b40 0x5632cacf98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3035593651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad09310 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- 172.21.15.16:0/3035593651 >> 172.21.15.16:0/3035593651 conn(0x5632cacbb310 msgr2=0x5632cacf88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- 172.21.15.16:0/3035593651 shutdown_connections 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- 172.21.15.16:0/3035593651 wait complete. 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.263+0000 7f3418b9b740 1 -- start start 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 Processor -- start 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf3b40 0x5632cad00c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 0x5632cad01160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b0f30 con 0x5632cacf25d0 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b0860 con 0x5632cacf3b40 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34101adaa0 con 0x5632cacfd370 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3418b9b740 10 client.0 _ll_register_callbacks cb 0x7f34000970b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42860/0 (socket says 172.21.15.16:42860) 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 -- 172.21.15.16:0/3114358714 learned_addr learned my addr 172.21.15.16:0/3114358714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf3b40 0x5632cad00c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 0x5632cad01160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 -- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 msgr2=0x5632cad01160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 0x5632cad01160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 -- 172.21.15.16:0/3114358714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf3b40 msgr2=0x5632cad00c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf3b40 0x5632cad00c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 -- 172.21.15.16:0/3114358714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632cad016a0 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f3414d5f640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 0x5632cad01160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f340ffff640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f340000be60 tx=0x7f3400000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d67b391d0f7821 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.264+0000 7f33ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000af000 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f34000cc130 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34000a3a00 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3400097400 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 10 client.16917 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632cad77600 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34000a3a00 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f340009a200 con 0x5632cacf25d0 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 10 client.16917 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 20 client.16917 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad77c70 0x5632cad7a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 4 client.16917 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.265+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632cac5fd70 con 0x5632cad77c70 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad77c70 0x5632cad7a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f340f7fe640 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad77c70 0x5632cad7a060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d1fdbdd18be5d4 server_cookie=a2e7db5d3c48b42c in_seq=0 out_seq=0 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f33ef7fe640 10 client.16917 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f33ef7fe640 1 -- 172.21.15.16:0/3114358714 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632cac5fd70 con 0x5632cad77c70 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f33ef7fe640 10 client.16917 handle_command_reply: tid=0 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f3418b9b740 1 client.16917 shutdown 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 msgr2=0x5632cad058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f340000be60 tx=0x7f3400000bc0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 shutdown_connections 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.266+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5632cad77c70 0x5632cad7a060 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f33ef7fe640 0 client.16917 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632cacfd370 0x5632cad01160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632cacf3b40 0x5632cad00c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 --2- 172.21.15.16:0/3114358714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632cacf25d0 0x5632cad058f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 >> 172.21.15.16:0/3114358714 conn(0x5632cacbb310 msgr2=0x5632cacfb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 shutdown_connections 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 1 -- 172.21.15.16:0/3114358714 wait complete. 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f3418b9b740 20 client.16917 trim_cache size 0 max 16384 2024-09-12T01:01:25.271 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 msgr2=0x7f33f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 0x7f33f80a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3408009ca0 tx=0x7f340802a950 comp rx=0 tx=0).stop 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 msgr2=0x7f34101009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f340000b4a0 tx=0x7f340000b960 comp rx=0 tx=0).stop 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.267+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 shutdown_connections 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f33f80a2cf0 0x7f33f80a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f341010c170 0x7f3410101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103660 0x7f3410100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 --2- 172.21.15.16:0/3328043190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410102d40 0x7f34101009a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 >> 172.21.15.16:0/3328043190 conn(0x7f34100fc7a0 msgr2=0x7f34100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 shutdown_connections 2024-09-12T01:01:25.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:25.268+0000 7f34167c2640 1 -- 172.21.15.16:0/3328043190 wait complete. 2024-09-12T01:01:25.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:25 smithi159 ceph-mon[35566]: pgmap v163: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:25 smithi099 ceph-mon[35654]: pgmap v163: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:27.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:01:27.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:01:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:27.325 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:27 smithi099 ceph-mon[35654]: pgmap v164: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-12T01:01:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:27 smithi159 ceph-mon[35566]: pgmap v164: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-12T01:01:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:27 smithi016 ceph-mon[30972]: pgmap v164: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-09-12T01:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 -- 172.21.15.16:0/1685459830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 msgr2=0x7f5a181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/1685459830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a181021c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009900 tx=0x7f5a0802a410 comp rx=0 tx=0).stop 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 -- 172.21.15.16:0/1685459830 shutdown_connections 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/1685459830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 0x7f5a1810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/1685459830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18102790 0x7f5a181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/1685459830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a181021c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 -- 172.21.15.16:0/1685459830 >> 172.21.15.16:0/1685459830 conn(0x7f5a180fd8c0 msgr2=0x7f5a180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 -- 172.21.15.16:0/1685459830 shutdown_connections 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.822+0000 7f5a1d76e640 1 -- 172.21.15.16:0/1685459830 wait complete. 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 Processor -- start 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 -- start start 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18101de0 0x7f5a18198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 0x7f5a1819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18111f30 con 0x7f5a18102790 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18111db0 con 0x7f5a18101de0 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a1d76e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181120b0 con 0x7f5a18106f10 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42884/0 (socket says 172.21.15.16:42884) 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 -- 172.21.15.16:0/2873081248 learned_addr learned my addr 172.21.15.16:0/2873081248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 0x7f5a1819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18101de0 0x7f5a18198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 -- 172.21.15.16:0/2873081248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 msgr2=0x7f5a1819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 0x7f5a1819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 -- 172.21.15.16:0/2873081248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18101de0 msgr2=0x7f5a18198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18101de0 0x7f5a18198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.823+0000 7f5a167fc640 1 -- 172.21.15.16:0/2873081248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1819d340 con 0x7f5a18102790 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c920 tx=0x7f5a0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f59fffff640 1 -- 172.21.15.16:0/2873081248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007c70 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f59fffff640 1 -- 172.21.15.16:0/2873081248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c007e10 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1819d630 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f59fffff640 1 -- 172.21.15.16:0/2873081248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c018890 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.824+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181a4f10 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.826+0000 7f59fffff640 1 -- 172.21.15.16:0/2873081248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a0c018a70 con 0x7f5a18102790 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.826+0000 7f5a1fb47740 1 Processor -- start 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.826+0000 7f5a1fb47740 1 -- start start 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa8340 0x5560e7ab6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7ab9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5560e7a0d280 con 0x5560e7aa7a20 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560e79b4b10 con 0x5560e7ab7150 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560e7a09240 con 0x5560e7aa8340 2024-09-12T01:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42894/0 (socket says 172.21.15.16:42894) 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 -- 172.21.15.16:0/3822464373 learned_addr learned my addr 172.21.15.16:0/3822464373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a167fc640 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7ab9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa8340 0x5560e7ab6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f59fffff640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 0x7f5a000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 -- 172.21.15.16:0/3822464373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa8340 msgr2=0x5560e7ab6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa8340 0x5560e7ab6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 -- 172.21.15.16:0/3822464373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 msgr2=0x5560e7ab9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7ab9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 -- 172.21.15.16:0/3822464373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e79c11f0 con 0x5560e7aa7a20 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 0x7f5a000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.827+0000 7f59fffff640 1 -- 172.21.15.16:0/2873081248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f5a0c0cf570 con 0x7f5a18102790 2024-09-12T01:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a177fe640 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5a10010440 tx=0x7f5a10010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5f563d48b17784 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f59fd7fa640 1 -- 172.21.15.16:0/3822464373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1001d400 con 0x5560e7aa7a20 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f59fd7fa640 1 -- 172.21.15.16:0/3822464373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a100134c0 con 0x5560e7aa7a20 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f59fd7fa640 1 -- 172.21.15.16:0/3822464373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a100118a0 con 0x5560e7aa7a20 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 -- 172.21.15.16:0/3822464373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 msgr2=0x5560e7aa7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5a10010440 tx=0x7f5a10010900 comp rx=0 tx=0).stop 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 0x7f5a000a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a10021000 tx=0x7f5a1001e000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 -- 172.21.15.16:0/3822464373 shutdown_connections 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7ab9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa8340 0x5560e7ab6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/3822464373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7aa7e00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.828+0000 7f5a1fb47740 1 -- 172.21.15.16:0/3822464373 >> 172.21.15.16:0/3822464373 conn(0x5560e7aa40f0 msgr2=0x5560e7aa44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- 172.21.15.16:0/3822464373 shutdown_connections 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- 172.21.15.16:0/3822464373 wait complete. 2024-09-12T01:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- start start 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 Processor -- start 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa8340 0x5560e7ab4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7ab4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5560e7ab9d20 con 0x5560e7aa7a20 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5560e79c7f10 con 0x5560e7aa8340 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560e7ab9ea0 con 0x5560e7ab7150 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a1fb47740 10 client.0 _ll_register_callbacks cb 0x5560e7990a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a16ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42898/0 (socket says 172.21.15.16:42898) 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a16ffd640 1 -- 172.21.15.16:0/2401189517 learned_addr learned my addr 172.21.15.16:0/2401189517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.829+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa8340 0x5560e7ab4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7ab4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 -- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 msgr2=0x5560e7ab4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7ab4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 -- 172.21.15.16:0/2401189517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa8340 msgr2=0x5560e7ab4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa8340 0x5560e7ab4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 -- 172.21.15.16:0/2401189517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e777e6a0 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009900 tx=0x7f5a080317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a02ef03a53ac50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f59faff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08035040 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f5a080319d0 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a100134c0 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a08004260 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a1fb47740 10 client.16929 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560e7ac5a10 con 0x5560e7aa7a20 2024-09-12T01:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.830+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a100134c0 con 0x5560e7aa7a20 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5a08004990 con 0x5560e7aa7a20 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 10 client.16929 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 20 client.16929 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7ac6e20 0x5560e7ac92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 4 client.16929 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e7a1a7f0 con 0x5560e7ac6e20 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.831+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7ac6e20 0x5560e7ac92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.832+0000 7f59faff5640 10 client.16929 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.832+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7ac6e20 0x5560e7ac92c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26f2b2ad73bed97 server_cookie=cb5cc933c3cae479 in_seq=0 out_seq=0 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.832+0000 7f59faff5640 1 -- 172.21.15.16:0/2401189517 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560e7a1a7f0 con 0x5560e7ac6e20 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.832+0000 7f59faff5640 10 client.16929 handle_command_reply: tid=0 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.832+0000 7f5a1fb47740 1 client.16929 shutdown 2024-09-12T01:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 msgr2=0x5560e7ab40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009900 tx=0x7f5a080317e0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 shutdown_connections 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7ac6e20 0x5560e7ac92c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7ab4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa8340 0x5560e7ab4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f59faff5640 0 client.16929 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2401189517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa7a20 0x5560e7ab40e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 >> 172.21.15.16:0/2401189517 conn(0x5560e7aa40f0 msgr2=0x5560e7aa9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 shutdown_connections 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2401189517 wait complete. 2024-09-12T01:01:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.833+0000 7f5a1fb47740 20 client.16929 trim_cache size 0 max 16384 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.850+0000 7f5a1fb47740 1 Processor -- start 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.850+0000 7f5a1fb47740 1 -- start start 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 0x5560e7ab5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7acf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181ab5a0 con 0x5560e7aa8340 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181ad0e0 con 0x5560e7ab7150 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181abc70 con 0x5560e7aa7a20 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42902/0 (socket says 172.21.15.16:42902) 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 -- 172.21.15.16:0/2432906937 learned_addr learned my addr 172.21.15.16:0/2432906937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7acf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 0x5560e7ab5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 -- 172.21.15.16:0/2432906937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 msgr2=0x5560e7ab5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 0x5560e7ab5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 -- 172.21.15.16:0/2432906937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 msgr2=0x5560e7acf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7acf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 -- 172.21.15.16:0/2432906937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e7a9e680 con 0x5560e7aa8340 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a177fe640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7acf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 0x5560e7ab5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.851+0000 7f5a167fc640 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c007590 tx=0x7f5a0c097e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce475999237c954e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f59fcff9640 1 -- 172.21.15.16:0/2432906937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c09bac0 con 0x5560e7aa8340 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f59fcff9640 1 -- 172.21.15.16:0/2432906937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c09fe70 con 0x5560e7aa8340 2024-09-12T01:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f59fcff9640 1 -- 172.21.15.16:0/2432906937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0ccf70 con 0x5560e7aa8340 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2432906937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 msgr2=0x5560e7acebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c007590 tx=0x7f5a0c097e90 comp rx=0 tx=0).stop 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2432906937 shutdown_connections 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7ab7150 0x5560e7acf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7acebe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/2432906937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7aa7a20 0x5560e7ab5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2432906937 >> 172.21.15.16:0/2432906937 conn(0x5560e7aa40f0 msgr2=0x5560e7ab5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2432906937 shutdown_connections 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.852+0000 7f5a1fb47740 1 -- 172.21.15.16:0/2432906937 wait complete. 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 -- start start 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 Processor -- start 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa7a20 0x5560e7abb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7abbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181aca10 con 0x5560e7aa8340 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181ac340 con 0x5560e7aa7a20 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181a95a0 con 0x5560e7ab7150 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a1fb47740 10 client.0 _ll_register_callbacks cb 0x7f5a08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42910/0 (socket says 172.21.15.16:42910) 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a167fc640 1 -- 172.21.15.16:0/910408288 learned_addr learned my addr 172.21.15.16:0/910408288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.853+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa7a20 0x5560e7abb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a177fe640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7abbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 -- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 msgr2=0x5560e7abbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7abbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 -- 172.21.15.16:0/910408288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa7a20 msgr2=0x5560e7abb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa7a20 0x5560e7abb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 -- 172.21.15.16:0/910408288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e7b30e50 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a167fc640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c09ba50 tx=0x7f5a0c099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ed36f357b2412c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f59faff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c09bac0 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f5a0c018d60 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c09fe70 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c00fb30 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a1fb47740 10 client.16935 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560e7b32130 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.854+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c09fe70 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5a0c0a42a0 con 0x5560e7aa8340 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 10 client.16935 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:27.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 20 client.16935 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7b32890 0x5560e7b34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 4 client.16935 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560e7a14d00 con 0x5560e7b32890 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7b32890 0x5560e7b34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f59faff5640 10 client.16935 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:27.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.855+0000 7f5a16ffd640 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7b32890 0x5560e7b34cb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f449ed66f1de217d server_cookie=59e56f126429f5a8 in_seq=0 out_seq=0 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.856+0000 7f59faff5640 1 -- 172.21.15.16:0/910408288 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560e7a14d00 con 0x5560e7b32890 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.856+0000 7f59faff5640 10 client.16935 handle_command_reply: tid=0 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.856+0000 7f5a1fb47740 1 client.16935 shutdown 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 msgr2=0x5560e7abb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c09ba50 tx=0x7f5a0c099bd0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 shutdown_connections 2024-09-12T01:01:27.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5560e7b32890 0x5560e7b34cb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560e7ab7150 0x5560e7abbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5560e7aa8340 0x5560e7abb690 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 --2- 172.21.15.16:0/910408288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5560e7aa7a20 0x5560e7abb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f59faff5640 0 client.16935 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 >> 172.21.15.16:0/910408288 conn(0x5560e7aa40f0 msgr2=0x5560e7aa3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 shutdown_connections 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 1 -- 172.21.15.16:0/910408288 wait complete. 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1fb47740 20 client.16935 trim_cache size 0 max 16384 2024-09-12T01:01:27.865 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "79efddb5-6e35-491f-97ba-26234429e913", 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 msgr2=0x7f5a000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 0x7f5a000a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5a10021000 tx=0x7f5a1001e000 comp rx=0 tx=0).stop 2024-09-12T01:01:27.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 msgr2=0x7f5a181988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.857+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c920 tx=0x7f5a0c00cde0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 shutdown_connections 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5a000a2cf0 0x7f5a000a51b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a18106f10 0x7f5a1819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181988d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 --2- 172.21.15.16:0/2873081248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a18101de0 0x7f5a18198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 >> 172.21.15.16:0/2873081248 conn(0x7f5a180fd8c0 msgr2=0x7f5a180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 shutdown_connections 2024-09-12T01:01:27.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:27.858+0000 7f5a1d76e640 1 -- 172.21.15.16:0/2873081248 wait complete. 2024-09-12T01:01:28.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:01:29.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:29.314 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:29 smithi016 ceph-mon[30972]: pgmap v165: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:29.314 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:29 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:01:29.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:29 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:29 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:01:29.315 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:29 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:29 smithi159 ceph-mon[35566]: pgmap v165: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:29 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:01:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:29 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:29 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:01:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:29 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:29 smithi099 ceph-mon[35654]: pgmap v165: 97 pgs: 97 active+clean; 5.0 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:01:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:29 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:01:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:29 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:29 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:01:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:29 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 -- 172.21.15.16:0/1033001119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c10c170 msgr2=0x7f962c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 --2- 172.21.15.16:0/1033001119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c10c170 0x7f962c10e610 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9624009ca0 tx=0x7f962402a950 comp rx=0 tx=0).stop 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 -- 172.21.15.16:0/1033001119 shutdown_connections 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 --2- 172.21.15.16:0/1033001119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c10c170 0x7f962c10e610 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 --2- 172.21.15.16:0/1033001119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c103660 0x7f962c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 --2- 172.21.15.16:0/1033001119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c102d40 0x7f962c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 -- 172.21.15.16:0/1033001119 >> 172.21.15.16:0/1033001119 conn(0x7f962c0fc7a0 msgr2=0x7f962c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.514+0000 7f9633841640 1 -- 172.21.15.16:0/1033001119 shutdown_connections 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.515+0000 7f9633841640 1 -- 172.21.15.16:0/1033001119 wait complete. 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.515+0000 7f9633841640 1 Processor -- start 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.515+0000 7f9633841640 1 -- start start 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.515+0000 7f9633841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.515+0000 7f9633841640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c103660 0x7f962c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 0x7f962c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f962c111f50 con 0x7f962c102d40 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f962c111dd0 con 0x7f962c103660 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1120d0 con 0x7f962c10c170 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56838/0 (socket says 172.21.15.16:56838) 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 -- 172.21.15.16:0/1656462264 learned_addr learned my addr 172.21.15.16:0/1656462264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9630ddc640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c103660 0x7f962c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9631dde640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 0x7f962c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 -- 172.21.15.16:0/1656462264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 msgr2=0x7f962c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 0x7f962c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 -- 172.21.15.16:0/1656462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c103660 msgr2=0x7f962c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c103660 0x7f962c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 -- 172.21.15.16:0/1656462264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c1a6920 con 0x7f962c102d40 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9631dde640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 0x7f962c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96315dd640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f962800c890 tx=0x7f962800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96227fc640 1 -- 172.21.15.16:0/1656462264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628004220 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1a6c10 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96227fc640 1 -- 172.21.15.16:0/1656462264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96280043c0 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f96227fc640 1 -- 172.21.15.16:0/1656462264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628004a00 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.516+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c1a7150 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f96227fc640 1 -- 172.21.15.16:0/1656462264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f96280079b0 con 0x7f962c102d40 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 Processor -- start 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 -- start start 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 0x55965b88a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b88ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55965b7e52e0 con 0x55965b87f4a0 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55965b78cce0 con 0x55965b88a680 2024-09-12T01:01:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.519+0000 7f9635c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55965b7e12a0 con 0x55965b87fdc0 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96227fc640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56846/0 (socket says 172.21.15.16:56846) 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96315dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 0x55965b88a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 -- 172.21.15.16:0/1146044073 learned_addr learned my addr 172.21.15.16:0/1146044073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9630ddc640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b88ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96227fc640 1 -- 172.21.15.16:0/1656462264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f96280cf120 con 0x7f962c102d40 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 -- 172.21.15.16:0/1146044073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 msgr2=0x55965b88a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 0x55965b88a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 -- 172.21.15.16:0/1146044073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 msgr2=0x55965b88ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b88ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 -- 172.21.15.16:0/1146044073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965b771440 con 0x55965b87f4a0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96315dd640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 0x55965b88a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9630ddc640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b88ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f962402f580 tx=0x7f962402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08d9ba37f96abc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f9631dde640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 0x7f96180a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f962402f620 tx=0x7f9624041000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96177fe640 1 -- 172.21.15.16:0/1146044073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962404c070 con 0x55965b87f4a0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.520+0000 7f96177fe640 1 -- 172.21.15.16:0/1146044073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9624003e30 con 0x55965b87f4a0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f96177fe640 1 -- 172.21.15.16:0/1146044073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034040 con 0x55965b87f4a0 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 -- 172.21.15.16:0/1146044073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 msgr2=0x55965b87f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f962402f580 tx=0x7f962402f5b0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 -- 172.21.15.16:0/1146044073 shutdown_connections 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b88ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87fdc0 0x55965b88a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1146044073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87f4a0 0x55965b87f880 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 -- 172.21.15.16:0/1146044073 >> 172.21.15.16:0/1146044073 conn(0x55965b87ba30 msgr2=0x55965b87be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 -- 172.21.15.16:0/1146044073 shutdown_connections 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.521+0000 7f9635c1a740 1 -- 172.21.15.16:0/1146044073 wait complete. 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 -- start start 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 Processor -- start 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b88a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b887830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55965b88d250 con 0x55965b87fdc0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55965b7a0050 con 0x55965b88a680 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55965b88d3d0 con 0x55965b87f4a0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9635c1a740 10 client.0 _ll_register_callbacks cb 0x55965b7e0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9630ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9630ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56850/0 (socket says 172.21.15.16:56850) 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.522+0000 7f9630ddc640 1 -- 172.21.15.16:0/2686172558 learned_addr learned my addr 172.21.15.16:0/2686172558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f96315dd640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b88a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 -- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 msgr2=0x55965b88a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9631dde640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b887830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b88a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 -- 172.21.15.16:0/2686172558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 msgr2=0x55965b887830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b887830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 -- 172.21.15.16:0/2686172558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965b5563c0 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f96315dd640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b88a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9630ddc640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f961c00b9a0 tx=0x7f961c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecd4547b1cba920 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c003dc0 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f961c011440 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9624003e30 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f961c005020 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9635c1a740 10 client.16947 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55965b898ca0 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.523+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9624003e30 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f961c011640 con 0x55965b87fdc0 2024-09-12T01:01:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 10 client.16947 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 20 client.16947 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b89a200 0x55965b89c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 4 client.16947 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55965b7f2850 con 0x55965b89a200 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.524+0000 7f96315dd640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b89a200 0x55965b89c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f96315dd640 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b89a200 0x55965b89c660 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e6049094d805d server_cookie=35b181185a5f94f5 in_seq=0 out_seq=0 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9614ff9640 10 client.16947 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9614ff9640 1 -- 172.21.15.16:0/2686172558 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55965b7f2850 con 0x55965b89a200 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9614ff9640 10 client.16947 handle_command_reply: tid=0 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 client.16947 shutdown 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 msgr2=0x55965b8872b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f961c00b9a0 tx=0x7f961c00be60 comp rx=0 tx=0).stop 2024-09-12T01:01:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 shutdown_connections 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b89a200 0x55965b89c660 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b887830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b8872b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9614ff9640 0 client.16947 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 --2- 172.21.15.16:0/2686172558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b88a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.525+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 >> 172.21.15.16:0/2686172558 conn(0x55965b87ba30 msgr2=0x55965b881710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.526+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 shutdown_connections 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.526+0000 7f9635c1a740 1 -- 172.21.15.16:0/2686172558 wait complete. 2024-09-12T01:01:29.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.526+0000 7f9635c1a740 20 client.16947 trim_cache size 0 max 16384 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 Processor -- start 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 -- start start 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b885b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 0x55965b8a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1afa60 con 0x55965b88a680 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1b15a0 con 0x55965b87fdc0 2024-09-12T01:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9635c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1b0130 con 0x55965b87f4a0 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9631dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9631dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56854/0 (socket says 172.21.15.16:56854) 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.543+0000 7f9631dde640 1 -- 172.21.15.16:0/1343067733 learned_addr learned my addr 172.21.15.16:0/1343067733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f96315dd640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b885b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9630ddc640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 0x55965b8a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 -- 172.21.15.16:0/1343067733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 msgr2=0x55965b885b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b885b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 -- 172.21.15.16:0/1343067733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 msgr2=0x55965b8a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 0x55965b8a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 -- 172.21.15.16:0/1343067733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965b87cae0 con 0x55965b88a680 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9630ddc640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 0x55965b8a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9631dde640 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f962402fde0 tx=0x7f9624004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c7194045b5415 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9616ffd640 1 -- 172.21.15.16:0/1343067733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962404c030 con 0x55965b88a680 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9616ffd640 1 -- 172.21.15.16:0/1343067733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9624056070 con 0x55965b88a680 2024-09-12T01:01:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.544+0000 7f9616ffd640 1 -- 172.21.15.16:0/1343067733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624034040 con 0x55965b88a680 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- 172.21.15.16:0/1343067733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 msgr2=0x55965b8a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f962402fde0 tx=0x7f9624004a60 comp rx=0 tx=0).stop 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- 172.21.15.16:0/1343067733 shutdown_connections 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b88a680 0x55965b8a2470 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b87fdc0 0x55965b8a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 --2- 172.21.15.16:0/1343067733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- 172.21.15.16:0/1343067733 >> 172.21.15.16:0/1343067733 conn(0x55965b87ba30 msgr2=0x55965b889330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- 172.21.15.16:0/1343067733 shutdown_connections 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- 172.21.15.16:0/1343067733 wait complete. 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 -- start start 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.545+0000 7f9635c1a740 1 Processor -- start 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b8906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b891120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1b0ed0 con 0x55965b87fdc0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1b0800 con 0x55965b88a680 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f962c1ada40 con 0x55965b87f4a0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9635c1a740 10 client.0 _ll_register_callbacks cb 0x7f961c0041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56866/0 (socket says 172.21.15.16:56866) 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 -- 172.21.15.16:0/693143215 learned_addr learned my addr 172.21.15.16:0/693143215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f96315dd640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b8906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9631dde640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b891120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 -- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 msgr2=0x55965b8906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b8906a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 -- 172.21.15.16:0/693143215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 msgr2=0x55965b891120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b891120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 -- 172.21.15.16:0/693143215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965b9041b0 con 0x55965b87fdc0 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f96315dd640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b8906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9631dde640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b891120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9630ddc640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f961c011c70 tx=0x7f961c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee244aff691c275 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.546+0000 7f9614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c027040 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f961c017040 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9624056070 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f961c005400 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 10 client.16953 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55965b905430 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9624056070 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f961c02a070 con 0x55965b87fdc0 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 10 client.16953 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 20 client.16953 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b905b50 0x55965b907f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 4 client.16953 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.547+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55965b7ecd60 con 0x55965b905b50 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f96315dd640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b905b50 0x55965b907f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f96315dd640 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b905b50 0x55965b907f40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9138bebc0538ba server_cookie=48003db66fa78ab5 in_seq=0 out_seq=0 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9614ff9640 10 client.16953 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9614ff9640 1 -- 172.21.15.16:0/693143215 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55965b7ecd60 con 0x55965b905b50 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9614ff9640 10 client.16953 handle_command_reply: tid=0 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 client.16953 shutdown 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 msgr2=0x55965b890be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f961c011c70 tx=0x7f961c00b510 comp rx=0 tx=0).stop 2024-09-12T01:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 shutdown_connections 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55965b905b50 0x55965b907f40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55965b88a680 0x55965b891120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55965b87fdc0 0x55965b890be0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9614ff9640 0 client.16953 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 --2- 172.21.15.16:0/693143215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55965b87f4a0 0x55965b8906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.548+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 >> 172.21.15.16:0/693143215 conn(0x55965b87ba30 msgr2=0x55965b888b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 shutdown_connections 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9635c1a740 1 -- 172.21.15.16:0/693143215 wait complete. 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9635c1a740 20 client.16953 trim_cache size 0 max 16384 2024-09-12T01:01:29.558 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:01:29.559 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:01:29.559 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:01:29.559 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 msgr2=0x7f96180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 0x7f96180a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f962402f620 tx=0x7f9624041000 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 msgr2=0x7f962c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f962800c890 tx=0x7f962800cd50 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 shutdown_connections 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96180a2cf0 0x7f96180a51b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f962c10c170 0x7f962c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f962c103660 0x7f962c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 --2- 172.21.15.16:0/1656462264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c102d40 0x7f962c1009a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 >> 172.21.15.16:0/1656462264 conn(0x7f962c0fc7a0 msgr2=0x7f962c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 shutdown_connections 2024-09-12T01:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:29.549+0000 7f9633841640 1 -- 172.21.15.16:0/1656462264 wait complete. 2024-09-12T01:01:30.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:30 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:01:30.354 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:01:30.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:01:30.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:30 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:01:30.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:30 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:01:30.609 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.123+0000 7fb532112640 1 -- 172.21.15.16:0/3048097243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 msgr2=0x7fb52c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.123+0000 7fb532112640 1 --2- 172.21.15.16:0/3048097243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c10ea90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb524009ca0 tx=0x7fb52402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.123+0000 7fb532112640 1 -- 172.21.15.16:0/3048097243 shutdown_connections 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 --2- 172.21.15.16:0/3048097243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c10ea90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 --2- 172.21.15.16:0/3048097243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c102790 0x7fb52c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 --2- 172.21.15.16:0/3048097243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c101de0 0x7fb52c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 -- 172.21.15.16:0/3048097243 >> 172.21.15.16:0/3048097243 conn(0x7fb52c0fd8c0 msgr2=0x7fb52c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 -- 172.21.15.16:0/3048097243 shutdown_connections 2024-09-12T01:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 -- 172.21.15.16:0/3048097243 wait complete. 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.124+0000 7fb532112640 1 Processor -- start 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 -- start start 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 0x7fb52c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102790 0x7fb52c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c111fc0 con 0x7fb52c106f10 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c111e40 con 0x7fb52c101de0 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb532112640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c112140 con 0x7fb52c102790 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56890/0 (socket says 172.21.15.16:56890) 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 -- 172.21.15.16:0/2243535565 learned_addr learned my addr 172.21.15.16:0/2243535565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52b7fe640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 0x7fb52c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52affd640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102790 0x7fb52c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 -- 172.21.15.16:0/2243535565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102790 msgr2=0x7fb52c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102790 0x7fb52c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 -- 172.21.15.16:0/2243535565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 msgr2=0x7fb52c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 0x7fb52c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52bfff640 1 -- 172.21.15.16:0/2243535565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19d430 con 0x7fb52c106f10 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.125+0000 7fb52b7fe640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 0x7fb52c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb52bfff640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb524003f10 tx=0x7fb524003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524031710 con 0x7fb52c106f10 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5240318b0 con 0x7fb52c106f10 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19d6c0 con 0x7fb52c106f10 2024-09-12T01:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402fb50 con 0x7fb52c106f10 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.126+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a5000 con 0x7fb52c106f10 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.127+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c1021c0 con 0x7fb52c106f10 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.128+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb52402fd30 con 0x7fb52c106f10 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.129+0000 7fb528ff9640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.129+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb5240f0980 con 0x7fb52c106f10 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.129+0000 7fb52b7fe640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.129+0000 7fb52b7fe640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c00b5f0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.133+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb524004040 con 0x7fb52c106f10 2024-09-12T01:01:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.345+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb52c10a0f0 con 0x7fb52c106f10 2024-09-12T01:01:31.345 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:31 smithi016 ceph-mon[30972]: pgmap v166: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.346+0000 7fb528ff9640 1 -- 172.21.15.16:0/2243535565 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fb5240bd680 con 0x7fb52c106f10 2024-09-12T01:01:31.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:01:31.346 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:01:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 msgr2=0x7fb5140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c00b5f0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 msgr2=0x7fb52c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb524003f10 tx=0x7fb524003f40 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 shutdown_connections 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb5140a2cf0 0x7fb5140a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c106f10 0x7fb52c19cd50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c102790 0x7fb52c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 --2- 172.21.15.16:0/2243535565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb52c101de0 0x7fb52c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 >> 172.21.15.16:0/2243535565 conn(0x7fb52c0fd8c0 msgr2=0x7fb52c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:31.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 shutdown_connections 2024-09-12T01:01:31.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:31.348+0000 7fb532112640 1 -- 172.21.15.16:0/2243535565 wait complete. 2024-09-12T01:01:31.365 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:31 smithi159 ceph-mon[35566]: pgmap v166: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:31 smithi099 ceph-mon[35654]: pgmap v166: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-09-12T01:01:32.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:01:32.306 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:32.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:32 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2243535565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:32.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2243535565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:32.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:32 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2243535565' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:01:32.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 -- 172.21.15.16:0/886361308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 msgr2=0x7f3ff010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/886361308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff010ec40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009cd0 tx=0x7f3fe802a910 comp rx=0 tx=0).stop 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 -- 172.21.15.16:0/886361308 shutdown_connections 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/886361308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff010ec40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/886361308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff01036e0 0x7f3ff0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/886361308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff0102d30 0x7f3ff0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 -- 172.21.15.16:0/886361308 >> 172.21.15.16:0/886361308 conn(0x7f3ff00fc7d0 msgr2=0x7f3ff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 -- 172.21.15.16:0/886361308 shutdown_connections 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 -- 172.21.15.16:0/886361308 wait complete. 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.832+0000 7f3ff5f43640 1 Processor -- start 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 -- start start 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 0x7f3ff01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff01036e0 0x7f3ff01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0112160 con 0x7f3ff01082a0 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0111fe0 con 0x7f3ff0102d30 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3ff5f43640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01122e0 con 0x7f3ff01036e0 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56912/0 (socket says 172.21.15.16:56912) 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 -- 172.21.15.16:0/2579858951 learned_addr learned my addr 172.21.15.16:0/2579858951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 0x7f3ff01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feeffd640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff01036e0 0x7f3ff01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 -- 172.21.15.16:0/2579858951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff01036e0 msgr2=0x7f3ff01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff01036e0 0x7f3ff01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 -- 172.21.15.16:0/2579858951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 msgr2=0x7f3ff01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 0x7f3ff01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3feffff640 1 -- 172.21.15.16:0/2579858951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff019af60 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.833+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 0x7f3ff01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3feffff640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005cc0 tx=0x7f3fe8031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3fecff9640 1 -- 172.21.15.16:0/2579858951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8031850 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3fecff9640 1 -- 172.21.15.16:0/2579858951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8004520 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff019b1f0 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3fecff9640 1 -- 172.21.15.16:0/2579858951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe802fe20 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.834+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff01a73d0 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.836+0000 7f3fecff9640 1 -- 172.21.15.16:0/2579858951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3fe8004040 con 0x7f3ff01082a0 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 Processor -- start 2024-09-12T01:01:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 -- start start 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8db40 0x559b62c8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b62bf22f0 con 0x559b62c8c5d0 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b62b99cf0 con 0x559b62c8db40 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3ff831c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559b62bee2b0 con 0x559b62c97370 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3fecff9640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 0x7f3fd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56914/0 (socket says 172.21.15.16:56914) 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3fef7fe640 1 -- 172.21.15.16:0/3437882495 learned_addr learned my addr 172.21.15.16:0/3437882495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3feffff640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 0x7f3fd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.837+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8db40 0x559b62c8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 -- 172.21.15.16:0/3437882495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 msgr2=0x559b62c99760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c99760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fecff9640 1 -- 172.21.15.16:0/2579858951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f3fe80f0a40 con 0x7f3ff01082a0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 -- 172.21.15.16:0/3437882495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8db40 msgr2=0x559b62c8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8db40 0x559b62c8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 -- 172.21.15.16:0/3437882495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b62b7e3a0 con 0x559b62c8c5d0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0007850 tx=0x7f3fe0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4f6d7f89ff499d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3feffff640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 0x7f3fd80a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80bcfd0 tx=0x7f3fe808f000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fd5ffb640 1 -- 172.21.15.16:0/3437882495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000d950 con 0x559b62c8c5d0 2024-09-12T01:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fd5ffb640 1 -- 172.21.15.16:0/3437882495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe0018070 con 0x559b62c8c5d0 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.838+0000 7f3fd5ffb640 1 -- 172.21.15.16:0/3437882495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00123e0 con 0x559b62c8c5d0 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- 172.21.15.16:0/3437882495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 msgr2=0x559b62c8ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0007850 tx=0x7f3fe0007d10 comp rx=0 tx=0).stop 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- 172.21.15.16:0/3437882495 shutdown_connections 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c8ca10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3437882495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8db40 0x559b62c8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- 172.21.15.16:0/3437882495 >> 172.21.15.16:0/3437882495 conn(0x559b62c55310 msgr2=0x559b62c88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- 172.21.15.16:0/3437882495 shutdown_connections 2024-09-12T01:01:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- 172.21.15.16:0/3437882495 wait complete. 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 -- start start 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.839+0000 7f3ff831c740 1 Processor -- start 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c96ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c944f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b62c9a070 con 0x559b62c8db40 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b62bad060 con 0x559b62c8c5d0 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559b62c9a1f0 con 0x559b62c97370 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3ff831c740 10 client.0 _ll_register_callbacks cb 0x559b629008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56928/0 (socket says 172.21.15.16:56928) 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 -- 172.21.15.16:0/4228523884 learned_addr learned my addr 172.21.15.16:0/4228523884 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3feeffd640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c96ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3feeffd640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 -- 172.21.15.16:0/4228523884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 msgr2=0x559b62c944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 -- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 msgr2=0x559b62c96ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c96ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 -- 172.21.15.16:0/4228523884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b629633c0 con 0x559b62c8db40 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3feeffd640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006f40 tx=0x7f3fe00136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85517e4f3d6752d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.840+0000 7f3fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000de20 con 0x559b62c8db40 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f3fe001c3f0 con 0x559b62c8db40 2024-09-12T01:01:32.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe0018070 con 0x559b62c8db40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fe001c5f0 con 0x559b62c8db40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 10 client.16971 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b62ca5ae0 con 0x559b62c8db40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe0018070 con 0x559b62c8db40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3fe00248d0 con 0x559b62c8db40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 10 client.16971 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 10 client.16971 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 20 client.16971 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62ca6f40 0x559b62ca9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 4 client.16971 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.841+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b62bff860 con 0x559b62ca6f40 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.842+0000 7f3feeffd640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62ca6f40 0x559b62ca9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.842+0000 7f3feeffd640 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62ca6f40 0x559b62ca9330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c09ae3516f1b7e77 server_cookie=4ebcce27e01366ed in_seq=0 out_seq=0 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.842+0000 7f3fceffd640 10 client.16971 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3fceffd640 1 -- 172.21.15.16:0/4228523884 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b62bff860 con 0x559b62ca6f40 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3fceffd640 10 client.16971 handle_command_reply: tid=0 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 client.16971 shutdown 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 msgr2=0x559b62c93f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0006f40 tx=0x7f3fe00136b0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 shutdown_connections 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62ca6f40 0x559b62ca9330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c97370 0x559b62c944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8db40 0x559b62c93f90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3fceffd640 0 client.16971 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 --2- 172.21.15.16:0/4228523884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c96ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 >> 172.21.15.16:0/4228523884 conn(0x559b62c55310 msgr2=0x559b62c8e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 shutdown_connections 2024-09-12T01:01:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 1 -- 172.21.15.16:0/4228523884 wait complete. 2024-09-12T01:01:32.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.843+0000 7f3ff831c740 20 client.16971 trim_cache size 0 max 16384 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 Processor -- start 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 -- start start 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c88e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62ca3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01afb70 con 0x559b62c97370 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b16b0 con 0x559b62c8c5d0 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3ff831c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b0240 con 0x559b62c8db40 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3feeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3feeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56934/0 (socket says 172.21.15.16:56934) 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.856+0000 7f3feeffd640 1 -- 172.21.15.16:0/3247848563 learned_addr learned my addr 172.21.15.16:0/3247848563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62ca3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c88e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 -- 172.21.15.16:0/3247848563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 msgr2=0x559b62ca3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62ca3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 -- 172.21.15.16:0/3247848563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 msgr2=0x559b62c88e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c88e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 -- 172.21.15.16:0/3247848563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b62c89960 con 0x559b62c97370 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3feeffd640 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3ff019bf40 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9096241ea22e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3fd57fa640 1 -- 172.21.15.16:0/3247848563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40185d0 con 0x559b62c97370 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3fd57fa640 1 -- 172.21.15.16:0/3247848563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe401c070 con 0x559b62c97370 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.857+0000 7f3fd57fa640 1 -- 172.21.15.16:0/3247848563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4017880 con 0x559b62c97370 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- 172.21.15.16:0/3247848563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 msgr2=0x559b62cae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3ff019bf40 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- 172.21.15.16:0/3247848563 shutdown_connections 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c97370 0x559b62cae990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62ca3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 --2- 172.21.15.16:0/3247848563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c8c5d0 0x559b62c88e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- 172.21.15.16:0/3247848563 >> 172.21.15.16:0/3247848563 conn(0x559b62c55310 msgr2=0x559b62c88700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- 172.21.15.16:0/3247848563 shutdown_connections 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- 172.21.15.16:0/3247848563 wait complete. 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 -- start start 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.858+0000 7f3ff831c740 1 Processor -- start 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62c9fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c97370 0x559b62c90bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b0fe0 con 0x559b62c8c5d0 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01b0910 con 0x559b62c97370 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01adb50 con 0x559b62c8db40 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3ff831c740 10 client.0 _ll_register_callbacks cb 0x7f3fe00076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56936/0 (socket says 172.21.15.16:56936) 2024-09-12T01:01:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 -- 172.21.15.16:0/757328878 learned_addr learned my addr 172.21.15.16:0/757328878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62c9fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c97370 0x559b62c90bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 -- 172.21.15.16:0/757328878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 msgr2=0x559b62c9fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62c9fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 -- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c97370 msgr2=0x559b62c90bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c97370 0x559b62c90bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 -- 172.21.15.16:0/757328878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b62c88510 con 0x559b62c8c5d0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3feeffd640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4016470 tx=0x7f3fe4016fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6871965c539918 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.859+0000 7f3fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4031020 con 0x559b62c8c5d0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f3fe4017b70 con 0x559b62c8c5d0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe401c070 con 0x559b62c8c5d0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fe403c050 con 0x559b62c8c5d0 2024-09-12T01:01:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 10 client.16977 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b62d11480 con 0x559b62c8c5d0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe401c070 con 0x559b62c8c5d0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3fe402a480 con 0x559b62c8c5d0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 10 client.16977 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 10 client.16977 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 20 client.16977 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62d11ae0 0x559b62d13f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 4 client.16977 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.860+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b62bf9d70 con 0x559b62d11ae0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62d11ae0 0x559b62d13f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fef7fe640 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62d11ae0 0x559b62d13f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a91fa95b96d3819e server_cookie=f413907490972bac in_seq=0 out_seq=0 2024-09-12T01:01:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fceffd640 10 client.16977 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fceffd640 1 -- 172.21.15.16:0/757328878 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b62bf9d70 con 0x559b62d11ae0 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fceffd640 10 client.16977 handle_command_reply: tid=0 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 client.16977 shutdown 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 msgr2=0x559b62c9f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4016470 tx=0x7f3fe4016fb0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 shutdown_connections 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559b62d11ae0 0x559b62d13f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b62c97370 0x559b62c90bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559b62c8db40 0x559b62c9fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 --2- 172.21.15.16:0/757328878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b62c8c5d0 0x559b62c9f890 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3fceffd640 0 client.16977 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:01:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.861+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 >> 172.21.15.16:0/757328878 conn(0x559b62c55310 msgr2=0x559b62c96010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 shutdown_connections 2024-09-12T01:01:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff831c740 1 -- 172.21.15.16:0/757328878 wait complete. 2024-09-12T01:01:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff831c740 20 client.16977 trim_cache size 0 max 16384 2024-09-12T01:01:32.871 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 msgr2=0x7f3fd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 0x7f3fd80a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80bcfd0 tx=0x7f3fe808f000 comp rx=0 tx=0).stop 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 msgr2=0x7f3ff019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005cc0 tx=0x7f3fe8031460 comp rx=0 tx=0).stop 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 shutdown_connections 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fd80a2cf0 0x7f3fd80a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff01082a0 0x7f3ff019a6f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff01036e0 0x7f3ff01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 --2- 172.21.15.16:0/2579858951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff0102d30 0x7f3ff01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 >> 172.21.15.16:0/2579858951 conn(0x7f3ff00fc7d0 msgr2=0x7f3ff0101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 shutdown_connections 2024-09-12T01:01:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:32.862+0000 7f3ff5f43640 1 -- 172.21.15.16:0/2579858951 wait complete. 2024-09-12T01:01:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:33 smithi159 ceph-mon[35566]: pgmap v167: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:33.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:33 smithi016 ceph-mon[30972]: pgmap v167: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:33.523 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:33 smithi099 ceph-mon[35654]: pgmap v167: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:33.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:01:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.434+0000 7fa086854640 1 -- 172.21.15.16:0/1424465722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0800745c0 msgr2=0x7fa0800749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.434+0000 7fa086854640 1 --2- 172.21.15.16:0/1424465722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0800745c0 0x7fa0800749a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa070009900 tx=0x7fa07002a480 comp rx=0 tx=0).stop 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 -- 172.21.15.16:0/1424465722 shutdown_connections 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 --2- 172.21.15.16:0/1424465722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa08010e420 0x7fa080110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 --2- 172.21.15.16:0/1424465722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 --2- 172.21.15.16:0/1424465722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0800745c0 0x7fa0800749a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 -- 172.21.15.16:0/1424465722 >> 172.21.15.16:0/1424465722 conn(0x7fa0800fd8c0 msgr2=0x7fa0800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 -- 172.21.15.16:0/1424465722 shutdown_connections 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 -- 172.21.15.16:0/1424465722 wait complete. 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 Processor -- start 2024-09-12T01:01:34.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.435+0000 7fa086854640 1 -- start start 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 0x7fa08019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa080114170 con 0x7fa08010e420 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa080113ff0 con 0x7fa080074ee0 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801142f0 con 0x7fa0800745c0 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56954/0 (socket says 172.21.15.16:56954) 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 -- 172.21.15.16:0/631271506 learned_addr learned my addr 172.21.15.16:0/631271506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07ffff640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 0x7fa08019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 -- 172.21.15.16:0/631271506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 msgr2=0x7fa08019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 0x7fa08019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 -- 172.21.15.16:0/631271506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 msgr2=0x7fa08019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 -- 172.21.15.16:0/631271506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019f620 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07ffff640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 0x7fa08019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa084df1640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e9d0 tx=0x7fa07800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07d7fa640 1 -- 172.21.15.16:0/631271506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07800cd80 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08019f910 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07d7fa640 1 -- 172.21.15.16:0/631271506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa078005020 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa07d7fa640 1 -- 172.21.15.16:0/631271506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780116d0 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.436+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801a71f0 con 0x7fa08010e420 2024-09-12T01:01:34.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa07d7fa640 1 -- 172.21.15.16:0/631271506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa0780051d0 con 0x7fa08010e420 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 Processor -- start 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 -- start start 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e387b40 0x557d6e386090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e386a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557d6e2ec2f0 con 0x557d6e391370 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557d6e293cf0 con 0x557d6e3865d0 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa088c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557d6e2e82b0 con 0x557d6e387b40 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.439+0000 7fa07d7fa640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 0x7fa0680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa084df1640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 0x7fa0680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e386a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56968/0 (socket says 172.21.15.16:56968) 2024-09-12T01:01:34.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 -- 172.21.15.16:0/641186812 learned_addr learned my addr 172.21.15.16:0/641186812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07d7fa640 1 -- 172.21.15.16:0/631271506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa0780cfac0 con 0x7fa08010e420 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e387b40 0x557d6e386090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 -- 172.21.15.16:0/641186812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e387b40 msgr2=0x557d6e386090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e387b40 0x557d6e386090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 -- 172.21.15.16:0/641186812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 msgr2=0x557d6e386a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e386a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 -- 172.21.15.16:0/641186812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6e2783a0 con 0x557d6e391370 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07ffff640 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e386a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa084df1640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 0x7fa0680a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa078098410 tx=0x7fa078091440 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa074012830 tx=0x7fa074009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93569c2968f4c32c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.440+0000 7fa0667fc640 1 -- 172.21.15.16:0/641186812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740107d0 con 0x557d6e391370 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa0667fc640 1 -- 172.21.15.16:0/641186812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa074010e50 con 0x557d6e391370 2024-09-12T01:01:34.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa0667fc640 1 -- 172.21.15.16:0/641186812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07401eca0 con 0x557d6e391370 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 -- 172.21.15.16:0/641186812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 msgr2=0x557d6e393760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa074012830 tx=0x7fa074009760 comp rx=0 tx=0).stop 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 -- 172.21.15.16:0/641186812 shutdown_connections 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e393760 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e386a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 --2- 172.21.15.16:0/641186812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e387b40 0x557d6e386090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 -- 172.21.15.16:0/641186812 >> 172.21.15.16:0/641186812 conn(0x557d6e34f310 msgr2=0x557d6e382c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 -- 172.21.15.16:0/641186812 shutdown_connections 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.441+0000 7fa088c2d740 1 -- 172.21.15.16:0/641186812 wait complete. 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 -- start start 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 Processor -- start 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e390d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e387b40 0x557d6e38e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557d6e394070 con 0x557d6e391370 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557d6e2a7060 con 0x557d6e387b40 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557d6e3941f0 con 0x557d6e3865d0 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa088c2d740 10 client.0 _ll_register_callbacks cb 0x557d6dffa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa07f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56974/0 (socket says 172.21.15.16:56974) 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa07f7fe640 1 -- 172.21.15.16:0/241405036 learned_addr learned my addr 172.21.15.16:0/241405036 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e390d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.442+0000 7fa07ffff640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e387b40 0x557d6e38e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 -- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 msgr2=0x557d6e390d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e390d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 -- 172.21.15.16:0/241405036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e387b40 msgr2=0x557d6e38e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e387b40 0x557d6e38e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 -- 172.21.15.16:0/241405036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6e05d3c0 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa07401e980 tx=0x7fa07401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7244081fd80fb38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07401ecc0 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa0740107a0 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa074010e50 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa07401ee60 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa088c2d740 10 client.16989 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d6e39fb10 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.443+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa074010e50 con 0x557d6e391370 2024-09-12T01:01:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa074027020 con 0x557d6e391370 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 10 client.16989 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 10 client.16989 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 20 client.16989 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e3a0e60 0x557d6e3a3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 4 client.16989 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6e2f9860 con 0x557d6e3a0e60 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa07ffff640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e3a0e60 0x557d6e3a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa07ffff640 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e3a0e60 0x557d6e3a3250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f087985c6a2d2919 server_cookie=484364175fb79b8 in_seq=0 out_seq=0 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.444+0000 7fa05f7fe640 10 client.16989 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa05f7fe640 1 -- 172.21.15.16:0/241405036 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d6e2f9860 con 0x557d6e3a0e60 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa05f7fe640 10 client.16989 handle_command_reply: tid=0 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 client.16989 shutdown 2024-09-12T01:01:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 msgr2=0x557d6e38e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa07401e980 tx=0x7fa07401b040 comp rx=0 tx=0).stop 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 shutdown_connections 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e3a0e60 0x557d6e3a3250 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e391370 0x557d6e38e610 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e387b40 0x557d6e38e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa05f7fe640 0 client.16989 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 --2- 172.21.15.16:0/241405036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e390d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 >> 172.21.15.16:0/241405036 conn(0x557d6e34f310 msgr2=0x557d6e388960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 shutdown_connections 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.445+0000 7fa088c2d740 1 -- 172.21.15.16:0/241405036 wait complete. 2024-09-12T01:01:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.446+0000 7fa088c2d740 20 client.16989 trim_cache size 0 max 16384 2024-09-12T01:01:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 Processor -- start 2024-09-12T01:01:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 -- start start 2024-09-12T01:01:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e39d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e391370 0x557d6e3a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801afab0 con 0x557d6e387b40 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801b15f0 con 0x557d6e391370 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.459+0000 7fa088c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801b0180 con 0x557d6e3865d0 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56982/0 (socket says 172.21.15.16:56982) 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 -- 172.21.15.16:0/3114346021 learned_addr learned my addr 172.21.15.16:0/3114346021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e391370 0x557d6e3a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e39d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 -- 172.21.15.16:0/3114346021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 msgr2=0x557d6e39d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e39d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 -- 172.21.15.16:0/3114346021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e391370 msgr2=0x557d6e3a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e391370 0x557d6e3a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 -- 172.21.15.16:0/3114346021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6e390990 con 0x557d6e387b40 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e39d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa07ffff640 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa070005ab0 tx=0x7fa070005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a90b5ef1af499d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa065ffb640 1 -- 172.21.15.16:0/3114346021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070035040 con 0x557d6e387b40 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa065ffb640 1 -- 172.21.15.16:0/3114346021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa070049070 con 0x557d6e387b40 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.460+0000 7fa065ffb640 1 -- 172.21.15.16:0/3114346021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070004c20 con 0x557d6e387b40 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- 172.21.15.16:0/3114346021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 msgr2=0x557d6e38d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa070005ab0 tx=0x7fa070005b50 comp rx=0 tx=0).stop 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- 172.21.15.16:0/3114346021 shutdown_connections 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e391370 0x557d6e3a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e38d000 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 --2- 172.21.15.16:0/3114346021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e3865d0 0x557d6e39d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- 172.21.15.16:0/3114346021 >> 172.21.15.16:0/3114346021 conn(0x557d6e34f310 msgr2=0x557d6e38c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- 172.21.15.16:0/3114346021 shutdown_connections 2024-09-12T01:01:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- 172.21.15.16:0/3114346021 wait complete. 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.461+0000 7fa088c2d740 1 -- start start 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 Processor -- start 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e3953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e391370 0x557d6e395e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801b0f20 con 0x557d6e387b40 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801b0850 con 0x557d6e3865d0 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0801ada90 con 0x557d6e391370 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa088c2d740 10 client.0 _ll_register_callbacks cb 0x7fa074005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56986/0 (socket says 172.21.15.16:56986) 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 -- 172.21.15.16:0/1344494801 learned_addr learned my addr 172.21.15.16:0/1344494801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e391370 0x557d6e395e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e3953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 -- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e391370 msgr2=0x557d6e395e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e391370 0x557d6e395e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 -- 172.21.15.16:0/1344494801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 msgr2=0x557d6e3953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e3953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.462+0000 7fa07ffff640 1 -- 172.21.15.16:0/1344494801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6e40ab40 con 0x557d6e387b40 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa07ffff640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa070006010 tx=0x7fa0700095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3523efd6c01f12f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070051020 con 0x557d6e387b40 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa070003800 con 0x557d6e387b40 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa070049070 con 0x557d6e387b40 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa070035040 con 0x557d6e387b40 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa088c2d740 10 client.16995 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:34.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d6e40bd70 con 0x557d6e387b40 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa070049070 con 0x557d6e387b40 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa07004d450 con 0x557d6e387b40 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa088c2d740 10 client.16995 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.463+0000 7fa088c2d740 10 client.16995 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa088c2d740 20 client.16995 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e40c4e0 0x557d6e40e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa088c2d740 4 client.16995 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d6e2f3d70 con 0x557d6e40c4e0 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e40c4e0 0x557d6e40e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa05f7fe640 10 client.16995 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa07f7fe640 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e40c4e0 0x557d6e40e900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83d62961b3172d75 server_cookie=f51df9bab178ae9f in_seq=0 out_seq=0 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa05f7fe640 1 -- 172.21.15.16:0/1344494801 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d6e2f3d70 con 0x557d6e40c4e0 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa05f7fe640 10 client.16995 handle_command_reply: tid=0 2024-09-12T01:01:34.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.464+0000 7fa088c2d740 1 client.16995 shutdown 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 msgr2=0x557d6e395900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa070006010 tx=0x7fa0700095f0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 shutdown_connections 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d6e391370 0x557d6e395e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x557d6e40c4e0 0x557d6e40e900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa05f7fe640 0 client.16995 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557d6e387b40 0x557d6e395900 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 --2- 172.21.15.16:0/1344494801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557d6e3865d0 0x557d6e3953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 >> 172.21.15.16:0/1344494801 conn(0x557d6e34f310 msgr2=0x557d6e38d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 shutdown_connections 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 1 -- 172.21.15.16:0/1344494801 wait complete. 2024-09-12T01:01:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa088c2d740 20 client.16995 trim_cache size 0 max 16384 2024-09-12T01:01:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 msgr2=0x7fa0680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 0x7fa0680a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa078098410 tx=0x7fa078091440 comp rx=0 tx=0).stop 2024-09-12T01:01:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 msgr2=0x7fa08019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.465+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e9d0 tx=0x7fa07800ee90 comp rx=0 tx=0).stop 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 shutdown_connections 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0680a2cf0 0x7fa0680a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa08010e420 0x7fa08019ef40 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa080074ee0 0x7fa08019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 --2- 172.21.15.16:0/631271506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0800745c0 0x7fa08019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 >> 172.21.15.16:0/631271506 conn(0x7fa0800fd8c0 msgr2=0x7fa0800fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 shutdown_connections 2024-09-12T01:01:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:34.466+0000 7fa086854640 1 -- 172.21.15.16:0/631271506 wait complete. 2024-09-12T01:01:35.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:35 smithi016 ceph-mon[30972]: pgmap v168: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:35.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:01:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:35 smithi159 ceph-mon[35566]: pgmap v168: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:35.431 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:35 smithi099 ceph-mon[35654]: pgmap v168: 97 pgs: 97 active+clean; 5.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.952+0000 7f1a661d3640 1 -- 172.21.15.16:0/2898663802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601082a0 msgr2=0x7f1a6010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.952+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2898663802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601082a0 0x7f1a6010ec40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009cd0 tx=0x7f1a5802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.952+0000 7f1a661d3640 1 -- 172.21.15.16:0/2898663802 shutdown_connections 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2898663802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601082a0 0x7f1a6010ec40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2898663802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a601036e0 0x7f1a60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2898663802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a60102d30 0x7f1a60103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 -- 172.21.15.16:0/2898663802 >> 172.21.15.16:0/2898663802 conn(0x7f1a600fc7d0 msgr2=0x7f1a600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 -- 172.21.15.16:0/2898663802 shutdown_connections 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 -- 172.21.15.16:0/2898663802 wait complete. 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 Processor -- start 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.953+0000 7f1a661d3640 1 -- start start 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a60102d30 0x7f1a60100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a601082a0 0x7f1a60101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601120a0 con 0x7f1a601036e0 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60111f20 con 0x7f1a601082a0 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a661d3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60112220 con 0x7f1a60102d30 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56994/0 (socket says 172.21.15.16:56994) 2024-09-12T01:01:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 -- 172.21.15.16:0/2100295131 learned_addr learned my addr 172.21.15.16:0/2100295131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a60102d30 0x7f1a60100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a601082a0 0x7f1a60101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 -- 172.21.15.16:0/2100295131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a60102d30 msgr2=0x7f1a60100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a60102d30 0x7f1a60100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 -- 172.21.15.16:0/2100295131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a601082a0 msgr2=0x7f1a60101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a601082a0 0x7f1a60101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 -- 172.21.15.16:0/2100295131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a601a6990 con 0x7f1a601036e0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.954+0000 7f1a5effd640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d740 tx=0x7f1a5400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.955+0000 7f1a5cff9640 1 -- 172.21.15.16:0/2100295131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54017070 con 0x7f1a601036e0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.955+0000 7f1a5cff9640 1 -- 172.21.15.16:0/2100295131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a54010460 con 0x7f1a601036e0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.955+0000 7f1a5cff9640 1 -- 172.21.15.16:0/2100295131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400f620 con 0x7f1a601036e0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.955+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a601a6c80 con 0x7f1a601036e0 2024-09-12T01:01:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.955+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a601a71c0 con 0x7f1a601036e0 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.957+0000 7f1a5cff9640 1 -- 172.21.15.16:0/2100295131 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1a5401a070 con 0x7f1a601036e0 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.957+0000 7f1a685ac740 1 Processor -- start 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.957+0000 7f1a685ac740 1 -- start start 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.957+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bcf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a685ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d81b282f0 con 0x560d81bc25d0 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a685ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d81acfcf0 con 0x560d81bcd370 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a685ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d81b242b0 con 0x560d81bc3b40 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57004/0 (socket says 172.21.15.16:57004) 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 -- 172.21.15.16:0/3176828811 learned_addr learned my addr 172.21.15.16:0/3176828811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5cff9640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 0x7f1a480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5effd640 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bcf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 -- 172.21.15.16:0/3176828811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 msgr2=0x560d81bc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 0x7f1a480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bc2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 -- 172.21.15.16:0/3176828811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 msgr2=0x560d81bcf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bcf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 -- 172.21.15.16:0/3176828811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d81ab43a0 con 0x560d81bc25d0 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5cff9640 1 -- 172.21.15.16:0/2100295131 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f1a540cee40 con 0x7f1a601036e0 2024-09-12T01:01:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.958+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ad00 tx=0x7f1a5000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d744ac8bde28f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a45ffb640 1 -- 172.21.15.16:0/3176828811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5001d070 con 0x560d81bc25d0 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 0x7f1a480a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a58005cc0 tx=0x7f1a5802a8e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a45ffb640 1 -- 172.21.15.16:0/3176828811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a50018b20 con 0x560d81bc25d0 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a45ffb640 1 -- 172.21.15.16:0/3176828811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50010bc0 con 0x560d81bc25d0 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 -- 172.21.15.16:0/3176828811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 msgr2=0x560d81bc2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000ad00 tx=0x7f1a5000f790 comp rx=0 tx=0).stop 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 -- 172.21.15.16:0/3176828811 shutdown_connections 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bcf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc25d0 0x560d81bc2a10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 --2- 172.21.15.16:0/3176828811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 -- 172.21.15.16:0/3176828811 >> 172.21.15.16:0/3176828811 conn(0x560d81b8b310 msgr2=0x560d81bbec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 -- 172.21.15.16:0/3176828811 shutdown_connections 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.959+0000 7f1a685ac740 1 -- 172.21.15.16:0/3176828811 wait complete. 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 -- start start 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 Processor -- start 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560d81bd0070 con 0x560d81bc3b40 2024-09-12T01:01:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d81ae3060 con 0x560d81bcd370 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d81bd01f0 con 0x560d81bc25d0 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a685ac740 10 client.0 _ll_register_callbacks cb 0x560d818368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57010/0 (socket says 172.21.15.16:57010) 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.960+0000 7f1a5effd640 1 -- 172.21.15.16:0/535317733 learned_addr learned my addr 172.21.15.16:0/535317733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 -- 172.21.15.16:0/535317733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 msgr2=0x560d81bccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 -- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 msgr2=0x560d81bca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 -- 172.21.15.16:0/535317733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d818993c0 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a5effd640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1a54097d00 tx=0x7f1a5409ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccad1cb53fbe4dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54021030 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f1a540979a0 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a50018b20 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a540a3fa0 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a685ac740 10 client.17007 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.961+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d81bdbb70 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a54064070 con 0x560d81bc3b40 2024-09-12T01:01:35.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a540a3b10 con 0x560d81bc3b40 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 10 client.17007 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 20 client.17007 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81bdd030 0x560d81bdf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 4 client.17007 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d81b35860 con 0x560d81bdd030 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.962+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81bdd030 0x560d81bdf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.963+0000 7f1a3effd640 10 client.17007 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.963+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81bdd030 0x560d81bdf420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff22d82994b3d1fc server_cookie=bdc6109ab5c2eda8 in_seq=0 out_seq=0 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.963+0000 7f1a3effd640 1 -- 172.21.15.16:0/535317733 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d81b35860 con 0x560d81bdd030 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a3effd640 10 client.17007 handle_command_reply: tid=0 2024-09-12T01:01:35.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 client.17007 shutdown 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 msgr2=0x560d81bca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f1a54097d00 tx=0x7f1a5409ba30 comp rx=0 tx=0).stop 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 shutdown_connections 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81bdd030 0x560d81bdf420 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bcd370 0x560d81bca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bc3b40 0x560d81bca000 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 --2- 172.21.15.16:0/535317733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 >> 172.21.15.16:0/535317733 conn(0x560d81b8b310 msgr2=0x560d81bc4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a3effd640 0 client.17007 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 shutdown_connections 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 1 -- 172.21.15.16:0/535317733 wait complete. 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.964+0000 7f1a685ac740 20 client.17007 trim_cache size 0 max 16384 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 Processor -- start 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 -- start start 2024-09-12T01:01:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 0x560d81bd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bbe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601af9f0 con 0x560d81bcd370 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601b1530 con 0x560d81bc25d0 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.977+0000 7f1a685ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601b00c0 con 0x560d81bc3b40 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57026/0 (socket says 172.21.15.16:57026) 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 -- 172.21.15.16:0/1149425024 learned_addr learned my addr 172.21.15.16:0/1149425024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 0x560d81bd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5effd640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bbe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 -- 172.21.15.16:0/1149425024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 msgr2=0x560d81bbe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bbe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 -- 172.21.15.16:0/1149425024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 msgr2=0x560d81bd9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 0x560d81bd9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 -- 172.21.15.16:0/1149425024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d81bbf180 con 0x560d81bcd370 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 0x560d81bd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1a58035000 tx=0x7f1a58009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfe899ed803971c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a457fa640 1 -- 172.21.15.16:0/1149425024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004240 con 0x560d81bcd370 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a457fa640 1 -- 172.21.15.16:0/1149425024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a58036040 con 0x560d81bcd370 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.978+0000 7f1a457fa640 1 -- 172.21.15.16:0/1149425024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58030ed0 con 0x560d81bcd370 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- 172.21.15.16:0/1149425024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 msgr2=0x560d81bbec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f1a58035000 tx=0x7f1a58009380 comp rx=0 tx=0).stop 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- 172.21.15.16:0/1149425024 shutdown_connections 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bbec40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc3b40 0x560d81bbe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 --2- 172.21.15.16:0/1149425024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc25d0 0x560d81bd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- 172.21.15.16:0/1149425024 >> 172.21.15.16:0/1149425024 conn(0x560d81b8b310 msgr2=0x560d81bc5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:35.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- 172.21.15.16:0/1149425024 shutdown_connections 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- 172.21.15.16:0/1149425024 wait complete. 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.979+0000 7f1a685ac740 1 -- start start 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 Processor -- start 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bd54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 0x560d81bd59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601b0e60 con 0x560d81bcd370 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601b0790 con 0x560d81bc3b40 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601ad9d0 con 0x560d81bc25d0 2024-09-12T01:01:35.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a685ac740 10 client.0 _ll_register_callbacks cb 0x7f1a540976a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57028/0 (socket says 172.21.15.16:57028) 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 0x560d81bd59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 -- 172.21.15.16:0/2286902309 learned_addr learned my addr 172.21.15.16:0/2286902309 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bd54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 -- 172.21.15.16:0/2286902309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 msgr2=0x560d81bd54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bd54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 -- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 msgr2=0x560d81bd59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 0x560d81bd59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5ffff640 1 -- 172.21.15.16:0/2286902309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d81bd29c0 con 0x560d81bcd370 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bd54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.980+0000 7f1a5effd640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 0x560d81bd59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:01:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a5ffff640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009430 tx=0x7f1a58009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a14a62e5b0293d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a580305a0 con 0x560d81bcd370 2024-09-12T01:01:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f1a58030740 con 0x560d81bcd370 2024-09-12T01:01:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a58036040 con 0x560d81bcd370 2024-09-12T01:01:35.999 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a5806a050 con 0x560d81bcd370 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 10 client.17013 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d81c48350 con 0x560d81bcd370 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a58036040 con 0x560d81bcd370 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a58030070 con 0x560d81bcd370 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 10 client.17013 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 10 client.17013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 20 client.17013 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:36.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81c489e0 0x560d81c4ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 4 client.17013 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.981+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d81b2fd70 con 0x560d81c489e0 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81c489e0 0x560d81c4ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a3effd640 10 client.17013 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a5f7fe640 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81c489e0 0x560d81c4ae00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a43167ecbe7883e server_cookie=40008772c579d27 in_seq=0 out_seq=0 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a3effd640 1 -- 172.21.15.16:0/2286902309 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d81b2fd70 con 0x560d81c489e0 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a3effd640 10 client.17013 handle_command_reply: tid=0 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.982+0000 7f1a685ac740 1 client.17013 shutdown 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 msgr2=0x560d81bd2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009430 tx=0x7f1a58009540 comp rx=0 tx=0).stop 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 shutdown_connections 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560d81c489e0 0x560d81c4ae00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560d81bcd370 0x560d81bd2480 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d81bc3b40 0x560d81bd59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 --2- 172.21.15.16:0/2286902309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d81bc25d0 0x560d81bd54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 >> 172.21.15.16:0/2286902309 conn(0x560d81b8b310 msgr2=0x560d81bc5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a3effd640 0 client.17013 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 shutdown_connections 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 1 -- 172.21.15.16:0/2286902309 wait complete. 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a685ac740 20 client.17013 trim_cache size 0 max 16384 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 msgr2=0x7f1a480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 0x7f1a480a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a58005cc0 tx=0x7f1a5802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 msgr2=0x7f1a60100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400d740 tx=0x7f1a5400db40 comp rx=0 tx=0).stop 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 shutdown_connections 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1a480a2cf0 0x7f1a480a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a601082a0 0x7f1a60101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a601036e0 0x7f1a60100ed0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 --2- 172.21.15.16:0/2100295131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a60102d30 0x7f1a60100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.983+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 >> 172.21.15.16:0/2100295131 conn(0x7f1a600fc7d0 msgr2=0x7f1a60106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.984+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 shutdown_connections 2024-09-12T01:01:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:35.984+0000 7f1a661d3640 1 -- 172.21.15.16:0/2100295131 wait complete. 2024-09-12T01:01:36.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:01:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:37 smithi159 ceph-mon[35566]: pgmap v169: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:37.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:37 smithi016 ceph-mon[30972]: pgmap v169: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2848736241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 msgr2=0x7f7bd0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2848736241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd0103120 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002a480 comp rx=0 tx=0).stop 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2848736241 shutdown_connections 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2848736241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd010c170 0x7f7bd010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2848736241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd0103660 0x7f7bd010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2848736241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd0103120 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2848736241 >> 172.21.15.16:0/2848736241 conn(0x7f7bd00fc7a0 msgr2=0x7f7bd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2848736241 shutdown_connections 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2848736241 wait complete. 2024-09-12T01:01:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 Processor -- start 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.470+0000 7f7bd69d3640 1 -- start start 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 0x7f7bd0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd010c170 0x7f7bd0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0111f70 con 0x7f7bd0102d40 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd0111df0 con 0x7f7bd010c170 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd69d3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01120f0 con 0x7f7bd0103660 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57044/0 (socket says 172.21.15.16:57044) 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 -- 172.21.15.16:0/2019140519 learned_addr learned my addr 172.21.15.16:0/2019140519 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd010c170 0x7f7bd0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 0x7f7bd0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 -- 172.21.15.16:0/2019140519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 msgr2=0x7f7bd0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 0x7f7bd0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 -- 172.21.15.16:0/2019140519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd010c170 msgr2=0x7f7bd0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd010c170 0x7f7bd0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 -- 172.21.15.16:0/2019140519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd01a6910 con 0x7f7bd0102d40 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 0x7f7bd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005ab0 tx=0x7f7bc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.471+0000 7f7bcdffb640 1 -- 172.21.15.16:0/2019140519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc003f030 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.472+0000 7f7bcdffb640 1 -- 172.21.15.16:0/2019140519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0003ce0 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.472+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd01a6ba0 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.472+0000 7f7bcdffb640 1 -- 172.21.15.16:0/2019140519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0035070 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.472+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a7080 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bcdffb640 1 -- 172.21.15.16:0/2019140519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7bc0042030 con 0x7f7bd0102d40 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 Processor -- start 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 -- start start 2024-09-12T01:01:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00007b40 0x563b00006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563afff6c2f0 con 0x563b00011370 2024-09-12T01:01:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563afff13cf0 con 0x563b000065d0 2024-09-12T01:01:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.474+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563afff682b0 con 0x563b00007b40 2024-09-12T01:01:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bcdffb640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 0x7f7bbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57058/0 (socket says 172.21.15.16:57058) 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 -- 172.21.15.16:0/682057186 learned_addr learned my addr 172.21.15.16:0/682057186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00007b40 0x563b00006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 0x7f7bbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bcdffb640 1 -- 172.21.15.16:0/2019140519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7bc00efde0 con 0x7f7bd0102d40 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 -- 172.21.15.16:0/682057186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00007b40 msgr2=0x563b00006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00007b40 0x563b00006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 -- 172.21.15.16:0/682057186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 msgr2=0x563b00006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 -- 172.21.15.16:0/682057186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563affef83a0 con 0x563b00011370 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800bb30 tx=0x7f7bb800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfb12dfd13ee2c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 0x7f7bbc0a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800bfd0 tx=0x7f7bc8005c20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc67fc640 1 -- 172.21.15.16:0/682057186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8017420 con 0x563b00011370 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc67fc640 1 -- 172.21.15.16:0/682057186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb8017aa0 con 0x563b00011370 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.475+0000 7f7bc67fc640 1 -- 172.21.15.16:0/682057186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800fb70 con 0x563b00011370 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- 172.21.15.16:0/682057186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 msgr2=0x563b00013760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800bb30 tx=0x7f7bb800d340 comp rx=0 tx=0).stop 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- 172.21.15.16:0/682057186 shutdown_connections 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00013760 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/682057186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00007b40 0x563b00006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- 172.21.15.16:0/682057186 >> 172.21.15.16:0/682057186 conn(0x563afffcf310 msgr2=0x563b00002c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- 172.21.15.16:0/682057186 shutdown_connections 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- 172.21.15.16:0/682057186 wait complete. 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.476+0000 7f7bd8dac740 1 -- start start 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 Processor -- start 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00010c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b0000e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b00014070 con 0x563b00007b40 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563afff27060 con 0x563b000065d0 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b000141f0 con 0x563b00011370 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd8dac740 10 client.0 _ll_register_callbacks cb 0x563affc7a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57062/0 (socket says 172.21.15.16:57062) 2024-09-12T01:01:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 -- 172.21.15.16:0/2061795873 learned_addr learned my addr 172.21.15.16:0/2061795873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b0000e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00010c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 -- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 msgr2=0x563b0000e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b0000e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 -- 172.21.15.16:0/2061795873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 msgr2=0x563b00010c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00010c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.477+0000 7f7bc7fff640 1 -- 172.21.15.16:0/2061795873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563affcdd3c0 con 0x563b00007b40 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800f790 tx=0x7f7bb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f70a00979c24a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8026020 con 0x563b00007b40 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7bb800dd80 con 0x563b00007b40 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb8017aa0 con 0x563b00007b40 2024-09-12T01:01:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bb8031050 con 0x563b00007b40 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 10 client.17025 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b0001fa30 con 0x563b00007b40 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb8017aa0 con 0x563b00007b40 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bb8021af0 con 0x563b00007b40 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 10 client.17025 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 20 client.17025 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b00020e90 0x563b00023280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 4 client.17025 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.478+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563afff79860 con 0x563b00020e90 2024-09-12T01:01:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.479+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b00020e90 0x563b00023280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.479+0000 7f7baf7fe640 10 client.17025 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.479+0000 7f7bcffff640 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b00020e90 0x563b00023280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14460084ae804a3f server_cookie=3c42c2f3150fb600 in_seq=0 out_seq=0 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7baf7fe640 1 -- 172.21.15.16:0/2061795873 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563afff79860 con 0x563b00020e90 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7baf7fe640 10 client.17025 handle_command_reply: tid=0 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 client.17025 shutdown 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 msgr2=0x563b0000df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800f790 tx=0x7f7bb8004730 comp rx=0 tx=0).stop 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 shutdown_connections 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b0000e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00007b40 0x563b0000df40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b00020e90 0x563b00023280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/2061795873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b000065d0 0x563b00010c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7baf7fe640 0 client.17025 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 >> 172.21.15.16:0/2061795873 conn(0x563afffcf310 msgr2=0x563b00008960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 shutdown_connections 2024-09-12T01:01:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 1 -- 172.21.15.16:0/2061795873 wait complete. 2024-09-12T01:01:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.480+0000 7f7bd8dac740 20 client.17025 trim_cache size 0 max 16384 2024-09-12T01:01:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.493+0000 7f7bd8dac740 1 Processor -- start 2024-09-12T01:01:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.493+0000 7f7bd8dac740 1 -- start start 2024-09-12T01:01:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b000065d0 0x563b0001d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00012760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01afa40 con 0x563b00011370 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01b1580 con 0x563b00007b40 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01b0110 con 0x563b000065d0 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57078/0 (socket says 172.21.15.16:57078) 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 -- 172.21.15.16:0/3498932804 learned_addr learned my addr 172.21.15.16:0/3498932804 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bcffff640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b000065d0 0x563b0001d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00012760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 -- 172.21.15.16:0/3498932804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b000065d0 msgr2=0x563b0001d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b000065d0 0x563b0001d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 -- 172.21.15.16:0/3498932804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 msgr2=0x563b00012760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00012760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 -- 172.21.15.16:0/3498932804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b00003df0 con 0x563b00011370 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00012760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005f30 tx=0x7f7bc800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df479213aa73eb4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bc5ffb640 1 -- 172.21.15.16:0/3498932804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8024070 con 0x563b00011370 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bc5ffb640 1 -- 172.21.15.16:0/3498932804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc800eb50 con 0x563b00011370 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.494+0000 7f7bc5ffb640 1 -- 172.21.15.16:0/3498932804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8029bb0 con 0x563b00011370 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3498932804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 msgr2=0x563b00012ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8005f30 tx=0x7f7bc800b540 comp rx=0 tx=0).stop 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3498932804 shutdown_connections 2024-09-12T01:01:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b00011370 0x563b00012ca0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00012760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3498932804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b000065d0 0x563b0001d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3498932804 >> 172.21.15.16:0/3498932804 conn(0x563afffcf310 msgr2=0x563b0000df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3498932804 shutdown_connections 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3498932804 wait complete. 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.495+0000 7f7bd8dac740 1 -- start start 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 Processor -- start 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00028080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b00018480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01b0eb0 con 0x563b000065d0 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01b07e0 con 0x563b00007b40 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd01ada20 con 0x563b00011370 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd8dac740 10 client.0 _ll_register_callbacks cb 0x7f7bc00b7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57088/0 (socket says 172.21.15.16:57088) 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 -- 172.21.15.16:0/3892014496 learned_addr learned my addr 172.21.15.16:0/3892014496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bd4f70640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b00018480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00028080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 -- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 msgr2=0x563b00018480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b00018480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 -- 172.21.15.16:0/3892014496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 msgr2=0x563b00028080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00028080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.496+0000 7f7bcffff640 1 -- 172.21.15.16:0/3892014496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b000189c0 con 0x563b000065d0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bcffff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc00f22f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e213a75768539e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0049040 con 0x563b000065d0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f7bc00b89a0 con 0x563b000065d0 2024-09-12T01:01:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc800eb50 con 0x563b000065d0 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bc00bca30 con 0x563b000065d0 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 10 client.17031 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b00019c90 con 0x563b000065d0 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc800eb50 con 0x563b000065d0 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bc003b460 con 0x563b000065d0 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 10 client.17031 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 20 client.17031 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b0008b940 0x563b0008dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 4 client.17031 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.497+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563afff73d70 con 0x563b0008b940 2024-09-12T01:01:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b0008b940 0x563b0008dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7baf7fe640 10 client.17031 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bc7fff640 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b0008b940 0x563b0008dd30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b8d3bf13c804639 server_cookie=a9e392458fa4940b in_seq=0 out_seq=0 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7baf7fe640 1 -- 172.21.15.16:0/3892014496 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563afff73d70 con 0x563b0008b940 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7baf7fe640 10 client.17031 handle_command_reply: tid=0 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 client.17031 shutdown 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 msgr2=0x563b00027b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc00f22f0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 shutdown_connections 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b00011370 0x563b00018480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.498+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b0008b940 0x563b0008dd30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b00007b40 0x563b00028080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 1 --2- 172.21.15.16:0/3892014496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b000065d0 0x563b00027b40 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 >> 172.21.15.16:0/3892014496 conn(0x563afffcf310 msgr2=0x563b0000f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7baf7fe640 0 client.17031 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:01:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 shutdown_connections 2024-09-12T01:01:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 1 -- 172.21.15.16:0/3892014496 wait complete. 2024-09-12T01:01:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd8dac740 20 client.17031 trim_cache size 0 max 16384 2024-09-12T01:01:37.510 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 msgr2=0x7f7bbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 0x7f7bbc0a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800bfd0 tx=0x7f7bc8005c20 comp rx=0 tx=0).stop 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 msgr2=0x7f7bd01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005ab0 tx=0x7f7bc00042f0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 shutdown_connections 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7bbc0a2cf0 0x7f7bbc0a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bd010c170 0x7f7bd0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd0103660 0x7f7bd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 --2- 172.21.15.16:0/2019140519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd0102d40 0x7f7bd01009a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:37.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 >> 172.21.15.16:0/2019140519 conn(0x7f7bd00fc7a0 msgr2=0x7f7bd00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:37.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 shutdown_connections 2024-09-12T01:01:37.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:37.499+0000 7f7bd69d3640 1 -- 172.21.15.16:0/2019140519 wait complete. 2024-09-12T01:01:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:37 smithi099 ceph-mon[35654]: pgmap v169: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:38.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:01:38.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 -- 172.21.15.16:0/1577402499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 msgr2=0x7fd1801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 --2- 172.21.15.16:0/1577402499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1801021c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd170009900 tx=0x7fd17002a410 comp rx=0 tx=0).stop 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 -- 172.21.15.16:0/1577402499 shutdown_connections 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 --2- 172.21.15.16:0/1577402499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 0x7fd18010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 --2- 172.21.15.16:0/1577402499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 0x7fd1801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 --2- 172.21.15.16:0/1577402499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1801021c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 -- 172.21.15.16:0/1577402499 >> 172.21.15.16:0/1577402499 conn(0x7fd1800fd8c0 msgr2=0x7fd1800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 -- 172.21.15.16:0/1577402499 shutdown_connections 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 -- 172.21.15.16:0/1577402499 wait complete. 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.090+0000 7fd1854c9640 1 Processor -- start 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 -- start start 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 0x7fd180074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 0x7fd180074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd180111fc0 con 0x7fd180101de0 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd180111e40 con 0x7fd180102790 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd1854c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd180112140 con 0x7fd180106f10 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 0x7fd180074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56090/0 (socket says 172.21.15.16:56090) 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 -- 172.21.15.16:0/2097951809 learned_addr learned my addr 172.21.15.16:0/2097951809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 0x7fd180074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 -- 172.21.15.16:0/2097951809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 msgr2=0x7fd180074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 0x7fd180074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 -- 172.21.15.16:0/2097951809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 msgr2=0x7fd180074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 0x7fd180074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.091+0000 7fd17effd640 1 -- 172.21.15.16:0/2097951809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1801a68e0 con 0x7fd180101de0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd17effd640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd170004770 tx=0x7fd170003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd177fff640 1 -- 172.21.15.16:0/2097951809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17003f070 con 0x7fd180101de0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd177fff640 1 -- 172.21.15.16:0/2097951809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd170031ca0 con 0x7fd180101de0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801a6b70 con 0x7fd180101de0 2024-09-12T01:01:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd177fff640 1 -- 172.21.15.16:0/2097951809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170035030 con 0x7fd180101de0 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.092+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801a7050 con 0x7fd180101de0 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.094+0000 7fd177fff640 1 -- 172.21.15.16:0/2097951809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd170042070 con 0x7fd180101de0 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.094+0000 7fd1878a2740 1 Processor -- start 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.094+0000 7fd1878a2740 1 -- start start 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd776b40 0x5557fd775090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd775a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5557fd6db2f0 con 0x5557fd780370 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557fd682cf0 con 0x5557fd776b40 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd1878a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5557fd6d72b0 con 0x5557fd7755d0 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd177fff640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 0x7fd16c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56100/0 (socket says 172.21.15.16:56100) 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 -- 172.21.15.16:0/1448512010 learned_addr learned my addr 172.21.15.16:0/1448512010 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 0x7fd16c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd177fff640 1 -- 172.21.15.16:0/2097951809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd1700efb40 con 0x7fd180101de0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17effd640 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd775a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd776b40 0x5557fd775090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 -- 172.21.15.16:0/1448512010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 msgr2=0x5557fd775a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd775a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 -- 172.21.15.16:0/1448512010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd776b40 msgr2=0x5557fd775090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd776b40 0x5557fd775090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.095+0000 7fd17e7fc640 1 -- 172.21.15.16:0/1448512010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fd6673a0 con 0x5557fd780370 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.096+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bcb0 tx=0x7fd1680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f6e2bf1b659e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.096+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 0x7fd16c0a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd178005cc0 tx=0x7fd17800ad50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.096+0000 7fd1757fa640 1 -- 172.21.15.16:0/1448512010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168004030 con 0x5557fd780370 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.096+0000 7fd1757fa640 1 -- 172.21.15.16:0/1448512010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1680041d0 con 0x5557fd780370 2024-09-12T01:01:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.096+0000 7fd1757fa640 1 -- 172.21.15.16:0/1448512010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd168011660 con 0x5557fd780370 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- 172.21.15.16:0/1448512010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 msgr2=0x5557fd782760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bcb0 tx=0x7fd1680107a0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- 172.21.15.16:0/1448512010 shutdown_connections 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd782760 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd775a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1448512010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd776b40 0x5557fd775090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- 172.21.15.16:0/1448512010 >> 172.21.15.16:0/1448512010 conn(0x5557fd73e310 msgr2=0x5557fd771c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- 172.21.15.16:0/1448512010 shutdown_connections 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- 172.21.15.16:0/1448512010 wait complete. 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- start start 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 Processor -- start 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 0x5557fd77fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5557fd783070 con 0x5557fd780370 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557fd696060 con 0x5557fd7755d0 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.097+0000 7fd1878a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5557fd7831f0 con 0x5557fd776b40 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd1878a2740 10 client.0 _ll_register_callbacks cb 0x5557fd3e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56104/0 (socket says 172.21.15.16:56104) 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 -- 172.21.15.16:0/1616216006 learned_addr learned my addr 172.21.15.16:0/1616216006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17effd640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 0x5557fd77fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 -- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 msgr2=0x5557fd77d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 -- 172.21.15.16:0/1616216006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 msgr2=0x5557fd77fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 0x5557fd77fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 -- 172.21.15.16:0/1616216006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fd44c3c0 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17effd640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 0x5557fd77fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd178012000 tx=0x7fd17800a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b33210a26b16118 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd1667fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178026070 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd178020860 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.098+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1680041d0 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd178032050 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 10 client.17043 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557fd78eb10 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1680041d0 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd17800f940 con 0x5557fd780370 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 10 client.17043 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 20 client.17043 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd78ff20 0x5557fd7923c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 4 client.17043 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557fd6e8860 con 0x5557fd78ff20 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.099+0000 7fd17effd640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd78ff20 0x5557fd7923c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd1667fc640 10 client.17043 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd17effd640 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd78ff20 0x5557fd7923c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47e3fad6717490c server_cookie=b4ead346f6012e4a in_seq=0 out_seq=0 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd1667fc640 1 -- 172.21.15.16:0/1616216006 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557fd6e8860 con 0x5557fd78ff20 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd1667fc640 10 client.17043 handle_command_reply: tid=0 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd1878a2740 1 client.17043 shutdown 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.100+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 msgr2=0x5557fd77d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd178012000 tx=0x7fd17800a4d0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 shutdown_connections 2024-09-12T01:01:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd78ff20 0x5557fd7923c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd780370 0x5557fd77d600 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 --2- 172.21.15.16:0/1616216006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd7755d0 0x5557fd77fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 >> 172.21.15.16:0/1616216006 conn(0x5557fd73e310 msgr2=0x5557fd777960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1667fc640 0 client.17043 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 shutdown_connections 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 1 -- 172.21.15.16:0/1616216006 wait complete. 2024-09-12T01:01:39.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.101+0000 7fd1878a2740 20 client.17043 trim_cache size 0 max 16384 2024-09-12T01:01:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 Processor -- start 2024-09-12T01:01:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 -- start start 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd77be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801af9c0 con 0x5557fd7755d0 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b1500 con 0x5557fd780370 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd1878a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b0090 con 0x5557fd776b40 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.114+0000 7fd17effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56114/0 (socket says 172.21.15.16:56114) 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 -- 172.21.15.16:0/943151449 learned_addr learned my addr 172.21.15.16:0/943151449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 -- 172.21.15.16:0/943151449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 msgr2=0x5557fd77b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd77be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 -- 172.21.15.16:0/943151449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 msgr2=0x5557fd77be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd77be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 -- 172.21.15.16:0/943151449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557fd77d6e0 con 0x5557fd7755d0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd17effd640 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd1700b8b10 tx=0x7fd1700dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6117abcf2dfefc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd174ff9640 1 -- 172.21.15.16:0/943151449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170049040 con 0x5557fd7755d0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd174ff9640 1 -- 172.21.15.16:0/943151449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1700ba440 con 0x5557fd7755d0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.115+0000 7fd174ff9640 1 -- 172.21.15.16:0/943151449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17003b5f0 con 0x5557fd7755d0 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- 172.21.15.16:0/943151449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 msgr2=0x5557fd78c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd1700b8b10 tx=0x7fd1700dc040 comp rx=0 tx=0).stop 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- 172.21.15.16:0/943151449 shutdown_connections 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd77be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd776b40 0x5557fd77b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 --2- 172.21.15.16:0/943151449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd7755d0 0x5557fd78c310 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- 172.21.15.16:0/943151449 >> 172.21.15.16:0/943151449 conn(0x5557fd73e310 msgr2=0x5557fd77cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- 172.21.15.16:0/943151449 shutdown_connections 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- 172.21.15.16:0/943151449 wait complete. 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.116+0000 7fd1878a2740 1 -- start start 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 Processor -- start 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd788a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd797910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b0e30 con 0x5557fd776b40 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b0760 con 0x5557fd780370 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ad9a0 con 0x5557fd7755d0 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1878a2740 10 client.0 _ll_register_callbacks cb 0x7fd17800af60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56130/0 (socket says 172.21.15.16:56130) 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 -- 172.21.15.16:0/592376383 learned_addr learned my addr 172.21.15.16:0/592376383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17effd640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd788a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17f7fe640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd797910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 -- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 msgr2=0x5557fd788a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd788a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 -- 172.21.15.16:0/592376383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 msgr2=0x5557fd797910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd797910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 -- 172.21.15.16:0/592376383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557fd797f00 con 0x5557fd776b40 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd17e7fc640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd168011010 tx=0x7fd16800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641d2da75ecd5635 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.117+0000 7fd1667fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16802c060 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fd168005450 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1700ba440 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd168005650 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 10 client.17049 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557fd7872e0 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1700ba440 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1680118a0 con 0x5557fd776b40 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 10 client.17049 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 20 client.17049 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd7fafe0 0x5557fd7fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 4 client.17049 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:01:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.118+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557fd782af0 con 0x5557fd7fafe0 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd17effd640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd7fafe0 0x5557fd7fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd17effd640 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd7fafe0 0x5557fd7fd400 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d48f2f34b82148 server_cookie=d454390df98e1255 in_seq=0 out_seq=0 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1667fc640 10 client.17049 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1667fc640 1 -- 172.21.15.16:0/592376383 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557fd782af0 con 0x5557fd7fafe0 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1667fc640 10 client.17049 handle_command_reply: tid=0 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 client.17049 shutdown 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 msgr2=0x5557fd7973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd168011010 tx=0x7fd16800bfd0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 shutdown_connections 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5557fd7fafe0 0x5557fd7fd400 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557fd780370 0x5557fd797910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557fd776b40 0x5557fd7973d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 --2- 172.21.15.16:0/592376383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5557fd7755d0 0x5557fd788a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 >> 172.21.15.16:0/592376383 conn(0x5557fd73e310 msgr2=0x5557fd77b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.119+0000 7fd1667fc640 0 client.17049 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 shutdown_connections 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1878a2740 1 -- 172.21.15.16:0/592376383 wait complete. 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1878a2740 20 client.17049 trim_cache size 0 max 16384 2024-09-12T01:01:39.130 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 msgr2=0x7fd16c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 0x7fd16c0a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd178005cc0 tx=0x7fd17800ad50 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 msgr2=0x7fd1800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd170004770 tx=0x7fd170003c10 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 shutdown_connections 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd16c0a2cf0 0x7fd16c0a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd180106f10 0x7fd180074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd180102790 0x7fd180074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 --2- 172.21.15.16:0/2097951809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd180101de0 0x7fd1800744f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:39.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 >> 172.21.15.16:0/2097951809 conn(0x7fd1800fd8c0 msgr2=0x7fd1800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 shutdown_connections 2024-09-12T01:01:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:39.120+0000 7fd1854c9640 1 -- 172.21.15.16:0/2097951809 wait complete. 2024-09-12T01:01:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:39 smithi159 ceph-mon[35566]: pgmap v170: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:39 smithi016 ceph-mon[30972]: pgmap v170: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:39 smithi099 ceph-mon[35654]: pgmap v170: 97 pgs: 97 active+clean; 5.3 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:40.930 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:01:40.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:01:41.203 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:41.325 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:41 smithi016 ceph-mon[30972]: pgmap v171: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:41 smithi159 ceph-mon[35566]: pgmap v171: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:41 smithi099 ceph-mon[35654]: pgmap v171: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 -- 172.21.15.16:0/76958735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101420 msgr2=0x7f84c410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 --2- 172.21.15.16:0/76958735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101420 0x7f84c410d950 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009cd0 tx=0x7f84bc02a8e0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 -- 172.21.15.16:0/76958735 shutdown_connections 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 --2- 172.21.15.16:0/76958735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4101420 0x7f84c410d950 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 --2- 172.21.15.16:0/76958735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4100a80 0x7f84c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 --2- 172.21.15.16:0/76958735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 -- 172.21.15.16:0/76958735 >> 172.21.15.16:0/76958735 conn(0x7f84c40fc7a0 msgr2=0x7f84c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 -- 172.21.15.16:0/76958735 shutdown_connections 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.689+0000 7f84ca076640 1 -- 172.21.15.16:0/76958735 wait complete. 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 Processor -- start 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 -- start start 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4101420 0x7f84c40fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c40fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4110e80 con 0x7f84c4100a80 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4110d00 con 0x7f84c4106a80 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84ca076640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c4111000 con 0x7f84c4101420 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.690+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56140/0 (socket says 172.21.15.16:56140) 2024-09-12T01:01:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 -- 172.21.15.16:0/2147036669 learned_addr learned my addr 172.21.15.16:0/2147036669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4101420 0x7f84c40fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c3fff640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c40fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 -- 172.21.15.16:0/2147036669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4101420 msgr2=0x7f84c40fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4101420 0x7f84c40fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 -- 172.21.15.16:0/2147036669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 msgr2=0x7f84c40fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c40fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 -- 172.21.15.16:0/2147036669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c41aada0 con 0x7f84c4100a80 2024-09-12T01:01:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c3fff640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c40fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c37fe640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d950 tx=0x7f84b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c0ff9640 1 -- 172.21.15.16:0/2147036669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4017070 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c0ff9640 1 -- 172.21.15.16:0/2147036669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4009c30 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c41aafd0 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84c0ff9640 1 -- 172.21.15.16:0/2147036669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400f6b0 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.691+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c41ab430 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c0ff9640 1 -- 172.21.15.16:0/2147036669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84b40109d0 con 0x7f84c4100a80 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 Processor -- start 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 -- start start 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f416f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f421d40 0x56378f424130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56378f37c280 con 0x56378f417480 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56378f323b10 con 0x56378f416b60 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84cc44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56378f378240 con 0x56378f421d40 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56144/0 (socket says 172.21.15.16:56144) 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c37fe640 1 -- 172.21.15.16:0/1869075111 learned_addr learned my addr 172.21.15.16:0/1869075111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c3fff640 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f416f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.694+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f421d40 0x56378f424130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c0ff9640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 -- 172.21.15.16:0/1869075111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f421d40 msgr2=0x56378f424130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f421d40 0x56378f424130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 -- 172.21.15.16:0/1869075111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 msgr2=0x56378f416f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f416f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 -- 172.21.15.16:0/1869075111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378f3301f0 con 0x56378f417480 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c3fff640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c0ff9640 1 -- 172.21.15.16:0/2147036669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84b40cefe0 con 0x7f84c4100a80 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc660 tx=0x7f84b4066120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7003c9205105800a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84a9ffb640 1 -- 172.21.15.16:0/1869075111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b409b030 con 0x56378f417480 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84a9ffb640 1 -- 172.21.15.16:0/1869075111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b4084a60 con 0x56378f417480 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.695+0000 7f84a9ffb640 1 -- 172.21.15.16:0/1869075111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40646e0 con 0x56378f417480 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- 172.21.15.16:0/1869075111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 msgr2=0x56378f421800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84c3fff640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004af0 tx=0x7f84bc0095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cc660 tx=0x7f84b4066120 comp rx=0 tx=0).stop 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- 172.21.15.16:0/1869075111 shutdown_connections 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f421d40 0x56378f424130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f417480 0x56378f421800 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1869075111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f416f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- 172.21.15.16:0/1869075111 >> 172.21.15.16:0/1869075111 conn(0x56378f4130f0 msgr2=0x56378f4134d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- 172.21.15.16:0/1869075111 shutdown_connections 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- 172.21.15.16:0/1869075111 wait complete. 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.696+0000 7f84cc44f740 1 -- start start 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 Processor -- start 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f41ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f41f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56378f424910 con 0x56378f421d40 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56378f336f10 con 0x56378f416b60 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56378f424a90 con 0x56378f417480 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84cc44f740 10 client.0 _ll_register_callbacks cb 0x56378f2ffa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56158/0 (socket says 172.21.15.16:56158) 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 -- 172.21.15.16:0/4027386514 learned_addr learned my addr 172.21.15.16:0/4027386514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c37fe640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f41ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f41f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 -- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 msgr2=0x56378f41f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f41f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 -- 172.21.15.16:0/4027386514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 msgr2=0x56378f41ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f41ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c3fff640 1 -- 172.21.15.16:0/4027386514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378f0ed6a0 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f41f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.697+0000 7f84c37fe640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f41ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84c3fff640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f84bc031ea0 tx=0x7f84bc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241e8ae611e9043 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0045d0 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84bc004770 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56378f3897f0 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84bc0305a0 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84cc44f740 10 client.17061 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56378f430510 con 0x56378f421d40 2024-09-12T01:01:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56378f3897f0 con 0x56378f421d40 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84bc030070 con 0x56378f421d40 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84cc44f740 10 client.17061 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.698+0000 7f84cc44f740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84cc44f740 20 client.17061 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f431920 0x56378f433d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84cc44f740 4 client.17061 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378f383d00 con 0x56378f431920 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84c37fe640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f431920 0x56378f433d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84c37fe640 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f431920 0x56378f433d70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f95c4f5c5174059 server_cookie=7153c79ba15e01f0 in_seq=0 out_seq=0 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.699+0000 7f84a2ffd640 10 client.17061 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84a2ffd640 1 -- 172.21.15.16:0/4027386514 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56378f383d00 con 0x56378f431920 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84a2ffd640 10 client.17061 handle_command_reply: tid=0 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 client.17061 shutdown 2024-09-12T01:01:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 msgr2=0x56378f41f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f84bc031ea0 tx=0x7f84bc02a8e0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 shutdown_connections 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f431920 0x56378f433d70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f41f690 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f41f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 --2- 172.21.15.16:0/4027386514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f41ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 >> 172.21.15.16:0/4027386514 conn(0x56378f4130f0 msgr2=0x56378f418e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84a2ffd640 0 client.17061 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 shutdown_connections 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 1 -- 172.21.15.16:0/4027386514 wait complete. 2024-09-12T01:01:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.700+0000 7f84cc44f740 20 client.17061 trim_cache size 0 max 16384 2024-09-12T01:01:41.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 Processor -- start 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 -- start start 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f420a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f439640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b3e70 con 0x56378f421d40 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b59b0 con 0x56378f416b60 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.718+0000 7f84cc44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b4540 con 0x56378f417480 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56168/0 (socket says 172.21.15.16:56168) 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 -- 172.21.15.16:0/913145576 learned_addr learned my addr 172.21.15.16:0/913145576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:41.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f439640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c37fe640 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f420a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 -- 172.21.15.16:0/913145576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 msgr2=0x56378f439640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f439640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 -- 172.21.15.16:0/913145576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 msgr2=0x56378f420a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f420a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 -- 172.21.15.16:0/913145576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378f4172b0 con 0x56378f421d40 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84c3fff640 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f84bc031df0 tx=0x7f84bc02af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333d20d1178bab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84a97fa640 1 -- 172.21.15.16:0/913145576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc05f030 con 0x56378f421d40 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84a97fa640 1 -- 172.21.15.16:0/913145576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84bc04c070 con 0x56378f421d40 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.719+0000 7f84a97fa640 1 -- 172.21.15.16:0/913145576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0309a0 con 0x56378f421d40 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 -- 172.21.15.16:0/913145576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 msgr2=0x56378f439b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f84bc031df0 tx=0x7f84bc02af20 comp rx=0 tx=0).stop 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 -- 172.21.15.16:0/913145576 shutdown_connections 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f421d40 0x56378f439b80 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f439640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 --2- 172.21.15.16:0/913145576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f416b60 0x56378f420a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 -- 172.21.15.16:0/913145576 >> 172.21.15.16:0/913145576 conn(0x56378f4130f0 msgr2=0x56378f420360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 -- 172.21.15.16:0/913145576 shutdown_connections 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.720+0000 7f84cc44f740 1 -- 172.21.15.16:0/913145576 wait complete. 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 -- start start 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 Processor -- start 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f4261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f421d40 0x56378f4266e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b52e0 con 0x56378f416b60 2024-09-12T01:01:41.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b4c10 con 0x56378f421d40 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84c41b1e50 con 0x56378f417480 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84cc44f740 10 client.0 _ll_register_callbacks cb 0x7f84bc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56176/0 (socket says 172.21.15.16:56176) 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84c37fe640 1 -- 172.21.15.16:0/1354669771 learned_addr learned my addr 172.21.15.16:0/1354669771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f4261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.721+0000 7f84c3fff640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f421d40 0x56378f4266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 -- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 msgr2=0x56378f4261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f4261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 -- 172.21.15.16:0/1354669771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f421d40 msgr2=0x56378f4266e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f421d40 0x56378f4266e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 -- 172.21.15.16:0/1354669771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378f49b860 con 0x56378f416b60 2024-09-12T01:01:41.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84c37fe640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f84b409b480 tx=0x7f84b4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c188a8c65fe1723d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4020040 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f84b40997f0 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84bc04c070 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84b409ae50 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84cc44f740 10 client.17067 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56378f49cbc0 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84bc04c070 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84b4084020 con 0x56378f416b60 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84cc44f740 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.722+0000 7f84cc44f740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:41.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84cc44f740 20 client.17067 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f49cf70 0x56378f49f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84cc44f740 4 client.17067 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56378f300110 con 0x56378f49cf70 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f49cf70 0x56378f49f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84c2ffd640 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f49cf70 0x56378f49f360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cea21f0ab0d8892 server_cookie=d9dbd6ccf87a6e37 in_seq=0 out_seq=0 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.723+0000 7f84a2ffd640 10 client.17067 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.724+0000 7f84a2ffd640 1 -- 172.21.15.16:0/1354669771 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56378f300110 con 0x56378f49cf70 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.724+0000 7f84a2ffd640 10 client.17067 handle_command_reply: tid=0 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.724+0000 7f84cc44f740 1 client.17067 shutdown 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.724+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 msgr2=0x56378f425c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.724+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f84b409b480 tx=0x7f84b4010d70 comp rx=0 tx=0).stop 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 shutdown_connections 2024-09-12T01:01:41.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56378f49cf70 0x56378f49f360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56378f421d40 0x56378f4266e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56378f417480 0x56378f4261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 --2- 172.21.15.16:0/1354669771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56378f416b60 0x56378f425c60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 >> 172.21.15.16:0/1354669771 conn(0x56378f4130f0 msgr2=0x56378f41f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84a2ffd640 0 client.17067 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 shutdown_connections 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 1 -- 172.21.15.16:0/1354669771 wait complete. 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84cc44f740 20 client.17067 trim_cache size 0 max 16384 2024-09-12T01:01:41.734 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "37fba28c-0a79-4de9-9583-9e2a80a9c399", 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 msgr2=0x7f84ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84bc004af0 tx=0x7f84bc0095c0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 msgr2=0x7f84c40fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d950 tx=0x7f84b400de10 comp rx=0 tx=0).stop 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 shutdown_connections 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84ac0a2cf0 0x7f84ac0a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84c4106a80 0x7f84c40fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84c4101420 0x7f84c40fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 --2- 172.21.15.16:0/2147036669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c4100a80 0x7f84c40fcd90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.725+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 >> 172.21.15.16:0/2147036669 conn(0x7f84c40fc7a0 msgr2=0x7f84c406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.726+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 shutdown_connections 2024-09-12T01:01:41.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:41.726+0000 7f84ca076640 1 -- 172.21.15.16:0/2147036669 wait complete. 2024-09-12T01:01:42.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:01:42.755 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:01:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:43 smithi016 ceph-mon[30972]: pgmap v172: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:43 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:01:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:43 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:01:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 -- 172.21.15.16:0/3275918303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d410c170 msgr2=0x7f63d410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 --2- 172.21.15.16:0/3275918303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d410c170 0x7f63d410e610 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009ca0 tx=0x7f63cc02a950 comp rx=0 tx=0).stop 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 -- 172.21.15.16:0/3275918303 shutdown_connections 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 --2- 172.21.15.16:0/3275918303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d410c170 0x7f63d410e610 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 --2- 172.21.15.16:0/3275918303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 --2- 172.21.15.16:0/3275918303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d4102d40 0x7f63d4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 -- 172.21.15.16:0/3275918303 >> 172.21.15.16:0/3275918303 conn(0x7f63d40fc7a0 msgr2=0x7f63d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 -- 172.21.15.16:0/3275918303 shutdown_connections 2024-09-12T01:01:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.256+0000 7f63dc571640 1 -- 172.21.15.16:0/3275918303 wait complete. 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 Processor -- start 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 -- start start 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d410c170 0x7f63d419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4111f70 con 0x7f63d4102d40 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4111df0 con 0x7f63d4103660 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dc571640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41120f0 con 0x7f63d410c170 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56188/0 (socket says 172.21.15.16:56188) 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 -- 172.21.15.16:0/1719061095 learned_addr learned my addr 172.21.15.16:0/1719061095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63dab0e640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d410c170 0x7f63d419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 -- 172.21.15.16:0/1719061095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d410c170 msgr2=0x7f63d419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d410c170 0x7f63d419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 -- 172.21.15.16:0/1719061095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 msgr2=0x7f63d419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63da30d640 1 -- 172.21.15.16:0/1719061095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d419f620 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.257+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63da30d640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f63d000d770 tx=0x7f63d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1719061095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0017070 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d419f910 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1719061095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d0005500 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1719061095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004e00 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.258+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d41a71f0 con 0x7f63d4102d40 2024-09-12T01:01:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.260+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1719061095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63d0004fa0 con 0x7f63d4102d40 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.260+0000 7f63de94a740 1 Processor -- start 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 -- start start 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af3015880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3022a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559af2f7b2e0 con 0x559af3015dc0 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559af2f22ce0 con 0x559af3020680 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63de94a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559af2f772a0 con 0x559af30154a0 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56200/0 (socket says 172.21.15.16:56200) 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63da30d640 1 -- 172.21.15.16:0/1168780961 learned_addr learned my addr 172.21.15.16:0/1168780961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63cb7fe640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 0x7f63c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63dab0e640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af3015880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3022a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.261+0000 7f63dab0e640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 0x7f63c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1719061095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f63d00ceea0 con 0x7f63d4102d40 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 -- 172.21.15.16:0/1168780961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 msgr2=0x559af3015880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af3015880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 -- 172.21.15.16:0/1168780961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 msgr2=0x559af3022a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3022a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 -- 172.21.15.16:0/1168780961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af2f07440 con 0x559af3015dc0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3022a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63dab0e640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af3015880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63da30d640 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f63d009b0f0 tx=0x7f63d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cbe2f28c125cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63c8ff9640 1 -- 172.21.15.16:0/1168780961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0020030 con 0x559af3015dc0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63c8ff9640 1 -- 172.21.15.16:0/1168780961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d009baf0 con 0x559af3015dc0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63c8ff9640 1 -- 172.21.15.16:0/1168780961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0009cc0 con 0x559af3015dc0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.262+0000 7f63dab0e640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 0x7f63c00a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009430 tx=0x7f63cc005c50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- 172.21.15.16:0/1168780961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 msgr2=0x559af3020140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f63d009b0f0 tx=0x7f63d0009c30 comp rx=0 tx=0).stop 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- 172.21.15.16:0/1168780961 shutdown_connections 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3022a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3020140 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 --2- 172.21.15.16:0/1168780961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af3015880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- 172.21.15.16:0/1168780961 >> 172.21.15.16:0/1168780961 conn(0x559af3011a30 msgr2=0x559af3011e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- 172.21.15.16:0/1168780961 shutdown_connections 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- 172.21.15.16:0/1168780961 wait complete. 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 -- start start 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.263+0000 7f63de94a740 1 Processor -- start 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af301d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af301d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559af3023250 con 0x559af30154a0 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559af2f36050 con 0x559af3015dc0 2024-09-12T01:01:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559af30233d0 con 0x559af3020680 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63de94a740 10 client.0 _ll_register_callbacks cb 0x559af2f76160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56206/0 (socket says 172.21.15.16:56206) 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 -- 172.21.15.16:0/3986202631 learned_addr learned my addr 172.21.15.16:0/3986202631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af301d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 -- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 msgr2=0x559af301d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63dab0e640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af301d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af301d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 -- 172.21.15.16:0/3986202631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 msgr2=0x559af301d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af301d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 -- 172.21.15.16:0/3986202631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af2cec3c0 con 0x559af30154a0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af301d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63da30d640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cc6d0 tx=0x7f63d0099900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac056bac7fd3dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.264+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0020030 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f63d0097a60 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d009baf0 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63d009b3f0 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 10 client.17079 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af302ebc0 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d009baf0 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f63d00605e0 con 0x559af30154a0 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 10 client.17079 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 20 client.17079 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af302ff10 0x559af3032300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 4 client.17079 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af2f88850 con 0x559af302ff10 2024-09-12T01:01:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.265+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af302ff10 0x559af3032300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.266+0000 7f63bdffb640 10 client.17079 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.266+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af302ff10 0x559af3032300 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1248b5767e988c server_cookie=e2769d3b28c25e7c in_seq=0 out_seq=0 2024-09-12T01:01:43.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63bdffb640 1 -- 172.21.15.16:0/3986202631 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559af2f88850 con 0x559af302ff10 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63bdffb640 10 client.17079 handle_command_reply: tid=0 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 client.17079 shutdown 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 msgr2=0x559af301ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f63d00cc6d0 tx=0x7f63d0099900 comp rx=0 tx=0).stop 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 shutdown_connections 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af302ff10 0x559af3032300 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af301d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af301d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 --2- 172.21.15.16:0/3986202631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af301ff70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 >> 172.21.15.16:0/3986202631 conn(0x559af3011a30 msgr2=0x559af3010b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63bdffb640 0 client.17079 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 shutdown_connections 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 1 -- 172.21.15.16:0/3986202631 wait complete. 2024-09-12T01:01:43.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.267+0000 7f63de94a740 20 client.17079 trim_cache size 0 max 16384 2024-09-12T01:01:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 Processor -- start 2024-09-12T01:01:43.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 -- start start 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af301d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3034f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41af980 con 0x559af3015dc0 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41b14c0 con 0x559af3020680 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63de94a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41b0050 con 0x559af30154a0 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63d9b0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63d9b0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56208/0 (socket says 172.21.15.16:56208) 2024-09-12T01:01:43.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.284+0000 7f63d9b0c640 1 -- 172.21.15.16:0/2141823558 learned_addr learned my addr 172.21.15.16:0/2141823558 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:43.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 -- 172.21.15.16:0/2141823558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 msgr2=0x559af301d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63da30d640 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af301d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af301d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 -- 172.21.15.16:0/2141823558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 msgr2=0x559af3034f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3034f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 -- 172.21.15.16:0/2141823558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af301b770 con 0x559af3015dc0 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63da30d640 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af301d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f63d41029a0 tx=0x7f63c400b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e792cdf711352f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63bffff640 1 -- 172.21.15.16:0/2141823558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c401f020 con 0x559af3015dc0 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63bffff640 1 -- 172.21.15.16:0/2141823558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c4004c90 con 0x559af3015dc0 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.285+0000 7f63bffff640 1 -- 172.21.15.16:0/2141823558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4012450 con 0x559af3015dc0 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- 172.21.15.16:0/2141823558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 msgr2=0x559af3010a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f63d41029a0 tx=0x7f63c400b7f0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- 172.21.15.16:0/2141823558 shutdown_connections 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3020680 0x559af3034f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af3015dc0 0x559af3010a60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- 172.21.15.16:0/2141823558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af30154a0 0x559af301d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- 172.21.15.16:0/2141823558 >> 172.21.15.16:0/2141823558 conn(0x559af3011a30 msgr2=0x559af3010350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- 172.21.15.16:0/2141823558 shutdown_connections 2024-09-12T01:01:43.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- 172.21.15.16:0/2141823558 wait complete. 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- start start 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 Processor -- start 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af3035000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af3091b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41b0df0 con 0x559af30154a0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41b0720 con 0x559af3015dc0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.286+0000 7f63de94a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41ad960 con 0x559af3020680 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63de94a740 10 client.0 _ll_register_callbacks cb 0x7f63d009b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56224/0 (socket says 172.21.15.16:56224) 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 -- 172.21.15.16:0/2873902598 learned_addr learned my addr 172.21.15.16:0/2873902598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af3035000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63dab0e640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af3091b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 -- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 msgr2=0x559af3091b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af3091b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 -- 172.21.15.16:0/2873902598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 msgr2=0x559af3035000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af3035000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 -- 172.21.15.16:0/2873902598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af30921b0 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63da30d640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f63d009b380 tx=0x7f63d009aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fc35ab165ebd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0020070 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f63d0097a60 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c4004c90 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63d0083050 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63de94a740 10 client.17085 fetch_fsmap learned FSMap version 28 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.287+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af309a6a0 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c4004c90 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f63d007d030 con 0x559af30154a0 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 10 client.17085 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:01:43.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:01:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 20 client.17085 populate_metadata read hostname 'smithi016' 2024-09-12T01:01:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af309abd0 0x559af309cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:01:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 4 client.17085 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:01:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559af301daf0 con 0x559af309abd0 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.288+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af309abd0 0x559af309cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63d9b0c640 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af309abd0 0x559af309cff0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf6eb24e5b7bb42 server_cookie=ec6c0efb22ea8a33 in_seq=0 out_seq=0 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63bdffb640 10 client.17085 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63bdffb640 1 -- 172.21.15.16:0/2873902598 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559af301daf0 con 0x559af309abd0 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63bdffb640 10 client.17085 handle_command_reply: tid=0 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 client.17085 shutdown 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 msgr2=0x559af3034ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f63d009b380 tx=0x7f63d009aa10 comp rx=0 tx=0).stop 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 shutdown_connections 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559af309abd0 0x559af309cff0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559af3020680 0x559af3091b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559af3015dc0 0x559af3035000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 --2- 172.21.15.16:0/2873902598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559af30154a0 0x559af3034ac0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 >> 172.21.15.16:0/2873902598 conn(0x559af3011a30 msgr2=0x559af3010b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63bdffb640 0 client.17085 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 shutdown_connections 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 1 -- 172.21.15.16:0/2873902598 wait complete. 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.289+0000 7f63de94a740 20 client.17085 trim_cache size 0 max 16384 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: "37fba28c-0a79-4de9-9583-9e2a80a9c399": { 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:01:43.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "37fba28c-0a79-4de9-9583-9e2a80a9c399", 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 msgr2=0x7f63c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 0x7f63c00a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009430 tx=0x7f63cc005c50 comp rx=0 tx=0).stop 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 msgr2=0x7f63d419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f63d000d770 tx=0x7f63d000dc30 comp rx=0 tx=0).stop 2024-09-12T01:01:43.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 shutdown_connections 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63c00a2cf0 0x7f63c00a51b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63d410c170 0x7f63d419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63d4103660 0x7f63d419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 --2- 172.21.15.16:0/1719061095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4102d40 0x7f63d419a670 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 >> 172.21.15.16:0/1719061095 conn(0x7f63d40fc7a0 msgr2=0x7f63d40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 shutdown_connections 2024-09-12T01:01:43.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:01:43.290+0000 7f63dc571640 1 -- 172.21.15.16:0/1719061095 wait complete. 2024-09-12T01:01:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:43 smithi159 ceph-mon[35566]: pgmap v172: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:43 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:01:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:43 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:01:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:43 smithi099 ceph-mon[35654]: pgmap v172: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:43 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:01:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:43 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:01:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:01:44.048 INFO:tasks.cephfs.filesystem:scrub status for tag:37fba28c-0a79-4de9-9583-9e2a80a9c399 - {'path': '/', 'tag': '37fba28c-0a79-4de9-9583-9e2a80a9c399', 'options': 'recursive,force'} 2024-09-12T01:01:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:45 smithi159 ceph-mon[35566]: pgmap v173: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:45 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:01:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:45 smithi016 ceph-mon[30972]: pgmap v173: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:45.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:45 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:01:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:45 smithi099 ceph-mon[35654]: pgmap v173: 97 pgs: 97 active+clean; 5.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-09-12T01:01:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:45 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:01:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:47 smithi159 ceph-mon[35566]: pgmap v174: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:47.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:47 smithi016 ceph-mon[30972]: pgmap v174: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:47 smithi099 ceph-mon[35654]: pgmap v174: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-12T01:01:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:49 smithi159 ceph-mon[35566]: pgmap v175: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:49.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:49 smithi016 ceph-mon[30972]: pgmap v175: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:49 smithi099 ceph-mon[35654]: pgmap v175: 97 pgs: 97 active+clean; 5.6 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:51 smithi159 ceph-mon[35566]: pgmap v176: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:51.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:51 smithi016 ceph-mon[30972]: pgmap v176: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:51 smithi099 ceph-mon[35654]: pgmap v176: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-09-12T01:01:53.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:53 smithi159 ceph-mon[35566]: pgmap v177: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:53.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:53 smithi016 ceph-mon[30972]: pgmap v177: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:53 smithi099 ceph-mon[35654]: pgmap v177: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:55 smithi159 ceph-mon[35566]: pgmap v178: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:55 smithi016 ceph-mon[30972]: pgmap v178: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:55 smithi099 ceph-mon[35654]: pgmap v178: 97 pgs: 97 active+clean; 5.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:56.064 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:127/600: open fd = 3 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:128/600: open fd = 3 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:129/600: open fd = 3 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.065 INFO:tasks.workunit.client.0.smithi016.stdout:130/600: open fd = 3 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:131/600: open fd = 3 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:132/600: open fd = 3 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:133/600: open fd = 3 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.066 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:134/600: open fd = 3 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:135/600: open fd = 3 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:136/600: open fd = 3 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.067 INFO:tasks.workunit.client.0.smithi016.stdout:137/600: open fd = 3 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:138/600: open fd = 3 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:139/600: open fd = 3 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:140/600: open fd = 3 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.068 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:141/600: open fd = 3 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:142/600: open fd = 3 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:143/600: open fd = 3 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.069 INFO:tasks.workunit.client.0.smithi016.stdout:144/600: open fd = 3 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:145/600: open fd = 3 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:146/600: open fd = 3 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:147/600: open fd = 3 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:148/600: open fd = 3 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:149/600: open fd = 3 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:150/600: open fd = 3 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:151/600: open fd = 3 2024-09-12T01:01:56.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:152/600: open fd = 3 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:153/600: open fd = 3 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:154/600: open fd = 3 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:155/600: open fd = 3 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:156/600: open fd = 3 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:157/600: open fd = 3 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:158/600: open fd = 3 2024-09-12T01:01:56.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:159/600: open fd = 3 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:160/600: open fd = 3 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:161/600: open fd = 3 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:162/600: open fd = 3 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:163/600: open fd = 3 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:164/600: open fd = 3 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:165/600: open fd = 3 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:166/600: open fd = 3 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:167/600: open fd = 3 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:168/600: open fd = 3 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.076 INFO:tasks.workunit.client.0.smithi016.stdout:169/600: open fd = 3 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:170/600: open fd = 3 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:171/600: open fd = 3 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:172/600: open fd = 3 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:173/600: open fd = 3 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:174/600: open fd = 3 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:175/600: open fd = 3 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:176/600: open fd = 3 2024-09-12T01:01:56.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:177/600: open fd = 3 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:178/600: open fd = 3 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:179/600: open fd = 3 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:180/600: open fd = 3 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:181/600: open fd = 3 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:182/600: open fd = 3 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:183/600: open fd = 3 2024-09-12T01:01:56.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:184/600: open fd = 3 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:185/600: open fd = 3 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:186/600: open fd = 3 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:187/600: open fd = 3 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:188/600: open fd = 3 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:56.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:01:57.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:57 smithi016 ceph-mon[30972]: pgmap v179: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-09-12T01:01:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:57 smithi099 ceph-mon[35654]: pgmap v179: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-09-12T01:01:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:57 smithi159 ceph-mon[35566]: pgmap v179: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-09-12T01:01:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:01:59 smithi016 ceph-mon[30972]: pgmap v180: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:01:59 smithi099 ceph-mon[35654]: pgmap v180: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:01:59.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:01:59 smithi159 ceph-mon[35566]: pgmap v180: 97 pgs: 97 active+clean; 6.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:01.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:01 smithi016 ceph-mon[30972]: pgmap v181: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-12T01:02:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:01 smithi099 ceph-mon[35654]: pgmap v181: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-12T01:02:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:01 smithi159 ceph-mon[35566]: pgmap v181: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-12T01:02:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:03 smithi016 ceph-mon[30972]: pgmap v182: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:03 smithi099 ceph-mon[35654]: pgmap v182: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:03 smithi159 ceph-mon[35566]: pgmap v182: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:05 smithi016 ceph-mon[30972]: pgmap v183: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:05.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:05 smithi099 ceph-mon[35654]: pgmap v183: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:05 smithi159 ceph-mon[35566]: pgmap v183: 97 pgs: 97 active+clean; 6.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-12T01:02:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:07 smithi016 ceph-mon[30972]: pgmap v184: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-09-12T01:02:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:07 smithi099 ceph-mon[35654]: pgmap v184: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-09-12T01:02:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:07 smithi159 ceph-mon[35566]: pgmap v184: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-09-12T01:02:08.323 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T01:02:08.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:02:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- 172.21.15.16:0/1101816415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 msgr2=0x7fcba4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 --2- 172.21.15.16:0/1101816415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba4074970 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009900 tx=0x7fcb9402a480 comp rx=0 tx=0).stop 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- 172.21.15.16:0/1101816415 shutdown_connections 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 --2- 172.21.15.16:0/1101816415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4112b90 0x7fcba4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 --2- 172.21.15.16:0/1101816415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4074eb0 0x7fcba4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 --2- 172.21.15.16:0/1101816415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba4074970 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- 172.21.15.16:0/1101816415 >> 172.21.15.16:0/1101816415 conn(0x7fcba40fd8c0 msgr2=0x7fcba40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- 172.21.15.16:0/1101816415 shutdown_connections 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- 172.21.15.16:0/1101816415 wait complete. 2024-09-12T01:02:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 Processor -- start 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.097+0000 7fcba955c640 1 -- start start 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4074eb0 0x7fcba41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4112b90 0x7fcba419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba41185c0 con 0x7fcba4074590 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4118440 con 0x7fcba4112b90 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba955c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4118740 con 0x7fcba4074eb0 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34750/0 (socket says 172.21.15.16:34750) 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 -- 172.21.15.16:0/1502805238 learned_addr learned my addr 172.21.15.16:0/1502805238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba37fe640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4112b90 0x7fcba419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcb9bfff640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4074eb0 0x7fcba41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 -- 172.21.15.16:0/1502805238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4074eb0 msgr2=0x7fcba41a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4074eb0 0x7fcba41a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 -- 172.21.15.16:0/1502805238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4112b90 msgr2=0x7fcba419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4112b90 0x7fcba419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 -- 172.21.15.16:0/1502805238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba419efe0 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba2ffd640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb94003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.098+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9403f070 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.099+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94031c90 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.099+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba419f270 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.099+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94035030 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.099+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41ab5c0 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.100+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba4074970 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.101+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb94042070 con 0x7fcba4074590 2024-09-12T01:02:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.102+0000 7fcba0ff9640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.102+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fcb940efcd0 con 0x7fcba4074590 2024-09-12T01:02:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.102+0000 7fcb9bfff640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.102+0000 7fcb9bfff640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 0x7fcb900a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcba40fff50 tx=0x7fcb8c00a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.106+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb940bc270 con 0x7fcba4074590 2024-09-12T01:02:09.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.318+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcba41ab790 con 0x7fcba4074590 2024-09-12T01:02:09.317 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:09 smithi016 ceph-mon[30972]: pgmap v185: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T01:02:09.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.318+0000 7fcba0ff9640 1 -- 172.21.15.16:0/1502805238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fcb940c0570 con 0x7fcba4074590 2024-09-12T01:02:09.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:02:09.318 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:02:09.319 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 msgr2=0x7fcb900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 0x7fcb900a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcba40fff50 tx=0x7fcb8c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 msgr2=0x7fcba41a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcb94006010 tx=0x7fcb94003ba0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 shutdown_connections 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba4112b90 0x7fcba419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcba4074eb0 0x7fcba41a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 --2- 172.21.15.16:0/1502805238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba4074590 0x7fcba41a46f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.320+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 >> 172.21.15.16:0/1502805238 conn(0x7fcba40fd8c0 msgr2=0x7fcba40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.321+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 shutdown_connections 2024-09-12T01:02:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:09.321+0000 7fcba955c640 1 -- 172.21.15.16:0/1502805238 wait complete. 2024-09-12T01:02:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:09 smithi099 ceph-mon[35654]: pgmap v185: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T01:02:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:09 smithi159 ceph-mon[35566]: pgmap v185: 97 pgs: 97 active+clean; 6.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-12T01:02:10.054 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T01:02:10.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T01:02:10.312 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:10.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:10 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1502805238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:10.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:10 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1502805238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:10.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:10 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1502805238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 -- 172.21.15.16:0/1095899475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c102790 msgr2=0x7f183c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 --2- 172.21.15.16:0/1095899475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c102790 0x7f183c1067e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1828009900 tx=0x7f182802a410 comp rx=0 tx=0).stop 2024-09-12T01:02:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 -- 172.21.15.16:0/1095899475 shutdown_connections 2024-09-12T01:02:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 --2- 172.21.15.16:0/1095899475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c106f10 0x7f183c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 --2- 172.21.15.16:0/1095899475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c102790 0x7f183c1067e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 --2- 172.21.15.16:0/1095899475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 0x7f183c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 -- 172.21.15.16:0/1095899475 >> 172.21.15.16:0/1095899475 conn(0x7f183c0fd8c0 msgr2=0x7f183c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 -- 172.21.15.16:0/1095899475 shutdown_connections 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.847+0000 7f1842ad6640 1 -- 172.21.15.16:0/1095899475 wait complete. 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 Processor -- start 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 -- start start 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 0x7f183c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c102790 0x7f183c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f183c111f30 con 0x7f183c106f10 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f183c111db0 con 0x7f183c102790 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1842ad6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1120b0 con 0x7f183c101de0 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34770/0 (socket says 172.21.15.16:34770) 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1841073640 1 -- 172.21.15.16:0/3460681729 learned_addr learned my addr 172.21.15.16:0/3460681729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.848+0000 7f1840872640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 0x7f183c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f183bfff640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c102790 0x7f183c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 -- 172.21.15.16:0/3460681729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 msgr2=0x7f183c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 0x7f183c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 -- 172.21.15.16:0/3460681729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c102790 msgr2=0x7f183c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c102790 0x7f183c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 -- 172.21.15.16:0/3460681729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c19f6a0 con 0x7f183c106f10 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1841073640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f183400cad0 tx=0x7f183400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1839ffb640 1 -- 172.21.15.16:0/3460681729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183400ae20 con 0x7f183c106f10 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1839ffb640 1 -- 172.21.15.16:0/3460681729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834005020 con 0x7f183c106f10 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c19f990 con 0x7f183c106f10 2024-09-12T01:02:10.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1839ffb640 1 -- 172.21.15.16:0/3460681729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183400f6f0 con 0x7f183c106f10 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.849+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1a7270 con 0x7f183c106f10 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1839ffb640 1 -- 172.21.15.16:0/3460681729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f18340051d0 con 0x7f183c106f10 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 Processor -- start 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 -- start start 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1839ffb640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 0x7f182c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575136b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575141400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55557509c2a0 con 0x555575141940 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555575043b10 con 0x555575137080 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f1844eaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555575098260 con 0x555575136760 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.852+0000 7f183bfff640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 0x7f182c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1839ffb640 1 -- 172.21.15.16:0/3460681729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f18340cf3a0 con 0x7f183c106f10 2024-09-12T01:02:10.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34774/0 (socket says 172.21.15.16:34774) 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 -- 172.21.15.16:0/938935692 learned_addr learned my addr 172.21.15.16:0/938935692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1840872640 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f183bfff640 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 -- 172.21.15.16:0/938935692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 msgr2=0x555575136b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575136b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 -- 172.21.15.16:0/938935692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 msgr2=0x555575141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575141400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 -- 172.21.15.16:0/938935692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555574e0db90 con 0x555575141940 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f183bfff640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 0x7f182c0a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1828009230 tx=0x7f1828031690 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1841073640 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f183409b930 tx=0x7f1834015e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9689ded2112011f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1826ffd640 1 -- 172.21.15.16:0/938935692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183401d040 con 0x555575141940 2024-09-12T01:02:10.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1826ffd640 1 -- 172.21.15.16:0/938935692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183401fce0 con 0x555575141940 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.853+0000 7f1826ffd640 1 -- 172.21.15.16:0/938935692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18340ccda0 con 0x555575141940 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- 172.21.15.16:0/938935692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 msgr2=0x555575143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f183409b930 tx=0x7f1834015e20 comp rx=0 tx=0).stop 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- 172.21.15.16:0/938935692 shutdown_connections 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575141940 0x555575143d30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 --2- 172.21.15.16:0/938935692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575136b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- 172.21.15.16:0/938935692 >> 172.21.15.16:0/938935692 conn(0x555575132cf0 msgr2=0x5555751330d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- 172.21.15.16:0/938935692 shutdown_connections 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- 172.21.15.16:0/938935692 wait complete. 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.854+0000 7f1844eaf740 1 -- start start 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 Processor -- start 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x55557513e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x55557513eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555575144510 con 0x555575137080 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5555750246e0 con 0x555575136760 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555575144690 con 0x555575141940 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1844eaf740 10 client.0 _ll_register_callbacks cb 0x555575048a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f183bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x55557513eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1841073640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1841073640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:50446/0 (socket says 172.21.15.16:50446) 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1841073640 1 -- 172.21.15.16:0/1589130710 learned_addr learned my addr 172.21.15.16:0/1589130710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.855+0000 7f1840872640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x55557513e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 -- 172.21.15.16:0/1589130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 msgr2=0x55557513e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x55557513e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 -- 172.21.15.16:0/1589130710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 msgr2=0x55557513eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x55557513eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 -- 172.21.15.16:0/1589130710 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557514da30 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1841073640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1834098370 tx=0x7f183409bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3909d4dc04c4d966 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f181bfff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183407f060 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183401fce0 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f18340ba210 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18340ba070 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1844eaf740 10 client.26422 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555575144510 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.856+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183401fce0 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f183409adf0 con 0x555575141940 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 10 client.26422 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:10.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 10 client.26422 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:02:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 20 client.26422 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575151540 0x555575153930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 4 client.26422 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555750a99d0 con 0x555575151540 2024-09-12T01:02:10.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1840872640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575151540 0x555575153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f1840872640 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575151540 0x555575153930 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858fc2dcac477dc8 server_cookie=44a0c42d1add0900 in_seq=0 out_seq=0 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.857+0000 7f181bfff640 10 client.26422 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f181bfff640 1 -- 172.21.15.16:0/1589130710 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555750a99d0 con 0x555575151540 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f181bfff640 10 client.26422 handle_command_reply: tid=0 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 client.26422 shutdown 2024-09-12T01:02:10.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 msgr2=0x55557513f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1834098370 tx=0x7f183409bae0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 shutdown_connections 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575151540 0x555575153930 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575141940 0x55557513f2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x55557513eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1589130710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x55557513e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f181bfff640 0 client.26422 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 >> 172.21.15.16:0/1589130710 conn(0x555575132cf0 msgr2=0x555575131e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 shutdown_connections 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 1 -- 172.21.15.16:0/1589130710 wait complete. 2024-09-12T01:02:10.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.858+0000 7f1844eaf740 20 client.26422 trim_cache size 0 max 16384 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.875+0000 7f1844eaf740 1 Processor -- start 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.875+0000 7f1844eaf740 1 -- start start 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575131d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575158f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1afaa0 con 0x555575132cf0 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1b15e0 con 0x555575137080 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1844eaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1b0170 con 0x555575136760 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34800/0 (socket says 172.21.15.16:34800) 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 -- 172.21.15.16:0/1826079781 learned_addr learned my addr 172.21.15.16:0/1826079781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f183bfff640 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575131d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 -- 172.21.15.16:0/1826079781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 msgr2=0x555575131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575131d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 -- 172.21.15.16:0/1826079781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 msgr2=0x555575158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 -- 172.21.15.16:0/1826079781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555575136590 con 0x555575132cf0 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f183bfff640 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575131d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f1840872640 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1830005d00 tx=0x7f183000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c198a03d140b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f18267fc640 1 -- 172.21.15.16:0/1826079781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830005220 con 0x555575132cf0 2024-09-12T01:02:10.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f18267fc640 1 -- 172.21.15.16:0/1826079781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830004c80 con 0x555575132cf0 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.876+0000 7f18267fc640 1 -- 172.21.15.16:0/1826079781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183001da50 con 0x555575132cf0 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- 172.21.15.16:0/1826079781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 msgr2=0x555575140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1830005d00 tx=0x7f183000b410 comp rx=0 tx=0).stop 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- 172.21.15.16:0/1826079781 shutdown_connections 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575137080 0x555575158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575136760 0x555575131d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1826079781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575132cf0 0x555575140730 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- 172.21.15.16:0/1826079781 >> 172.21.15.16:0/1826079781 conn(0x555575153af0 msgr2=0x555575131650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- 172.21.15.16:0/1826079781 shutdown_connections 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- 172.21.15.16:0/1826079781 wait complete. 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 -- start start 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.877+0000 7f1844eaf740 1 Processor -- start 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575132cf0 0x555575145580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x555575145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1b0f10 con 0x555575137080 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1b0840 con 0x555575136760 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f183c1ada80 con 0x555575132cf0 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1844eaf740 10 client.0 _ll_register_callbacks cb 0x7f183409b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34812/0 (socket says 172.21.15.16:34812) 2024-09-12T01:02:10.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 -- 172.21.15.16:0/1498656266 learned_addr learned my addr 172.21.15.16:0/1498656266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1840872640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575132cf0 0x555575145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f183bfff640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x555575145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 -- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575132cf0 msgr2=0x555575145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575132cf0 0x555575145580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 -- 172.21.15.16:0/1498656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 msgr2=0x555575145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x555575145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.878+0000 7f1841073640 1 -- 172.21.15.16:0/1498656266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557512e260 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1841073640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1834097e00 tx=0x7f18340ba440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2cb4db4f7c0d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183407f040 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f1834098010 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1830004c80 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f183401d040 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 10 client.17109 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555751bbf20 con 0x555575137080 2024-09-12T01:02:10.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830004c80 con 0x555575137080 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1834053080 con 0x555575137080 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 10 client.17109 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 10 client.17109 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 20 client.17109 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575141940 0x5555751be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 4 client.17109 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.879+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5555750a3d20 con 0x555575141940 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.880+0000 7f1840872640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575141940 0x5555751be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.880+0000 7f1840872640 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575141940 0x5555751be4d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbdbe8e455e7017 server_cookie=189ed7f0666e6f1 in_seq=0 out_seq=0 2024-09-12T01:02:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.880+0000 7f181bfff640 10 client.17109 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.889+0000 7f181bfff640 1 -- 172.21.15.16:0/1498656266 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5555750a3d20 con 0x555575141940 2024-09-12T01:02:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.889+0000 7f181bfff640 10 client.17109 handle_command_reply: tid=0 2024-09-12T01:02:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.889+0000 7f1844eaf740 1 client.17109 shutdown 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 msgr2=0x555575146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1834097e00 tx=0x7f18340ba440 comp rx=0 tx=0).stop 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 shutdown_connections 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555575141940 0x5555751be4d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555575137080 0x555575146000 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555575136760 0x555575145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f181bfff640 0 client.17109 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 --2- 172.21.15.16:0/1498656266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555575132cf0 0x555575145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 >> 172.21.15.16:0/1498656266 conn(0x555575153af0 msgr2=0x55557513f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 shutdown_connections 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 1 -- 172.21.15.16:0/1498656266 wait complete. 2024-09-12T01:02:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1844eaf740 20 client.17109 trim_cache size 0 max 16384 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 22, 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "4c083005": { 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 22, 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:02:10.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.1, 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:10.894 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 msgr2=0x7f182c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 0x7f182c0a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1828009230 tx=0x7f1828031690 comp rx=0 tx=0).stop 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 msgr2=0x7f183c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f183400cad0 tx=0x7f183400cf90 comp rx=0 tx=0).stop 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.890+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 shutdown_connections 2024-09-12T01:02:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f182c0a2cf0 0x7f182c0a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f183c106f10 0x7f183c19efc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f183c102790 0x7f183c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 --2- 172.21.15.16:0/3460681729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f183c101de0 0x7f183c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 >> 172.21.15.16:0/3460681729 conn(0x7f183c0fd8c0 msgr2=0x7f183c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 shutdown_connections 2024-09-12T01:02:10.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:10.891+0000 7f1842ad6640 1 -- 172.21.15.16:0/3460681729 wait complete. 2024-09-12T01:02:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:11 smithi016 ceph-mon[30972]: pgmap v186: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-09-12T01:02:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:11 smithi099 ceph-mon[35654]: pgmap v186: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-09-12T01:02:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:11 smithi159 ceph-mon[35566]: pgmap v186: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 70 KiB/s wr, 34 op/s 2024-09-12T01:02:11.638 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4c083005', quiesce took 3.3 seconds. Will release after: 16.8 2024-09-12T01:02:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:13 smithi099 ceph-mon[35654]: pgmap v187: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:13 smithi159 ceph-mon[35566]: pgmap v187: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:13.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:13 smithi016 ceph-mon[30972]: pgmap v187: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:14.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:02:14.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 -- 172.21.15.16:0/922544696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4102790 msgr2=0x7f80b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 --2- 172.21.15.16:0/922544696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4102790 0x7f80b41067e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c02a410 comp rx=0 tx=0).stop 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 -- 172.21.15.16:0/922544696 shutdown_connections 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 --2- 172.21.15.16:0/922544696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4106f10 0x7f80b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 --2- 172.21.15.16:0/922544696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4102790 0x7f80b41067e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 --2- 172.21.15.16:0/922544696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b4101de0 0x7f80b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 -- 172.21.15.16:0/922544696 >> 172.21.15.16:0/922544696 conn(0x7f80b40fd8c0 msgr2=0x7f80b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 -- 172.21.15.16:0/922544696 shutdown_connections 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.959+0000 7f80b884e640 1 -- 172.21.15.16:0/922544696 wait complete. 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 Processor -- start 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 -- start start 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4102790 0x7f80b419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b4106f10 0x7f80b419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4111f70 con 0x7f80b4101de0 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4111df0 con 0x7f80b4106f10 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b884e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41120f0 con 0x7f80b4102790 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34832/0 (socket says 172.21.15.16:34832) 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 -- 172.21.15.16:0/1084036141 learned_addr learned my addr 172.21.15.16:0/1084036141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80abfff640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4102790 0x7f80b419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 -- 172.21.15.16:0/1084036141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4102790 msgr2=0x7f80b419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4102790 0x7f80b419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 -- 172.21.15.16:0/1084036141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b4106f10 msgr2=0x7f80b419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b4106f10 0x7f80b419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.960+0000 7f80b259c640 1 -- 172.21.15.16:0/1084036141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b419f670 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80b259c640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c890 tx=0x7f80a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80ab7fe640 1 -- 172.21.15.16:0/1084036141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004220 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80ab7fe640 1 -- 172.21.15.16:0/1084036141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a40043c0 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b419f960 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80ab7fe640 1 -- 172.21.15.16:0/1084036141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40049e0 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.961+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41a7240 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.963+0000 7f80ab7fe640 1 -- 172.21.15.16:0/1084036141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f80a40079b0 con 0x7f80b4101de0 2024-09-12T01:02:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 Processor -- start 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 -- start start 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x55676826b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x55676826da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567681c62e0 con 0x5567682604a0 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55676816dce0 con 0x556768260dc0 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80bac27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567681c22a0 con 0x55676826b680 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80ab7fe640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 0x7f80a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80b2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80b2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34848/0 (socket says 172.21.15.16:34848) 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3897547543 learned_addr learned my addr 172.21.15.16:0/3897547543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80ab7fe640 1 -- 172.21.15.16:0/1084036141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f80a40cf530 con 0x7f80b4101de0 2024-09-12T01:02:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80abfff640 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x55676826da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80b259c640 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x55676826b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.964+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 0x7f80a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3897547543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 msgr2=0x55676826da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x55676826da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3897547543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 msgr2=0x55676826b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x55676826b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3897547543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556768152440 con 0x5567682604a0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f80ac013850 tx=0x7f80ac013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e820f46d769e7065 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80a8ff9640 1 -- 172.21.15.16:0/3897547543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac020070 con 0x5567682604a0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80a8ff9640 1 -- 172.21.15.16:0/3897547543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac00d8a0 con 0x5567682604a0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80a8ff9640 1 -- 172.21.15.16:0/3897547543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00e5a0 con 0x5567682604a0 2024-09-12T01:02:14.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.965+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 0x7f80a00a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80ac01d000 tx=0x7f80ac019000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- 172.21.15.16:0/3897547543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 msgr2=0x556768260880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f80ac013850 tx=0x7f80ac013d10 comp rx=0 tx=0).stop 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- 172.21.15.16:0/3897547543 shutdown_connections 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x55676826da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x55676826b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 --2- 172.21.15.16:0/3897547543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x556768260880 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- 172.21.15.16:0/3897547543 >> 172.21.15.16:0/3897547543 conn(0x55676825ca30 msgr2=0x55676825ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- 172.21.15.16:0/3897547543 shutdown_connections 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- 172.21.15.16:0/3897547543 wait complete. 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 -- start start 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 Processor -- start 2024-09-12T01:02:14.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.966+0000 7f80bac27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x55676826aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55676826b680 0x556768268800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55676826e250 con 0x556768260dc0 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556768181050 con 0x55676826b680 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55676826e3d0 con 0x5567682604a0 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 10 client.0 _ll_register_callbacks cb 0x5567681c1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34858/0 (socket says 172.21.15.16:34858) 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 -- 172.21.15.16:0/1572299827 learned_addr learned my addr 172.21.15.16:0/1572299827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55676826b680 0x556768268800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80b259c640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x55676826aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 -- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 msgr2=0x55676826aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x55676826aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 -- 172.21.15.16:0/1572299827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55676826b680 msgr2=0x556768268800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55676826b680 0x556768268800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 -- 172.21.15.16:0/1572299827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556767f373c0 con 0x556768260dc0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80abfff640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f809c031550 tx=0x7f809c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90976b02c789d7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f8099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c035040 con 0x556768260dc0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80ac00d8a0 con 0x556768260dc0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.967+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f809c005020 con 0x556768260dc0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f809c005220 con 0x556768260dc0 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 10 client.17121 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:14.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556768279c20 con 0x556768260dc0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80ac00d8a0 con 0x556768260dc0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f809c003680 con 0x556768260dc0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 10 client.17121 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 20 client.17121 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55676827b0e0 0x55676827d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 4 client.17121 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567681d3850 con 0x55676827b0e0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.968+0000 7f80b259c640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55676827b0e0 0x55676827d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.969+0000 7f8099ffb640 10 client.17121 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.969+0000 7f80b259c640 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55676827b0e0 0x55676827d520 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7aa21c126769005 server_cookie=350a1e6b027e45a2 in_seq=0 out_seq=0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f8099ffb640 1 -- 172.21.15.16:0/1572299827 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567681d3850 con 0x55676827b0e0 2024-09-12T01:02:14.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f8099ffb640 10 client.17121 handle_command_reply: tid=0 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 client.17121 shutdown 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 msgr2=0x5567682682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f809c031550 tx=0x7f809c004270 comp rx=0 tx=0).stop 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 shutdown_connections 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55676827b0e0 0x55676827d520 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55676826b680 0x556768268800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556768260dc0 0x5567682682a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 --2- 172.21.15.16:0/1572299827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x55676826aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f8099ffb640 0 client.17121 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 >> 172.21.15.16:0/1572299827 conn(0x55676825ca30 msgr2=0x55676825bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 shutdown_connections 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 1 -- 172.21.15.16:0/1572299827 wait complete. 2024-09-12T01:02:14.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.970+0000 7f80bac27740 20 client.17121 trim_cache size 0 max 16384 2024-09-12T01:02:14.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 Processor -- start 2024-09-12T01:02:14.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 -- start start 2024-09-12T01:02:14.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768282df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x556768283330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41afb00 con 0x5567682604a0 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b1640 con 0x556768260dc0 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80bac27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b01d0 con 0x55676826b680 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34866/0 (socket says 172.21.15.16:34866) 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80b259c640 1 -- 172.21.15.16:0/2219317347 learned_addr learned my addr 172.21.15.16:0/2219317347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x556768283330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.988+0000 7f80abfff640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768282df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 -- 172.21.15.16:0/2219317347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 msgr2=0x556768283330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x556768283330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 -- 172.21.15.16:0/2219317347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 msgr2=0x556768282df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:14.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768282df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 -- 172.21.15.16:0/2219317347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567682602d0 con 0x5567682604a0 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80abfff640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768282df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f80b259c640 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f80a4097ee0 tx=0x7f80a40cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea849e8ea77c8362 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f809bfff640 1 -- 172.21.15.16:0/2219317347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a401d040 con 0x5567682604a0 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f809bfff640 1 -- 172.21.15.16:0/2219317347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a40b3070 con 0x5567682604a0 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.989+0000 7f809bfff640 1 -- 172.21.15.16:0/2219317347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4007ca0 con 0x5567682604a0 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- 172.21.15.16:0/2219317347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 msgr2=0x5567682774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f80a4097ee0 tx=0x7f80a40cf040 comp rx=0 tx=0).stop 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- 172.21.15.16:0/2219317347 shutdown_connections 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55676826b680 0x556768283330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768282df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- 172.21.15.16:0/2219317347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567682604a0 0x5567682774b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- 172.21.15.16:0/2219317347 >> 172.21.15.16:0/2219317347 conn(0x55676825ca30 msgr2=0x556768263900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:14.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- 172.21.15.16:0/2219317347 shutdown_connections 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- 172.21.15.16:0/2219317347 wait complete. 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- start start 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 Processor -- start 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x5567682738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768273df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.990+0000 7f80bac27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b0f70 con 0x55676826b680 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80bac27740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b08a0 con 0x556768260dc0 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80bac27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41adae0 con 0x5567682604a0 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80bac27740 10 client.0 _ll_register_callbacks cb 0x7f809c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:15.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34876/0 (socket says 172.21.15.16:34876) 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3219785429 learned_addr learned my addr 172.21.15.16:0/3219785429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b259c640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x5567682738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80abfff640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768273df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 msgr2=0x5567682738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x5567682738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3219785429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 msgr2=0x556768273df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768273df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 -- 172.21.15.16:0/3219785429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567682653a0 con 0x55676826b680 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80abfff640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768273df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b259c640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x5567682738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80b2d9d640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f80ac004a80 tx=0x7f80ac007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1282699115b80b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f8099ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac031020 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f80ac00e560 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.991+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a40b3070 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f80ac03c050 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 10 client.17127 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567682e5910 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a40b3070 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f80ac00f040 con 0x55676826b680 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 10 client.17127 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 20 client.17127 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567682e5f10 0x5567682e8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 4 client.17127 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567681cdd60 con 0x5567682e5f10 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.992+0000 7f80b259c640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567682e5f10 0x5567682e8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80b259c640 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567682e5f10 0x5567682e8330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f5d8ad3ef22133 server_cookie=e94c0c6d5a5b1ff2 in_seq=0 out_seq=0 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f8099ffb640 10 client.17127 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f8099ffb640 1 -- 172.21.15.16:0/3219785429 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5567681cdd60 con 0x5567682e5f10 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f8099ffb640 10 client.17127 handle_command_reply: tid=0 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 client.17127 shutdown 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 msgr2=0x556768264b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f80ac004a80 tx=0x7f80ac007e50 comp rx=0 tx=0).stop 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 shutdown_connections 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567682e5f10 0x5567682e8330 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55676826b680 0x556768264b70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556768260dc0 0x556768273df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 --2- 172.21.15.16:0/3219785429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567682604a0 0x5567682738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 >> 172.21.15.16:0/3219785429 conn(0x55676825ca30 msgr2=0x556768262fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f8099ffb640 0 client.17127 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 shutdown_connections 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 1 -- 172.21.15.16:0/3219785429 wait complete. 2024-09-12T01:02:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.993+0000 7f80bac27740 20 client.17127 trim_cache size 0 max 16384 2024-09-12T01:02:15.005 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:02:15.005 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:02:15.005 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:02:15.005 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 msgr2=0x7f80a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 0x7f80a00a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80ac01d000 tx=0x7f80ac019000 comp rx=0 tx=0).stop 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 msgr2=0x7f80b419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f80a400c890 tx=0x7f80a400cd50 comp rx=0 tx=0).stop 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 shutdown_connections 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f80a00a2cf0 0x7f80a00a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b4106f10 0x7f80b419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80b4102790 0x7f80b419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 --2- 172.21.15.16:0/1084036141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80b4101de0 0x7f80b419a6c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 >> 172.21.15.16:0/1084036141 conn(0x7f80b40fd8c0 msgr2=0x7f80b40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 shutdown_connections 2024-09-12T01:02:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:14.994+0000 7f80b884e640 1 -- 172.21.15.16:0/1084036141 wait complete. 2024-09-12T01:02:15.463 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:15 smithi016 ceph-mon[30972]: pgmap v188: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:15 smithi099 ceph-mon[35654]: pgmap v188: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:15 smithi159 ceph-mon[35566]: pgmap v188: 97 pgs: 97 active+clean; 6.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-09-12T01:02:15.701 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:02:15.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:02:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 -- 172.21.15.16:0/4253950244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150102790 msgr2=0x7f71501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 --2- 172.21.15.16:0/4253950244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150102790 0x7f71501067e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f7144009900 tx=0x7f714402a410 comp rx=0 tx=0).stop 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 -- 172.21.15.16:0/4253950244 shutdown_connections 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 --2- 172.21.15.16:0/4253950244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 --2- 172.21.15.16:0/4253950244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150102790 0x7f71501067e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 --2- 172.21.15.16:0/4253950244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150101de0 0x7f71501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 -- 172.21.15.16:0/4253950244 >> 172.21.15.16:0/4253950244 conn(0x7f71500fd8c0 msgr2=0x7f71500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 -- 172.21.15.16:0/4253950244 shutdown_connections 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.406+0000 7f715636f640 1 -- 172.21.15.16:0/4253950244 wait complete. 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 Processor -- start 2024-09-12T01:02:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 -- start start 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150102790 0x7f71501a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7150111f90 con 0x7f7150101de0 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7150111e10 con 0x7f7150106f10 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f715636f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7150112110 con 0x7f7150102790 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f714ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f714ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34892/0 (socket says 172.21.15.16:34892) 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.407+0000 7f714ffff640 1 -- 172.21.15.16:0/827168580 learned_addr learned my addr 172.21.15.16:0/827168580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714f7fe640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150102790 0x7f71501a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f715490c640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 -- 172.21.15.16:0/827168580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150102790 msgr2=0x7f71501a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150102790 0x7f71501a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 -- 172.21.15.16:0/827168580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 msgr2=0x7f715019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 -- 172.21.15.16:0/827168580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715019acd0 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f715490c640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714ffff640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f714000b4a0 tx=0x7f714000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140014070 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715019afc0 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7140010af0 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140017d10 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.408+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501a72b0 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.409+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71501021c0 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.411+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7140017070 con 0x7f7150101de0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.411+0000 7f714d7fa640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 0x7f71380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.411+0000 7f714f7fe640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 0x7f71380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.411+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f71400cea70 con 0x7f7150101de0 2024-09-12T01:02:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.412+0000 7f714f7fe640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 0x7f71380a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7144000c00 tx=0x7f71440058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.414+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f714009b7d0 con 0x7f7150101de0 2024-09-12T01:02:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.625+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f715019bad0 con 0x7f7150101de0 2024-09-12T01:02:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.626+0000 7f714d7fa640 1 -- 172.21.15.16:0/827168580 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f714009b0d0 con 0x7f7150101de0 2024-09-12T01:02:16.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:02:16.626 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:02:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:02:16.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 msgr2=0x7f71380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:16.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 0x7f71380a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7144000c00 tx=0x7f71440058d0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 msgr2=0x7f71501a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f714000b4a0 tx=0x7f714000b960 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 shutdown_connections 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f71380a2cf0 0x7f71380a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7150106f10 0x7f715019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150102790 0x7f71501a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 --2- 172.21.15.16:0/827168580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7150101de0 0x7f71501a03e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 >> 172.21.15.16:0/827168580 conn(0x7f71500fd8c0 msgr2=0x7f71500fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 shutdown_connections 2024-09-12T01:02:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:16.628+0000 7f715636f640 1 -- 172.21.15.16:0/827168580 wait complete. 2024-09-12T01:02:17.438 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:17 smithi016 ceph-mon[30972]: pgmap v189: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-12T01:02:17.438 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:17 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/827168580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:17.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:02:17.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:17 smithi099 ceph-mon[35654]: pgmap v189: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-12T01:02:17.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:17 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/827168580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:17 smithi159 ceph-mon[35566]: pgmap v189: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-09-12T01:02:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:17 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/827168580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:17.745 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:18.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 -- 172.21.15.16:0/1202083969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 msgr2=0x7f66801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 --2- 172.21.15.16:0/1202083969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f66801021c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f667c009900 tx=0x7f667c02a410 comp rx=0 tx=0).stop 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 -- 172.21.15.16:0/1202083969 shutdown_connections 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 --2- 172.21.15.16:0/1202083969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680106f10 0x7f668010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 --2- 172.21.15.16:0/1202083969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680102790 0x7f66801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 --2- 172.21.15.16:0/1202083969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f66801021c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 -- 172.21.15.16:0/1202083969 >> 172.21.15.16:0/1202083969 conn(0x7f66800fd8c0 msgr2=0x7f66800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 -- 172.21.15.16:0/1202083969 shutdown_connections 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.378+0000 7f66878d5640 1 -- 172.21.15.16:0/1202083969 wait complete. 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 Processor -- start 2024-09-12T01:02:18.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 -- start start 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680102790 0x7f6680198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680106f10 0x7f668019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6680111f70 con 0x7f6680101de0 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6680111df0 con 0x7f6680102790 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f66878d5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66801120f0 con 0x7f6680106f10 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34916/0 (socket says 172.21.15.16:34916) 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f6685671640 1 -- 172.21.15.16:0/312837074 learned_addr learned my addr 172.21.15.16:0/312837074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f6684e70640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680102790 0x7f6680198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.379+0000 7f6685e72640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680106f10 0x7f668019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 -- 172.21.15.16:0/312837074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680106f10 msgr2=0x7f668019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680106f10 0x7f668019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 -- 172.21.15.16:0/312837074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680102790 msgr2=0x7f6680198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680102790 0x7f6680198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 -- 172.21.15.16:0/312837074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668019d400 con 0x7f6680101de0 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f6685671640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f667c006010 tx=0x7f667c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f66767fc640 1 -- 172.21.15.16:0/312837074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c03f030 con 0x7f6680101de0 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f66767fc640 1 -- 172.21.15.16:0/312837074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c003ce0 con 0x7f6680101de0 2024-09-12T01:02:18.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668019d690 con 0x7f6680101de0 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f66767fc640 1 -- 172.21.15.16:0/312837074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c035070 con 0x7f6680101de0 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.380+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a4fd0 con 0x7f6680101de0 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.383+0000 7f66767fc640 1 -- 172.21.15.16:0/312837074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f667c042030 con 0x7f6680101de0 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.383+0000 7f6689cae740 1 Processor -- start 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.383+0000 7f6689cae740 1 -- start start 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 0x55c398428a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c398439b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c39838e2f0 con 0x55c398429b40 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c398335cf0 con 0x55c398437780 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6689cae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c39838a2b0 con 0x55c3984285d0 2024-09-12T01:02:18.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34920/0 (socket says 172.21.15.16:34920) 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 -- 172.21.15.16:0/2020201589 learned_addr learned my addr 172.21.15.16:0/2020201589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6684e70640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c398439b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685671640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 0x55c398428a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f66767fc640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 0x7f666c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 -- 172.21.15.16:0/2020201589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 msgr2=0x55c398428a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 0x55c398428a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 -- 172.21.15.16:0/2020201589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 msgr2=0x55c398439b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c398439b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 -- 172.21.15.16:0/2020201589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39831a3a0 con 0x55c398429b40 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685671640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 0x55c398428a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:18.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6684e70640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c398439b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f66767fc640 1 -- 172.21.15.16:0/312837074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f667c0efa00 con 0x7f6680101de0 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 0x7f666c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.384+0000 7f6685e72640 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6678009e60 tx=0x7f6678010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732e611207160398 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f666b7fe640 1 -- 172.21.15.16:0/2020201589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678021070 con 0x55c398429b40 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f666b7fe640 1 -- 172.21.15.16:0/2020201589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667801b400 con 0x55c398429b40 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f666b7fe640 1 -- 172.21.15.16:0/2020201589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678016070 con 0x55c398429b40 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 -- 172.21.15.16:0/2020201589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 msgr2=0x55c398428090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6685e72640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 0x7f666c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6678011010 tx=0x7f6678011b70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6678009e60 tx=0x7f6678010740 comp rx=0 tx=0).stop 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 -- 172.21.15.16:0/2020201589 shutdown_connections 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c398439b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3984285d0 0x55c398428a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 --2- 172.21.15.16:0/2020201589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398429b40 0x55c398428090 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.385+0000 7f6689cae740 1 -- 172.21.15.16:0/2020201589 >> 172.21.15.16:0/2020201589 conn(0x55c3983f1310 msgr2=0x55c398424c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.386+0000 7f6689cae740 1 -- 172.21.15.16:0/2020201589 shutdown_connections 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.386+0000 7f6689cae740 1 -- 172.21.15.16:0/2020201589 wait complete. 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.386+0000 7f6689cae740 1 -- start start 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.386+0000 7f6689cae740 1 Processor -- start 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 0x55c398437170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398434420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c39843a480 con 0x55c398437780 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c398349060 con 0x55c3984285d0 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c39843a600 con 0x55c398429b40 2024-09-12T01:02:18.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 10 client.0 _ll_register_callbacks cb 0x55c39809c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34936/0 (socket says 172.21.15.16:34936) 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 -- 172.21.15.16:0/4293270513 learned_addr learned my addr 172.21.15.16:0/4293270513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685671640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 0x55c398437170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6684e70640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398434420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 -- 172.21.15.16:0/4293270513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 msgr2=0x55c398434420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398434420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 -- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 msgr2=0x55c398437170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 0x55c398437170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 -- 172.21.15.16:0/4293270513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3980ff3c0 con 0x55c398437780 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685671640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 0x55c398437170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6685e72640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6678010780 tx=0x7f6678016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89a9d453b2f7543 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035060 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667801b400 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.387+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f6678011ca0 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6678021070 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 10 client.17145 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c398445f20 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667801b400 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f667802f030 con 0x55c398437780 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 10 client.17145 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 10 client.17145 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 20 client.17145 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c398447380 0x55c398449770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 4 client.17145 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:18.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c39839b860 con 0x55c398447380 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.388+0000 7f6685671640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c398447380 0x55c398449770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.389+0000 7f6685671640 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c398447380 0x55c398449770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=239746d832fbaa3e server_cookie=ff99c61712d5219d in_seq=0 out_seq=0 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.389+0000 7f6668ff9640 10 client.17145 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.389+0000 7f6668ff9640 1 -- 172.21.15.16:0/4293270513 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c39839b860 con 0x55c398447380 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.389+0000 7f6668ff9640 10 client.17145 handle_command_reply: tid=0 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.389+0000 7f6689cae740 1 client.17145 shutdown 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 msgr2=0x55c398434a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6678010780 tx=0x7f6678016000 comp rx=0 tx=0).stop 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 shutdown_connections 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c398447380 0x55c398449770 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c398437780 0x55c398434a10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398434420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 --2- 172.21.15.16:0/4293270513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c3984285d0 0x55c398437170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 >> 172.21.15.16:0/4293270513 conn(0x55c3983f1310 msgr2=0x55c39842a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6668ff9640 0 client.17145 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:02:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 shutdown_connections 2024-09-12T01:02:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 1 -- 172.21.15.16:0/4293270513 wait complete. 2024-09-12T01:02:18.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.390+0000 7f6689cae740 20 client.17145 trim_cache size 0 max 16384 2024-09-12T01:02:18.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 Processor -- start 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 -- start start 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398443720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c39844ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ab680 con 0x55c3984285d0 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ad1c0 con 0x55c398437780 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6689cae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66801abd50 con 0x55c398429b40 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45396/0 (socket says 172.21.15.16:45396) 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685671640 1 -- 172.21.15.16:0/2054204132 learned_addr learned my addr 172.21.15.16:0/2054204132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6684e70640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398443720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685e72640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c39844ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685671640 1 -- 172.21.15.16:0/2054204132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 msgr2=0x55c398443720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.404+0000 7f6685671640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398443720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6685671640 1 -- 172.21.15.16:0/2054204132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 msgr2=0x55c39844ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6685671640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c39844ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6685671640 1 -- 172.21.15.16:0/2054204132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c398429780 con 0x55c3984285d0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6684e70640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398443720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6685e72640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c39844ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f6685671640 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdc3f4c2a124a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f666affd640 1 -- 172.21.15.16:0/2054204132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0f6030 con 0x55c3984285d0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f666affd640 1 -- 172.21.15.16:0/2054204132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c0bc710 con 0x55c3984285d0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.405+0000 7f666affd640 1 -- 172.21.15.16:0/2054204132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c049040 con 0x55c3984285d0 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- 172.21.15.16:0/2054204132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 msgr2=0x55c398424e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c02a410 comp rx=0 tx=0).stop 2024-09-12T01:02:18.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- 172.21.15.16:0/2054204132 shutdown_connections 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398437780 0x55c39844ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398429b40 0x55c398443720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 --2- 172.21.15.16:0/2054204132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c398424e10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- 172.21.15.16:0/2054204132 >> 172.21.15.16:0/2054204132 conn(0x55c3983f1310 msgr2=0x55c398424700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- 172.21.15.16:0/2054204132 shutdown_connections 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- 172.21.15.16:0/2054204132 wait complete. 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 -- start start 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.406+0000 7f6689cae740 1 Processor -- start 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 0x55c3984311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398437780 0x55c3984316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66801acaf0 con 0x55c3984285d0 2024-09-12T01:02:18.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ac420 con 0x55c398429b40 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66801a9660 con 0x55c398437780 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6689cae740 10 client.0 _ll_register_callbacks cb 0x7f667801f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45412/0 (socket says 172.21.15.16:45412) 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 -- 172.21.15.16:0/3608091934 learned_addr learned my addr 172.21.15.16:0/3608091934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6684e70640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 0x55c3984311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685e72640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398437780 0x55c3984316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 -- 172.21.15.16:0/3608091934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398437780 msgr2=0x55c3984316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398437780 0x55c3984316f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 -- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 msgr2=0x55c3984311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 0x55c3984311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 -- 172.21.15.16:0/3608091934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c398431f20 con 0x55c3984285d0 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6684e70640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 0x55c3984311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:18.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6685671640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ab0 tx=0x7f667c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cc83e9929f5b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:18.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.407+0000 7f6668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:18.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:18 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:02:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0d9020 con 0x55c3984285d0 2024-09-12T01:02:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c0bc710 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f667c0b8680 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f667c0f6030 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 10 client.17151 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3984b1f00 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c0bc710 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f667c0bc270 con 0x55c3984285d0 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 10 client.17151 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 10 client.17151 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 20 client.17151 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c3984b2580 0x55c3984b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 4 client.17151 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.408+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c398395d70 con 0x55c3984b2580 2024-09-12T01:02:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6684e70640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c3984b2580 0x55c3984b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6668ff9640 10 client.17151 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6684e70640 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c3984b2580 0x55c3984b49a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d2d0051362d94b4 server_cookie=3a51862c133a0bd6 in_seq=0 out_seq=0 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6668ff9640 1 -- 172.21.15.16:0/3608091934 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c398395d70 con 0x55c3984b2580 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6668ff9640 10 client.17151 handle_command_reply: tid=0 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.409+0000 7f6689cae740 1 client.17151 shutdown 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 msgr2=0x55c39843ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f667c005ab0 tx=0x7f667c004460 comp rx=0 tx=0).stop 2024-09-12T01:02:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 shutdown_connections 2024-09-12T01:02:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c3984b2580 0x55c3984b49a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c398437780 0x55c3984316f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c398429b40 0x55c3984311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6668ff9640 0 client.17151 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 --2- 172.21.15.16:0/3608091934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3984285d0 0x55c39843ffd0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 >> 172.21.15.16:0/3608091934 conn(0x55c3983f1310 msgr2=0x55c3984234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 shutdown_connections 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 1 -- 172.21.15.16:0/3608091934 wait complete. 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f6689cae740 20 client.17151 trim_cache size 0 max 16384 2024-09-12T01:02:18.424 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 msgr2=0x7f666c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 0x7f666c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6678011010 tx=0x7f6678011b70 comp rx=0 tx=0).stop 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 msgr2=0x7f6680198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.410+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f667c006010 tx=0x7f667c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 shutdown_connections 2024-09-12T01:02:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f666c0a2cf0 0x7f666c0a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6680106f10 0x7f668019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6680102790 0x7f6680198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 --2- 172.21.15.16:0/312837074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6680101de0 0x7f6680198450 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 >> 172.21.15.16:0/312837074 conn(0x7f66800fd8c0 msgr2=0x7f66800fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 shutdown_connections 2024-09-12T01:02:18.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:18.411+0000 7f66878d5640 1 -- 172.21.15.16:0/312837074 wait complete. 2024-09-12T01:02:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:18 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:02:18.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:18 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:02:19.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:19 smithi016 ceph-mon[30972]: pgmap v190: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-09-12T01:02:19.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:02:19.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:02:19.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:02:19.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:02:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:19 smithi099 ceph-mon[35654]: pgmap v190: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-09-12T01:02:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:02:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:02:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:02:19.525 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:19 smithi159 ceph-mon[35566]: pgmap v190: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-09-12T01:02:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:02:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:02:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 -- 172.21.15.16:0/2707625921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca30102bf0 msgr2=0x7fca30102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2707625921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca30102bf0 0x7fca30102fd0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fca20009900 tx=0x7fca2002a480 comp rx=0 tx=0).stop 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 -- 172.21.15.16:0/2707625921 shutdown_connections 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2707625921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca3010bfd0 0x7fca3010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2707625921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 0x7fca3010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2707625921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca30102bf0 0x7fca30102fd0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 -- 172.21.15.16:0/2707625921 >> 172.21.15.16:0/2707625921 conn(0x7fca300fc630 msgr2=0x7fca300fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 -- 172.21.15.16:0/2707625921 shutdown_connections 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.136+0000 7fca34ba0640 1 -- 172.21.15.16:0/2707625921 wait complete. 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 Processor -- start 2024-09-12T01:02:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 -- start start 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca30102bf0 0x7fca30100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 0x7fca30100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca30111e70 con 0x7fca3010bfd0 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca30111cf0 con 0x7fca30103510 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca34ba0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fca30111ff0 con 0x7fca30102bf0 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45424/0 (socket says 172.21.15.16:45424) 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca2ffff640 1 -- 172.21.15.16:0/2858608750 learned_addr learned my addr 172.21.15.16:0/2858608750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.137+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca30102bf0 0x7fca30100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2effd640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 0x7fca30100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 -- 172.21.15.16:0/2858608750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca30102bf0 msgr2=0x7fca30100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca30102bf0 0x7fca30100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 -- 172.21.15.16:0/2858608750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 msgr2=0x7fca30100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 0x7fca30100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 -- 172.21.15.16:0/2858608750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca301a68c0 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b970 tx=0x7fca2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2cff9640 1 -- 172.21.15.16:0/2858608750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28019070 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca301a6bb0 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2cff9640 1 -- 172.21.15.16:0/2858608750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2800cb40 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca2cff9640 1 -- 172.21.15.16:0/2858608750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28007650 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.138+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca301a70a0 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.140+0000 7fca2cff9640 1 -- 172.21.15.16:0/2858608750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fca2801c070 con 0x7fca3010bfd0 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 Processor -- start 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 -- start start 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab230dd20 0x55dab230c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2cff9640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 0x7fca180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 0x55dab2319940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dab22722f0 con 0x55dab230c7b0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dab2219cf0 con 0x55dab2317550 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca381db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dab226e2b0 con 0x55dab230dd20 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45430/0 (socket says 172.21.15.16:45430) 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2957628787 learned_addr learned my addr 172.21.15.16:0/2957628787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2effd640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 0x7fca180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2cff9640 1 -- 172.21.15.16:0/2858608750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fca280ce910 con 0x7fca3010bfd0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2effd640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab230dd20 0x55dab230c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 0x55dab2319940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2957628787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab230dd20 msgr2=0x55dab230c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab230dd20 0x55dab230c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2957628787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 msgr2=0x55dab2319940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.141+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 0x55dab2319940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2957628787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab21fe3a0 con 0x55dab230c7b0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 0x55dab2319940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca2effd640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 0x7fca180a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca300fdad0 tx=0x7fca2400c3e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca20009900 tx=0x7fca200316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42318e617e30572e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca15ffb640 1 -- 172.21.15.16:0/2957628787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20040070 con 0x55dab230c7b0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca15ffb640 1 -- 172.21.15.16:0/2957628787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20005020 con 0x55dab230c7b0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.142+0000 7fca15ffb640 1 -- 172.21.15.16:0/2957628787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20035040 con 0x55dab230c7b0 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- 172.21.15.16:0/2957628787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 msgr2=0x55dab230cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fca20009900 tx=0x7fca200316e0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- 172.21.15.16:0/2957628787 shutdown_connections 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab2317550 0x55dab2319940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab230cbf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 --2- 172.21.15.16:0/2957628787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab230dd20 0x55dab230c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- 172.21.15.16:0/2957628787 >> 172.21.15.16:0/2957628787 conn(0x55dab22d5310 msgr2=0x55dab2308e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- 172.21.15.16:0/2957628787 shutdown_connections 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- 172.21.15.16:0/2957628787 wait complete. 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 -- start start 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.143+0000 7fca381db740 1 Processor -- start 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2314190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2314710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dab231a1f0 con 0x55dab230c7b0 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dab222d060 con 0x55dab230dd20 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dab231a370 con 0x55dab2317550 2024-09-12T01:02:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca381db740 10 client.0 _ll_register_callbacks cb 0x55dab1f808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45446/0 (socket says 172.21.15.16:45446) 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 -- 172.21.15.16:0/3312264530 learned_addr learned my addr 172.21.15.16:0/3312264530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2effd640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2314190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2ffff640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2314710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 -- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 msgr2=0x55dab2314710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2314710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 -- 172.21.15.16:0/3312264530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 msgr2=0x55dab2314190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2314190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 -- 172.21.15.16:0/3312264530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab1fe33c0 con 0x55dab230c7b0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fca20031ba0 tx=0x7fca2002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cb5b532a86a187 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca0effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20040070 con 0x55dab230c7b0 2024-09-12T01:02:20.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.144+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fca20004b40 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca20005020 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca20056050 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 10 client.17163 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dab2325bc0 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca20005020 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fca20044400 con 0x55dab230c7b0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 10 client.17163 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 10 client.17163 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 20 client.17163 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2326f10 0x55dab2329300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 4 client.17163 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab227f860 con 0x55dab2326f10 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.145+0000 7fca2effd640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2326f10 0x55dab2329300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.146+0000 7fca2effd640 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2326f10 0x55dab2329300 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a61fa57e36f1ace server_cookie=193110d2e3cb317c in_seq=0 out_seq=0 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.146+0000 7fca0effd640 10 client.17163 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca0effd640 1 -- 172.21.15.16:0/3312264530 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dab227f860 con 0x55dab2326f10 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca0effd640 10 client.17163 handle_command_reply: tid=0 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 client.17163 shutdown 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 msgr2=0x55dab2316ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fca20031ba0 tx=0x7fca2002a480 comp rx=0 tx=0).stop 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 shutdown_connections 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2314710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2314190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2326f10 0x55dab2329300 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 --2- 172.21.15.16:0/3312264530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab2316ee0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 >> 172.21.15.16:0/3312264530 conn(0x55dab22d5310 msgr2=0x55dab230eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca0effd640 0 client.17163 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:02:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 shutdown_connections 2024-09-12T01:02:20.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 1 -- 172.21.15.16:0/3312264530 wait complete. 2024-09-12T01:02:20.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.147+0000 7fca381db740 20 client.17163 trim_cache size 0 max 16384 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.160+0000 7fca381db740 1 Processor -- start 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.160+0000 7fca381db740 1 -- start start 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.160+0000 7fca381db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230c7b0 0x55dab2308ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.160+0000 7fca381db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.160+0000 7fca381db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2303b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca381db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca301af8c0 con 0x55dab230dd20 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca381db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b1400 con 0x55dab230c7b0 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca381db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fca301aff90 con 0x55dab2317550 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45454/0 (socket says 172.21.15.16:45454) 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 -- 172.21.15.16:0/2651945906 learned_addr learned my addr 172.21.15.16:0/2651945906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230c7b0 0x55dab2308ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2303b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 -- 172.21.15.16:0/2651945906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 msgr2=0x55dab2303b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2303b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 -- 172.21.15.16:0/2651945906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230c7b0 msgr2=0x55dab2308ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230c7b0 0x55dab2308ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 -- 172.21.15.16:0/2651945906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab2313180 con 0x55dab230dd20 2024-09-12T01:02:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.161+0000 7fca2effd640 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fca24006c60 tx=0x7fca2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13044134acc1905 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca157fa640 1 -- 172.21.15.16:0/2651945906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2401f040 con 0x55dab230dd20 2024-09-12T01:02:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca157fa640 1 -- 172.21.15.16:0/2651945906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2402b070 con 0x55dab230dd20 2024-09-12T01:02:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca157fa640 1 -- 172.21.15.16:0/2651945906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400f040 con 0x55dab230dd20 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 -- 172.21.15.16:0/2651945906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 msgr2=0x55dab23234c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fca24006c60 tx=0x7fca2401a000 comp rx=0 tx=0).stop 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 -- 172.21.15.16:0/2651945906 shutdown_connections 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2303b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230dd20 0x55dab23234c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 --2- 172.21.15.16:0/2651945906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230c7b0 0x55dab2308ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.163+0000 7fca381db740 1 -- 172.21.15.16:0/2651945906 >> 172.21.15.16:0/2651945906 conn(0x55dab22d5310 msgr2=0x55dab23088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- 172.21.15.16:0/2651945906 shutdown_connections 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- 172.21.15.16:0/2651945906 wait complete. 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- start start 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 Processor -- start 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2310d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2311290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b0d30 con 0x55dab230c7b0 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca301b0660 con 0x55dab230dd20 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.164+0000 7fca381db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fca301ad8a0 con 0x55dab2317550 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca381db740 10 client.0 _ll_register_callbacks cb 0x7fca200315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45456/0 (socket says 172.21.15.16:45456) 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2509237560 learned_addr learned my addr 172.21.15.16:0/2509237560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2effd640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2310d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2ffff640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2311290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 msgr2=0x55dab2311290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2311290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2509237560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 msgr2=0x55dab2310d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2310d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 -- 172.21.15.16:0/2509237560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab23118d0 con 0x55dab230c7b0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca2f7fe640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fca20009900 tx=0x7fca20005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832606935e5d718f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca0effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2005e020 con 0x55dab230c7b0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fca20044c40 con 0x55dab230c7b0 2024-09-12T01:02:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.165+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2402b070 con 0x55dab230c7b0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca20069050 con 0x55dab230c7b0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 10 client.17169 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dab23916d0 con 0x55dab230c7b0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2402b070 con 0x55dab230c7b0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fca2003c440 con 0x55dab230c7b0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 10 client.17169 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 10 client.17169 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 20 client.17169 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2391e40 0x55dab2394260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 4 client.17169 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dab2279d70 con 0x55dab2391e40 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.166+0000 7fca2effd640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2391e40 0x55dab2394260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:20.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca2effd640 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2391e40 0x55dab2394260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4785f8a03af50e36 server_cookie=bc1ed00b2560d3fb in_seq=0 out_seq=0 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca0effd640 10 client.17169 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca0effd640 1 -- 172.21.15.16:0/2509237560 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dab2279d70 con 0x55dab2391e40 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca0effd640 10 client.17169 handle_command_reply: tid=0 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 client.17169 shutdown 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 msgr2=0x55dab231fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fca20009900 tx=0x7fca20005750 comp rx=0 tx=0).stop 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 shutdown_connections 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab2317550 0x55dab2311290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dab230dd20 0x55dab2310d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dab2391e40 0x55dab2394260 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 --2- 172.21.15.16:0/2509237560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dab230c7b0 0x55dab231fb80 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca0effd640 0 client.17169 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.167+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 >> 172.21.15.16:0/2509237560 conn(0x55dab22d5310 msgr2=0x55dab2307780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 shutdown_connections 2024-09-12T01:02:20.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca381db740 1 -- 172.21.15.16:0/2509237560 wait complete. 2024-09-12T01:02:20.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca381db740 20 client.17169 trim_cache size 0 max 16384 2024-09-12T01:02:20.172 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 msgr2=0x7fca180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 0x7fca180a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca300fdad0 tx=0x7fca2400c3e0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 msgr2=0x7fca301012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fca2800b970 tx=0x7fca2800be30 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 shutdown_connections 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fca180a2cf0 0x7fca180a51b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3010bfd0 0x7fca301012d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca30103510 0x7fca30100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 --2- 172.21.15.16:0/2858608750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca30102bf0 0x7fca30100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.168+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 >> 172.21.15.16:0/2858608750 conn(0x7fca300fc630 msgr2=0x7fca300fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.169+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 shutdown_connections 2024-09-12T01:02:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:20.169+0000 7fca34ba0640 1 -- 172.21.15.16:0/2858608750 wait complete. 2024-09-12T01:02:20.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:02:21.116 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:21.419 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:21 smithi016 ceph-mon[30972]: pgmap v191: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:02:21.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:21 smithi099 ceph-mon[35654]: pgmap v191: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:02:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:21 smithi159 ceph-mon[35566]: pgmap v191: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:02:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 -- 172.21.15.16:0/4039196659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 msgr2=0x7f48dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 --2- 172.21.15.16:0/4039196659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc10e610 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009ca0 tx=0x7f48d402a950 comp rx=0 tx=0).stop 2024-09-12T01:02:21.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 -- 172.21.15.16:0/4039196659 shutdown_connections 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 --2- 172.21.15.16:0/4039196659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc10e610 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 --2- 172.21.15.16:0/4039196659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 --2- 172.21.15.16:0/4039196659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 0x7f48dc1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 -- 172.21.15.16:0/4039196659 >> 172.21.15.16:0/4039196659 conn(0x7f48dc0fc7a0 msgr2=0x7f48dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 -- 172.21.15.16:0/4039196659 shutdown_connections 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.621+0000 7f48e3191640 1 -- 172.21.15.16:0/4039196659 wait complete. 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 Processor -- start 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 -- start start 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 0x7f48dc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc111fb0 con 0x7f48dc10c1a0 2024-09-12T01:02:21.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc111e30 con 0x7f48dc103720 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e3191640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc112130 con 0x7f48dc102e00 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e172e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e172e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45472/0 (socket says 172.21.15.16:45472) 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e172e640 1 -- 172.21.15.16:0/3958052032 learned_addr learned my addr 172.21.15.16:0/3958052032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 0x7f48dc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.622+0000 7f48e172e640 1 -- 172.21.15.16:0/3958052032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 msgr2=0x7f48dc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e172e640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 0x7f48dc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e172e640 1 -- 172.21.15.16:0/3958052032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 msgr2=0x7f48dc100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e172e640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e172e640 1 -- 172.21.15.16:0/3958052032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc1a68b0 con 0x7f48dc10c1a0 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:21.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e172e640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f48d4006010 tx=0x7f48d402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48d1ffb640 1 -- 172.21.15.16:0/3958052032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4034040 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48d1ffb640 1 -- 172.21.15.16:0/3958052032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d4003e00 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc1a6b40 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48d1ffb640 1 -- 172.21.15.16:0/3958052032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d40456e0 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.623+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc1a7070 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.625+0000 7f48d1ffb640 1 -- 172.21.15.16:0/3958052032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f48d4037020 con 0x7f48dc10c1a0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 Processor -- start 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 -- start start 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c5c53412f0 con 0x55c5c53db5d0 2024-09-12T01:02:21.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5c52e8cf0 con 0x55c5c53dcb40 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.626+0000 7f48e556a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c5c533d2b0 con 0x55c5c53e6370 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45480/0 (socket says 172.21.15.16:45480) 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 -- 172.21.15.16:0/1478888576 learned_addr learned my addr 172.21.15.16:0/1478888576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48d1ffb640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 0x7f48c40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48d3fff640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e172e640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 0x7f48c40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48d3fff640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 -- 172.21.15.16:0/1478888576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 msgr2=0x55c5c53e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48d1ffb640 1 -- 172.21.15.16:0/3958052032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f48d40f04e0 con 0x7f48dc10c1a0 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 -- 172.21.15.16:0/1478888576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 msgr2=0x55c5c53db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53db090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 -- 172.21.15.16:0/1478888576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c52cd3a0 con 0x55c5c53db5d0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48d3fff640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f48d800cc60 tx=0x7f48d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b0eedc5161f131 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48e172e640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 0x7f48c40a5230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48d40b8f40 tx=0x7f48d408eaa0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48c2ffd640 1 -- 172.21.15.16:0/1478888576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8014070 con 0x55c5c53db5d0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48c2ffd640 1 -- 172.21.15.16:0/1478888576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d80047a0 con 0x55c5c53db5d0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.627+0000 7f48c2ffd640 1 -- 172.21.15.16:0/1478888576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8010760 con 0x55c5c53db5d0 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 -- 172.21.15.16:0/1478888576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 msgr2=0x55c5c53dba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f48d800cc60 tx=0x7f48d800e720 comp rx=0 tx=0).stop 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 -- 172.21.15.16:0/1478888576 shutdown_connections 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53dba10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 --2- 172.21.15.16:0/1478888576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 -- 172.21.15.16:0/1478888576 >> 172.21.15.16:0/1478888576 conn(0x55c5c53a4310 msgr2=0x55c5c53d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 -- 172.21.15.16:0/1478888576 shutdown_connections 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.628+0000 7f48e556a740 1 -- 172.21.15.16:0/1478888576 wait complete. 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 -- start start 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 Processor -- start 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c5c53e9070 con 0x55c5c53dcb40 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5c52fc060 con 0x55c5c53db5d0 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c5c53e91f0 con 0x55c5c53e6370 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e556a740 10 client.0 _ll_register_callbacks cb 0x55c5c504f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45496/0 (socket says 172.21.15.16:45496) 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e0f2d640 1 -- 172.21.15.16:0/3109176745 learned_addr learned my addr 172.21.15.16:0/3109176745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.629+0000 7f48e0f2d640 1 -- 172.21.15.16:0/3109176745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 msgr2=0x55c5c53e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e0f2d640 1 -- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 msgr2=0x55c5c53e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e0f2d640 1 -- 172.21.15.16:0/3109176745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c50b23c0 con 0x55c5c53dcb40 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f48d800ec20 tx=0x7f48d8010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f99187b39ca1154 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8027060 con 0x55c5c53dcb40 2024-09-12T01:02:21.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d80047a0 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f48d8022410 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48d8014070 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e556a740 10 client.17181 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5c53f4ae0 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d80047a0 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f48d800f450 con 0x55c5c53dcb40 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e556a740 10 client.17181 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.630+0000 7f48e556a740 10 client.17181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48e556a740 20 client.17181 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c53f5e90 0x55c5c53f8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48e556a740 4 client.17181 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c534e860 con 0x55c5c53f5e90 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c53f5e90 0x55c5c53f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48bbfff640 10 client.17181 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:02:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.631+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c53f5e90 0x55c5c53f8280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc9e86a224bae124 server_cookie=4a29bb15cacd6762 in_seq=0 out_seq=0 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48bbfff640 1 -- 172.21.15.16:0/3109176745 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5c534e860 con 0x55c5c53f5e90 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48bbfff640 10 client.17181 handle_command_reply: tid=0 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 client.17181 shutdown 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 msgr2=0x55c5c53e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f48d800ec20 tx=0x7f48d8010b90 comp rx=0 tx=0).stop 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 shutdown_connections 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c53f5e90 0x55c5c53f8280 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53e2f90 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 --2- 172.21.15.16:0/3109176745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48bbfff640 0 client.17181 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:02:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 >> 172.21.15.16:0/3109176745 conn(0x55c5c53a4310 msgr2=0x55c5c53dd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 shutdown_connections 2024-09-12T01:02:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 1 -- 172.21.15.16:0/3109176745 wait complete. 2024-09-12T01:02:21.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.632+0000 7f48e556a740 20 client.17181 trim_cache size 0 max 16384 2024-09-12T01:02:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 Processor -- start 2024-09-12T01:02:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 -- start start 2024-09-12T01:02:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1af980 con 0x55c5c53db5d0 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1b14c0 con 0x55c5c53dcb40 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e556a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1b0050 con 0x55c5c53e6370 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45502/0 (socket says 172.21.15.16:45502) 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 -- 172.21.15.16:0/3936229797 learned_addr learned my addr 172.21.15.16:0/3936229797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 -- 172.21.15.16:0/3936229797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 msgr2=0x55c5c53fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 -- 172.21.15.16:0/3936229797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 msgr2=0x55c5c53e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.646+0000 7f48d3fff640 1 -- 172.21.15.16:0/3936229797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c53d6310 con 0x55c5c53db5d0 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48d3fff640 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48cc000c00 tx=0x7f48cc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a01f87d86bc764b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48c27fc640 1 -- 172.21.15.16:0/3936229797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc017d20 con 0x55c5c53db5d0 2024-09-12T01:02:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48c27fc640 1 -- 172.21.15.16:0/3936229797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc023070 con 0x55c5c53db5d0 2024-09-12T01:02:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48c27fc640 1 -- 172.21.15.16:0/3936229797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc016610 con 0x55c5c53db5d0 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 -- 172.21.15.16:0/3936229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 msgr2=0x55c5c53f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f48cc000c00 tx=0x7f48cc0047d0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 -- 172.21.15.16:0/3936229797 shutdown_connections 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53dcb40 0x55c5c53e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 --2- 172.21.15.16:0/3936229797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53db5d0 0x55c5c53f2370 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.647+0000 7f48e556a740 1 -- 172.21.15.16:0/3936229797 >> 172.21.15.16:0/3936229797 conn(0x55c5c53a4310 msgr2=0x55c5c53e1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- 172.21.15.16:0/3936229797 shutdown_connections 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- 172.21.15.16:0/3936229797 wait complete. 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- start start 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 Processor -- start 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1b0df0 con 0x55c5c53dcb40 2024-09-12T01:02:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1b0720 con 0x55c5c53db5d0 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.648+0000 7f48e556a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc1ad960 con 0x55c5c53e6370 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e556a740 10 client.0 _ll_register_callbacks cb 0x7f48cc00a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45514/0 (socket says 172.21.15.16:45514) 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 -- 172.21.15.16:0/4189810741 learned_addr learned my addr 172.21.15.16:0/4189810741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48d3fff640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48d3fff640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 -- 172.21.15.16:0/4189810741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 msgr2=0x55c5c53e00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 -- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 msgr2=0x55c5c53eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53eea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 -- 172.21.15.16:0/4189810741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c53d8960 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48d3fff640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e0f2d640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f48d8000c00 tx=0x7f48d800fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6a2b1528c04cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d801c070 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f48d8023480 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48cc023070 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f48d8027070 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e556a740 10 client.17187 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.649+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5c5460580 con 0x55c5c53dcb40 2024-09-12T01:02:21.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc023070 con 0x55c5c53dcb40 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f48d8023c10 con 0x55c5c53dcb40 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 10 client.17187 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 20 client.17187 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c5460cd0 0x55c5c54630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 4 client.17187 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5c5348d70 con 0x55c5c5460cd0 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.650+0000 7f48d3fff640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c5460cd0 0x55c5c54630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48bbfff640 10 client.17187 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48d3fff640 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c5460cd0 0x55c5c54630f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5737ca80c2c1ec5a server_cookie=ece9990584d7d13d in_seq=0 out_seq=0 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48bbfff640 1 -- 172.21.15.16:0/4189810741 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5c5348d70 con 0x55c5c5460cd0 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48bbfff640 10 client.17187 handle_command_reply: tid=0 2024-09-12T01:02:21.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 client.17187 shutdown 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 msgr2=0x55c5c53dfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f48d8000c00 tx=0x7f48d800fa90 comp rx=0 tx=0).stop 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 shutdown_connections 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c5c5460cd0 0x55c5c54630f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5c53e6370 0x55c5c53e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c5c53dcb40 0x55c5c53dfbb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 --2- 172.21.15.16:0/4189810741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5c53db5d0 0x55c5c53eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48bbfff640 0 client.17187 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 >> 172.21.15.16:0/4189810741 conn(0x55c5c53a4310 msgr2=0x55c5c53e2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 shutdown_connections 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 1 -- 172.21.15.16:0/4189810741 wait complete. 2024-09-12T01:02:21.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.651+0000 7f48e556a740 20 client.17187 trim_cache size 0 max 16384 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 msgr2=0x7f48c40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 0x7f48c40a5230 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f48d40b8f40 tx=0x7f48d408eaa0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 msgr2=0x7f48dc1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f48d4006010 tx=0x7f48d402f710 comp rx=0 tx=0).stop 2024-09-12T01:02:21.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 shutdown_connections 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48c40a2d70 0x7f48c40a5230 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48dc10c1a0 0x7f48dc1014e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48dc103720 0x7f48dc100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 --2- 172.21.15.16:0/3958052032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f48dc102e00 0x7f48dc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 >> 172.21.15.16:0/3958052032 conn(0x7f48dc0fc7a0 msgr2=0x7f48dc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 shutdown_connections 2024-09-12T01:02:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:21.652+0000 7f48e3191640 1 -- 172.21.15.16:0/3958052032 wait complete. 2024-09-12T01:02:22.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:02:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 -- 172.21.15.16:0/536333591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 msgr2=0x7f35e410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 --2- 172.21.15.16:0/536333591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e410e600 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009ca0 tx=0x7f35dc02a950 comp rx=0 tx=0).stop 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 -- 172.21.15.16:0/536333591 shutdown_connections 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 --2- 172.21.15.16:0/536333591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e410e600 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 --2- 172.21.15.16:0/536333591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 --2- 172.21.15.16:0/536333591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 0x7f35e4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 -- 172.21.15.16:0/536333591 >> 172.21.15.16:0/536333591 conn(0x7f35e40fc7a0 msgr2=0x7f35e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 -- 172.21.15.16:0/536333591 shutdown_connections 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.154+0000 7f35ec727640 1 -- 172.21.15.16:0/536333591 wait complete. 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 Processor -- start 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 -- start start 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 0x7f35e419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4111f20 con 0x7f35e410c180 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4111da0 con 0x7f35e4102da0 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ec727640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41120a0 con 0x7f35e41036c0 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45534/0 (socket says 172.21.15.16:45534) 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 -- 172.21.15.16:0/114992524 learned_addr learned my addr 172.21.15.16:0/114992524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 0x7f35e419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 -- 172.21.15.16:0/114992524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 msgr2=0x7f35e419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 -- 172.21.15.16:0/114992524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 msgr2=0x7f35e419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 0x7f35e419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35eacc4640 1 -- 172.21.15.16:0/114992524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e419f570 con 0x7f35e410c180 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.155+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35eacc4640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02ae70 tx=0x7f35dc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35db7fe640 1 -- 172.21.15.16:0/114992524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc034040 con 0x7f35e410c180 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35db7fe640 1 -- 172.21.15.16:0/114992524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35dc030920 con 0x7f35e410c180 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35db7fe640 1 -- 172.21.15.16:0/114992524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc045480 con 0x7f35e410c180 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e419f800 con 0x7f35e410c180 2024-09-12T01:02:23.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.156+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41a7140 con 0x7f35e410c180 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.158+0000 7f35db7fe640 1 -- 172.21.15.16:0/114992524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f35dc037020 con 0x7f35e410c180 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.158+0000 7f35eeb00740 1 Processor -- start 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 -- start start 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac309090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac3095d0 0x555aac309a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35db7fe640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 0x7f35d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555aac26f2f0 con 0x555aac314370 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555aac216cf0 con 0x555aac3095d0 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eeb00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555aac26b2b0 con 0x555aac30ab40 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35eacc4640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 0x7f35d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45548/0 (socket says 172.21.15.16:45548) 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3744387975 learned_addr learned my addr 172.21.15.16:0/3744387975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac3095d0 0x555aac309a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac309090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.159+0000 7f35db7fe640 1 -- 172.21.15.16:0/114992524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f35dc0f0270 con 0x7f35e410c180 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3744387975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 msgr2=0x555aac309090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac309090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3744387975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac3095d0 msgr2=0x555aac309a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac3095d0 0x555aac309a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3744387975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aac1fb3a0 con 0x555aac314370 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35eacc4640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 0x7f35d00a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0bc570 tx=0x7f35dc04e000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f35d40138f0 tx=0x7f35d400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced9a07e280c4316 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35d8ff9640 1 -- 172.21.15.16:0/3744387975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004220 con 0x555aac314370 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35d8ff9640 1 -- 172.21.15.16:0/3744387975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40043c0 con 0x555aac314370 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.160+0000 7f35d8ff9640 1 -- 172.21.15.16:0/3744387975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003810 con 0x555aac314370 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- 172.21.15.16:0/3744387975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 msgr2=0x555aac316760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f35d40138f0 tx=0x7f35d400b420 comp rx=0 tx=0).stop 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- 172.21.15.16:0/3744387975 shutdown_connections 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac316760 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac3095d0 0x555aac309a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3744387975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac309090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- 172.21.15.16:0/3744387975 >> 172.21.15.16:0/3744387975 conn(0x555aac2d2310 msgr2=0x555aac305c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- 172.21.15.16:0/3744387975 shutdown_connections 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- 172.21.15.16:0/3744387975 wait complete. 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- start start 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 Processor -- start 2024-09-12T01:02:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac3095d0 0x555aac313d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac311020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555aac317070 con 0x555aac314370 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.161+0000 7f35eeb00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555aac22a060 con 0x555aac30ab40 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35eeb00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555aac3171f0 con 0x555aac3095d0 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35eeb00740 10 client.0 _ll_register_callbacks cb 0x555aabf7d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45558/0 (socket says 172.21.15.16:45558) 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 -- 172.21.15.16:0/533838643 learned_addr learned my addr 172.21.15.16:0/533838643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac3095d0 0x555aac313d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac311020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 -- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac3095d0 msgr2=0x555aac313d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac3095d0 0x555aac313d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 -- 172.21.15.16:0/533838643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 msgr2=0x555aac311020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac311020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 -- 172.21.15.16:0/533838643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aabfe03c0 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f35d4013fd0 tx=0x7f35d400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24944a8a7505634 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4013560 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f35d4003800 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.162+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d40043c0 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35d4003c40 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 10 client.17199 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aac322b10 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40043c0 con 0x555aac314370 2024-09-12T01:02:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35d401e7d0 con 0x555aac314370 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 10 client.17199 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 10 client.17199 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 20 client.17199 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac323f10 0x555aac326300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 4 client.17199 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.163+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aac27c860 con 0x555aac323f10 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.164+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac323f10 0x555aac326300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.164+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac323f10 0x555aac326300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8779ed168d936283 server_cookie=d3b73631f3b4aa1c in_seq=0 out_seq=0 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.164+0000 7f35cdffb640 10 client.17199 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35cdffb640 1 -- 172.21.15.16:0/533838643 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555aac27c860 con 0x555aac323f10 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35cdffb640 10 client.17199 handle_command_reply: tid=0 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 client.17199 shutdown 2024-09-12T01:02:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 msgr2=0x555aac311610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f35d4013fd0 tx=0x7f35d400ee90 comp rx=0 tx=0).stop 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 shutdown_connections 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac314370 0x555aac311610 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac323f10 0x555aac326300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac311020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 --2- 172.21.15.16:0/533838643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac3095d0 0x555aac313d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 >> 172.21.15.16:0/533838643 conn(0x555aac2d2310 msgr2=0x555aac30b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35cdffb640 0 client.17199 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 shutdown_connections 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 1 -- 172.21.15.16:0/533838643 wait complete. 2024-09-12T01:02:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.165+0000 7f35eeb00740 20 client.17199 trim_cache size 0 max 16384 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.179+0000 7f35eeb00740 1 Processor -- start 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 -- start start 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac320310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac314370 0x555aac32b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41af9b0 con 0x555aac3095d0 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41b14f0 con 0x555aac314370 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35eeb00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41b0080 con 0x555aac30ab40 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45570/0 (socket says 172.21.15.16:45570) 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 -- 172.21.15.16:0/2814368358 learned_addr learned my addr 172.21.15.16:0/2814368358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac320310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 -- 172.21.15.16:0/2814368358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 msgr2=0x555aac320310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac320310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 -- 172.21.15.16:0/2814368358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac314370 msgr2=0x555aac32b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac314370 0x555aac32b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 -- 172.21.15.16:0/2814368358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aac30a760 con 0x555aac3095d0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.180+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005cd0 tx=0x7f35d4013360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad6620d56279a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35cffff640 1 -- 172.21.15.16:0/2814368358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400b8c0 con 0x555aac3095d0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35cffff640 1 -- 172.21.15.16:0/2814368358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40043a0 con 0x555aac3095d0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35cffff640 1 -- 172.21.15.16:0/2814368358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d401e9b0 con 0x555aac3095d0 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 -- 172.21.15.16:0/2814368358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 msgr2=0x555aac305e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005cd0 tx=0x7f35d4013360 comp rx=0 tx=0).stop 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 -- 172.21.15.16:0/2814368358 shutdown_connections 2024-09-12T01:02:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac314370 0x555aac32b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac30ab40 0x555aac320310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 --2- 172.21.15.16:0/2814368358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac305e10 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.181+0000 7f35eeb00740 1 -- 172.21.15.16:0/2814368358 >> 172.21.15.16:0/2814368358 conn(0x555aac2d2310 msgr2=0x555aac305700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- 172.21.15.16:0/2814368358 shutdown_connections 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- 172.21.15.16:0/2814368358 wait complete. 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- start start 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 Processor -- start 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac30e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac314370 0x555aac30ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41b0e20 con 0x555aac3095d0 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41b0750 con 0x555aac30ab40 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41ad990 con 0x555aac314370 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35eeb00740 10 client.0 _ll_register_callbacks cb 0x7f35e000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:23.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35e9cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45574/0 (socket says 172.21.15.16:45574) 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.182+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3410240082 learned_addr learned my addr 172.21.15.16:0/3410240082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac314370 0x555aac30ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac30e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac314370 msgr2=0x555aac30ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac314370 0x555aac30ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3410240082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 msgr2=0x555aac30e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac30e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 -- 172.21.15.16:0/3410240082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aac38daf0 con 0x555aac3095d0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35e9cc2640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005ad0 tx=0x7f35d4013880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcb6c14151cd02d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d403e030 con 0x555aac3095d0 2024-09-12T01:02:23.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f35d402a040 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d40043a0 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35d4003de0 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35eeb00740 10 client.17205 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555aac38ed70 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.183+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40043a0 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35d4035070 con 0x555aac3095d0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 10 client.17205 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 20 client.17205 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac38f1a0 0x555aac391590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 4 client.17205 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555aac276d70 con 0x555aac38f1a0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac38f1a0 0x555aac391590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35ea4c3640 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac38f1a0 0x555aac391590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f6f4d732acef032 server_cookie=d0041f17bd1d6927 in_seq=0 out_seq=0 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35cdffb640 10 client.17205 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35cdffb640 1 -- 172.21.15.16:0/3410240082 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555aac276d70 con 0x555aac38f1a0 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35cdffb640 10 client.17205 handle_command_reply: tid=0 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.184+0000 7f35eeb00740 1 client.17205 shutdown 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 msgr2=0x555aac30e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005ad0 tx=0x7f35d4013880 comp rx=0 tx=0).stop 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 shutdown_connections 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aac314370 0x555aac30ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555aac38f1a0 0x555aac391590 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555aac30ab40 0x555aac30e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 --2- 172.21.15.16:0/3410240082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555aac3095d0 0x555aac30e180 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 >> 172.21.15.16:0/3410240082 conn(0x555aac2d2310 msgr2=0x555aac3044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35cdffb640 0 client.17205 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 shutdown_connections 2024-09-12T01:02:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 1 -- 172.21.15.16:0/3410240082 wait complete. 2024-09-12T01:02:23.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35eeb00740 20 client.17205 trim_cache size 0 max 16384 2024-09-12T01:02:23.197 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 msgr2=0x7f35d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 0x7f35d00a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0bc570 tx=0x7f35dc04e000 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 msgr2=0x7f35e419ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02ae70 tx=0x7f35dc02f5b0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 shutdown_connections 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f35d00a2d70 0x7f35d00a5230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e410c180 0x7f35e419ee90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e41036c0 0x7f35e419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 --2- 172.21.15.16:0/114992524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35e4102da0 0x7f35e419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.185+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 >> 172.21.15.16:0/114992524 conn(0x7f35e40fc7a0 msgr2=0x7f35e4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.186+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 shutdown_connections 2024-09-12T01:02:23.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:23.186+0000 7f35ec727640 1 -- 172.21.15.16:0/114992524 wait complete. 2024-09-12T01:02:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:23 smithi016 ceph-mon[30972]: pgmap v192: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:23.418 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:23 smithi159 ceph-mon[35566]: pgmap v192: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:23.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:23 smithi099 ceph-mon[35654]: pgmap v192: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:23.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:02:24.136 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 -- 172.21.15.16:0/1398904279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304100a80 msgr2=0x7f9304100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 --2- 172.21.15.16:0/1398904279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304100a80 0x7f9304100ee0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009930 tx=0x7f92f802a420 comp rx=0 tx=0).stop 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 -- 172.21.15.16:0/1398904279 shutdown_connections 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 --2- 172.21.15.16:0/1398904279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304101420 0x7f930410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 --2- 172.21.15.16:0/1398904279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304100a80 0x7f9304100ee0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 --2- 172.21.15.16:0/1398904279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304106a80 0x7f9304106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 -- 172.21.15.16:0/1398904279 >> 172.21.15.16:0/1398904279 conn(0x7f93040fc7a0 msgr2=0x7f93040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 -- 172.21.15.16:0/1398904279 shutdown_connections 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.602+0000 7f930a0f8640 1 -- 172.21.15.16:0/1398904279 wait complete. 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.603+0000 7f930a0f8640 1 Processor -- start 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.603+0000 7f930a0f8640 1 -- start start 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304100a80 0x7f93041973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304106a80 0x7f930419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9304110e80 con 0x7f9304101420 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9304110d00 con 0x7f9304106a80 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9304111000 con 0x7f9304100a80 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45582/0 (socket says 172.21.15.16:45582) 2024-09-12T01:02:24.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 -- 172.21.15.16:0/4027242975 learned_addr learned my addr 172.21.15.16:0/4027242975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9303fff640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304106a80 0x7f930419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f93037fe640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304100a80 0x7f93041973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 -- 172.21.15.16:0/4027242975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304100a80 msgr2=0x7f93041973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304100a80 0x7f93041973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 -- 172.21.15.16:0/4027242975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304106a80 msgr2=0x7f930419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304106a80 0x7f930419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 -- 172.21.15.16:0/4027242975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930419c350 con 0x7f9304101420 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9302ffd640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009230 tx=0x7f92f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9300ff9640 1 -- 172.21.15.16:0/4027242975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f803f030 con 0x7f9304101420 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9300ff9640 1 -- 172.21.15.16:0/4027242975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f8003ad0 con 0x7f9304101420 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930419c5e0 con 0x7f9304101420 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.604+0000 7f9300ff9640 1 -- 172.21.15.16:0/4027242975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8035070 con 0x7f9304101420 2024-09-12T01:02:24.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.605+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93041a3f20 con 0x7f9304101420 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9300ff9640 1 -- 172.21.15.16:0/4027242975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f92f8042030 con 0x7f9304101420 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 Processor -- start 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 -- start start 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4914655d0 0x55f491465a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 0x55f491474970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f4913cb2f0 con 0x55f491466b40 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f491372cf0 con 0x55f4914655d0 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f930c4d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4913c72b0 con 0x55f491472580 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45586/0 (socket says 172.21.15.16:45586) 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9303fff640 1 -- 172.21.15.16:0/2343029764 learned_addr learned my addr 172.21.15.16:0/2343029764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:24.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9302ffd640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 0x55f491474970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.607+0000 7f9300ff9640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 0x7f92ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f93037fe640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4914655d0 0x55f491465a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 -- 172.21.15.16:0/2343029764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 msgr2=0x55f491474970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 0x55f491474970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 -- 172.21.15.16:0/2343029764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4914655d0 msgr2=0x55f491465a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4914655d0 0x55f491465a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 -- 172.21.15.16:0/2343029764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4913573a0 con 0x55f491466b40 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9302ffd640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 0x55f491474970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 0x7f92ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9300ff9640 1 -- 172.21.15.16:0/4027242975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f92f80ef910 con 0x7f9304101420 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f9303fff640 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f92fc010510 tx=0x7f92fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de6631d29512fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f92e9ffb640 1 -- 172.21.15.16:0/2343029764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc01d400 con 0x55f491466b40 2024-09-12T01:02:24.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f92e9ffb640 1 -- 172.21.15.16:0/2343029764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc01da80 con 0x55f491466b40 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.608+0000 7f92e9ffb640 1 -- 172.21.15.16:0/2343029764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc01c7e0 con 0x55f491466b40 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- 172.21.15.16:0/2343029764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 msgr2=0x55f491465090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f92fc010510 tx=0x7f92fc009f90 comp rx=0 tx=0).stop 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f9303fff640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 0x7f92ec0a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f92fc021000 tx=0x7f92fc01e000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- 172.21.15.16:0/2343029764 shutdown_connections 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f491472580 0x55f491474970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4914655d0 0x55f491465a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2343029764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491466b40 0x55f491465090 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- 172.21.15.16:0/2343029764 >> 172.21.15.16:0/2343029764 conn(0x55f49142e310 msgr2=0x55f491461c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- 172.21.15.16:0/2343029764 shutdown_connections 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- 172.21.15.16:0/2343029764 wait complete. 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.609+0000 7f930c4d1740 1 -- start start 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 Processor -- start 2024-09-12T01:02:24.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f491471f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f491475280 con 0x55f491472580 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f491386060 con 0x55f491466b40 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f491475400 con 0x55f4914655d0 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f930c4d1740 10 client.0 _ll_register_callbacks cb 0x55f4910d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45594/0 (socket says 172.21.15.16:45594) 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 -- 172.21.15.16:0/2542018148 learned_addr learned my addr 172.21.15.16:0/2542018148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9302ffd640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f93037fe640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f491471f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 -- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 msgr2=0x55f491471f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f491471f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 -- 172.21.15.16:0/2542018148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 msgr2=0x55f49146f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 -- 172.21.15.16:0/2542018148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49113c3c0 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f9303fff640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f92fc01c3e0 tx=0x7f92fc01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3125d65e11a2c6e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.610+0000 7f92e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc01cd10 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f92fc01d400 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc01da80 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92fc01d600 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 10 client.17217 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f491480d20 con 0x55f491472580 2024-09-12T01:02:24.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc01da80 con 0x55f491472580 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92fc00f040 con 0x55f491472580 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 20 client.17217 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f491482070 0x55f491484460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 4 client.17217 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.611+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4913d8860 con 0x55f491482070 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.612+0000 7f93037fe640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f491482070 0x55f491484460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.612+0000 7f92e2ffd640 10 client.17217 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.612+0000 7f93037fe640 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f491482070 0x55f491484460 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e438a235f36441d8 server_cookie=1ff4ca2800358ba4 in_seq=0 out_seq=0 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2542018148 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4913d8860 con 0x55f491482070 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f92e2ffd640 10 client.17217 handle_command_reply: tid=0 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 client.17217 shutdown 2024-09-12T01:02:24.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 msgr2=0x55f49146f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f92fc01c3e0 tx=0x7f92fc01c870 comp rx=0 tx=0).stop 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 shutdown_connections 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f491482070 0x55f491484460 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146f820 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f92e2ffd640 0 client.17217 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2542018148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f491471f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 >> 172.21.15.16:0/2542018148 conn(0x55f49142e310 msgr2=0x55f491467960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 shutdown_connections 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 1 -- 172.21.15.16:0/2542018148 wait complete. 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.613+0000 7f930c4d1740 20 client.17217 trim_cache size 0 max 16384 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 Processor -- start 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 -- start start 2024-09-12T01:02:24.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ab5e0 con 0x55f491472580 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ad120 con 0x55f491466b40 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f930c4d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93041abcb0 con 0x55f4914655d0 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45604/0 (socket says 172.21.15.16:45604) 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.626+0000 7f9303fff640 1 -- 172.21.15.16:0/1209585205 learned_addr learned my addr 172.21.15.16:0/1209585205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f93037fe640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9302ffd640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 -- 172.21.15.16:0/1209585205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 msgr2=0x55f49147e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 -- 172.21.15.16:0/1209585205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 msgr2=0x55f49146e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 -- 172.21.15.16:0/1209585205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f491471ba0 con 0x55f491472580 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f93037fe640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f9303fff640 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009c20 tx=0x7f92fc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931948df759bd189 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f92e97fa640 1 -- 172.21.15.16:0/1209585205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc046070 con 0x55f491472580 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f92e97fa640 1 -- 172.21.15.16:0/1209585205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc02b070 con 0x55f491472580 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.627+0000 7f92e97fa640 1 -- 172.21.15.16:0/1209585205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc01da00 con 0x55f491472580 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- 172.21.15.16:0/1209585205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 msgr2=0x55f491489c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009c20 tx=0x7f92fc01c8e0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- 172.21.15.16:0/1209585205 shutdown_connections 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f491489c20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 --2- 172.21.15.16:0/1209585205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- 172.21.15.16:0/1209585205 >> 172.21.15.16:0/1209585205 conn(0x55f49142e310 msgr2=0x55f49146db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- 172.21.15.16:0/1209585205 shutdown_connections 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- 172.21.15.16:0/1209585205 wait complete. 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 -- start start 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.628+0000 7f930c4d1740 1 Processor -- start 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93041aca50 con 0x55f491472580 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ac380 con 0x55f491466b40 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93041a95c0 con 0x55f4914655d0 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f930c4d1740 10 client.0 _ll_register_callbacks cb 0x7f92fc020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45612/0 (socket says 172.21.15.16:45612) 2024-09-12T01:02:24.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 -- 172.21.15.16:0/2747822043 learned_addr learned my addr 172.21.15.16:0/2747822043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9302ffd640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f93037fe640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 -- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 msgr2=0x55f49147ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 -- 172.21.15.16:0/2747822043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 msgr2=0x55f49146c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 -- 172.21.15.16:0/2747822043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49146ce30 con 0x55f491472580 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f9303fff640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009640 tx=0x7f92fc01d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d59a7e35cd5589d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.629+0000 7f92e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc046040 con 0x55f491472580 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f92fc01d7a0 con 0x55f491472580 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc02b070 con 0x55f491472580 2024-09-12T01:02:24.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92fc052050 con 0x55f491472580 2024-09-12T01:02:24.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 10 client.17223 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:24.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4914ef120 con 0x55f491472580 2024-09-12T01:02:24.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92fc02b070 con 0x55f491472580 2024-09-12T01:02:24.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92fc049070 con 0x55f491472580 2024-09-12T01:02:24.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 10 client.17223 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:24.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:24.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 20 client.17223 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f4914ef780 0x55f4914f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 4 client.17223 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4913d2d70 con 0x55f4914ef780 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.630+0000 7f93037fe640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f4914ef780 0x55f4914f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f93037fe640 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f4914ef780 0x55f4914f1b70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650cfaaa5a072309 server_cookie=89c75b7bd772fd11 in_seq=0 out_seq=0 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f92e2ffd640 10 client.17223 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f92e2ffd640 1 -- 172.21.15.16:0/2747822043 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4913d2d70 con 0x55f4914ef780 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f92e2ffd640 10 client.17223 handle_command_reply: tid=0 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 client.17223 shutdown 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 msgr2=0x55f49146c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009640 tx=0x7f92fc01d400 comp rx=0 tx=0).stop 2024-09-12T01:02:24.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 shutdown_connections 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f4914ef780 0x55f4914f1b70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f92e2ffd640 0 client.17223 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f491472580 0x55f49146c630 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f491466b40 0x55f49146c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 --2- 172.21.15.16:0/2747822043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4914655d0 0x55f49147ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 >> 172.21.15.16:0/2747822043 conn(0x55f49142e310 msgr2=0x55f49146e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 shutdown_connections 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.631+0000 7f930c4d1740 1 -- 172.21.15.16:0/2747822043 wait complete. 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930c4d1740 20 client.17223 trim_cache size 0 max 16384 2024-09-12T01:02:24.646 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 msgr2=0x7f92ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 0x7f92ec0a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f92fc021000 tx=0x7f92fc01e000 comp rx=0 tx=0).stop 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 msgr2=0x7f93041978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009230 tx=0x7f92f8004270 comp rx=0 tx=0).stop 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 shutdown_connections 2024-09-12T01:02:24.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f92ec0a2cf0 0x7f92ec0a51b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304106a80 0x7f930419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9304101420 0x7f93041978e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 --2- 172.21.15.16:0/4027242975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9304100a80 0x7f93041973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 >> 172.21.15.16:0/4027242975 conn(0x7f93040fc7a0 msgr2=0x7f93040fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 shutdown_connections 2024-09-12T01:02:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:24.632+0000 7f930a0f8640 1 -- 172.21.15.16:0/4027242975 wait complete. 2024-09-12T01:02:25.382 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:25 smithi016 ceph-mon[30972]: pgmap v193: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:25.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:25 smithi099 ceph-mon[35654]: pgmap v193: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:25 smithi159 ceph-mon[35566]: pgmap v193: 97 pgs: 97 active+clean; 6.6 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 KiB/s wr, 9 op/s 2024-09-12T01:02:26.415 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:02:26.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:02:26.688 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 -- 172.21.15.16:0/4204927964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0101420 msgr2=0x7fb7b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 --2- 172.21.15.16:0/4204927964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0101420 0x7fb7b010d950 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009cd0 tx=0x7fb7a802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 -- 172.21.15.16:0/4204927964 shutdown_connections 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 --2- 172.21.15.16:0/4204927964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0101420 0x7fb7b010d950 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 --2- 172.21.15.16:0/4204927964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0100a80 0x7fb7b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 --2- 172.21.15.16:0/4204927964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 0x7fb7b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 -- 172.21.15.16:0/4204927964 >> 172.21.15.16:0/4204927964 conn(0x7fb7b00fc7a0 msgr2=0x7fb7b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 -- 172.21.15.16:0/4204927964 shutdown_connections 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.182+0000 7fb7b6841640 1 -- 172.21.15.16:0/4204927964 wait complete. 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 Processor -- start 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 -- start start 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0101420 0x7fb7b01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 0x7fb7b019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0111fa0 con 0x7fb7b0100a80 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0111e20 con 0x7fb7b0106a80 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.183+0000 7fb7b6841640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0112120 con 0x7fb7b0101420 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45630/0 (socket says 172.21.15.16:45630) 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 -- 172.21.15.16:0/1623256743 learned_addr learned my addr 172.21.15.16:0/1623256743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 0x7fb7b019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0101420 0x7fb7b01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 -- 172.21.15.16:0/1623256743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0101420 msgr2=0x7fb7b01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0101420 0x7fb7b01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 -- 172.21.15.16:0/1623256743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 msgr2=0x7fb7b019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 0x7fb7b019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 -- 172.21.15.16:0/1623256743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b019d360 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7affff640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d950 tx=0x7fb7a000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7ad7fa640 1 -- 172.21.15.16:0/1623256743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0017070 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7ad7fa640 1 -- 172.21.15.16:0/1623256743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a000de80 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b019d650 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7ad7fa640 1 -- 172.21.15.16:0/1623256743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000f620 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.184+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b01a4f30 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7ad7fa640 1 -- 172.21.15.16:0/1623256743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb7a0009c30 con 0x7fb7b0100a80 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 Processor -- start 2024-09-12T01:02:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 -- start start 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f951e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f963540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e5f8b7280 con 0x562e5f952360 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e5f85eb10 con 0x562e5f951a40 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e5f8b3240 con 0x562e5f961150 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45634/0 (socket says 172.21.15.16:45634) 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7affff640 1 -- 172.21.15.16:0/844755807 learned_addr learned my addr 172.21.15.16:0/844755807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.187+0000 7fb7ad7fa640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 0x7fb7980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f963540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f951e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 0x7fb7980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7ad7fa640 1 -- 172.21.15.16:0/1623256743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7a00ceef0 con 0x7fb7b0100a80 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 -- 172.21.15.16:0/844755807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 msgr2=0x562e5f963540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f963540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 -- 172.21.15.16:0/844755807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 msgr2=0x562e5f951e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f951e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 -- 172.21.15.16:0/844755807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5f86b1f0 con 0x562e5f952360 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f963540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7affff640 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb7a009b140 tx=0x7fb7a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab03a8764e7b27a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 0x7fb7980a5230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004ab0 tx=0x7fb7a8004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7967fc640 1 -- 172.21.15.16:0/844755807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0020070 con 0x562e5f952360 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7967fc640 1 -- 172.21.15.16:0/844755807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a009bb40 con 0x562e5f952360 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.188+0000 7fb7967fc640 1 -- 172.21.15.16:0/844755807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0097ab0 con 0x562e5f952360 2024-09-12T01:02:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/844755807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 msgr2=0x562e5f960c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb7a009b140 tx=0x7fb7a0000bc0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/844755807 shutdown_connections 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f963540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f960c10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/844755807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f951e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/844755807 >> 172.21.15.16:0/844755807 conn(0x562e5f94e0f0 msgr2=0x562e5f94e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/844755807 shutdown_connections 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/844755807 wait complete. 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.189+0000 7fb7b8c1a740 1 -- start start 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 Processor -- start 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f95eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e5f963cc0 con 0x562e5f952360 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e5f871f10 con 0x562e5f951a40 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e5f963e40 con 0x562e5f961150 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b8c1a740 10 client.0 _ll_register_callbacks cb 0x562e5f83aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45638/0 (socket says 172.21.15.16:45638) 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3130052644 learned_addr learned my addr 172.21.15.16:0/3130052644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7affff640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f95eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 msgr2=0x562e5f95eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f95eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3130052644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 msgr2=0x562e5f95e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3130052644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5f6286a0 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4012780 tx=0x7fb7a4012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00d83a7ae497648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.190+0000 7fb78f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4010dc0 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a009bb40 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7a40047a0 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7a40049a0 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 10 client.17235 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e5f96fa40 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a009bb40 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7a400e600 con 0x562e5f952360 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 10 client.17235 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 20 client.17235 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f970f00 0x562e5f9733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 4 client.17235 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.191+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e5f8c47f0 con 0x562e5f970f00 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.192+0000 7fb7affff640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f970f00 0x562e5f9733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.192+0000 7fb78f7fe640 10 client.17235 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.192+0000 7fb7affff640 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f970f00 0x562e5f9733a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6658240ab01b213 server_cookie=ab5a1c54248de93c in_seq=0 out_seq=0 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3130052644 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e5f8c47f0 con 0x562e5f970f00 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb78f7fe640 10 client.17235 handle_command_reply: tid=0 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 client.17235 shutdown 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 msgr2=0x562e5f95e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4012780 tx=0x7fb7a4012c40 comp rx=0 tx=0).stop 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 shutdown_connections 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f970f00 0x562e5f9733a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f95eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f95e5d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb78f7fe640 0 client.17235 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3130052644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 >> 172.21.15.16:0/3130052644 conn(0x562e5f94e0f0 msgr2=0x562e5f953e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 shutdown_connections 2024-09-12T01:02:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3130052644 wait complete. 2024-09-12T01:02:27.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.193+0000 7fb7b8c1a740 20 client.17235 trim_cache size 0 max 16384 2024-09-12T01:02:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 Processor -- start 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 -- start start 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ab600 con 0x562e5f952360 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ad140 con 0x562e5f951a40 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01abcd0 con 0x562e5f961150 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45652/0 (socket says 172.21.15.16:45652) 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 -- 172.21.15.16:0/1051776876 learned_addr learned my addr 172.21.15.16:0/1051776876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7affff640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 -- 172.21.15.16:0/1051776876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 msgr2=0x562e5f9791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9791b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 -- 172.21.15.16:0/1051776876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 msgr2=0x562e5f95fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7af7fe640 1 -- 172.21.15.16:0/1051776876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5f948680 con 0x562e5f952360 2024-09-12T01:02:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.211+0000 7fb7affff640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400ea20 tx=0x7fb7a40108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf6ffcb3a4de16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb795ffb640 1 -- 172.21.15.16:0/1051776876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400ee90 con 0x562e5f952360 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb795ffb640 1 -- 172.21.15.16:0/1051776876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4030070 con 0x562e5f952360 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb795ffb640 1 -- 172.21.15.16:0/1051776876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40239a0 con 0x562e5f952360 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/1051776876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 msgr2=0x562e5f978c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400ea20 tx=0x7fb7a40108e0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/1051776876 shutdown_connections 2024-09-12T01:02:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f978c70 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/1051776876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f95fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.212+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/1051776876 >> 172.21.15.16:0/1051776876 conn(0x562e5f94e0f0 msgr2=0x562e5f95f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/1051776876 shutdown_connections 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/1051776876 wait complete. 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- start start 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 Processor -- start 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f9655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01aca70 con 0x562e5f952360 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01ac3a0 con 0x562e5f951a40 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b01a95e0 con 0x562e5f961150 2024-09-12T01:02:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.213+0000 7fb7b8c1a740 10 client.0 _ll_register_callbacks cb 0x7fb7a400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45660/0 (socket says 172.21.15.16:45660) 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3149966838 learned_addr learned my addr 172.21.15.16:0/3149966838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7b4dde640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7affff640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f9655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 msgr2=0x562e5f9dac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9dac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3149966838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 msgr2=0x562e5f9655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f9655a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 -- 172.21.15.16:0/3149966838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5f9db180 con 0x562e5f952360 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7af7fe640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400eb40 tx=0x7fb7a400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08d3a57adb15c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb78f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4038060 con 0x562e5f952360 2024-09-12T01:02:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a4030070 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fb7a400e3b0 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7a400e5d0 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7b8c1a740 10 client.17241 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.214+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e5f9dc4c0 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4030070 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7a403d580 con 0x562e5f952360 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 10 client.17241 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 20 client.17241 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f9dcaf0 0x562e5f9def10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 4 client.17241 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e5f8bed00 con 0x562e5f9dcaf0 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.215+0000 7fb7affff640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f9dcaf0 0x562e5f9def10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.216+0000 7fb78f7fe640 10 client.17241 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.216+0000 7fb7affff640 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f9dcaf0 0x562e5f9def10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b6a59f1b8c97f6 server_cookie=ece9e12829d79b0 in_seq=0 out_seq=0 2024-09-12T01:02:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb78f7fe640 1 -- 172.21.15.16:0/3149966838 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e5f8bed00 con 0x562e5f9dcaf0 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb78f7fe640 10 client.17241 handle_command_reply: tid=0 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 client.17241 shutdown 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 msgr2=0x562e5f965ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400eb40 tx=0x7fb7a400eec0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 shutdown_connections 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e5f9dcaf0 0x562e5f9def10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5f961150 0x562e5f9dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb78f7fe640 0 client.17241 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e5f952360 0x562e5f965ae0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 --2- 172.21.15.16:0/3149966838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e5f951a40 0x562e5f9655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 >> 172.21.15.16:0/3149966838 conn(0x562e5f94e0f0 msgr2=0x562e5f94d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 shutdown_connections 2024-09-12T01:02:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 1 -- 172.21.15.16:0/3149966838 wait complete. 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.217+0000 7fb7b8c1a740 20 client.17241 trim_cache size 0 max 16384 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a5266971-7b9c-4d3f-9fe4-16186de24312", 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:02:27.221 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 msgr2=0x7fb7980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 0x7fb7980a5230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004ab0 tx=0x7fb7a8004a40 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 msgr2=0x7fb7b01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000d950 tx=0x7fb7a000dd50 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 shutdown_connections 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb7980a2d70 0x7fb7980a5230 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b0106a80 0x7fb7b019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb7b0101420 0x7fb7b01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 --2- 172.21.15.16:0/1623256743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b0100a80 0x7fb7b01983b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 >> 172.21.15.16:0/1623256743 conn(0x7fb7b00fc7a0 msgr2=0x7fb7b00fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 shutdown_connections 2024-09-12T01:02:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:27.218+0000 7fb7b6841640 1 -- 172.21.15.16:0/1623256743 wait complete. 2024-09-12T01:02:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:27 smithi016 ceph-mon[30972]: pgmap v194: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-12T01:02:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:27 smithi099 ceph-mon[35654]: pgmap v194: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-12T01:02:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:27 smithi159 ceph-mon[35566]: pgmap v194: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.1 KiB/s wr, 13 op/s 2024-09-12T01:02:27.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:02:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:28.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:02:28.463 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:28 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:02:28.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:28 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:28 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:02:28.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:28 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:28 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:02:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:28 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:28 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:02:28.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:28 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:28 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:02:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:28 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:28 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:02:28.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:28 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:02:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1367720260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0106a70 msgr2=0x7fc6e0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1367720260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0106a70 0x7fc6e0106e50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009930 tx=0x7fc6d402a420 comp rx=0 tx=0).stop 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1367720260 shutdown_connections 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1367720260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0101410 0x7fc6e010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1367720260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0100a70 0x7fc6e0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1367720260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0106a70 0x7fc6e0106e50 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1367720260 >> 172.21.15.16:0/1367720260 conn(0x7fc6e00fc790 msgr2=0x7fc6e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1367720260 shutdown_connections 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.755+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1367720260 wait complete. 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 Processor -- start 2024-09-12T01:02:28.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 -- start start 2024-09-12T01:02:28.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0100a70 0x7fc6e00fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0106a70 0x7fc6e00fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110e30 con 0x7fc6e0101410 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110cb0 con 0x7fc6e0100a70 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.756+0000 7fc6e6e09640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110fb0 con 0x7fc6e0106a70 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56018/0 (socket says 172.21.15.16:56018) 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 -- 172.21.15.16:0/1947994082 learned_addr learned my addr 172.21.15.16:0/1947994082 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0100a70 0x7fc6e00fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0106a70 0x7fc6e00fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 -- 172.21.15.16:0/1947994082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0106a70 msgr2=0x7fc6e00fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0106a70 0x7fc6e00fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 -- 172.21.15.16:0/1947994082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0100a70 msgr2=0x7fc6e00fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0100a70 0x7fc6e00fce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 -- 172.21.15.16:0/1947994082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e01a6980 con 0x7fc6e0101410 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6dffff640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00d950 tx=0x7fc6cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6ddffb640 1 -- 172.21.15.16:0/1947994082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc017030 con 0x7fc6e0101410 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6ddffb640 1 -- 172.21.15.16:0/1947994082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc004020 con 0x7fc6e0101410 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6ddffb640 1 -- 172.21.15.16:0/1947994082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0047a0 con 0x7fc6e0101410 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.757+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e01a6c70 con 0x7fc6e0101410 2024-09-12T01:02:28.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.758+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a7160 con 0x7fc6e0101410 2024-09-12T01:02:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6ddffb640 1 -- 172.21.15.16:0/1947994082 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc6cc0041d0 con 0x7fc6e0101410 2024-09-12T01:02:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6e91e2740 1 Processor -- start 2024-09-12T01:02:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6e91e2740 1 -- start start 2024-09-12T01:02:28.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.760+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a6f680 0x55e9e0a71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9e09ca2e0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9e0971ce0 con 0x55e9e0a644a0 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9e09c62a0 con 0x55e9e0a6f680 2024-09-12T01:02:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56020/0 (socket says 172.21.15.16:56020) 2024-09-12T01:02:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 -- 172.21.15.16:0/2301511663 learned_addr learned my addr 172.21.15.16:0/2301511663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6dffff640 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a6f680 0x55e9e0a71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 -- 172.21.15.16:0/2301511663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a6f680 msgr2=0x55e9e0a71a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a6f680 0x55e9e0a71a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 -- 172.21.15.16:0/2301511663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 msgr2=0x55e9e0a64880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6ddffb640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 0x7fc6d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a64880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 -- 172.21.15.16:0/2301511663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e0956440 con 0x55e9e0a64dc0 2024-09-12T01:02:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 0x7fc6d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6ddffb640 1 -- 172.21.15.16:0/1947994082 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc6cc0cee00 con 0x7fc6e0101410 2024-09-12T01:02:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.761+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402a940 tx=0x7fc6d4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1edb99574b1975b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6caffd640 1 -- 172.21.15.16:0/2301511663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4040070 con 0x55e9e0a64dc0 2024-09-12T01:02:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6caffd640 1 -- 172.21.15.16:0/2301511663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d40319f0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6caffd640 1 -- 172.21.15.16:0/2301511663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4035040 con 0x55e9e0a64dc0 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2301511663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 msgr2=0x55e9e0a6f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402a940 tx=0x7fc6d4005c00 comp rx=0 tx=0).stop 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.762+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 0x7fc6d00a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d8004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2301511663 shutdown_connections 2024-09-12T01:02:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a6f680 0x55e9e0a71a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6f140 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2301511663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2301511663 >> 172.21.15.16:0/2301511663 conn(0x55e9e0a60a30 msgr2=0x55e9e0a60e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2301511663 shutdown_connections 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2301511663 wait complete. 2024-09-12T01:02:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.763+0000 7fc6e91e2740 1 -- start start 2024-09-12T01:02:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 Processor -- start 2024-09-12T01:02:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a6ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 0x55e9e0a6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e9e0a72250 con 0x55e9e0a6f680 2024-09-12T01:02:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9e0985050 con 0x55e9e0a64dc0 2024-09-12T01:02:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9e0a723d0 con 0x55e9e0a644a0 2024-09-12T01:02:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e91e2740 10 client.0 _ll_register_callbacks cb 0x55e9e09c5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:38368/0 (socket says 172.21.15.16:38368) 2024-09-12T01:02:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6dffff640 1 -- 172.21.15.16:0/2411859935 learned_addr learned my addr 172.21.15.16:0/2411859935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e4ba5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a6ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.764+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 0x55e9e0a6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 -- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 msgr2=0x55e9e0a6ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a6ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 -- 172.21.15.16:0/2411859935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 msgr2=0x55e9e0a6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 0x55e9e0a6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 -- 172.21.15.16:0/2411859935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e073b3c0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 0x55e9e0a6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6dffff640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0cc5c0 tx=0x7fc6cc09b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea02529cd52f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6bffff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:02:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0cc9e0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc6cc09a780 con 0x55e9e0a64dc0 2024-09-12T01:02:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d40319f0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6cc09ba30 con 0x55e9e0a64dc0 2024-09-12T01:02:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6e91e2740 10 client.26405 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.765+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9e0a7dae0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d40319f0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6cc0b4120 con 0x55e9e0a64dc0 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 10 client.26405 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 20 client.26405 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a7efa0 0x55e9e0a81390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 4 client.26405 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e09d7850 con 0x55e9e0a7efa0 2024-09-12T01:02:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.766+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a7efa0 0x55e9e0a81390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a7efa0 0x55e9e0a81390 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacff6054b056f70 server_cookie=4a1df80bc298e872 in_seq=0 out_seq=0 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6bffff640 10 client.26405 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6bffff640 1 -- 172.21.15.16:0/2411859935 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9e09d7850 con 0x55e9e0a7efa0 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6bffff640 10 client.26405 handle_command_reply: tid=0 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6e91e2740 1 client.26405 shutdown 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 msgr2=0x55e9e0a6c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.767+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0cc5c0 tx=0x7fc6cc09b100 comp rx=0 tx=0).stop 2024-09-12T01:02:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 shutdown_connections 2024-09-12T01:02:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a7efa0 0x55e9e0a81390 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a6f680 0x55e9e0a6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a6c130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6bffff640 0 client.26405 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/2411859935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a6ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 >> 172.21.15.16:0/2411859935 conn(0x55e9e0a60a30 msgr2=0x55e9e0a66760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 shutdown_connections 2024-09-12T01:02:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 1 -- 172.21.15.16:0/2411859935 wait complete. 2024-09-12T01:02:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.768+0000 7fc6e91e2740 20 client.26405 trim_cache size 0 max 16384 2024-09-12T01:02:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.786+0000 7fc6e91e2740 1 Processor -- start 2024-09-12T01:02:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.786+0000 7fc6e91e2740 1 -- start start 2024-09-12T01:02:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a60a30 0x55e9e0a677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a86c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01afa80 con 0x55e9e0a64dc0 2024-09-12T01:02:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01b15c0 con 0x55e9e0a644a0 2024-09-12T01:02:28.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01b0150 con 0x55e9e0a60a30 2024-09-12T01:02:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e53a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e53a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56048/0 (socket says 172.21.15.16:56048) 2024-09-12T01:02:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e53a6640 1 -- 172.21.15.16:0/3739488374 learned_addr learned my addr 172.21.15.16:0/3739488374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a86c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a60a30 0x55e9e0a677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 -- 172.21.15.16:0/3739488374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a60a30 msgr2=0x55e9e0a677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a60a30 0x55e9e0a677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 -- 172.21.15.16:0/3739488374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 msgr2=0x55e9e0a87150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6dffff640 1 -- 172.21.15.16:0/3739488374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e0a60860 con 0x55e9e0a644a0 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.787+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.788+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a86c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009fd0 tx=0x7fc6cc09b230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd3d3051a36d0f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.788+0000 7fc6ca7fc640 1 -- 172.21.15.16:0/3739488374 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc09f6e0 con 0x55e9e0a644a0 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.788+0000 7fc6ca7fc640 1 -- 172.21.15.16:0/3739488374 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc0a3fa0 con 0x55e9e0a644a0 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.788+0000 7fc6ca7fc640 1 -- 172.21.15.16:0/3739488374 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc09a780 con 0x55e9e0a644a0 2024-09-12T01:02:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3739488374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 msgr2=0x55e9e0a86c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a86c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009fd0 tx=0x7fc6cc09b230 comp rx=0 tx=0).stop 2024-09-12T01:02:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3739488374 shutdown_connections 2024-09-12T01:02:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a87150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a644a0 0x55e9e0a86c10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3739488374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a60a30 0x55e9e0a677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3739488374 >> 172.21.15.16:0/3739488374 conn(0x55e9e0a6c7d0 msgr2=0x55e9e0a670e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3739488374 shutdown_connections 2024-09-12T01:02:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3739488374 wait complete. 2024-09-12T01:02:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 -- start start 2024-09-12T01:02:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.789+0000 7fc6e91e2740 1 Processor -- start 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 0x55e9e0a77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01b0ef0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01b0820 con 0x55e9e0a60a30 2024-09-12T01:02:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56058/0 (socket says 172.21.15.16:56058) 2024-09-12T01:02:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 -- 172.21.15.16:0/3417002452 learned_addr learned my addr 172.21.15.16:0/3417002452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01ada60 con 0x55e9e0a644a0 2024-09-12T01:02:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e91e2740 10 client.0 _ll_register_callbacks cb 0x7fc6cc0d2440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 0x55e9e0a77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 -- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 msgr2=0x55e9e0a77b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a77b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 -- 172.21.15.16:0/3417002452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 msgr2=0x55e9e0a77630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 0x55e9e0a77630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e53a6640 1 -- 172.21.15.16:0/3417002452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e0a67d30 con 0x55e9e0a64dc0 2024-09-12T01:02:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 0x55e9e0a77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:02:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.790+0000 7fc6dffff640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6e53a6640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80116e0 tx=0x7fc6d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc0ce977976b138 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6bffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d802b420 con 0x55e9e0a64dc0 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6cc0a3fa0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fc6d802b5c0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc6d802b7c0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6e91e2740 10 client.17259 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9e0a616c0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.791+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6cc0a3fa0 con 0x55e9e0a64dc0 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc6d8013620 con 0x55e9e0a64dc0 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 10 client.17259 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 20 client.17259 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a6f680 0x55e9e0aec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 4 client.17259 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:02:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9e09d1d60 con 0x55e9e0a6f680 2024-09-12T01:02:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.792+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a6f680 0x55e9e0aec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6bffff640 10 client.17259 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e4ba5640 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a6f680 0x55e9e0aec8e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1512a90fa3e8243f server_cookie=4fb7aa68074bd26e in_seq=0 out_seq=0 2024-09-12T01:02:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6bffff640 1 -- 172.21.15.16:0/3417002452 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9e09d1d60 con 0x55e9e0a6f680 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6bffff640 10 client.17259 handle_command_reply: tid=0 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 client.17259 shutdown 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 msgr2=0x55e9e0a725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80116e0 tx=0x7fc6d8004ab0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 shutdown_connections 2024-09-12T01:02:28.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e9e0a6f680 0x55e9e0aec8e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6bffff640 0 client.17259 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9e0a64dc0 0x55e9e0a725e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9e0a644a0 0x55e9e0a77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 --2- 172.21.15.16:0/3417002452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9e0a60a30 0x55e9e0a77630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 >> 172.21.15.16:0/3417002452 conn(0x55e9e0a6c7d0 msgr2=0x55e9e0a66760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 shutdown_connections 2024-09-12T01:02:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 1 -- 172.21.15.16:0/3417002452 wait complete. 2024-09-12T01:02:28.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.793+0000 7fc6e91e2740 20 client.17259 trim_cache size 0 max 16384 2024-09-12T01:02:28.811 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:02:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:02:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:02:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: "a5266971-7b9c-4d3f-9fe4-16186de24312": { 2024-09-12T01:02:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:02:28.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "a5266971-7b9c-4d3f-9fe4-16186de24312", 2024-09-12T01:02:28.813 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:02:28.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:28.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:28.813 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:02:28.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 msgr2=0x7fc6d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 0x7fc6d00a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d8004a40 comp rx=0 tx=0).stop 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 msgr2=0x7fc6e00fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00d950 tx=0x7fc6cc00de10 comp rx=0 tx=0).stop 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 shutdown_connections 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc6d00a2cf0 0x7fc6d00a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6e0106a70 0x7fc6e00fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e0101410 0x7fc6e00fd3a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 --2- 172.21.15.16:0/1947994082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e0100a70 0x7fc6e00fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:28.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 >> 172.21.15.16:0/1947994082 conn(0x7fc6e00fc790 msgr2=0x7fc6e006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:28.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 shutdown_connections 2024-09-12T01:02:28.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:28.794+0000 7fc6e6e09640 1 -- 172.21.15.16:0/1947994082 wait complete. 2024-09-12T01:02:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:29.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:29 smithi016 ceph-mon[30972]: pgmap v195: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T01:02:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:29 smithi099 ceph-mon[35654]: pgmap v195: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T01:02:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 -- 172.21.15.16:0/2952580097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994106f10 msgr2=0x7fa99410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 --2- 172.21.15.16:0/2952580097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994106f10 0x7fa99410ea90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009ca0 tx=0x7fa98c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 -- 172.21.15.16:0/2952580097 shutdown_connections 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 --2- 172.21.15.16:0/2952580097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994106f10 0x7fa99410ea90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 --2- 172.21.15.16:0/2952580097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994102790 0x7fa9941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 --2- 172.21.15.16:0/2952580097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 0x7fa9941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 -- 172.21.15.16:0/2952580097 >> 172.21.15.16:0/2952580097 conn(0x7fa9940fd8c0 msgr2=0x7fa9940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 -- 172.21.15.16:0/2952580097 shutdown_connections 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.590+0000 7fa99afe7640 1 -- 172.21.15.16:0/2952580097 wait complete. 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 Processor -- start 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 -- start start 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 0x7fa9940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994106f10 0x7fa994074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa994111f70 con 0x7fa994102790 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa994111df0 con 0x7fa994101de0 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa99afe7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941120f0 con 0x7fa994106f10 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa993fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56076/0 (socket says 172.21.15.16:56076) 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa993fff640 1 -- 172.21.15.16:0/4127147785 learned_addr learned my addr 172.21.15.16:0/4127147785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.591+0000 7fa998d83640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 0x7fa9940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa999584640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994106f10 0x7fa994074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 -- 172.21.15.16:0/4127147785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994106f10 msgr2=0x7fa994074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994106f10 0x7fa994074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 -- 172.21.15.16:0/4127147785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 msgr2=0x7fa9940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 0x7fa9940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 -- 172.21.15.16:0/4127147785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9941a68b0 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa993fff640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d770 tx=0x7fa98000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980017070 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980005500 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9941a6ba0 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980004e00 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.592+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a7090 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.593+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9941021c0 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.594+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa980004fa0 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.595+0000 7fa991ffb640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 0x7fa9840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.595+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7fa9800cedf0 con 0x7fa994102790 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.595+0000 7fa998d83640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 0x7fa9840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.596+0000 7fa998d83640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 0x7fa9840a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa98800bbe0 tx=0x7fa98800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.599+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98009ba40 con 0x7fa994102790 2024-09-12T01:02:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:29 smithi159 ceph-mon[35566]: pgmap v195: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-12T01:02:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.812+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa99410a120 con 0x7fa994102790 2024-09-12T01:02:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.813+0000 7fa991ffb640 1 -- 172.21.15.16:0/4127147785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa98009b340 con 0x7fa994102790 2024-09-12T01:02:29.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:02:29.812 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:02:29.813 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:02:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 msgr2=0x7fa9840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 0x7fa9840a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa98800bbe0 tx=0x7fa98800b390 comp rx=0 tx=0).stop 2024-09-12T01:02:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 msgr2=0x7fa994074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d770 tx=0x7fa98000dc30 comp rx=0 tx=0).stop 2024-09-12T01:02:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 shutdown_connections 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa9840a2cf0 0x7fa9840a51b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa994106f10 0x7fa994074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa994102790 0x7fa994074a30 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 --2- 172.21.15.16:0/4127147785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa994101de0 0x7fa9940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 >> 172.21.15.16:0/4127147785 conn(0x7fa9940fd8c0 msgr2=0x7fa9940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 shutdown_connections 2024-09-12T01:02:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:29.815+0000 7fa99afe7640 1 -- 172.21.15.16:0/4127147785 wait complete. 2024-09-12T01:02:30.122 INFO:tasks.cephfs.filesystem:scrub status for tag:a5266971-7b9c-4d3f-9fe4-16186de24312 - {'path': '/', 'tag': 'a5266971-7b9c-4d3f-9fe4-16186de24312', 'options': 'recursive,force'} 2024-09-12T01:02:30.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:30 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:02:30.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:30 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4127147785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:30.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:30 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:02:30.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:30 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4127147785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:30.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:30 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:02:30.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:30 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4127147785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:02:30.851 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 4c083005 --release --await' 2024-09-12T01:02:30.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 4c083005 --release --await 2024-09-12T01:02:31.117 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:02:31.495 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:31 smithi016 ceph-mon[30972]: pgmap v196: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:02:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:31 smithi099 ceph-mon[35654]: pgmap v196: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:02:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- 172.21.15.16:0/365647472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 msgr2=0x7f916010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 --2- 172.21.15.16:0/365647472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f916010ea90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9158009ca0 tx=0x7f915802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- 172.21.15.16:0/365647472 shutdown_connections 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 --2- 172.21.15.16:0/365647472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f916010ea90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 --2- 172.21.15.16:0/365647472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 0x7f91601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 --2- 172.21.15.16:0/365647472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 0x7f91601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- 172.21.15.16:0/365647472 >> 172.21.15.16:0/365647472 conn(0x7f91600fd8c0 msgr2=0x7f91600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- 172.21.15.16:0/365647472 shutdown_connections 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- 172.21.15.16:0/365647472 wait complete. 2024-09-12T01:02:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 Processor -- start 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.588+0000 7f9166b89640 1 -- start start 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 0x7f91600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 0x7f9160074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9160111f70 con 0x7f9160106f10 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9160111df0 con 0x7f9160102790 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9166b89640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91601120f0 con 0x7f9160101de0 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56088/0 (socket says 172.21.15.16:56088) 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 -- 172.21.15.16:0/717917045 learned_addr learned my addr 172.21.15.16:0/717917045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f915ffff640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 0x7f9160074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9164925640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 0x7f91600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 -- 172.21.15.16:0/717917045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 msgr2=0x7f91600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 0x7f91600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 -- 172.21.15.16:0/717917045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 msgr2=0x7f9160074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 0x7f9160074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 -- 172.21.15.16:0/717917045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601a6910 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f9165126640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9158031a20 tx=0x7f9158031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.589+0000 7f915dffb640 1 -- 172.21.15.16:0/717917045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158003c30 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.590+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91601a6ba0 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.590+0000 7f915dffb640 1 -- 172.21.15.16:0/717917045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9158003dd0 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.590+0000 7f915dffb640 1 -- 172.21.15.16:0/717917045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915802fcf0 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.590+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a70e0 con 0x7f9160106f10 2024-09-12T01:02:31.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f915dffb640 1 -- 172.21.15.16:0/717917045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9158004050 con 0x7f9160106f10 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 Processor -- start 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 -- start start 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d58440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d65630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b765cbd620 con 0x55b765d58980 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b765c64e50 con 0x55b765d63240 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9168f62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b765cb95e0 con 0x55b765d58060 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9164925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9164925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56102/0 (socket says 172.21.15.16:56102) 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f9164925640 1 -- 172.21.15.16:0/1586289801 learned_addr learned my addr 172.21.15.16:0/1586289801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f915ffff640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d58440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.593+0000 7f915dffb640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 0x7f91480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f915ffff640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d65630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 -- 172.21.15.16:0/1586289801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 msgr2=0x55b765d58440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d58440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 -- 172.21.15.16:0/1586289801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 msgr2=0x55b765d65630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d65630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 -- 172.21.15.16:0/1586289801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b765c55d40 con 0x55b765d58980 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f915ffff640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f915dffb640 1 -- 172.21.15.16:0/717917045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f91580f0660 con 0x7f9160106f10 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9165126640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 0x7f91480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9164925640 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f915000cbe0 tx=0x7f915000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb90370383ea6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9146ffd640 1 -- 172.21.15.16:0/1586289801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150010630 con 0x55b765d58980 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9146ffd640 1 -- 172.21.15.16:0/1586289801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9150010cb0 con 0x55b765d58980 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.594+0000 7f9146ffd640 1 -- 172.21.15.16:0/1586289801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915001ac20 con 0x55b765d58980 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- 172.21.15.16:0/1586289801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 msgr2=0x55b765d62d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9165126640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 0x7f91480a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91580319f0 tx=0x7f915802a8c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f915000cbe0 tx=0x7f915000e720 comp rx=0 tx=0).stop 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- 172.21.15.16:0/1586289801 shutdown_connections 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d65630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58980 0x55b765d62d00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 --2- 172.21.15.16:0/1586289801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d58440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- 172.21.15.16:0/1586289801 >> 172.21.15.16:0/1586289801 conn(0x55b765d545f0 msgr2=0x55b765d549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- 172.21.15.16:0/1586289801 shutdown_connections 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- 172.21.15.16:0/1586289801 wait complete. 2024-09-12T01:02:31.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 -- start start 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.595+0000 7f9168f62740 1 Processor -- start 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d60180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b765d65e10 con 0x55b765d63240 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b765c45f10 con 0x55b765d58980 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b765d65f90 con 0x55b765d58060 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9168f62740 10 client.0 _ll_register_callbacks cb 0x55b765c69db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56112/0 (socket says 172.21.15.16:56112) 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 -- 172.21.15.16:0/3485896217 learned_addr learned my addr 172.21.15.16:0/3485896217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:31.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f9164925640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d60180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 -- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 msgr2=0x55b765d60180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d60180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 -- 172.21.15.16:0/3485896217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 msgr2=0x55b765d606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 -- 172.21.15.16:0/3485896217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7659cc580 con 0x55b765d63240 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f915ffff640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f915400bd00 tx=0x7f915400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8c786dcc3c6ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f913ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.596+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154005220 con 0x55b765d63240 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f91540053c0 con 0x55b765d63240 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9150010cb0 con 0x55b765d63240 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91540055c0 con 0x55b765d63240 2024-09-12T01:02:31.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 10 client.17277 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b765a96960 con 0x55b765d63240 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9150010cb0 con 0x55b765d63240 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91540128d0 con 0x55b765d63240 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 10 client.17277 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 10 client.17277 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 20 client.17277 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765d75110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 4 client.17277 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.597+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b765ccab90 con 0x55b765d72d20 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.598+0000 7f9164925640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765d75110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.598+0000 7f9164925640 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765d75110 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14dfac61fdde5164 server_cookie=730cfe50e7e9039 in_seq=0 out_seq=0 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.598+0000 7f913ffff640 10 client.17277 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f913ffff640 1 -- 172.21.15.16:0/3485896217 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b765ccab90 con 0x55b765d72d20 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f913ffff640 10 client.17277 handle_command_reply: tid=0 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 client.17277 shutdown 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 msgr2=0x55b765d60c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f915400bd00 tx=0x7f915400b390 comp rx=0 tx=0).stop 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 shutdown_connections 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765d75110 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f913ffff640 0 client.17277 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d63240 0x55b765d60c00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 --2- 172.21.15.16:0/3485896217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58060 0x55b765d60180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 >> 172.21.15.16:0/3485896217 conn(0x55b765d545f0 msgr2=0x55b765d5a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 shutdown_connections 2024-09-12T01:02:31.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 1 -- 172.21.15.16:0/3485896217 wait complete. 2024-09-12T01:02:31.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.599+0000 7f9168f62740 20 client.17277 trim_cache size 0 max 16384 2024-09-12T01:02:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:31 smithi159 ceph-mon[35566]: pgmap v196: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:02:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 Processor -- start 2024-09-12T01:02:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 -- start start 2024-09-12T01:02:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d63240 0x55b765d7af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91601af9a0 con 0x55b765d58060 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91601b14e0 con 0x55b765d58980 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9168f62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91601b0070 con 0x55b765d63240 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56122/0 (socket says 172.21.15.16:56122) 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f915ffff640 1 -- 172.21.15.16:0/4118694327 learned_addr learned my addr 172.21.15.16:0/4118694327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f9164925640 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.617+0000 7f915ffff640 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d63240 0x55b765d7af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 -- 172.21.15.16:0/4118694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d63240 msgr2=0x55b765d7af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d63240 0x55b765d7af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 -- 172.21.15.16:0/4118694327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 msgr2=0x55b765d7a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d7a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 -- 172.21.15.16:0/4118694327 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b765d5b010 con 0x55b765d58060 2024-09-12T01:02:31.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f915ffff640 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f915400b620 tx=0x7f915400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e858f9ceb20071 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f91467fc640 1 -- 172.21.15.16:0/4118694327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154034070 con 0x55b765d58060 2024-09-12T01:02:31.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f91467fc640 1 -- 172.21.15.16:0/4118694327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9154012890 con 0x55b765d58060 2024-09-12T01:02:31.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.618+0000 7f91467fc640 1 -- 172.21.15.16:0/4118694327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915402c690 con 0x55b765d58060 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- 172.21.15.16:0/4118694327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 msgr2=0x55b765d62050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f915400b620 tx=0x7f915400b5b0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- 172.21.15.16:0/4118694327 shutdown_connections 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d63240 0x55b765d7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d58980 0x55b765d7a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 --2- 172.21.15.16:0/4118694327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d62050 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- 172.21.15.16:0/4118694327 >> 172.21.15.16:0/4118694327 conn(0x55b765d545f0 msgr2=0x55b765d53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- 172.21.15.16:0/4118694327 shutdown_connections 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- 172.21.15.16:0/4118694327 wait complete. 2024-09-12T01:02:31.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 -- start start 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.619+0000 7f9168f62740 1 Processor -- start 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 0x55b765d67ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d67ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91601b0e10 con 0x55b765d58060 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91601b0740 con 0x55b765d63240 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91601ad980 con 0x55b765d58980 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9168f62740 10 client.0 _ll_register_callbacks cb 0x7f915400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56124/0 (socket says 172.21.15.16:56124) 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 -- 172.21.15.16:0/3525332164 learned_addr learned my addr 172.21.15.16:0/3525332164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9164925640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 0x55b765d67ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d67ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 -- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 msgr2=0x55b765d67ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 0x55b765d67ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 -- 172.21.15.16:0/3525332164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 msgr2=0x55b765d67ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d67ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f915ffff640 1 -- 172.21.15.16:0/3525332164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b765d68630 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.620+0000 7f9164925640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 0x55b765d67ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f915ffff640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9154015010 tx=0x7f915400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a65efe7ab3d28ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915403e060 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f9154039480 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9154012890 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9154034040 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 10 client.17283 fetch_fsmap learned FSMap version 28 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b765dde850 con 0x55b765d58060 2024-09-12T01:02:31.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9154012890 con 0x55b765d58060 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9154020430 con 0x55b765d58060 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 10 client.17283 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 10 client.17283 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 20 client.17283 populate_metadata read hostname 'smithi016' 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765de0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 4 client.17283 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4c083005", "release": true, "await": true}] 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.621+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4c083005", "release": true, "await": true}) -- 0x55b765cc50a0 con 0x55b765d72d20 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.622+0000 7f9164925640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765de0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.622+0000 7f9164925640 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765de0f50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc6b99cc46d18d33 server_cookie=d552d04a8a89c057 in_seq=0 out_seq=0 2024-09-12T01:02:31.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.622+0000 7f913ffff640 10 client.17283 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f913ffff640 1 -- 172.21.15.16:0/3525332164 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b765cc50a0 con 0x55b765d72d20 2024-09-12T01:02:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f913ffff640 10 client.17283 handle_command_reply: tid=0 2024-09-12T01:02:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 client.17283 shutdown 2024-09-12T01:02:31.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 msgr2=0x55b765d6b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9154015010 tx=0x7f915400b810 comp rx=0 tx=0).stop 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 shutdown_connections 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b765d72d20 0x55b765de0f50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b765d63240 0x55b765d67ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b765d58980 0x55b765d67ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f913ffff640 0 client.17283 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 --2- 172.21.15.16:0/3525332164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b765d58060 0x55b765d6b770 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.625+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 >> 172.21.15.16:0/3525332164 conn(0x55b765d545f0 msgr2=0x55b765d52f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 shutdown_connections 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9168f62740 1 -- 172.21.15.16:0/3525332164 wait complete. 2024-09-12T01:02:31.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9168f62740 20 client.17283 trim_cache size 0 max 16384 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 24, 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "4c083005": { 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 24, 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:02:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 140.1, 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:02:31.629 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:02:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 msgr2=0x7f91480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 0x7f91480a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91580319f0 tx=0x7f915802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 msgr2=0x7f9160074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:02:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9158031a20 tx=0x7f9158031a50 comp rx=0 tx=0).stop 2024-09-12T01:02:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 shutdown_connections 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f91480a2cf0 0x7f91480a51b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9160106f10 0x7f9160074f70 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9160102790 0x7f9160074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.626+0000 7f9166b89640 1 --2- 172.21.15.16:0/717917045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9160101de0 0x7f91600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.627+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 >> 172.21.15.16:0/717917045 conn(0x7f91600fd8c0 msgr2=0x7f91600fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.627+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 shutdown_connections 2024-09-12T01:02:31.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:02:31.627+0000 7f9166b89640 1 -- 172.21.15.16:0/717917045 wait complete. 2024-09-12T01:02:32.396 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4c083005', total seconds elapsed: 24.1 2024-09-12T01:02:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:33 smithi099 ceph-mon[35654]: pgmap v197: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:33 smithi159 ceph-mon[35566]: pgmap v197: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:33 smithi016 ceph-mon[30972]: pgmap v197: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:35 smithi099 ceph-mon[35654]: pgmap v198: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:35 smithi159 ceph-mon[35566]: pgmap v198: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:35 smithi016 ceph-mon[30972]: pgmap v198: 97 pgs: 97 active+clean; 6.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T01:02:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:37 smithi099 ceph-mon[35654]: pgmap v199: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T01:02:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:37 smithi159 ceph-mon[35566]: pgmap v199: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T01:02:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:37 smithi016 ceph-mon[30972]: pgmap v199: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-12T01:02:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:39 smithi099 ceph-mon[35654]: pgmap v200: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T01:02:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:39 smithi159 ceph-mon[35566]: pgmap v200: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T01:02:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:39 smithi016 ceph-mon[30972]: pgmap v200: 97 pgs: 97 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-12T01:02:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:41 smithi099 ceph-mon[35654]: pgmap v201: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T01:02:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:41 smithi159 ceph-mon[35566]: pgmap v201: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T01:02:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:41 smithi016 ceph-mon[30972]: pgmap v201: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 KiB/s wr, 16 op/s 2024-09-12T01:02:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:43 smithi099 ceph-mon[35654]: pgmap v202: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:43 smithi159 ceph-mon[35566]: pgmap v202: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:43 smithi016 ceph-mon[30972]: pgmap v202: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:45 smithi099 ceph-mon[35654]: pgmap v203: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:45 smithi159 ceph-mon[35566]: pgmap v203: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:45 smithi016 ceph-mon[30972]: pgmap v203: 97 pgs: 97 active+clean; 6.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-09-12T01:02:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:47 smithi099 ceph-mon[35654]: pgmap v204: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T01:02:47.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:47 smithi159 ceph-mon[35566]: pgmap v204: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T01:02:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:47 smithi016 ceph-mon[30972]: pgmap v204: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-09-12T01:02:49.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:49 smithi016 ceph-mon[30972]: pgmap v205: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:49 smithi099 ceph-mon[35654]: pgmap v205: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:49.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:49 smithi159 ceph-mon[35566]: pgmap v205: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:51.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:51 smithi099 ceph-mon[35654]: pgmap v206: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:51 smithi159 ceph-mon[35566]: pgmap v206: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:51.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:51 smithi016 ceph-mon[30972]: pgmap v206: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:53 smithi099 ceph-mon[35654]: pgmap v207: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:53 smithi159 ceph-mon[35566]: pgmap v207: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:53 smithi016 ceph-mon[30972]: pgmap v207: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:55 smithi099 ceph-mon[35654]: pgmap v208: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:55.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:55 smithi159 ceph-mon[35566]: pgmap v208: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:55.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:55 smithi016 ceph-mon[30972]: pgmap v208: 97 pgs: 97 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T01:02:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:57 smithi099 ceph-mon[35654]: pgmap v209: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:57 smithi159 ceph-mon[35566]: pgmap v209: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:57.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:57 smithi016 ceph-mon[30972]: pgmap v209: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-09-12T01:02:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:02:59 smithi099 ceph-mon[35654]: pgmap v210: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:02:59.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:02:59 smithi159 ceph-mon[35566]: pgmap v210: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:02:59.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:02:59 smithi016 ceph-mon[30972]: pgmap v210: 97 pgs: 97 active+clean; 7.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:00.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:03:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 -- 172.21.15.16:0/1973971325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90102790 msgr2=0x7f4c901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 --2- 172.21.15.16:0/1973971325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90102790 0x7f4c901067e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8402a410 comp rx=0 tx=0).stop 2024-09-12T01:03:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 -- 172.21.15.16:0/1973971325 shutdown_connections 2024-09-12T01:03:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 --2- 172.21.15.16:0/1973971325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90106f10 0x7f4c9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 --2- 172.21.15.16:0/1973971325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90102790 0x7f4c901067e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 --2- 172.21.15.16:0/1973971325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 0x7f4c901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 -- 172.21.15.16:0/1973971325 >> 172.21.15.16:0/1973971325 conn(0x7f4c900fd8c0 msgr2=0x7f4c900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 -- 172.21.15.16:0/1973971325 shutdown_connections 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.922+0000 7f4c958e8640 1 -- 172.21.15.16:0/1973971325 wait complete. 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 Processor -- start 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 -- start start 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 0x7f4c900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90102790 0x7f4c90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111f70 con 0x7f4c90106f10 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111df0 con 0x7f4c90102790 2024-09-12T01:03:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c958e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901120f0 con 0x7f4c90101de0 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c8f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39038/0 (socket says 172.21.15.16:39038) 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/3690929767 learned_addr learned my addr 172.21.15.16:0/3690929767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c8effd640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 0x7f4c900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.923+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90102790 0x7f4c90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/3690929767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 msgr2=0x7f4c900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 0x7f4c900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/3690929767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90102790 msgr2=0x7f4c90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90102790 0x7f4c90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 -- 172.21.15.16:0/3690929767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c901a6890 con 0x7f4c90106f10 2024-09-12T01:03:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800cce0 tx=0x7f4c8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c948e6640 1 -- 172.21.15.16:0/3690929767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800f450 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c901a6b80 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c948e6640 1 -- 172.21.15.16:0/3690929767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8800fad0 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c948e6640 1 -- 172.21.15.16:0/3690929767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88018c10 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.924+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c901a70c0 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 Processor -- start 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c948e6640 1 -- 172.21.15.16:0/3690929767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4c8800f5f0 con 0x7f4c90106f10 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 -- start start 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b726f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586b71d52e0 con 0x5586b726fdc0 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586b717cce0 con 0x5586b726f4a0 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586b71d12a0 con 0x5586b727a680 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39044/0 (socket says 172.21.15.16:39044) 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c8effd640 1 -- 172.21.15.16:0/2865127297 learned_addr learned my addr 172.21.15.16:0/2865127297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.927+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 -- 172.21.15.16:0/2865127297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 msgr2=0x5586b727ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b726f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 -- 172.21.15.16:0/2865127297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 msgr2=0x5586b726f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b726f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 -- 172.21.15.16:0/2865127297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b7161440 con 0x5586b726fdc0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b726f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c948e6640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 0x7f4c780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8effd640 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009d20 tx=0x7f4c80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cca59babe6abb5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 0x7f4c780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c948e6640 1 -- 172.21.15.16:0/3690929767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4c880cfaa0 con 0x7f4c90106f10 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c75ffb640 1 -- 172.21.15.16:0/2865127297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8001e070 con 0x5586b726fdc0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c75ffb640 1 -- 172.21.15.16:0/2865127297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80019b10 con 0x5586b726fdc0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.928+0000 7f4c75ffb640 1 -- 172.21.15.16:0/2865127297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000ebf0 con 0x5586b726fdc0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- 172.21.15.16:0/2865127297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 msgr2=0x5586b727a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 0x7f4c780a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c90075f80 tx=0x7f4c8402a410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009d20 tx=0x7f4c80011810 comp rx=0 tx=0).stop 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- 172.21.15.16:0/2865127297 shutdown_connections 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727a140 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/2865127297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b726f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- 172.21.15.16:0/2865127297 >> 172.21.15.16:0/2865127297 conn(0x5586b726ba30 msgr2=0x5586b726be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- 172.21.15.16:0/2865127297 shutdown_connections 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- 172.21.15.16:0/2865127297 wait complete. 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.929+0000 7f4c97cc1740 1 -- start start 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 Processor -- start 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 0x5586b7277250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 0x5586b7277790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586b727d250 con 0x5586b726f4a0 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5586b7190050 con 0x5586b727a680 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586b727d3d0 con 0x5586b726fdc0 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c97cc1740 10 client.0 _ll_register_callbacks cb 0x5586b71d0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39052/0 (socket says 172.21.15.16:39052) 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 -- 172.21.15.16:0/1768273336 learned_addr learned my addr 172.21.15.16:0/1768273336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 0x5586b7277250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 0x5586b7277790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 -- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 msgr2=0x5586b7277250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 0x5586b7277250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 -- 172.21.15.16:0/1768273336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 msgr2=0x5586b7277790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 0x5586b7277790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8effd640 1 -- 172.21.15.16:0/1768273336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b6f463c0 con 0x5586b726f4a0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 0x5586b7277790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.930+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 0x5586b7277250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c8effd640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4c80011d10 tx=0x7f4c80019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc9b62fc88a9251 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8001e070 con 0x5586b726f4a0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4c8000eaf0 con 0x5586b726f4a0 2024-09-12T01:03:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c80019b10 con 0x5586b726f4a0 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c80031050 con 0x5586b726f4a0 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c97cc1740 10 client.17295 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586b7288b40 con 0x5586b726f4a0 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c80019b10 con 0x5586b726f4a0 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c800046c0 con 0x5586b726f4a0 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c97cc1740 10 client.17295 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.931+0000 7f4c97cc1740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c97cc1740 20 client.17295 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b7289fa0 0x5586b728c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c97cc1740 4 client.17295 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586b71e2850 con 0x5586b7289fa0 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b7289fa0 0x5586b728c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c6effd640 10 client.17295 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.932+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b7289fa0 0x5586b728c390 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efcbacf88a73409 server_cookie=112cb616581a7946 in_seq=0 out_seq=0 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c6effd640 1 -- 172.21.15.16:0/1768273336 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586b71e2850 con 0x5586b7289fa0 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c6effd640 10 client.17295 handle_command_reply: tid=0 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 client.17295 shutdown 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 msgr2=0x5586b7279fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4c80011d10 tx=0x7f4c80019660 comp rx=0 tx=0).stop 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 shutdown_connections 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b7289fa0 0x5586b728c390 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b727a680 0x5586b7277790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b726fdc0 0x5586b7277250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/1768273336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726f4a0 0x5586b7279fa0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 >> 172.21.15.16:0/1768273336 conn(0x5586b726ba30 msgr2=0x5586b726ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.933+0000 7f4c6effd640 0 client.17295 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.934+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 shutdown_connections 2024-09-12T01:03:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.934+0000 7f4c97cc1740 1 -- 172.21.15.16:0/1768273336 wait complete. 2024-09-12T01:03:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.934+0000 7f4c97cc1740 20 client.17295 trim_cache size 0 max 16384 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 Processor -- start 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 -- start start 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b7276a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b7292090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901af930 con 0x5586b726fdc0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901b1470 con 0x5586b726f4a0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.968+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901b0000 con 0x5586b727a680 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39058/0 (socket says 172.21.15.16:39058) 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/3195303184 learned_addr learned my addr 172.21.15.16:0/3195303184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b7292090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8effd640 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b7276a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/3195303184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 msgr2=0x5586b7292090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b7292090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/3195303184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 msgr2=0x5586b7276a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b7276a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/3195303184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b726fbf0 con 0x5586b726fdc0 2024-09-12T01:03:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c84042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773895a3eaefeff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c757fa640 1 -- 172.21.15.16:0/3195303184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84056070 con 0x5586b726fdc0 2024-09-12T01:03:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c757fa640 1 -- 172.21.15.16:0/3195303184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840319f0 con 0x5586b726fdc0 2024-09-12T01:03:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.969+0000 7f4c757fa640 1 -- 172.21.15.16:0/3195303184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84035040 con 0x5586b726fdc0 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 -- 172.21.15.16:0/3195303184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 msgr2=0x5586b7291b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c84042000 comp rx=0 tx=0).stop 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 -- 172.21.15.16:0/3195303184 shutdown_connections 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b7292090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b7291b50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/3195303184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b7276a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 -- 172.21.15.16:0/3195303184 >> 172.21.15.16:0/3195303184 conn(0x5586b726ba30 msgr2=0x5586b726ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 -- 172.21.15.16:0/3195303184 shutdown_connections 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.970+0000 7f4c97cc1740 1 -- 172.21.15.16:0/3195303184 wait complete. 2024-09-12T01:03:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 -- start start 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 Processor -- start 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b727f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901b0da0 con 0x5586b726fdc0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901b06d0 con 0x5586b726f4a0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901ad910 con 0x5586b727a680 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c8e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39074/0 (socket says 172.21.15.16:39074) 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c97cc1740 10 client.0 _ll_register_callbacks cb 0x7f4c8001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/694718707 learned_addr learned my addr 172.21.15.16:0/694718707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.971+0000 7f4c8effd640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b727f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8f7fe640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 msgr2=0x5586b727fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/694718707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 msgr2=0x5586b727f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b727f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 -- 172.21.15.16:0/694718707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b72f2b20 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8effd640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b727f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c8e7fc640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4c84054040 tx=0x7f4c84003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a6763910a00e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c6effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8405f060 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c840319f0 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f4c84003800 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.972+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c84056070 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 10 client.17301 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586b72f3df0 con 0x5586b726fdc0 2024-09-12T01:03:00.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c840319f0 con 0x5586b726fdc0 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c8403a040 con 0x5586b726fdc0 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 10 client.17301 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 20 client.17301 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b72f4530 0x5586b72f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 4 client.17301 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586b71dcd60 con 0x5586b72f4530 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.973+0000 7f4c8effd640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b72f4530 0x5586b72f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c8effd640 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b72f4530 0x5586b72f6950 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd72b313150a64c server_cookie=ae6c5521dae3c1ef in_seq=0 out_seq=0 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c6effd640 10 client.17301 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c6effd640 1 -- 172.21.15.16:0/694718707 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586b71dcd60 con 0x5586b72f4530 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c6effd640 10 client.17301 handle_command_reply: tid=0 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 client.17301 shutdown 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 msgr2=0x5586b727f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4c84054040 tx=0x7f4c84003b50 comp rx=0 tx=0).stop 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 shutdown_connections 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5586b72f4530 0x5586b72f6950 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586b727a680 0x5586b727fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b726fdc0 0x5586b727f8b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 --2- 172.21.15.16:0/694718707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5586b726f4a0 0x5586b727f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 >> 172.21.15.16:0/694718707 conn(0x5586b726ba30 msgr2=0x5586b7278ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c6effd640 0 client.17301 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 shutdown_connections 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.974+0000 7f4c97cc1740 1 -- 172.21.15.16:0/694718707 wait complete. 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c97cc1740 20 client.17301 trim_cache size 0 max 16384 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:00.978 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 msgr2=0x7f4c780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 0x7f4c780a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4c90075f80 tx=0x7f4c8402a410 comp rx=0 tx=0).stop 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 msgr2=0x7f4c90074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800cce0 tx=0x7f4c8800a620 comp rx=0 tx=0).stop 2024-09-12T01:03:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 shutdown_connections 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c780a2cf0 0x7f4c780a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c90106f10 0x7f4c90074f70 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c90102790 0x7f4c90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 --2- 172.21.15.16:0/3690929767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c90101de0 0x7f4c900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 >> 172.21.15.16:0/3690929767 conn(0x7f4c900fd8c0 msgr2=0x7f4c900fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 shutdown_connections 2024-09-12T01:03:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:00.975+0000 7f4c958e8640 1 -- 172.21.15.16:0/3690929767 wait complete. 2024-09-12T01:03:01.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:01 smithi016 ceph-mon[30972]: pgmap v211: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-12T01:03:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:01 smithi099 ceph-mon[35654]: pgmap v211: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-12T01:03:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:01 smithi159 ceph-mon[35566]: pgmap v211: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-09-12T01:03:01.904 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:03:01.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 -- 172.21.15.16:0/3031522321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0112b90 msgr2=0x7f18a0114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 --2- 172.21.15.16:0/3031522321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0112b90 0x7f18a0114f80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1898009ca0 tx=0x7f189802a950 comp rx=0 tx=0).stop 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 -- 172.21.15.16:0/3031522321 shutdown_connections 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 --2- 172.21.15.16:0/3031522321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0112b90 0x7f18a0114f80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 --2- 172.21.15.16:0/3031522321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 0x7f18a0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 --2- 172.21.15.16:0/3031522321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0074590 0x7f18a0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.672+0000 7f18a7847640 1 -- 172.21.15.16:0/3031522321 >> 172.21.15.16:0/3031522321 conn(0x7f18a00fd8c0 msgr2=0x7f18a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.673+0000 7f18a7847640 1 -- 172.21.15.16:0/3031522321 shutdown_connections 2024-09-12T01:03:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.673+0000 7f18a7847640 1 -- 172.21.15.16:0/3031522321 wait complete. 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 Processor -- start 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 -- start start 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 0x7f18a01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 0x7f18a019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0118630 con 0x7f18a0074590 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01184b0 con 0x7f18a0112b90 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a7847640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01187b0 con 0x7f18a0074eb0 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a55e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a55e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39102/0 (socket says 172.21.15.16:39102) 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a55e3640 1 -- 172.21.15.16:0/71674585 learned_addr learned my addr 172.21.15.16:0/71674585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.674+0000 7f18a4de2640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 0x7f18a01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a5de4640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 0x7f18a019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 -- 172.21.15.16:0/71674585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 msgr2=0x7f18a01a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 0x7f18a01a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 -- 172.21.15.16:0/71674585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 msgr2=0x7f18a019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 0x7f18a019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 -- 172.21.15.16:0/71674585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a019f110 con 0x7f18a0074590 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a5de4640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 0x7f18a019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a55e3640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f189c00b4a0 tx=0x7f189c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c015020 con 0x7f18a0074590 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189c011d10 con 0x7f18a0074590 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a019f400 con 0x7f18a0074590 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c00f2e0 con 0x7f18a0074590 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.675+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01ab6f0 con 0x7f18a0074590 2024-09-12T01:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.676+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a00749f0 con 0x7f18a0074590 2024-09-12T01:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.677+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f189c018020 con 0x7f18a0074590 2024-09-12T01:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.678+0000 7f18967fc640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 0x7f188c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.678+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7f189c0ceb80 con 0x7f18a0074590 2024-09-12T01:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.678+0000 7f18a4de2640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 0x7f188c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.679+0000 7f18a4de2640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 0x7f188c0a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18a01000e0 tx=0x7f1890009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:02.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.682+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f189c01f020 con 0x7f18a0074590 2024-09-12T01:03:02.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.893+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18a0104fd0 con 0x7f18a0074590 2024-09-12T01:03:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.894+0000 7f18967fc640 1 -- 172.21.15.16:0/71674585 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f189c0b9e20 con 0x7f18a0074590 2024-09-12T01:03:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:02.893 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:02.893 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 msgr2=0x7f188c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 0x7f188c0a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18a01000e0 tx=0x7f1890009110 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 msgr2=0x7f18a01a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f189c00b4a0 tx=0x7f189c00b8a0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 shutdown_connections 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f188c0a2d70 0x7f188c0a5230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18a0112b90 0x7f18a019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18a0074eb0 0x7f18a01a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 --2- 172.21.15.16:0/71674585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18a0074590 0x7f18a01a4820 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 >> 172.21.15.16:0/71674585 conn(0x7f18a00fd8c0 msgr2=0x7f18a00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 shutdown_connections 2024-09-12T01:03:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:02.897+0000 7f18a7847640 1 -- 172.21.15.16:0/71674585 wait complete. 2024-09-12T01:03:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:03 smithi099 ceph-mon[35654]: pgmap v212: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:03 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/71674585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:03.575 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:03 smithi016 ceph-mon[30972]: pgmap v212: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:03.575 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:03 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/71674585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:03.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:03:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:03 smithi159 ceph-mon[35566]: pgmap v212: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:03 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/71674585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 -- 172.21.15.16:0/2213452938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14101de0 msgr2=0x7faf141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2213452938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14101de0 0x7faf141021c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf0402a410 comp rx=0 tx=0).stop 2024-09-12T01:03:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 -- 172.21.15.16:0/2213452938 shutdown_connections 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2213452938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14106f10 0x7faf1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2213452938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14102790 0x7faf141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2213452938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14101de0 0x7faf141021c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.356+0000 7faf1a42c640 1 -- 172.21.15.16:0/2213452938 >> 172.21.15.16:0/2213452938 conn(0x7faf140fd8c0 msgr2=0x7faf140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.357+0000 7faf1a42c640 1 -- 172.21.15.16:0/2213452938 shutdown_connections 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.357+0000 7faf1a42c640 1 -- 172.21.15.16:0/2213452938 wait complete. 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.357+0000 7faf1a42c640 1 Processor -- start 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.357+0000 7faf1a42c640 1 -- start start 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14101de0 0x7faf141984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14102790 0x7faf14198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf14111f80 con 0x7faf14106f10 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf14111e00 con 0x7faf14101de0 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf1a42c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112100 con 0x7faf14102790 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39120/0 (socket says 172.21.15.16:39120) 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 -- 172.21.15.16:0/2392762671 learned_addr learned my addr 172.21.15.16:0/2392762671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf13fff640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14101de0 0x7faf141984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf137fe640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14102790 0x7faf14198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 -- 172.21.15.16:0/2392762671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14102790 msgr2=0x7faf14198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14102790 0x7faf14198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 -- 172.21.15.16:0/2392762671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14101de0 msgr2=0x7faf141984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14101de0 0x7faf141984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 -- 172.21.15.16:0/2392762671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1419d490 con 0x7faf14106f10 2024-09-12T01:03:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.358+0000 7faf189c9640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00cad0 tx=0x7faf0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.359+0000 7faf117fa640 1 -- 172.21.15.16:0/2392762671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00ae20 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.359+0000 7faf117fa640 1 -- 172.21.15.16:0/2392762671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0c005020 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.359+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1419d780 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.359+0000 7faf117fa640 1 -- 172.21.15.16:0/2392762671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00f6f0 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.359+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a5060 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf117fa640 1 -- 172.21.15.16:0/2392762671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faf0c0051d0 con 0x7faf14106f10 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 Processor -- start 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 -- start start 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f225f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f225fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641f21c52f0 con 0x5641f226a370 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f216ccf0 con 0x5641f225f5d0 2024-09-12T01:03:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.361+0000 7faf1c805740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641f21c12b0 con 0x5641f2260b40 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39126/0 (socket says 172.21.15.16:39126) 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 -- 172.21.15.16:0/1761291028 learned_addr learned my addr 172.21.15.16:0/1761291028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf117fa640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 0x7faefc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f225f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf13fff640 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f225fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf189c9640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 0x7faefc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 -- 172.21.15.16:0/1761291028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 msgr2=0x5641f225f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f225f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 -- 172.21.15.16:0/1761291028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 msgr2=0x5641f225fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f225fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 -- 172.21.15.16:0/1761291028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f21513a0 con 0x5641f226a370 2024-09-12T01:03:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf117fa640 1 -- 172.21.15.16:0/2392762671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7faf0c0cf400 con 0x7faf14106f10 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf137fe640 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7faf08013920 tx=0x7faf0800e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ec9f66a3f8d159 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faf189c9640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 0x7faefc0a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf0c09b880 tx=0x7faf0c005960 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faefa7fc640 1 -- 172.21.15.16:0/1761291028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800b910 con 0x5641f226a370 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faefa7fc640 1 -- 172.21.15.16:0/1761291028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08005020 con 0x5641f226a370 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.362+0000 7faefa7fc640 1 -- 172.21.15.16:0/1761291028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08004a00 con 0x5641f226a370 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 -- 172.21.15.16:0/1761291028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 msgr2=0x5641f226c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7faf08013920 tx=0x7faf0800e430 comp rx=0 tx=0).stop 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 -- 172.21.15.16:0/1761291028 shutdown_connections 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f226a370 0x5641f226c760 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f225fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 --2- 172.21.15.16:0/1761291028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f225f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 -- 172.21.15.16:0/1761291028 >> 172.21.15.16:0/1761291028 conn(0x5641f2228310 msgr2=0x5641f225bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 -- 172.21.15.16:0/1761291028 shutdown_connections 2024-09-12T01:03:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.363+0000 7faf1c805740 1 -- 172.21.15.16:0/1761291028 wait complete. 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 -- start start 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 Processor -- start 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f2267030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f226a370 0x5641f2267640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641f226d010 con 0x5641f225f5d0 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f2180060 con 0x5641f226a370 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641f226d190 con 0x5641f2260b40 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf1c805740 10 client.0 _ll_register_callbacks cb 0x5641f1ed38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39130/0 (socket says 172.21.15.16:39130) 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf13fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f2267030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.364+0000 7faf137fe640 1 -- 172.21.15.16:0/241993020 learned_addr learned my addr 172.21.15.16:0/241993020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 -- 172.21.15.16:0/241993020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 msgr2=0x5641f2267030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f2267030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 -- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f226a370 msgr2=0x5641f2267640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f226a370 0x5641f2267640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 -- 172.21.15.16:0/241993020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f1f363c0 con 0x5641f225f5d0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf137fe640 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b6d0 tx=0x7faf0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ba0a0404e7ac97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf080053d0 con 0x5641f225f5d0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7faf08005570 con 0x5641f225f5d0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf08005020 con 0x5641f225f5d0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf08030050 con 0x5641f225f5d0 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf1c805740 10 client.17319 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641f2278ae0 con 0x5641f225f5d0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.365+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08005020 con 0x5641f225f5d0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf08004b50 con 0x5641f225f5d0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 10 client.17319 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 10 client.17319 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 20 client.17319 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f227c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 4 client.17319 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f21d2860 con 0x5641f2279e30 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.366+0000 7faf13fff640 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f227c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faef37fe640 10 client.17319 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faf13fff640 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f227c220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6f7969495a353e3 server_cookie=ab7ccc0fee81e9b8 in_seq=0 out_seq=0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faef37fe640 1 -- 172.21.15.16:0/241993020 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641f21d2860 con 0x5641f2279e30 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faef37fe640 10 client.17319 handle_command_reply: tid=0 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faf1c805740 1 client.17319 shutdown 2024-09-12T01:03:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 msgr2=0x5641f2269d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b6d0 tx=0x7faf0800ed70 comp rx=0 tx=0).stop 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.367+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 shutdown_connections 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f227c220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f226a370 0x5641f2267640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f2260b40 0x5641f2267030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faef37fe640 0 client.17319 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 --2- 172.21.15.16:0/241993020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2269d80 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 >> 172.21.15.16:0/241993020 conn(0x5641f2228310 msgr2=0x5641f2261960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 shutdown_connections 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 1 -- 172.21.15.16:0/241993020 wait complete. 2024-09-12T01:03:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.368+0000 7faf1c805740 20 client.17319 trim_cache size 0 max 16384 2024-09-12T01:03:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 Processor -- start 2024-09-12T01:03:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 -- start start 2024-09-12T01:03:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f2260b40 0x5641f227f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ad8c0 con 0x5641f225f5d0 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf141af400 con 0x5641f2260b40 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf1c805740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf141adf90 con 0x5641f226a370 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39144/0 (socket says 172.21.15.16:39144) 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.381+0000 7faf137fe640 1 -- 172.21.15.16:0/670910809 learned_addr learned my addr 172.21.15.16:0/670910809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 -- 172.21.15.16:0/670910809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 msgr2=0x5641f227f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 -- 172.21.15.16:0/670910809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f2260b40 msgr2=0x5641f227f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f2260b40 0x5641f227f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 -- 172.21.15.16:0/670910809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f226a0f0 con 0x5641f225f5d0 2024-09-12T01:03:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faf137fe640 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7faf08013fd0 tx=0x7faf0801e560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f9b657da7fcca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faef9ffb640 1 -- 172.21.15.16:0/670910809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08022070 con 0x5641f225f5d0 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faef9ffb640 1 -- 172.21.15.16:0/670910809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0802d070 con 0x5641f225f5d0 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.382+0000 7faef9ffb640 1 -- 172.21.15.16:0/670910809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005020 con 0x5641f225f5d0 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- 172.21.15.16:0/670910809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 msgr2=0x5641f2276430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7faf08013fd0 tx=0x7faf0801e560 comp rx=0 tx=0).stop 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- 172.21.15.16:0/670910809 shutdown_connections 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f2260b40 0x5641f227f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 --2- 172.21.15.16:0/670910809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f225f5d0 0x5641f2276430 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- 172.21.15.16:0/670910809 >> 172.21.15.16:0/670910809 conn(0x5641f2228310 msgr2=0x5641f2266140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- 172.21.15.16:0/670910809 shutdown_connections 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- 172.21.15.16:0/670910809 wait complete. 2024-09-12T01:03:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 -- start start 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.383+0000 7faf1c805740 1 Processor -- start 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f227f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf141aed30 con 0x5641f2260b40 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ae660 con 0x5641f225f5d0 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ab8a0 con 0x5641f226a370 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 10 client.0 _ll_register_callbacks cb 0x7faf04005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39150/0 (socket says 172.21.15.16:39150) 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 -- 172.21.15.16:0/3097792787 learned_addr learned my addr 172.21.15.16:0/3097792787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf137fe640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f227f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf137fe640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 -- 172.21.15.16:0/3097792787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 msgr2=0x5641f227cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 -- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 msgr2=0x5641f227f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f227f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 -- 172.21.15.16:0/3097792787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f227d510 con 0x5641f2260b40 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf137fe640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf13fff640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf04009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595d1a34f9541495 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0404a030 con 0x5641f2260b40 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8477+0+0 (secure 0 0 0) 0x7faf04005020 con 0x5641f2260b40 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.384+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0802d070 con 0x5641f2260b40 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faf04054050 con 0x5641f2260b40 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 10 client.17325 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641f22e4560 con 0x5641f2260b40 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0802d070 con 0x5641f2260b40 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faf04004830 con 0x5641f2260b40 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 10 client.17325 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 10 client.17325 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 20 client.17325 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f22e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 4 client.17325 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641f21ccd70 con 0x5641f2279e30 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.385+0000 7faf137fe640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f22e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faef37fe640 10 client.17325 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf137fe640 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f22e6d50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86f78ad4af9b34e9 server_cookie=99c619fd5292a50e in_seq=0 out_seq=0 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faef37fe640 1 -- 172.21.15.16:0/3097792787 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641f21ccd70 con 0x5641f2279e30 2024-09-12T01:03:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faef37fe640 10 client.17325 handle_command_reply: tid=0 2024-09-12T01:03:04.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 client.17325 shutdown 2024-09-12T01:03:04.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 msgr2=0x5641f227ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf04009880 comp rx=0 tx=0).stop 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 shutdown_connections 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5641f2279e30 0x5641f22e6d50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641f226a370 0x5641f227cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641f2260b40 0x5641f227ca60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 --2- 172.21.15.16:0/3097792787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f225f5d0 0x5641f227f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 >> 172.21.15.16:0/3097792787 conn(0x5641f2228310 msgr2=0x5641f225a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.386+0000 7faef37fe640 0 client.17325 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 shutdown_connections 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1c805740 1 -- 172.21.15.16:0/3097792787 wait complete. 2024-09-12T01:03:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1c805740 20 client.17325 trim_cache size 0 max 16384 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 msgr2=0x7faefc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 0x7faefc0a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf0c09b880 tx=0x7faf0c005960 comp rx=0 tx=0).stop 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 msgr2=0x7faf1419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00cad0 tx=0x7faf0c00cf90 comp rx=0 tx=0).stop 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 shutdown_connections 2024-09-12T01:03:04.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faefc0a2d70 0x7faefc0a5230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf14106f10 0x7faf1419cdb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf14102790 0x7faf14198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 --2- 172.21.15.16:0/2392762671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf14101de0 0x7faf141984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 >> 172.21.15.16:0/2392762671 conn(0x7faf140fd8c0 msgr2=0x7faf140fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 shutdown_connections 2024-09-12T01:03:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:04.387+0000 7faf1a42c640 1 -- 172.21.15.16:0/2392762671 wait complete. 2024-09-12T01:03:05.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:03:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:05 smithi016 ceph-mon[30972]: pgmap v213: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:05.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:05 smithi099 ceph-mon[35654]: pgmap v213: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:05.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:05 smithi159 ceph-mon[35566]: pgmap v213: 97 pgs: 97 active+clean; 7.6 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 -- 172.21.15.16:0/771107232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764106f10 msgr2=0x7fe76410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 --2- 172.21.15.16:0/771107232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764106f10 0x7fe76410ea90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009ca0 tx=0x7fe75c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 -- 172.21.15.16:0/771107232 shutdown_connections 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 --2- 172.21.15.16:0/771107232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764106f10 0x7fe76410ea90 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 --2- 172.21.15.16:0/771107232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764102790 0x7fe7641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 --2- 172.21.15.16:0/771107232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764101de0 0x7fe7641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 -- 172.21.15.16:0/771107232 >> 172.21.15.16:0/771107232 conn(0x7fe7640fd8c0 msgr2=0x7fe7640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 -- 172.21.15.16:0/771107232 shutdown_connections 2024-09-12T01:03:05.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.873+0000 7fe769c0c640 1 -- 172.21.15.16:0/771107232 wait complete. 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 Processor -- start 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 -- start start 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764102790 0x7fe764198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764106f10 0x7fe76419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe764111f90 con 0x7fe764101de0 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe764111e10 con 0x7fe764102790 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.874+0000 7fe769c0c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe764112110 con 0x7fe764106f10 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39160/0 (socket says 172.21.15.16:39160) 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 -- 172.21.15.16:0/4088391677 learned_addr learned my addr 172.21.15.16:0/4088391677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe762ffd640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764102790 0x7fe764198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe763fff640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764106f10 0x7fe76419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 -- 172.21.15.16:0/4088391677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764106f10 msgr2=0x7fe76419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764106f10 0x7fe76419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 -- 172.21.15.16:0/4088391677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764102790 msgr2=0x7fe764198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764102790 0x7fe764198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 -- 172.21.15.16:0/4088391677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76419d3e0 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe7637fe640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d950 tx=0x7fe75400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe760ff9640 1 -- 172.21.15.16:0/4088391677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754017070 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76419d6d0 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe760ff9640 1 -- 172.21.15.16:0/4088391677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75400de80 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe760ff9640 1 -- 172.21.15.16:0/4088391677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75400f620 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.875+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a4fb0 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.878+0000 7fe760ff9640 1 -- 172.21.15.16:0/4088391677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe754009c30 con 0x7fe764101de0 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.878+0000 7fe76bfe5740 1 Processor -- start 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.878+0000 7fe76bfe5740 1 -- start start 2024-09-12T01:03:05.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 0x556e5c0d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0e3370 0x556e5c0e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e5c03e2f0 con 0x556e5c0d85d0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556e5bfe5cf0 con 0x556e5c0d9b40 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556e5c03a2b0 con 0x556e5c0e3370 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe760ff9640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 0x7fe74c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe7637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 0x556e5c0d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:52590/0 (socket says 172.21.15.16:52590) 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 -- 172.21.15.16:0/1410818427 learned_addr learned my addr 172.21.15.16:0/1410818427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 0x7fe74c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe760ff9640 1 -- 172.21.15.16:0/4088391677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7fe7540cf180 con 0x7fe764101de0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe762ffd640 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0e3370 0x556e5c0e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 -- 172.21.15.16:0/1410818427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0e3370 msgr2=0x556e5c0e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0e3370 0x556e5c0e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 -- 172.21.15.16:0/1410818427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 msgr2=0x556e5c0d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 0x556e5c0d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe763fff640 1 -- 172.21.15.16:0/1410818427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5bfca3a0 con 0x556e5c0d9b40 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.879+0000 7fe7637fe640 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 0x556e5c0d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.880+0000 7fe763fff640 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005fe0 tx=0x7fe75c0316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df9e7520b9d7c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.880+0000 7fe763fff640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 0x7fe74c0a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe75c035000 tx=0x7fe75c048000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.880+0000 7fe749ffb640 1 -- 172.21.15.16:0/1410818427 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c04e030 con 0x556e5c0d9b40 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.880+0000 7fe749ffb640 1 -- 172.21.15.16:0/1410818427 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c036040 con 0x556e5c0d9b40 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.880+0000 7fe749ffb640 1 -- 172.21.15.16:0/1410818427 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c004da0 con 0x556e5c0d9b40 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1410818427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 msgr2=0x556e5c0d8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005fe0 tx=0x7fe75c0316e0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1410818427 shutdown_connections 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0e3370 0x556e5c0e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0d85d0 0x556e5c0d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1410818427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0d8090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1410818427 >> 172.21.15.16:0/1410818427 conn(0x556e5c0a1310 msgr2=0x556e5c0d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1410818427 shutdown_connections 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1410818427 wait complete. 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 -- start start 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 Processor -- start 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.881+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e5c0e6010 con 0x556e5c0e3370 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556e5bff9060 con 0x556e5c0d85d0 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556e5c0e6190 con 0x556e5c0d9b40 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 10 client.0 _ll_register_callbacks cb 0x556e5bd4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39176/0 (socket says 172.21.15.16:39176) 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 -- 172.21.15.16:0/2725491281 learned_addr learned my addr 172.21.15.16:0/2725491281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe7637fe640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe762ffd640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 -- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 msgr2=0x556e5c0e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 -- 172.21.15.16:0/2725491281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 msgr2=0x556e5c0e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 -- 172.21.15.16:0/2725491281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5bdaf3c0 con 0x556e5c0e3370 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe763fff640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe75c02ade0 tx=0x7fe75c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c961f477572bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c056020 con 0x556e5c0e3370 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7fe75c02fe20 con 0x556e5c0e3370 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.882+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c036040 con 0x556e5c0e3370 2024-09-12T01:03:05.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe75c061050 con 0x556e5c0e3370 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 10 client.17337 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e5c0e6010 con 0x556e5c0e3370 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c036040 con 0x556e5c0e3370 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe75c04a620 con 0x556e5c0e3370 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 10 client.17337 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 10 client.17337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 20 client.17337 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c0f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 4 client.17337 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e5c04b860 con 0x556e5c0f2c80 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.883+0000 7fe7637fe640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c0f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.884+0000 7fe7637fe640 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c0f5070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c457b4bf22db5619 server_cookie=43225543798f7b79 in_seq=0 out_seq=0 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.884+0000 7fe742ffd640 10 client.17337 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.884+0000 7fe742ffd640 1 -- 172.21.15.16:0/2725491281 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e5c04b860 con 0x556e5c0f2c80 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.884+0000 7fe742ffd640 10 client.17337 handle_command_reply: tid=0 2024-09-12T01:03:05.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 client.17337 shutdown 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 msgr2=0x556e5c0e07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe75c02ade0 tx=0x7fe75c004b40 comp rx=0 tx=0).stop 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 shutdown_connections 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c0f5070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0e07f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe742ffd640 0 client.17337 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2725491281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 >> 172.21.15.16:0/2725491281 conn(0x556e5c0a1310 msgr2=0x556e5c0da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 shutdown_connections 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2725491281 wait complete. 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.885+0000 7fe76bfe5740 20 client.17337 trim_cache size 0 max 16384 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.898+0000 7fe76bfe5740 1 Processor -- start 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 -- start start 2024-09-12T01:03:05.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d85d0 0x556e5c0d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641ad900 con 0x556e5c0e3370 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641af440 con 0x556e5c0d9b40 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641adfd0 con 0x556e5c0d85d0 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39190/0 (socket says 172.21.15.16:39190) 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 -- 172.21.15.16:0/2117691425 learned_addr learned my addr 172.21.15.16:0/2117691425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe762ffd640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe7637fe640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d85d0 0x556e5c0d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 -- 172.21.15.16:0/2117691425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d85d0 msgr2=0x556e5c0d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d85d0 0x556e5c0d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 -- 172.21.15.16:0/2117691425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 msgr2=0x556e5c0e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0e1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 -- 172.21.15.16:0/2117691425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5c0e5ad0 con 0x556e5c0e3370 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe762ffd640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.899+0000 7fe763fff640 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe75c004c50 tx=0x7fe75c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cefaeb563b1bd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe7497fa640 1 -- 172.21.15.16:0/2117691425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c056040 con 0x556e5c0e3370 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe7497fa640 1 -- 172.21.15.16:0/2117691425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c036040 con 0x556e5c0e3370 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe7497fa640 1 -- 172.21.15.16:0/2117691425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c046040 con 0x556e5c0e3370 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2117691425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 msgr2=0x556e5c0db120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe75c004c50 tx=0x7fe75c005fd0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2117691425 shutdown_connections 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0db120 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d9b40 0x556e5c0e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/2117691425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d85d0 0x556e5c0d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2117691425 >> 172.21.15.16:0/2117691425 conn(0x556e5c0a1310 msgr2=0x556e5c0e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2117691425 shutdown_connections 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.900+0000 7fe76bfe5740 1 -- 172.21.15.16:0/2117691425 wait complete. 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 -- start start 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 Processor -- start 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641aed70 con 0x556e5c0e3370 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641ae6a0 con 0x556e5c0d85d0 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641ab900 con 0x556e5c0d9b40 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe76bfe5740 10 client.0 _ll_register_callbacks cb 0x7fe754010c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.901+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39196/0 (socket says 172.21.15.16:39196) 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 -- 172.21.15.16:0/1972054713 learned_addr learned my addr 172.21.15.16:0/1972054713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe762ffd640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe7637fe640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 -- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 msgr2=0x556e5c0dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 -- 172.21.15.16:0/1972054713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 msgr2=0x556e5c0ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 -- 172.21.15.16:0/1972054713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5c0ddcd0 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe763fff640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe75c004d30 tx=0x7fe75c04ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293c04e021d8b658 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe742ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c06a060 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7fe75c04a480 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c036040 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe75c056040 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe76bfe5740 10 client.17343 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.902+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e5c15d9d0 con 0x556e5c0e3370 2024-09-12T01:03:05.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75c036040 con 0x556e5c0e3370 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe75c06e440 con 0x556e5c0e3370 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 10 client.17343 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 10 client.17343 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 20 client.17343 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c15ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 4 client.17343 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e5c045d70 con 0x556e5c0f2c80 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe7637fe640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c15ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:05.944 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe7637fe640 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c15ffe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce498ff5c5f1a18d server_cookie=b0cd8f964be8478d in_seq=0 out_seq=0 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.903+0000 7fe742ffd640 10 client.17343 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe742ffd640 1 -- 172.21.15.16:0/1972054713 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e5c045d70 con 0x556e5c0f2c80 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe742ffd640 10 client.17343 handle_command_reply: tid=0 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 client.17343 shutdown 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 msgr2=0x556e5c0dd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe75c004d30 tx=0x7fe75c04ab10 comp rx=0 tx=0).stop 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 shutdown_connections 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x556e5c0f2c80 0x556e5c15ffe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e5c0e3370 0x556e5c0dd4a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe742ffd640 0 client.17343 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e5c0d9b40 0x556e5c0dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 --2- 172.21.15.16:0/1972054713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556e5c0d85d0 0x556e5c0ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 >> 172.21.15.16:0/1972054713 conn(0x556e5c0a1310 msgr2=0x556e5c0e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 shutdown_connections 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 1 -- 172.21.15.16:0/1972054713 wait complete. 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.904+0000 7fe76bfe5740 20 client.17343 trim_cache size 0 max 16384 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 msgr2=0x7fe74c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 0x7fe74c0a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe75c035000 tx=0x7fe75c048000 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 msgr2=0x7fe764198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d950 tx=0x7fe75400dd50 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 shutdown_connections 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe74c0a2cf0 0x7fe74c0a51b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe764106f10 0x7fe76419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe764102790 0x7fe764198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 --2- 172.21.15.16:0/4088391677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe764101de0 0x7fe764198430 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 >> 172.21.15.16:0/4088391677 conn(0x7fe7640fd8c0 msgr2=0x7fe7640feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 shutdown_connections 2024-09-12T01:03:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:05.905+0000 7fe769c0c640 1 -- 172.21.15.16:0/4088391677 wait complete. 2024-09-12T01:03:06.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:06 smithi099 ceph-mon[35654]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:03:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:06 smithi159 ceph-mon[35566]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:03:06.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:06 smithi016 ceph-mon[30972]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T01:03:06.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:03:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:07 smithi016 ceph-mon[30972]: pgmap v215: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:07.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.499+0000 7f390cda2640 1 -- 172.21.15.16:0/1633064702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908102790 msgr2=0x7f39081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.499+0000 7f390cda2640 1 --2- 172.21.15.16:0/1633064702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908102790 0x7f39081067e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 -- 172.21.15.16:0/1633064702 shutdown_connections 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 --2- 172.21.15.16:0/1633064702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908106f10 0x7f390810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 --2- 172.21.15.16:0/1633064702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908102790 0x7f39081067e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 --2- 172.21.15.16:0/1633064702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908101de0 0x7f39081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 -- 172.21.15.16:0/1633064702 >> 172.21.15.16:0/1633064702 conn(0x7f39080fd8c0 msgr2=0x7f39080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 -- 172.21.15.16:0/1633064702 shutdown_connections 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 -- 172.21.15.16:0/1633064702 wait complete. 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.500+0000 7f390cda2640 1 Processor -- start 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 -- start start 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908101de0 0x7f3908198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 0x7f3908198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111f70 con 0x7f3908106f10 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111df0 con 0x7f3908102790 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390cda2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39081120f0 con 0x7f3908101de0 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39210/0 (socket says 172.21.15.16:39210) 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 -- 172.21.15.16:0/3884444185 learned_addr learned my addr 172.21.15.16:0/3884444185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f390659c640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908101de0 0x7f3908198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3905d9b640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 0x7f3908198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 -- 172.21.15.16:0/3884444185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908101de0 msgr2=0x7f3908198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908101de0 0x7f3908198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 -- 172.21.15.16:0/3884444185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 msgr2=0x7f3908198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 0x7f3908198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3906d9d640 1 -- 172.21.15.16:0/3884444185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819d400 con 0x7f3908106f10 2024-09-12T01:03:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.501+0000 7f3905d9b640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 0x7f3908198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f3906d9d640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f390000eaa0 tx=0x7f390000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f38ff7fe640 1 -- 172.21.15.16:0/3884444185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390000cdb0 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390819d6f0 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f38ff7fe640 1 -- 172.21.15.16:0/3884444185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3900005020 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f38ff7fe640 1 -- 172.21.15.16:0/3884444185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39000116b0 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.502+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a4fd0 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.504+0000 7f38ff7fe640 1 -- 172.21.15.16:0/3884444185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3900005560 con 0x7f3908106f10 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.504+0000 7f390f17b740 1 Processor -- start 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.504+0000 7f390f17b740 1 -- start start 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c1791b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c178812f0 con 0x556c1791b5d0 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c17828cf0 con 0x556c1792a780 2024-09-12T01:03:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390f17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c1787d2b0 con 0x556c1791cb40 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39212/0 (socket says 172.21.15.16:39212) 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 -- 172.21.15.16:0/1935048442 learned_addr learned my addr 172.21.15.16:0/1935048442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f38ff7fe640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 0x7f38f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f3905d9b640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f3905d9b640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c1791b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f3906d9d640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 0x7f38f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 -- 172.21.15.16:0/1935048442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 msgr2=0x556c1791b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c1791b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 -- 172.21.15.16:0/1935048442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 msgr2=0x556c1792cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 -- 172.21.15.16:0/1935048442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c1780d3a0 con 0x556c1791b5d0 2024-09-12T01:03:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f3905d9b640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c1791b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f38ff7fe640 1 -- 172.21.15.16:0/3884444185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f39000cf6d0 con 0x7f3908106f10 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.505+0000 7f390659c640 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009cf0 tx=0x7f38f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb7c3c6564e9095 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f38fcff9640 1 -- 172.21.15.16:0/1935048442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004230 con 0x556c1791b5d0 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f3906d9d640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 0x7f38f40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f390009b860 tx=0x7f390001c040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f38fcff9640 1 -- 172.21.15.16:0/1935048442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80043d0 con 0x556c1791b5d0 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f38fcff9640 1 -- 172.21.15.16:0/1935048442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80198c0 con 0x556c1791b5d0 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 -- 172.21.15.16:0/1935048442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 msgr2=0x556c1791ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009cf0 tx=0x7f38f8011810 comp rx=0 tx=0).stop 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 -- 172.21.15.16:0/1935048442 shutdown_connections 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1791ba10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 --2- 172.21.15.16:0/1935048442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c1791b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.506+0000 7f390f17b740 1 -- 172.21.15.16:0/1935048442 >> 172.21.15.16:0/1935048442 conn(0x556c178e4310 msgr2=0x556c17917c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- 172.21.15.16:0/1935048442 shutdown_connections 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- 172.21.15.16:0/1935048442 wait complete. 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- start start 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 Processor -- start 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 0x556c179273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1792a780 0x556c17927980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c1792d480 con 0x556c1791b5d0 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c1783c060 con 0x556c1791cb40 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c1792d600 con 0x556c1792a780 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390f17b740 10 client.0 _ll_register_callbacks cb 0x556c1758f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39226/0 (socket says 172.21.15.16:39226) 2024-09-12T01:03:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f3905d9b640 1 -- 172.21.15.16:0/362384695 learned_addr learned my addr 172.21.15.16:0/362384695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.507+0000 7f390659c640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 0x556c179273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1792a780 0x556c17927980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 -- 172.21.15.16:0/362384695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1792a780 msgr2=0x556c17927980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1792a780 0x556c17927980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 -- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 msgr2=0x556c179273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 0x556c179273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 -- 172.21.15.16:0/362384695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c175f23c0 con 0x556c1791b5d0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f390659c640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 0x556c179273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f3905d9b640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f38f0033000 tx=0x7f38f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa81e2e6e0d56b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f38edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0045020 con 0x556c1791b5d0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f80043d0 con 0x556c1791b5d0 2024-09-12T01:03:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f38f00037f0 con 0x556c1791b5d0 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38f0050050 con 0x556c1791b5d0 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f390f17b740 10 client.17355 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c17938e40 con 0x556c1791b5d0 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.508+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80043d0 con 0x556c1791b5d0 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38f003a050 con 0x556c1791b5d0 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 10 client.17355 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 10 client.17355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 20 client.17355 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c1793a300 0x556c1793c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 4 client.17355 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c1788e860 con 0x556c1793a300 2024-09-12T01:03:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390659c640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c1793a300 0x556c1793c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f38edffb640 10 client.17355 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.509+0000 7f390659c640 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c1793a300 0x556c1793c760 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9b209b1c870b5b7 server_cookie=feb646f967060807 in_seq=0 out_seq=0 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f38edffb640 1 -- 172.21.15.16:0/362384695 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c1788e860 con 0x556c1793a300 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f38edffb640 10 client.17355 handle_command_reply: tid=0 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 client.17355 shutdown 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 msgr2=0x556c1792a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f38f0033000 tx=0x7f38f002a410 comp rx=0 tx=0).stop 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 shutdown_connections 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c1793a300 0x556c1793c760 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1792a780 0x556c17927980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f38edffb640 0 client.17355 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1791cb40 0x556c179273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 --2- 172.21.15.16:0/362384695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c1792a130 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 >> 172.21.15.16:0/362384695 conn(0x556c178e4310 msgr2=0x556c1791d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 shutdown_connections 2024-09-12T01:03:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 1 -- 172.21.15.16:0/362384695 wait complete. 2024-09-12T01:03:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.510+0000 7f390f17b740 20 client.17355 trim_cache size 0 max 16384 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 Processor -- start 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 -- start start 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17926ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39081ab780 con 0x556c1791b5d0 2024-09-12T01:03:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39081ad2c0 con 0x556c1792a780 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f390f17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39081abe50 con 0x556c1791cb40 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39232/0 (socket says 172.21.15.16:39232) 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 -- 172.21.15.16:0/4203071923 learned_addr learned my addr 172.21.15.16:0/4203071923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 -- 172.21.15.16:0/4203071923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 msgr2=0x556c17926ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17926ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 -- 172.21.15.16:0/4203071923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 msgr2=0x556c1792bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 -- 172.21.15.16:0/4203071923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c17918cf0 con 0x556c1791b5d0 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.524+0000 7f3905d9b640 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f38f00092a0 tx=0x7f38f0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a34a283e261d8df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f38effff640 1 -- 172.21.15.16:0/4203071923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0045040 con 0x556c1791b5d0 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f38effff640 1 -- 172.21.15.16:0/4203071923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f0005030 con 0x556c1791b5d0 2024-09-12T01:03:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f38effff640 1 -- 172.21.15.16:0/4203071923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0035040 con 0x556c1791b5d0 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 -- 172.21.15.16:0/4203071923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 msgr2=0x556c17936720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f38f00092a0 tx=0x7f38f0005b50 comp rx=0 tx=0).stop 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 -- 172.21.15.16:0/4203071923 shutdown_connections 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c1792bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17926ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 --2- 172.21.15.16:0/4203071923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17936720 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 -- 172.21.15.16:0/4203071923 >> 172.21.15.16:0/4203071923 conn(0x556c178e4310 msgr2=0x556c17925cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 -- 172.21.15.16:0/4203071923 shutdown_connections 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.525+0000 7f390f17b740 1 -- 172.21.15.16:0/4203071923 wait complete. 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 -- start start 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 Processor -- start 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17932890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c17932df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39081acbf0 con 0x556c1791b5d0 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39081ac520 con 0x556c1792a780 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39081a9780 con 0x556c1791cb40 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390f17b740 10 client.0 _ll_register_callbacks cb 0x7f38f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39234/0 (socket says 172.21.15.16:39234) 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 -- 172.21.15.16:0/712949470 learned_addr learned my addr 172.21.15.16:0/712949470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390659c640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17932890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 -- 172.21.15.16:0/712949470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 msgr2=0x556c17932890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17932890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 -- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 msgr2=0x556c17932df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c17932df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f3905d9b640 1 -- 172.21.15.16:0/712949470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c17941b10 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.526+0000 7f390659c640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17932890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f3905d9b640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f38f0000c00 tx=0x7f38f003c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e89183f9b404fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0035040 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f0005030 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f38f00051e0 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38f00053e0 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 10 client.17361 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c179a5210 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f0005030 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38f005c070 con 0x556c1791b5d0 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 10 client.17361 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 10 client.17361 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 20 client.17361 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c179a5880 0x556c179a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 4 client.17361 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.527+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c17888d70 con 0x556c179a5880 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390659c640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c179a5880 0x556c179a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390659c640 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c179a5880 0x556c179a7c70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8549db8ad2b3a9cc server_cookie=f4b7f580e01019 in_seq=0 out_seq=0 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f38edffb640 10 client.17361 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f38edffb640 1 -- 172.21.15.16:0/712949470 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c17888d70 con 0x556c179a5880 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f38edffb640 10 client.17361 handle_command_reply: tid=0 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 client.17361 shutdown 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 msgr2=0x556c17925020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f38f0000c00 tx=0x7f38f003c410 comp rx=0 tx=0).stop 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 shutdown_connections 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x556c179a5880 0x556c179a7c70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c1792a780 0x556c17932df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c1791cb40 0x556c17932890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 --2- 172.21.15.16:0/712949470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c1791b5d0 0x556c17925020 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 >> 172.21.15.16:0/712949470 conn(0x556c178e4310 msgr2=0x556c17916500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.528+0000 7f38edffb640 0 client.17361 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 shutdown_connections 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390f17b740 1 -- 172.21.15.16:0/712949470 wait complete. 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390f17b740 20 client.17361 trim_cache size 0 max 16384 2024-09-12T01:03:07.533 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 msgr2=0x7f38f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 0x7f38f40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f390009b860 tx=0x7f390001c040 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 msgr2=0x7f390819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f390000eaa0 tx=0x7f390000ef60 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 shutdown_connections 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38f40a2cf0 0x7f38f40a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3908106f10 0x7f390819cd20 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3908102790 0x7f3908198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 --2- 172.21.15.16:0/3884444185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3908101de0 0x7f3908198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 >> 172.21.15.16:0/3884444185 conn(0x7f39080fd8c0 msgr2=0x7f39080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 shutdown_connections 2024-09-12T01:03:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:07.529+0000 7f390cda2640 1 -- 172.21.15.16:0/3884444185 wait complete. 2024-09-12T01:03:07.535 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:07 smithi099 ceph-mon[35654]: pgmap v215: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:07 smithi159 ceph-mon[35566]: pgmap v215: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:08.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:03:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 -- 172.21.15.16:0/2322672144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 msgr2=0x7f26bc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 --2- 172.21.15.16:0/2322672144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc107b90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009930 tx=0x7f26b002a420 comp rx=0 tx=0).stop 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 -- 172.21.15.16:0/2322672144 shutdown_connections 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 --2- 172.21.15.16:0/2322672144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 --2- 172.21.15.16:0/2322672144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc107b90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 --2- 172.21.15.16:0/2322672144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 -- 172.21.15.16:0/2322672144 >> 172.21.15.16:0/2322672144 conn(0x7f26bc0fc7d0 msgr2=0x7f26bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 -- 172.21.15.16:0/2322672144 shutdown_connections 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.165+0000 7f26c2994640 1 -- 172.21.15.16:0/2322672144 wait complete. 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 Processor -- start 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 -- start start 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1120a0 con 0x7f26bc1036e0 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc111f20 con 0x7f26bc1082a0 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c2994640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc112220 con 0x7f26bc102d30 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45550/0 (socket says 172.21.15.16:45550) 2024-09-12T01:03:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26bbfff640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26bb7fe640 1 -- 172.21.15.16:0/353251161 learned_addr learned my addr 172.21.15.16:0/353251161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.166+0000 7f26c0f31640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 -- 172.21.15.16:0/353251161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 msgr2=0x7f26bc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 -- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 msgr2=0x7f26bc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 -- 172.21.15.16:0/353251161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc19d590 con 0x7f26bc1082a0 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26bbfff640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c0f31640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc19ceb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009fd0 tx=0x7f26b400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26b97fa640 1 -- 172.21.15.16:0/353251161 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b400ce20 con 0x7f26bc1082a0 2024-09-12T01:03:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc19d880 con 0x7f26bc1082a0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26b97fa640 1 -- 172.21.15.16:0/353251161 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b4010430 con 0x7f26bc1082a0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a5160 con 0x7f26bc1082a0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.167+0000 7f26b97fa640 1 -- 172.21.15.16:0/353251161 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b400f620 con 0x7f26bc1082a0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 Processor -- start 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26b97fa640 1 -- 172.21.15.16:0/353251161 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26b4010950 con 0x7f26bc1082a0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 -- start start 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f44b40 0x562922f43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 0x562922f50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562922ea92f0 con 0x562922f435d0 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562922e50cf0 con 0x562922f44b40 2024-09-12T01:03:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562922ea52b0 con 0x562922f4e370 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45554/0 (socket says 172.21.15.16:45554) 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26bbfff640 1 -- 172.21.15.16:0/1585950146 learned_addr learned my addr 172.21.15.16:0/1585950146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f44b40 0x562922f43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.170+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 0x562922f50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 -- 172.21.15.16:0/1585950146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 msgr2=0x562922f50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 0x562922f50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 -- 172.21.15.16:0/1585950146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f44b40 msgr2=0x562922f43090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f44b40 0x562922f43090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 -- 172.21.15.16:0/1585950146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562922e353a0 con 0x562922f435d0 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 0x562922f50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26b97fa640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 0x7f26a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 0x7f26a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0115c0 tx=0x7f26ac011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04e0d04473b80c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26b97fa640 1 -- 172.21.15.16:0/353251161 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f26b40cf740 con 0x7f26bc1082a0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26a27fc640 1 -- 172.21.15.16:0/1585950146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00f640 con 0x562922f435d0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26a27fc640 1 -- 172.21.15.16:0/1585950146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac00fcc0 con 0x562922f435d0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.171+0000 7f26a27fc640 1 -- 172.21.15.16:0/1585950146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac019cd0 con 0x562922f435d0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1585950146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 msgr2=0x562922f43a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0115c0 tx=0x7f26ac011a80 comp rx=0 tx=0).stop 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 0x7f26a40a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b002a420 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1585950146 shutdown_connections 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f4e370 0x562922f50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f43a10 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1585950146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f44b40 0x562922f43090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1585950146 >> 172.21.15.16:0/1585950146 conn(0x562922f0c310 msgr2=0x562922f3fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1585950146 shutdown_connections 2024-09-12T01:03:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1585950146 wait complete. 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 -- start start 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.172+0000 7f26c4d6d740 1 Processor -- start 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562922f51070 con 0x562922f435d0 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562922e64060 con 0x562922f4e370 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562922f511f0 con 0x562922f44b40 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c4d6d740 10 client.0 _ll_register_callbacks cb 0x562922bb78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45560/0 (socket says 172.21.15.16:45560) 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 -- 172.21.15.16:0/1593285647 learned_addr learned my addr 172.21.15.16:0/1593285647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c0f31640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 -- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 msgr2=0x562922f4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 -- 172.21.15.16:0/1593285647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 msgr2=0x562922f4b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f4b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 -- 172.21.15.16:0/1593285647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922c1a3c0 con 0x562922f435d0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26c0f31640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f4b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f26ac019c00 tx=0x7f26ac019c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55ca6bb761d7fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.173+0000 7f269b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac026020 con 0x562922f435d0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac00fcc0 con 0x562922f435d0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f26ac004250 con 0x562922f435d0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26ac031050 con 0x562922f435d0 2024-09-12T01:03:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 10 client.17373 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562922f5cac0 con 0x562922f435d0 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac00fcc0 con 0x562922f435d0 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26ac003790 con 0x562922f435d0 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 10 client.17373 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 10 client.17373 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 20 client.17373 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922f5df20 0x562922f60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 4 client.17373 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.174+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562922eb6860 con 0x562922f5df20 2024-09-12T01:03:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.175+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922f5df20 0x562922f60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.175+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922f5df20 0x562922f60310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec5d165ff55f5c56 server_cookie=e2f29b42bf97a3a6 in_seq=0 out_seq=0 2024-09-12T01:03:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.175+0000 7f269b7fe640 10 client.17373 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f269b7fe640 1 -- 172.21.15.16:0/1593285647 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562922eb6860 con 0x562922f5df20 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f269b7fe640 10 client.17373 handle_command_reply: tid=0 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 client.17373 shutdown 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 msgr2=0x562922f4dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f26ac019c00 tx=0x7f26ac019c30 comp rx=0 tx=0).stop 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 shutdown_connections 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922f5df20 0x562922f60310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f4b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1593285647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f4dd20 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 >> 172.21.15.16:0/1593285647 conn(0x562922f0c310 msgr2=0x562922f45930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f269b7fe640 0 client.17373 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 shutdown_connections 2024-09-12T01:03:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1593285647 wait complete. 2024-09-12T01:03:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.176+0000 7f26c4d6d740 20 client.17373 trim_cache size 0 max 16384 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.189+0000 7f26c4d6d740 1 Processor -- start 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.189+0000 7f26c4d6d740 1 -- start start 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f435d0 0x562922f3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ad960 con 0x562922f4e370 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1af4a0 con 0x562922f435d0 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ae030 con 0x562922f44b40 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45578/0 (socket says 172.21.15.16:45578) 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 -- 172.21.15.16:0/781785457 learned_addr learned my addr 172.21.15.16:0/781785457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26bbfff640 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f435d0 0x562922f3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 -- 172.21.15.16:0/781785457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 msgr2=0x562922f46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 -- 172.21.15.16:0/781785457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f435d0 msgr2=0x562922f3ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f435d0 0x562922f3ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 -- 172.21.15.16:0/781785457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562922f4b8e0 con 0x562922f4e370 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.190+0000 7f26c0f31640 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26b40ccf10 tx=0x7f26b409b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3252bce6e30035 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26a1ffb640 1 -- 172.21.15.16:0/781785457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b40b9070 con 0x562922f4e370 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26a1ffb640 1 -- 172.21.15.16:0/781785457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b40105e0 con 0x562922f4e370 2024-09-12T01:03:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26a1ffb640 1 -- 172.21.15.16:0/781785457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4086050 con 0x562922f4e370 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 -- 172.21.15.16:0/781785457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 msgr2=0x562922f46660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26b40ccf10 tx=0x7f26b409b8a0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 -- 172.21.15.16:0/781785457 shutdown_connections 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f4e370 0x562922f46660 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/781785457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f435d0 0x562922f3ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 -- 172.21.15.16:0/781785457 >> 172.21.15.16:0/781785457 conn(0x562922f0c310 msgr2=0x562922f3e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 -- 172.21.15.16:0/781785457 shutdown_connections 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.191+0000 7f26c4d6d740 1 -- 172.21.15.16:0/781785457 wait complete. 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 -- start start 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 Processor -- start 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1aedd0 con 0x562922f435d0 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ae700 con 0x562922f4e370 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc1ab960 con 0x562922f44b40 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26c4d6d740 10 client.0 _ll_register_callbacks cb 0x7f26ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45580/0 (socket says 172.21.15.16:45580) 2024-09-12T01:03:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26bbfff640 1 -- 172.21.15.16:0/1219390773 learned_addr learned my addr 172.21.15.16:0/1219390773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.192+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26c0f31640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 -- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 msgr2=0x562922f47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 -- 172.21.15.16:0/1219390773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 msgr2=0x562922f48020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f48020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 -- 172.21.15.16:0/1219390773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562922f48660 con 0x562922f435d0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26bbfff640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f26ac003d20 tx=0x7f26ac0198a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81c098510fda014 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f269b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac026040 con 0x562922f435d0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f26ac0052c0 con 0x562922f435d0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b40105e0 con 0x562922f435d0 2024-09-12T01:03:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26ac003e60 con 0x562922f435d0 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26c4d6d740 10 client.17379 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562922fc88a0 con 0x562922f435d0 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.193+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b40105e0 con 0x562922f435d0 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26ac022480 con 0x562922f435d0 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 10 client.17379 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 10 client.17379 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 20 client.17379 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922fc9040 0x562922fcb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 4 client.17379 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562922eb0d70 con 0x562922fc9040 2024-09-12T01:03:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922fc9040 0x562922fcb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f26bb7fe640 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922fc9040 0x562922fcb460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41edf38d526ad7fb server_cookie=bed9dcf8a711a5e in_seq=0 out_seq=0 2024-09-12T01:03:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.194+0000 7f269b7fe640 10 client.17379 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f269b7fe640 1 -- 172.21.15.16:0/1219390773 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562922eb0d70 con 0x562922fc9040 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f269b7fe640 10 client.17379 handle_command_reply: tid=0 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 client.17379 shutdown 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 msgr2=0x562922f56a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f26ac003d20 tx=0x7f26ac0198a0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 shutdown_connections 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x562922fc9040 0x562922fcb460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562922f4e370 0x562922f48020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562922f44b40 0x562922f47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f269b7fe640 0 client.17379 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 --2- 172.21.15.16:0/1219390773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562922f435d0 0x562922f56a20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 >> 172.21.15.16:0/1219390773 conn(0x562922f0c310 msgr2=0x562922f3ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 shutdown_connections 2024-09-12T01:03:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 1 -- 172.21.15.16:0/1219390773 wait complete. 2024-09-12T01:03:09.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.195+0000 7f26c4d6d740 20 client.17379 trim_cache size 0 max 16384 2024-09-12T01:03:09.201 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 msgr2=0x7f26a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 0x7f26a40a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b002a420 comp rx=0 tx=0).stop 2024-09-12T01:03:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 msgr2=0x7f26bc19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc19ceb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009fd0 tx=0x7f26b400ef10 comp rx=0 tx=0).stop 2024-09-12T01:03:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 shutdown_connections 2024-09-12T01:03:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26a40a2cf0 0x7f26a40a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26bc1082a0 0x7f26bc19ceb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26bc1036e0 0x7f26bc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 --2- 172.21.15.16:0/353251161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26bc102d30 0x7f26bc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 >> 172.21.15.16:0/353251161 conn(0x7f26bc0fc7d0 msgr2=0x7f26bc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 shutdown_connections 2024-09-12T01:03:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:09.196+0000 7f26c2994640 1 -- 172.21.15.16:0/353251161 wait complete. 2024-09-12T01:03:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:09 smithi016 ceph-mon[30972]: pgmap v216: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:09 smithi099 ceph-mon[35654]: pgmap v216: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:09 smithi159 ceph-mon[35566]: pgmap v216: 97 pgs: 97 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:03:09.726 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T01:03:09.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:09.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:03:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1207014374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 msgr2=0x7fc7c010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1207014374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c010ec40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009cd0 tx=0x7fc7b802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1207014374 shutdown_connections 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1207014374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c010ec40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1207014374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 0x7fc7c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1207014374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 0x7fc7c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1207014374 >> 172.21.15.16:0/1207014374 conn(0x7fc7c00fc7d0 msgr2=0x7fc7c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1207014374 shutdown_connections 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.498+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1207014374 wait complete. 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 Processor -- start 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 -- start start 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 0x7fc7c0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 0x7fc7c0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c01120a0 con 0x7fc7c01082a0 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0111f20 con 0x7fc7c01036e0 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7c56ca640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0112220 con 0x7fc7c0102d30 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45606/0 (socket says 172.21.15.16:45606) 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7bf7fe640 1 -- 172.21.15.16:0/1441109885 learned_addr learned my addr 172.21.15.16:0/1441109885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:10.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.499+0000 7fc7beffd640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 0x7fc7c0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7be7fc640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 0x7fc7c0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 -- 172.21.15.16:0/1441109885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 msgr2=0x7fc7c0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 0x7fc7c0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 -- 172.21.15.16:0/1441109885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 msgr2=0x7fc7c0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 0x7fc7c0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 -- 172.21.15.16:0/1441109885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c01a69f0 con 0x7fc7c01082a0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7bf7fe640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005cc0 tx=0x7fc7b80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8003e00 con 0x7fc7c01082a0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b8004520 con 0x7fc7c01082a0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c01a6c80 con 0x7fc7c01082a0 2024-09-12T01:03:10.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8030db0 con 0x7fc7c01082a0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.500+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c01a71c0 con 0x7fc7c01082a0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.501+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c0103110 con 0x7fc7c01082a0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.502+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc7b80046d0 con 0x7fc7c01082a0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.503+0000 7fc7b7fff640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 0x7fc7ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.503+0000 7fc7beffd640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 0x7fc7ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.503+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7fc7b802e030 con 0x7fc7c01082a0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.504+0000 7fc7beffd640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 0x7fc7ac0a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009bd0 tx=0x7fc7b0005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.508+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7b804b060 con 0x7fc7c01082a0 2024-09-12T01:03:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.726+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7c010a2b0 con 0x7fc7c01082a0 2024-09-12T01:03:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.727+0000 7fc7b7fff640 1 -- 172.21.15.16:0/1441109885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fc7b80bd750 con 0x7fc7c01082a0 2024-09-12T01:03:10.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:10.727 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 msgr2=0x7fc7ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 0x7fc7ac0a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009bd0 tx=0x7fc7b0005f90 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 msgr2=0x7fc7c0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005cc0 tx=0x7fc7b80093d0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 shutdown_connections 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7ac0a2cf0 0x7fc7ac0a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c01082a0 0x7fc7c0101410 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7c01036e0 0x7fc7c0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 --2- 172.21.15.16:0/1441109885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7c0102d30 0x7fc7c0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 >> 172.21.15.16:0/1441109885 conn(0x7fc7c00fc7d0 msgr2=0x7fc7c0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 shutdown_connections 2024-09-12T01:03:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:10.729+0000 7fc7c56ca640 1 -- 172.21.15.16:0/1441109885 wait complete. 2024-09-12T01:03:11.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:11 smithi016 ceph-mon[30972]: pgmap v217: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:11.489 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:11 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1441109885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:11.519 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T01:03:11.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T01:03:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:11 smithi099 ceph-mon[35654]: pgmap v217: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:11 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1441109885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:11 smithi159 ceph-mon[35566]: pgmap v217: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:11 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1441109885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 -- 172.21.15.16:0/999166338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 msgr2=0x7f5c5810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/999166338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5810e610 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009ca0 tx=0x7f5c5002a950 comp rx=0 tx=0).stop 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 -- 172.21.15.16:0/999166338 shutdown_connections 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/999166338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5810e610 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/999166338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 0x7f5c5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/999166338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 -- 172.21.15.16:0/999166338 >> 172.21.15.16:0/999166338 conn(0x7f5c580fc7a0 msgr2=0x7f5c580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 -- 172.21.15.16:0/999166338 shutdown_connections 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.769+0000 7f5c5d68b640 1 -- 172.21.15.16:0/999166338 wait complete. 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 Processor -- start 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 -- start start 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 0x7f5c581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111fd0 con 0x7f5c5810c170 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111e50 con 0x7f5c58102d40 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c5d68b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58112150 con 0x7f5c58103660 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.770+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45624/0 (socket says 172.21.15.16:45624) 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 0x7f5c581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 -- 172.21.15.16:0/370327030 learned_addr learned my addr 172.21.15.16:0/370327030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 -- 172.21.15.16:0/370327030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 msgr2=0x7f5c581989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 0x7f5c581989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 -- 172.21.15.16:0/370327030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 msgr2=0x7f5c58198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 -- 172.21.15.16:0/370327030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5819d420 con 0x7f5c5810c170 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c577fe640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002ae70 tx=0x7f5c5002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c3ffff640 1 -- 172.21.15.16:0/370327030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50034040 con 0x7f5c5810c170 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5819d6b0 con 0x7f5c5810c170 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c3ffff640 1 -- 172.21.15.16:0/370327030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c50030920 con 0x7f5c5810c170 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c3ffff640 1 -- 172.21.15.16:0/370327030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50045480 con 0x7f5c5810c170 2024-09-12T01:03:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.771+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581a4ff0 con 0x7f5c5810c170 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c3ffff640 1 -- 172.21.15.16:0/370327030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5c50037020 con 0x7f5c5810c170 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 Processor -- start 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 -- start start 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c3ffff640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 0x7f5c400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c3ffff640 1 -- 172.21.15.16:0/370327030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f5c500f0270 con 0x7f5c5810c170 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e7b40 0x55dddb1e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c567fc640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 0x7f5c400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dddb14c2f0 con 0x55dddb1f1370 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dddb0f3cf0 con 0x55dddb1e65d0 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.774+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dddb1482b0 con 0x55dddb1e7b40 2024-09-12T01:03:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45632/0 (socket says 172.21.15.16:45632) 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 -- 172.21.15.16:0/2429210363 learned_addr learned my addr 172.21.15.16:0/2429210363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e7b40 0x55dddb1e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c567fc640 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 -- 172.21.15.16:0/2429210363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e7b40 msgr2=0x55dddb1e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e7b40 0x55dddb1e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 -- 172.21.15.16:0/2429210363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 msgr2=0x55dddb1e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 -- 172.21.15.16:0/2429210363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddb0d83a0 con 0x55dddb1f1370 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c567fc640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 0x7f5c400a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c5810d490 tx=0x7f5c4c00b2e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5c50032000 tx=0x7f5c500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe87f6eb292001 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c3d7fa640 1 -- 172.21.15.16:0/2429210363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500d5070 con 0x55dddb1f1370 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c3d7fa640 1 -- 172.21.15.16:0/2429210363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c500098d0 con 0x55dddb1f1370 2024-09-12T01:03:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.775+0000 7f5c3d7fa640 1 -- 172.21.15.16:0/2429210363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5002c040 con 0x55dddb1f1370 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 -- 172.21.15.16:0/2429210363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 msgr2=0x55dddb1f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5c50032000 tx=0x7f5c500095c0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 -- 172.21.15.16:0/2429210363 shutdown_connections 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb1f3760 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/2429210363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e7b40 0x55dddb1e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 -- 172.21.15.16:0/2429210363 >> 172.21.15.16:0/2429210363 conn(0x55dddb1af310 msgr2=0x55dddb1e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 -- 172.21.15.16:0/2429210363 shutdown_connections 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.776+0000 7f5c5fa64740 1 -- 172.21.15.16:0/2429210363 wait complete. 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 -- start start 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 Processor -- start 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dddb1f3e30 con 0x55dddb1e65d0 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dddb107060 con 0x55dddb1e7b40 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dddb1f3fb0 con 0x55dddb1f1370 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c5fa64740 10 client.0 _ll_register_callbacks cb 0x55dddae5a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45636/0 (socket says 172.21.15.16:45636) 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 -- 172.21.15.16:0/3227491788 learned_addr learned my addr 172.21.15.16:0/3227491788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c577fe640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c567fc640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 -- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 msgr2=0x55dddb1ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 -- 172.21.15.16:0/3227491788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 msgr2=0x55dddb1ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c56ffd640 1 -- 172.21.15.16:0/3227491788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddaebd3c0 con 0x55dddb1e65d0 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.777+0000 7f5c567fc640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:11.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5c48010500 tx=0x7f5c48010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbdb0fb20669cce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48018070 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f5c48014640 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dddb159860 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c48024050 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 10 client.17397 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dddb1ff920 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dddb159860 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c480129f0 con 0x55dddb1e65d0 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 10 client.17397 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 20 client.17397 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:11.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb200dd0 0x55dddb203250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 4 client.17397 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.778+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dddb153d70 con 0x55dddb200dd0 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.779+0000 7f5c567fc640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb200dd0 0x55dddb203250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.779+0000 7f5c3aff5640 10 client.17397 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.779+0000 7f5c567fc640 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb200dd0 0x55dddb203250 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1569887db15a460a server_cookie=84ff3f7a0f461b3b in_seq=0 out_seq=0 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c3aff5640 1 -- 172.21.15.16:0/3227491788 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dddb153d70 con 0x55dddb200dd0 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c3aff5640 10 client.17397 handle_command_reply: tid=0 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 client.17397 shutdown 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 msgr2=0x55dddb1f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5c48010500 tx=0x7f5c48010900 comp rx=0 tx=0).stop 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 shutdown_connections 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb200dd0 0x55dddb203250 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/3227491788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e65d0 0x55dddb1f0e10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c3aff5640 0 client.17397 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 >> 172.21.15.16:0/3227491788 conn(0x55dddb1af310 msgr2=0x55dddb1e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 shutdown_connections 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 1 -- 172.21.15.16:0/3227491788 wait complete. 2024-09-12T01:03:11.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.780+0000 7f5c5fa64740 20 client.17397 trim_cache size 0 max 16384 2024-09-12T01:03:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.793+0000 7f5c5fa64740 1 Processor -- start 2024-09-12T01:03:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 -- start start 2024-09-12T01:03:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e65d0 0x55dddb1e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581ad930 con 0x55dddb1f1370 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581af470 con 0x55dddb1e7b40 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581ae000 con 0x55dddb1e65d0 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45642/0 (socket says 172.21.15.16:45642) 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 -- 172.21.15.16:0/4152915267 learned_addr learned my addr 172.21.15.16:0/4152915267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e65d0 0x55dddb1e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c567fc640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 -- 172.21.15.16:0/4152915267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e65d0 msgr2=0x55dddb1e9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e65d0 0x55dddb1e9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 -- 172.21.15.16:0/4152915267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 msgr2=0x55dddb1e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c577fe640 1 -- 172.21.15.16:0/4152915267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddb1eeb80 con 0x55dddb1f1370 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.794+0000 7f5c567fc640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c577fe640 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5c500bc800 tx=0x7f5c50003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9af42f15beec08f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c3cff9640 1 -- 172.21.15.16:0/4152915267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c500d5040 con 0x55dddb1f1370 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c3cff9640 1 -- 172.21.15.16:0/4152915267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c500bcf70 con 0x55dddb1f1370 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c3cff9640 1 -- 172.21.15.16:0/4152915267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5002c040 con 0x55dddb1f1370 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 -- 172.21.15.16:0/4152915267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 msgr2=0x55dddb208b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5c500bc800 tx=0x7f5c50003f10 comp rx=0 tx=0).stop 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 -- 172.21.15.16:0/4152915267 shutdown_connections 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1f1370 0x55dddb208b20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e7b40 0x55dddb1e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/4152915267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1e65d0 0x55dddb1e9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.795+0000 7f5c5fa64740 1 -- 172.21.15.16:0/4152915267 >> 172.21.15.16:0/4152915267 conn(0x55dddb1af310 msgr2=0x55dddb1ed1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- 172.21.15.16:0/4152915267 shutdown_connections 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- 172.21.15.16:0/4152915267 wait complete. 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- start start 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 Processor -- start 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581aeda0 con 0x55dddb1e7b40 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581ae6d0 con 0x55dddb1e65d0 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.796+0000 7f5c5fa64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581ab930 con 0x55dddb1f1370 2024-09-12T01:03:11.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45646/0 (socket says 172.21.15.16:45646) 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 -- 172.21.15.16:0/876408302 learned_addr learned my addr 172.21.15.16:0/876408302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c5fa64740 10 client.0 _ll_register_callbacks cb 0x7f5c48007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c577fe640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 -- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 msgr2=0x55dddb1eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 -- 172.21.15.16:0/876408302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 msgr2=0x55dddb1f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 -- 172.21.15.16:0/876408302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddb1eb800 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c567fc640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003c60 tx=0x7f5c4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6655973861210f66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c011d20 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8565+0+0 (secure 0 0 0) 0x7f5c4c01d430 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.797+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c500bcf70 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c4c034050 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 10 client.17403 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dddb26b980 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c500bcf70 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c4c02f5b0 con 0x55dddb1e7b40 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 10 client.17403 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 20 client.17403 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb26c180 0x55dddb26e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 4 client.17403 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.798+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dddb0d0170 con 0x55dddb26c180 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb26c180 0x55dddb26e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb26c180 0x55dddb26e5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2109b0fd82b04f server_cookie=1239eb84e84377cb in_seq=0 out_seq=0 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c3aff5640 10 client.17403 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c3aff5640 1 -- 172.21.15.16:0/876408302 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dddb0d0170 con 0x55dddb26c180 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c3aff5640 10 client.17403 handle_command_reply: tid=0 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 client.17403 shutdown 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 msgr2=0x55dddb1eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003c60 tx=0x7f5c4c004730 comp rx=0 tx=0).stop 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 shutdown_connections 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dddb26c180 0x55dddb26e5a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddb1f1370 0x55dddb1eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dddb1e7b40 0x55dddb1eac80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 --2- 172.21.15.16:0/876408302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dddb1e65d0 0x55dddb1f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 >> 172.21.15.16:0/876408302 conn(0x55dddb1af310 msgr2=0x55dddb1fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.799+0000 7f5c3aff5640 0 client.17403 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 shutdown_connections 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5fa64740 1 -- 172.21.15.16:0/876408302 wait complete. 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5fa64740 20 client.17403 trim_cache size 0 max 16384 2024-09-12T01:03:11.812 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 msgr2=0x7f5c400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 0x7f5c400a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c5810d490 tx=0x7f5c4c00b2e0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 msgr2=0x7f5c5819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.800+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5c5002ae70 tx=0x7f5c5002f5b0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 shutdown_connections 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c400a2cf0 0x7f5c400a51b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c5810c170 0x7f5c5819cd40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c58103660 0x7f5c581989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 --2- 172.21.15.16:0/370327030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c58102d40 0x7f5c58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 >> 172.21.15.16:0/370327030 conn(0x7f5c580fc7a0 msgr2=0x7f5c580fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:11.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 shutdown_connections 2024-09-12T01:03:11.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:11.801+0000 7f5c5d68b640 1 -- 172.21.15.16:0/370327030 wait complete. 2024-09-12T01:03:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 -- 172.21.15.16:0/418682883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c102790 msgr2=0x7fde7c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 --2- 172.21.15.16:0/418682883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c102790 0x7fde7c1067e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fde70009900 tx=0x7fde7002a410 comp rx=0 tx=0).stop 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 -- 172.21.15.16:0/418682883 shutdown_connections 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 --2- 172.21.15.16:0/418682883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c106f10 0x7fde7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 --2- 172.21.15.16:0/418682883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c102790 0x7fde7c1067e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 --2- 172.21.15.16:0/418682883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c101de0 0x7fde7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.028+0000 7fde80d76640 1 -- 172.21.15.16:0/418682883 >> 172.21.15.16:0/418682883 conn(0x7fde7c0fd8c0 msgr2=0x7fde7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.029+0000 7fde80d76640 1 -- 172.21.15.16:0/418682883 shutdown_connections 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.029+0000 7fde80d76640 1 -- 172.21.15.16:0/418682883 wait complete. 2024-09-12T01:03:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.029+0000 7fde80d76640 1 Processor -- start 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.029+0000 7fde80d76640 1 -- start start 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c102790 0x7fde7c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106f10 0x7fde7c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c111f70 con 0x7fde7c101de0 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c111df0 con 0x7fde7c106f10 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1120f0 con 0x7fde7c102790 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45672/0 (socket says 172.21.15.16:45672) 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 -- 172.21.15.16:0/4093774923 learned_addr learned my addr 172.21.15.16:0/4093774923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106f10 0x7fde7c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde79d9b640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c102790 0x7fde7c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 -- 172.21.15.16:0/4093774923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c102790 msgr2=0x7fde7c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c102790 0x7fde7c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 -- 172.21.15.16:0/4093774923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106f10 msgr2=0x7fde7c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106f10 0x7fde7c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 -- 172.21.15.16:0/4093774923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c19d420 con 0x7fde7c101de0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde7a59c640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00d950 tx=0x7fde6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde6b7fe640 1 -- 172.21.15.16:0/4093774923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c017070 con 0x7fde7c101de0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c19d710 con 0x7fde7c101de0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde6b7fe640 1 -- 172.21.15.16:0/4093774923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c009c30 con 0x7fde7c101de0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.030+0000 7fde6b7fe640 1 -- 172.21.15.16:0/4093774923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00f6b0 con 0x7fde7c101de0 2024-09-12T01:03:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.031+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1a4ff0 con 0x7fde7c101de0 2024-09-12T01:03:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde6b7fe640 1 -- 172.21.15.16:0/4093774923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde6c0109d0 con 0x7fde7c101de0 2024-09-12T01:03:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 Processor -- start 2024-09-12T01:03:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 -- start start 2024-09-12T01:03:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3184e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe31851140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe317a52a0 con 0x55fe3183f660 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fe3174cb10 con 0x55fe3184ed50 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde8314f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe317a1260 con 0x55fe3183ff80 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45682/0 (socket says 172.21.15.16:45682) 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde7ad9d640 1 -- 172.21.15.16:0/2594885520 learned_addr learned my addr 172.21.15.16:0/2594885520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde7a59c640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3184e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.034+0000 7fde79d9b640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe31851140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 -- 172.21.15.16:0/2594885520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 msgr2=0x55fe3184e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3184e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 -- 172.21.15.16:0/2594885520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 msgr2=0x55fe31851140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe31851140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 -- 172.21.15.16:0/2594885520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe31516b90 con 0x55fe3183f660 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7a59c640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3184e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde6b7fe640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 0x7fde640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7a59c640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 0x7fde640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde6b7fe640 1 -- 172.21.15.16:0/4093774923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fde6c0cf110 con 0x7fde7c101de0 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fde74009f40 tx=0x7fde74010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09f64251e55aaa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde68ff9640 1 -- 172.21.15.16:0/2594885520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74003b70 con 0x55fe3183f660 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde68ff9640 1 -- 172.21.15.16:0/2594885520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde74003d10 con 0x55fe3183f660 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.035+0000 7fde68ff9640 1 -- 172.21.15.16:0/2594885520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74011a40 con 0x55fe3183f660 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 -- 172.21.15.16:0/2594885520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 msgr2=0x55fe3183fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde7a59c640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 0x7fde640a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cc910 tx=0x7fde6c0d2c60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fde74009f40 tx=0x7fde74010860 comp rx=0 tx=0).stop 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 -- 172.21.15.16:0/2594885520 shutdown_connections 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe31851140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3184e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 --2- 172.21.15.16:0/2594885520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe3183fa40 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 -- 172.21.15.16:0/2594885520 >> 172.21.15.16:0/2594885520 conn(0x55fe3183bcf0 msgr2=0x55fe3183c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 -- 172.21.15.16:0/2594885520 shutdown_connections 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.036+0000 7fde8314f740 1 -- 172.21.15.16:0/2594885520 wait complete. 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 -- start start 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 Processor -- start 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 0x55fe3184e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3184c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fe31851980 con 0x55fe3183ff80 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fe3172d6e0 con 0x55fe3184ed50 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe31851b00 con 0x55fe3183f660 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde8314f740 10 client.0 _ll_register_callbacks cb 0x55fe31751a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde79d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde79d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45690/0 (socket says 172.21.15.16:45690) 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde79d9b640 1 -- 172.21.15.16:0/2382824414 learned_addr learned my addr 172.21.15.16:0/2382824414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 0x55fe3184e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.037+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3184c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 -- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 msgr2=0x55fe3184e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 0x55fe3184e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 -- 172.21.15.16:0/2382824414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 msgr2=0x55fe3184c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3184c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 -- 172.21.15.16:0/2382824414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3184c930 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 0x55fe3184e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde79d9b640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fde7002a930 tx=0x7fde700317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a57c9b7099b5dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70035040 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde74003d10 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fde70004240 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde70031a10 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde8314f740 10 client.17415 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.038+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe31861d10 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde74003d10 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde70004ac0 con 0x55fe3183ff80 2024-09-12T01:03:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 10 client.17415 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 10 client.17415 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 20 client.17415 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 4 client.17415 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe317b29d0 con 0x55fe318631d0 2024-09-12T01:03:13.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.039+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318655f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9ac9f26e17d56b server_cookie=178a6e38246b1fa2 in_seq=0 out_seq=0 2024-09-12T01:03:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde61ffb640 10 client.17415 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde61ffb640 1 -- 172.21.15.16:0/2382824414 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe317b29d0 con 0x55fe318631d0 2024-09-12T01:03:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde61ffb640 10 client.17415 handle_command_reply: tid=0 2024-09-12T01:03:13.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde8314f740 1 client.17415 shutdown 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 msgr2=0x55fe3184bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.040+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fde7002a930 tx=0x7fde700317e0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 shutdown_connections 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318655f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde61ffb640 0 client.17415 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3184c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183ff80 0x55fe3184bc60 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 --2- 172.21.15.16:0/2382824414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183f660 0x55fe3184e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 >> 172.21.15.16:0/2382824414 conn(0x55fe3183bcf0 msgr2=0x55fe318419b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 shutdown_connections 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 1 -- 172.21.15.16:0/2382824414 wait complete. 2024-09-12T01:03:13.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.041+0000 7fde8314f740 20 client.17415 trim_cache size 0 max 16384 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.062+0000 7fde8314f740 1 Processor -- start 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.062+0000 7fde8314f740 1 -- start start 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3186aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3186b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ab5f0 con 0x55fe3183f660 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ad130 con 0x55fe3184ed50 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde8314f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1abcc0 con 0x55fe3183ff80 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde79d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3186aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45696/0 (socket says 172.21.15.16:45696) 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 -- 172.21.15.16:0/3309335254 learned_addr learned my addr 172.21.15.16:0/3309335254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3186b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 -- 172.21.15.16:0/3309335254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 msgr2=0x55fe3186aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3186aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 -- 172.21.15.16:0/3309335254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 msgr2=0x55fe3186b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3186b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 -- 172.21.15.16:0/3309335254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3183fdb0 con 0x55fe3183f660 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fde740116a0 tx=0x7fde740115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe6c35866ad8f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.063+0000 7fde63fff640 1 -- 172.21.15.16:0/3309335254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74024040 con 0x55fe3183f660 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde63fff640 1 -- 172.21.15.16:0/3309335254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde740116d0 con 0x55fe3183f660 2024-09-12T01:03:13.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde63fff640 1 -- 172.21.15.16:0/3309335254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740052a0 con 0x55fe3183f660 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 -- 172.21.15.16:0/3309335254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 msgr2=0x55fe318418a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fde740116a0 tx=0x7fde740115a0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 -- 172.21.15.16:0/3309335254 shutdown_connections 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3184ed50 0x55fe3186b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3186aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 --2- 172.21.15.16:0/3309335254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3183f660 0x55fe318418a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 -- 172.21.15.16:0/3309335254 >> 172.21.15.16:0/3309335254 conn(0x55fe3183bcf0 msgr2=0x55fe31841190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 -- 172.21.15.16:0/3309335254 shutdown_connections 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.064+0000 7fde8314f740 1 -- 172.21.15.16:0/3309335254 wait complete. 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 -- start start 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 Processor -- start 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3183f660 0x55fe3185b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3185be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1aca60 con 0x55fe3184ed50 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1ac390 con 0x55fe3183f660 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1a95f0 con 0x55fe3183ff80 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde8314f740 10 client.0 _ll_register_callbacks cb 0x7fde70009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde7ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45698/0 (socket says 172.21.15.16:45698) 2024-09-12T01:03:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde7ad9d640 1 -- 172.21.15.16:0/1156191416 learned_addr learned my addr 172.21.15.16:0/1156191416 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde79d9b640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3185be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.065+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3183f660 0x55fe3185b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 -- 172.21.15.16:0/1156191416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3183f660 msgr2=0x55fe3185b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3183f660 0x55fe3185b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 -- 172.21.15.16:0/1156191416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 msgr2=0x55fe318cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 -- 172.21.15.16:0/1156191416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe318cd3d0 con 0x55fe3183ff80 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde79d9b640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3185be20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55fe3184d210 tx=0x7fde70000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e5d0bbd64114a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70052040 con 0x55fe3183ff80 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde740116d0 con 0x55fe3183ff80 2024-09-12T01:03:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.066+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fde70004b70 con 0x55fe3183ff80 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde70035040 con 0x55fe3183ff80 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 10 client.26680 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe318ce540 con 0x55fe3183ff80 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde740116d0 con 0x55fe3183ff80 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde700607f0 con 0x55fe3183ff80 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 10 client.26680 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 10 client.26680 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 20 client.26680 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 4 client.26680 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55fe317acd20 con 0x55fe318631d0 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.067+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.068+0000 7fde7ad9d640 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318d0d20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dad9240e604f07a server_cookie=a42920eaffbe7a82 in_seq=0 out_seq=0 2024-09-12T01:03:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.068+0000 7fde61ffb640 10 client.26680 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde61ffb640 1 -- 172.21.15.16:0/1156191416 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fe317acd20 con 0x55fe318631d0 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde61ffb640 10 client.26680 handle_command_reply: tid=0 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 client.26680 shutdown 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 msgr2=0x55fe3185be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3185be20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55fe3184d210 tx=0x7fde70000bc0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 shutdown_connections 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fe318631d0 0x55fe318d0d20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fe3184ed50 0x55fe318cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe3183ff80 0x55fe3185be20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 --2- 172.21.15.16:0/1156191416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fe3183f660 0x55fe3185b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde61ffb640 0 client.26680 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 >> 172.21.15.16:0/1156191416 conn(0x55fe3183bcf0 msgr2=0x55fe3183ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 shutdown_connections 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.078+0000 7fde8314f740 1 -- 172.21.15.16:0/1156191416 wait complete. 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde8314f740 20 client.26680 trim_cache size 0 max 16384 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:03:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 26, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "4f7e6111": { 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 26, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 136.6, 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:03:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:13.081 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 msgr2=0x7fde640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 0x7fde640a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cc910 tx=0x7fde6c0d2c60 comp rx=0 tx=0).stop 2024-09-12T01:03:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 msgr2=0x7fde7c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00d950 tx=0x7fde6c00de10 comp rx=0 tx=0).stop 2024-09-12T01:03:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 shutdown_connections 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fde640a2cf0 0x7fde640a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde7c106f10 0x7fde7c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c102790 0x7fde7c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 --2- 172.21.15.16:0/4093774923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c101de0 0x7fde7c198470 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 >> 172.21.15.16:0/4093774923 conn(0x7fde7c0fd8c0 msgr2=0x7fde7c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 shutdown_connections 2024-09-12T01:03:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:13.079+0000 7fde80d76640 1 -- 172.21.15.16:0/4093774923 wait complete. 2024-09-12T01:03:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:13 smithi016 ceph-mon[30972]: pgmap v218: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:13 smithi016 ceph-mon[30972]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:03:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:13 smithi099 ceph-mon[35654]: pgmap v218: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:13 smithi099 ceph-mon[35654]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:03:13.538 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:03:13.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:03:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:13 smithi159 ceph-mon[35566]: pgmap v218: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 36 op/s 2024-09-12T01:03:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:13 smithi159 ceph-mon[35566]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T01:03:13.834 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4f7e6111', quiesce took 4.1 seconds. Will release after: 12.500000000000002 2024-09-12T01:03:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- 172.21.15.16:0/3928297617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 msgr2=0x7f280810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 --2- 172.21.15.16:0/3928297617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f280810e620 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009900 tx=0x7f27fc02a480 comp rx=0 tx=0).stop 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- 172.21.15.16:0/3928297617 shutdown_connections 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 --2- 172.21.15.16:0/3928297617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f280810e620 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 --2- 172.21.15.16:0/3928297617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 0x7f280810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 --2- 172.21.15.16:0/3928297617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 0x7f28081031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- 172.21.15.16:0/3928297617 >> 172.21.15.16:0/3928297617 conn(0x7f28080fc7d0 msgr2=0x7f28080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- 172.21.15.16:0/3928297617 shutdown_connections 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- 172.21.15.16:0/3928297617 wait complete. 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 Processor -- start 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.386+0000 7f280f18c640 1 -- start start 2024-09-12T01:03:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 0x7f2808100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 0x7f2808100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111fd0 con 0x7f280810c1b0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2808111e50 con 0x7f2808102dd0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280f18c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2808112150 con 0x7f28081036f0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45712/0 (socket says 172.21.15.16:45712) 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 -- 172.21.15.16:0/2656837574 learned_addr learned my addr 172.21.15.16:0/2656837574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f2807fff640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 0x7f2808100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280cf28640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 0x7f2808100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 -- 172.21.15.16:0/2656837574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 msgr2=0x7f2808100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 0x7f2808100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 -- 172.21.15.16:0/2656837574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 msgr2=0x7f2808100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 0x7f2808100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.387+0000 7f280d729640 1 -- 172.21.15.16:0/2656837574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28081a68f0 con 0x7f280810c1b0 2024-09-12T01:03:14.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f280d729640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f27fc004770 tx=0x7f27fc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f2805ffb640 1 -- 172.21.15.16:0/2656837574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc03f070 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f2805ffb640 1 -- 172.21.15.16:0/2656837574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27fc031c90 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28081a6b80 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f2805ffb640 1 -- 172.21.15.16:0/2656837574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc035030 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.388+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a70b0 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.390+0000 7f2805ffb640 1 -- 172.21.15.16:0/2656837574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f27fc042070 con 0x7f280810c1b0 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.390+0000 7f2811565740 1 Processor -- start 2024-09-12T01:03:14.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 -- start start 2024-09-12T01:03:14.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a33cac6280 con 0x55a33cb6bd40 2024-09-12T01:03:14.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33ca6db10 con 0x55a33cb61480 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2811565740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a33cac2240 con 0x55a33cb60b60 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2805ffb640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 0x7f27f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45728/0 (socket says 172.21.15.16:45728) 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2807fff640 1 -- 172.21.15.16:0/828873951 learned_addr learned my addr 172.21.15.16:0/828873951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f280d729640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 0x7f27f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f280cf28640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2807fff640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.391+0000 7f2805ffb640 1 -- 172.21.15.16:0/2656837574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f27fc0efab0 con 0x7f280810c1b0 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 -- 172.21.15.16:0/828873951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 msgr2=0x55a33cb60f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb60f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 -- 172.21.15.16:0/828873951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 msgr2=0x55a33cb6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb6b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 -- 172.21.15.16:0/828873951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33ca7a1f0 con 0x55a33cb6bd40 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f280cf28640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f2807fff640 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012720 tx=0x7f27f800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9679625e1c6eef10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f280d729640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 0x7f27f00a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f27fc08f120 tx=0x7f27fc08e0a0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f27eeffd640 1 -- 172.21.15.16:0/828873951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8012ca0 con 0x55a33cb6bd40 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f27eeffd640 1 -- 172.21.15.16:0/828873951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f8012e40 con 0x55a33cb6bd40 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.392+0000 7f27eeffd640 1 -- 172.21.15.16:0/828873951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f80049a0 con 0x55a33cb6bd40 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- 172.21.15.16:0/828873951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 msgr2=0x55a33cb6e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012720 tx=0x7f27f800b6e0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- 172.21.15.16:0/828873951 shutdown_connections 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb6e130 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 --2- 172.21.15.16:0/828873951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- 172.21.15.16:0/828873951 >> 172.21.15.16:0/828873951 conn(0x55a33cb5d0f0 msgr2=0x55a33cb5d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- 172.21.15.16:0/828873951 shutdown_connections 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- 172.21.15.16:0/828873951 wait complete. 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 -- start start 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.393+0000 7f2811565740 1 Processor -- start 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a33cb6e910 con 0x55a33cb6bd40 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a33ca80f10 con 0x55a33cb61480 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a33cb6ea90 con 0x55a33cb60b60 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2811565740 10 client.0 _ll_register_callbacks cb 0x55a33ca49a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:14.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45734/0 (socket says 172.21.15.16:45734) 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 -- 172.21.15.16:0/3205691958 learned_addr learned my addr 172.21.15.16:0/3205691958 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f280cf28640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 -- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 msgr2=0x55a33cb68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 -- 172.21.15.16:0/3205691958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 msgr2=0x55a33cb69120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb69120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.394+0000 7f2807fff640 1 -- 172.21.15.16:0/3205691958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33c8376a0 con 0x55a33cb6bd40 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f2807fff640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012cd0 tx=0x7f27f800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3943593f0423c73f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8028040 con 0x55a33cb6bd40 2024-09-12T01:03:14.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f27f8010b50 con 0x55a33cb6bd40 2024-09-12T01:03:14.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a33cad37f0 con 0x55a33cb6bd40 2024-09-12T01:03:14.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27f8033050 con 0x55a33cb6bd40 2024-09-12T01:03:14.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f2811565740 10 client.17433 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:14.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a33cb7a540 con 0x55a33cb6bd40 2024-09-12T01:03:14.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a33cad37f0 con 0x55a33cb6bd40 2024-09-12T01:03:14.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.395+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27f801f470 con 0x55a33cb6bd40 2024-09-12T01:03:14.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 10 client.17433 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 20 client.17433 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cb7b890 0x55a33cb7dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 4 client.17433 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a33cacdd00 con 0x55a33cb7b890 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f280cf28640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cb7b890 0x55a33cb7dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f280cf28640 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cb7b890 0x55a33cb7dc80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a898ecfeff3b6a server_cookie=efe0e7be6d9d40c2 in_seq=0 out_seq=0 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.396+0000 7f27e7fff640 10 client.17433 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f27e7fff640 1 -- 172.21.15.16:0/3205691958 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a33cacdd00 con 0x55a33cb7b890 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f27e7fff640 10 client.17433 handle_command_reply: tid=0 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 client.17433 shutdown 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 msgr2=0x55a33cb69660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012cd0 tx=0x7f27f800b5b0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 shutdown_connections 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cb7b890 0x55a33cb7dc80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb69660 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 --2- 172.21.15.16:0/3205691958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f27e7fff640 0 client.17433 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.397+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 >> 172.21.15.16:0/3205691958 conn(0x55a33cb5d0f0 msgr2=0x55a33cb62dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.398+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 shutdown_connections 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.398+0000 7f2811565740 1 -- 172.21.15.16:0/3205691958 wait complete. 2024-09-12T01:03:14.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.398+0000 7f2811565740 20 client.17433 trim_cache size 0 max 16384 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 Processor -- start 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 -- start start 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28081afab0 con 0x55a33cb6bd40 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b15f0 con 0x55a33cb61480 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2811565740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b0180 con 0x55a33cb60b60 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45742/0 (socket says 172.21.15.16:45742) 2024-09-12T01:03:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2807fff640 1 -- 172.21.15.16:0/3158859069 learned_addr learned my addr 172.21.15.16:0/3158859069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f2807fff640 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.416+0000 7f280cf28640 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 -- 172.21.15.16:0/3158859069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 msgr2=0x55a33cb67dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb67dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 -- 172.21.15.16:0/3158859069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 msgr2=0x55a33cb83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 -- 172.21.15.16:0/3158859069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a33cb612b0 con 0x55a33cb6bd40 2024-09-12T01:03:14.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f2807fff640 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f27f8001eb0 tx=0x7f27f80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a926afefb51013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f27ee7fc640 1 -- 172.21.15.16:0/3158859069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f801f470 con 0x55a33cb6bd40 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f27ee7fc640 1 -- 172.21.15.16:0/3158859069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f801f610 con 0x55a33cb6bd40 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.417+0000 7f27ee7fc640 1 -- 172.21.15.16:0/3158859069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f80038e0 con 0x55a33cb6bd40 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- 172.21.15.16:0/3158859069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 msgr2=0x55a33cb83980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f27f8001eb0 tx=0x7f27f80056d0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- 172.21.15.16:0/3158859069 shutdown_connections 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb6bd40 0x55a33cb83980 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb61480 0x55a33cb83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 --2- 172.21.15.16:0/3158859069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb60b60 0x55a33cb67dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- 172.21.15.16:0/3158859069 >> 172.21.15.16:0/3158859069 conn(0x55a33cb5d0f0 msgr2=0x55a33cb6ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- 172.21.15.16:0/3158859069 shutdown_connections 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- 172.21.15.16:0/3158859069 wait complete. 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 -- start start 2024-09-12T01:03:14.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.418+0000 7f2811565740 1 Processor -- start 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb60b60 0x55a33cb73f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb6bd40 0x55a33cb651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b0f20 con 0x55a33cb61480 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b0850 con 0x55a33cb60b60 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28081adab0 con 0x55a33cb6bd40 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2811565740 10 client.0 _ll_register_callbacks cb 0x7f2800009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45748/0 (socket says 172.21.15.16:45748) 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 -- 172.21.15.16:0/716468699 learned_addr learned my addr 172.21.15.16:0/716468699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2807fff640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb60b60 0x55a33cb73f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f2807fff640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb6bd40 0x55a33cb651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 -- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb6bd40 msgr2=0x55a33cb651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb6bd40 0x55a33cb651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 -- 172.21.15.16:0/716468699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb60b60 msgr2=0x55a33cb73f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb60b60 0x55a33cb73f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.419+0000 7f280cf28640 1 -- 172.21.15.16:0/716468699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a33cb658e0 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f280cf28640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55a33cb68470 tx=0x7f280001f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e66c0575e312a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f27e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280001fce0 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f2800010840 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f801f610 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2800010a40 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f2811565740 10 client.17439 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a33cbe5ee0 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.420+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f801f610 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f280002e400 con 0x55a33cb61480 2024-09-12T01:03:14.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 10 client.17439 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 20 client.17439 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cbe6710 0x55a33cbe8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 4 client.17439 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a33ca4a110 con 0x55a33cbe6710 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2807fff640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cbe6710 0x55a33cbe8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f2807fff640 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cbe6710 0x55a33cbe8b00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5ae19953ba8773 server_cookie=93b90a6be3d4aa4a in_seq=0 out_seq=0 2024-09-12T01:03:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.421+0000 7f27e7fff640 10 client.17439 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.422+0000 7f27e7fff640 1 -- 172.21.15.16:0/716468699 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a33ca4a110 con 0x55a33cbe6710 2024-09-12T01:03:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.422+0000 7f27e7fff640 10 client.17439 handle_command_reply: tid=0 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.422+0000 7f2811565740 1 client.17439 shutdown 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 msgr2=0x55a33cb74460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x55a33cb68470 tx=0x7f280001f520 comp rx=0 tx=0).stop 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 shutdown_connections 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a33cbe6710 0x55a33cbe8b00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a33cb6bd40 0x55a33cb651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a33cb61480 0x55a33cb74460 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f27e7fff640 0 client.17439 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 --2- 172.21.15.16:0/716468699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a33cb60b60 0x55a33cb73f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 >> 172.21.15.16:0/716468699 conn(0x55a33cb5d0f0 msgr2=0x55a33cb5ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 shutdown_connections 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 1 -- 172.21.15.16:0/716468699 wait complete. 2024-09-12T01:03:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.423+0000 7f2811565740 20 client.17439 trim_cache size 0 max 16384 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b326e490-55c0-4687-9b06-57a13eddf11b", 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:14.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 msgr2=0x7f27f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 0x7f27f00a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f27fc08f120 tx=0x7f27fc08e0a0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 msgr2=0x7f28081014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f27fc004770 tx=0x7f27fc003b90 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 shutdown_connections 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27f00a2d70 0x7f27f00a5230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280810c1b0 0x7f28081014b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28081036f0 0x7f2808100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 --2- 172.21.15.16:0/2656837574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808102dd0 0x7f2808100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 >> 172.21.15.16:0/2656837574 conn(0x7f28080fc7d0 msgr2=0x7f28080fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 shutdown_connections 2024-09-12T01:03:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:14.424+0000 7f280f18c640 1 -- 172.21.15.16:0/2656837574 wait complete. 2024-09-12T01:03:15.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: pgmap v220: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:15 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:03:15.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: pgmap v220: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:15 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: pgmap v220: 97 pgs: 97 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:15 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.883+0000 7f7fdfb52640 1 -- 172.21.15.16:0/925001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 msgr2=0x7f7fd810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.883+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/925001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd810d950 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009cd0 tx=0x7f7fd002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- 172.21.15.16:0/925001862 shutdown_connections 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/925001862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd810d950 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/925001862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/925001862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 0x7f7fd8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- 172.21.15.16:0/925001862 >> 172.21.15.16:0/925001862 conn(0x7f7fd80fc7a0 msgr2=0x7f7fd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- 172.21.15.16:0/925001862 shutdown_connections 2024-09-12T01:03:15.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- 172.21.15.16:0/925001862 wait complete. 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 Processor -- start 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- start start 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 0x7f7fd819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8110e80 con 0x7f7fd8101420 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8110d00 con 0x7f7fd8106a80 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.884+0000 7f7fdfb52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8111000 con 0x7f7fd8100a80 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45772/0 (socket says 172.21.15.16:45772) 2024-09-12T01:03:15.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/3127391222 learned_addr learned my addr 172.21.15.16:0/3127391222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 0x7f7fd819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/3127391222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 msgr2=0x7f7fd819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/3127391222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 msgr2=0x7f7fd819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 0x7f7fd819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/3127391222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd819f5b0 con 0x7f7fd8101420 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d950 tx=0x7f7fc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fceffd640 1 -- 172.21.15.16:0/3127391222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8017070 con 0x7f7fd8101420 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fceffd640 1 -- 172.21.15.16:0/3127391222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc8009c30 con 0x7f7fd8101420 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd819f8a0 con 0x7f7fd8101420 2024-09-12T01:03:15.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fceffd640 1 -- 172.21.15.16:0/3127391222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800f6b0 con 0x7f7fd8101420 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.885+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a7180 con 0x7f7fd8101420 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fceffd640 1 -- 172.21.15.16:0/3127391222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7fc80109d0 con 0x7f7fd8101420 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 Processor -- start 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 -- start start 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b59754a0 0x5584b5975880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fceffd640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 0x7f7fc40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b5982a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584b58db2e0 con 0x5584b5975dc0 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5584b5882ce0 con 0x5584b59754a0 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.888+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5584b58d72a0 con 0x5584b5980680 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fceffd640 1 -- 172.21.15.16:0/3127391222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7fc80cf0f0 con 0x7f7fd8101420 2024-09-12T01:03:15.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 0x7f7fc40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45774/0 (socket says 172.21.15.16:45774) 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/425392978 learned_addr learned my addr 172.21.15.16:0/425392978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b59754a0 0x5584b5975880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b5982a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/425392978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 msgr2=0x5584b5982a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b5982a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/425392978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b59754a0 msgr2=0x5584b5975880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b59754a0 0x5584b5975880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 -- 172.21.15.16:0/425392978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b5867440 con 0x5584b5975dc0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b5982a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80cc940 tx=0x7f7fc8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65554e31f1e1ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 0x7f7fc40a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005d90 tx=0x7f7fd4005d20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fc3fff640 1 -- 172.21.15.16:0/425392978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8020070 con 0x5584b5975dc0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fc3fff640 1 -- 172.21.15.16:0/425392978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc809bdb0 con 0x5584b5975dc0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.889+0000 7f7fc3fff640 1 -- 172.21.15.16:0/425392978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80a4320 con 0x5584b5975dc0 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/425392978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 msgr2=0x5584b5980140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7fc80cc940 tx=0x7f7fc8010d70 comp rx=0 tx=0).stop 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/425392978 shutdown_connections 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b5982a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5975dc0 0x5584b5980140 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/425392978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b59754a0 0x5584b5975880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/425392978 >> 172.21.15.16:0/425392978 conn(0x5584b5971a30 msgr2=0x5584b5971e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/425392978 shutdown_connections 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.890+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/425392978 wait complete. 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 -- start start 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 Processor -- start 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b597d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b597d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584b5983250 con 0x5584b59754a0 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5584b5896050 con 0x5584b5975dc0 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5584b59833d0 con 0x5584b5980680 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fe1f2b740 10 client.0 _ll_register_callbacks cb 0x5584b58d6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:15.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fdd8ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fdd8ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45780/0 (socket says 172.21.15.16:45780) 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/3629213090 learned_addr learned my addr 172.21.15.16:0/3629213090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b597d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.891+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b597d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 msgr2=0x5584b597d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b597d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/3629213090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 msgr2=0x5584b597d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b597d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/3629213090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b564c3c0 con 0x5584b59754a0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40109b0 tx=0x7f7fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c195c53099362894 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4021070 con 0x5584b59754a0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc809bdb0 con 0x5584b59754a0 2024-09-12T01:03:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7fd400fcd0 con 0x5584b59754a0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fd402c050 con 0x5584b59754a0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 10 client.17451 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584b598ecc0 con 0x5584b59754a0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc809bdb0 con 0x5584b59754a0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fd401c440 con 0x5584b59754a0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 10 client.17451 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.892+0000 7f7fe1f2b740 20 client.17451 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59900d0 0x5584b5992500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fe1f2b740 4 client.17451 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584b58e8850 con 0x5584b59900d0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59900d0 0x5584b5992500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fc17fa640 10 client.17451 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.893+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59900d0 0x5584b5992500 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9560a23fa131be3 server_cookie=30045fa01e63f3 in_seq=0 out_seq=0 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3629213090 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584b58e8850 con 0x5584b59900d0 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fc17fa640 10 client.17451 handle_command_reply: tid=0 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 client.17451 shutdown 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 msgr2=0x5584b5980030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40109b0 tx=0x7f7fd40046c0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 shutdown_connections 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59900d0 0x5584b5992500 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5980680 0x5584b597d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b597d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fc17fa640 0 client.17451 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3629213090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5980030 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 >> 172.21.15.16:0/3629213090 conn(0x5584b5971a30 msgr2=0x5584b5970b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 shutdown_connections 2024-09-12T01:03:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3629213090 wait complete. 2024-09-12T01:03:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.894+0000 7f7fe1f2b740 20 client.17451 trim_cache size 0 max 16384 2024-09-12T01:03:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 Processor -- start 2024-09-12T01:03:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 -- start start 2024-09-12T01:03:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5975dc0 0x5584b5998360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 0x5584b597a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81afae0 con 0x5584b59754a0 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81b1620 con 0x5584b5980680 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81b01b0 con 0x5584b5975dc0 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fdd8ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.912+0000 7f7fdd8ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45786/0 (socket says 172.21.15.16:45786) 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/4068718873 learned_addr learned my addr 172.21.15.16:0/4068718873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 0x5584b597a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5975dc0 0x5584b5998360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/4068718873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5975dc0 msgr2=0x5584b5998360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5975dc0 0x5584b5998360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/4068718873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 msgr2=0x5584b597a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 0x5584b597a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 -- 172.21.15.16:0/4068718873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b597c290 con 0x5584b59754a0 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 0x5584b597a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400f010 tx=0x7f7fd4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcf1221c0246b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fc37fe640 1 -- 172.21.15.16:0/4068718873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4034020 con 0x5584b59754a0 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fc37fe640 1 -- 172.21.15.16:0/4068718873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd401c5b0 con 0x5584b59754a0 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.913+0000 7f7fc37fe640 1 -- 172.21.15.16:0/4068718873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4021070 con 0x5584b59754a0 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/4068718873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 msgr2=0x5584b5997e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400f010 tx=0x7f7fd4005a10 comp rx=0 tx=0).stop 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/4068718873 shutdown_connections 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5980680 0x5584b597a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b5975dc0 0x5584b5998360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/4068718873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b59754a0 0x5584b5997e20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/4068718873 >> 172.21.15.16:0/4068718873 conn(0x5584b5971a30 msgr2=0x5584b597f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/4068718873 shutdown_connections 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/4068718873 wait complete. 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 -- start start 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.914+0000 7f7fe1f2b740 1 Processor -- start 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b59754a0 0x5584b59864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b5986a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81b0f50 con 0x5584b5980680 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81b0880 con 0x5584b5975dc0 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd81adae0 con 0x5584b59754a0 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fe1f2b740 10 client.0 _ll_register_callbacks cb 0x7f7fd4014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45802/0 (socket says 172.21.15.16:45802) 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 -- 172.21.15.16:0/3575431913 learned_addr learned my addr 172.21.15.16:0/3575431913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fdd0ed640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b5986a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b59754a0 0x5584b59864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 -- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b59754a0 msgr2=0x5584b59864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b59754a0 0x5584b59864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 -- 172.21.15.16:0/3575431913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 msgr2=0x5584b5986a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b5986a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.915+0000 7f7fde0ef640 1 -- 172.21.15.16:0/3575431913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b597dbf0 con 0x5584b5980680 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fde0ef640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009cd0 tx=0x7f7fd002afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04d67f1f29351ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd00307a0 con 0x5584b5980680 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7fd0030940 con 0x5584b5980680 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd401c5b0 con 0x5584b5980680 2024-09-12T01:03:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0030b40 con 0x5584b5980680 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 10 client.17457 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584b59fb040 con 0x5584b5980680 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd401c5b0 con 0x5584b5980680 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fd0050030 con 0x5584b5980680 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 10 client.17457 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 20 client.17457 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59fb810 0x5584b59fdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 4 client.17457 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.916+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584b58e2d60 con 0x5584b59fb810 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59fb810 0x5584b59fdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fdd8ee640 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59fb810 0x5584b59fdc30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5211e66ee218c3de server_cookie=81fc6db53f1da632 in_seq=0 out_seq=0 2024-09-12T01:03:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fc17fa640 10 client.17457 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fc17fa640 1 -- 172.21.15.16:0/3575431913 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584b58e2d60 con 0x5584b59fb810 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fc17fa640 10 client.17457 handle_command_reply: tid=0 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 client.17457 shutdown 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 msgr2=0x5584b5986f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009cd0 tx=0x7f7fd002afa0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 shutdown_connections 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5584b59fb810 0x5584b59fdc30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b5980680 0x5584b5986f50 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5584b5975dc0 0x5584b5986a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.917+0000 7f7fe1f2b740 1 --2- 172.21.15.16:0/3575431913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b59754a0 0x5584b59864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fc17fa640 0 client.17457 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 >> 172.21.15.16:0/3575431913 conn(0x5584b5971a30 msgr2=0x5584b5970370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 shutdown_connections 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fe1f2b740 1 -- 172.21.15.16:0/3575431913 wait complete. 2024-09-12T01:03:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fe1f2b740 20 client.17457 trim_cache size 0 max 16384 2024-09-12T01:03:15.929 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:15.929 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:03:15.929 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:03:15.929 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 msgr2=0x7f7fc40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 0x7f7fc40a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005d90 tx=0x7f7fd4005d20 comp rx=0 tx=0).stop 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 msgr2=0x7f7fd819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800d950 tx=0x7f7fc800de10 comp rx=0 tx=0).stop 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 shutdown_connections 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc40a2cf0 0x7f7fc40a51b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd8106a80 0x7f7fd819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd8101420 0x7f7fd819ab40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 --2- 172.21.15.16:0/3127391222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd8100a80 0x7f7fd819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 >> 172.21.15.16:0/3127391222 conn(0x7f7fd80fc7a0 msgr2=0x7f7fd806a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 shutdown_connections 2024-09-12T01:03:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:15.918+0000 7f7fdfb52640 1 -- 172.21.15.16:0/3127391222 wait complete. 2024-09-12T01:03:16.639 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:03:16.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:17 smithi016 ceph-mon[30972]: pgmap v221: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.390+0000 7f5562983640 1 -- 172.21.15.16:0/4125954664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 msgr2=0x7f555c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.390+0000 7f5562983640 1 --2- 172.21.15.16:0/4125954664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c10e610 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5554009ca0 tx=0x7f555402a950 comp rx=0 tx=0).stop 2024-09-12T01:03:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.390+0000 7f5562983640 1 -- 172.21.15.16:0/4125954664 shutdown_connections 2024-09-12T01:03:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.390+0000 7f5562983640 1 --2- 172.21.15.16:0/4125954664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c10e610 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 --2- 172.21.15.16:0/4125954664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c103660 0x7f555c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 --2- 172.21.15.16:0/4125954664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c102d40 0x7f555c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- 172.21.15.16:0/4125954664 >> 172.21.15.16:0/4125954664 conn(0x7f555c0fc7a0 msgr2=0x7f555c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- 172.21.15.16:0/4125954664 shutdown_connections 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- 172.21.15.16:0/4125954664 wait complete. 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 Processor -- start 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- start start 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c102d40 0x7f555c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c103660 0x7f555c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111fd0 con 0x7f555c10c170 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f555c111e50 con 0x7f555c102d40 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.391+0000 7f5562983640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f555c112150 con 0x7f555c103660 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45812/0 (socket says 172.21.15.16:45812) 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 -- 172.21.15.16:0/2596259379 learned_addr learned my addr 172.21.15.16:0/2596259379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5553fff640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c103660 0x7f555c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f555bfff640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c102d40 0x7f555c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 -- 172.21.15.16:0/2596259379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c103660 msgr2=0x7f555c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c103660 0x7f555c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 -- 172.21.15.16:0/2596259379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c102d40 msgr2=0x7f555c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c102d40 0x7f555c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 -- 172.21.15.16:0/2596259379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c1a6950 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5560f20640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f555402ae70 tx=0x7f555402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554034040 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554030920 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c1a6be0 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554045480 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.392+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1a70c0 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.393+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c103120 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.394+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5554037020 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.395+0000 7f5559ffb640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 0x7f55480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.395+0000 7f555bfff640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 0x7f55480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.395+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f55540f0360 con 0x7f555c10c170 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.396+0000 7f555bfff640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 0x7f55480a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f554c005fd0 tx=0x7f554c005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.399+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55540bd090 con 0x7f555c10c170 2024-09-12T01:03:17.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:17 smithi099 ceph-mon[35654]: pgmap v221: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:17.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.610+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f555c107210 con 0x7f555c10c170 2024-09-12T01:03:17.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.611+0000 7f5559ffb640 1 -- 172.21.15.16:0/2596259379 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f55540bc990 con 0x7f555c10c170 2024-09-12T01:03:17.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:17.611 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:17.611 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 msgr2=0x7f55480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 0x7f55480a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f554c005fd0 tx=0x7f554c005850 comp rx=0 tx=0).stop 2024-09-12T01:03:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 msgr2=0x7f555c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f555402ae70 tx=0x7f555402f5b0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 shutdown_connections 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f55480a2cf0 0x7f55480a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c10c170 0x7f555c101420 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f555c103660 0x7f555c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 --2- 172.21.15.16:0/2596259379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f555c102d40 0x7f555c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.613+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 >> 172.21.15.16:0/2596259379 conn(0x7f555c0fc7a0 msgr2=0x7f555c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.614+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 shutdown_connections 2024-09-12T01:03:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:17.614+0000 7f5562983640 1 -- 172.21.15.16:0/2596259379 wait complete. 2024-09-12T01:03:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:17 smithi159 ceph-mon[35566]: pgmap v221: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:18.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:03:18.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:18 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2596259379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:18.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:18 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2596259379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:18.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:18 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2596259379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 -- 172.21.15.16:0/38838827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c102790 msgr2=0x7f175c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 --2- 172.21.15.16:0/38838827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c102790 0x7f175c1067e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f174402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 -- 172.21.15.16:0/38838827 shutdown_connections 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 --2- 172.21.15.16:0/38838827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c106f10 0x7f175c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 --2- 172.21.15.16:0/38838827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c102790 0x7f175c1067e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 --2- 172.21.15.16:0/38838827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 0x7f175c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 -- 172.21.15.16:0/38838827 >> 172.21.15.16:0/38838827 conn(0x7f175c0fd8c0 msgr2=0x7f175c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 -- 172.21.15.16:0/38838827 shutdown_connections 2024-09-12T01:03:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.151+0000 7f1760d75640 1 -- 172.21.15.16:0/38838827 wait complete. 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 Processor -- start 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 -- start start 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 0x7f175c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c102790 0x7f175c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f175c111f80 con 0x7f175c106f10 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f175c111e00 con 0x7f175c102790 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f1760d75640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f175c112100 con 0x7f175c101de0 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f175ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f175ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46140/0 (socket says 172.21.15.16:46140) 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f175ad9d640 1 -- 172.21.15.16:0/1360230827 learned_addr learned my addr 172.21.15.16:0/1360230827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.152+0000 7f175a59c640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 0x7f175c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f1753fff640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c102790 0x7f175c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 -- 172.21.15.16:0/1360230827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 msgr2=0x7f175c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 0x7f175c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 -- 172.21.15.16:0/1360230827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c102790 msgr2=0x7f175c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c102790 0x7f175c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 -- 172.21.15.16:0/1360230827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c19f650 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f175ad9d640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f175400cce0 tx=0x7f175400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f17537fe640 1 -- 172.21.15.16:0/1360230827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175400f450 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f17537fe640 1 -- 172.21.15.16:0/1360230827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f175400fad0 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175c19f940 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f17537fe640 1 -- 172.21.15.16:0/1360230827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1754018c10 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.153+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175c1a7220 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f17537fe640 1 -- 172.21.15.16:0/1360230827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f175400f5f0 con 0x7f175c106f10 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 Processor -- start 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 -- start start 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b36741d2f0 con 0x55b3674b8b40 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b3673c4cf0 con 0x55b3674b75d0 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f176314e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b3674192b0 con 0x55b3674c2370 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f1753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f1753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46154/0 (socket says 172.21.15.16:46154) 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.156+0000 7f1753fff640 1 -- 172.21.15.16:0/2103727595 learned_addr learned my addr 172.21.15.16:0/2103727595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f17537fe640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 0x7f17480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f175a59c640 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f175ad9d640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 0x7f17480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f17537fe640 1 -- 172.21.15.16:0/1360230827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f17540cf9d0 con 0x7f175c106f10 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 -- 172.21.15.16:0/2103727595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 msgr2=0x55b3674c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 -- 172.21.15.16:0/2103727595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 msgr2=0x55b3674b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 -- 172.21.15.16:0/2103727595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3673a93a0 con 0x55b3674b8b40 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1753fff640 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x55b3674b8070 tx=0x7f17440095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f911c93e19260d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f175ad9d640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 0x7f17480a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17540cd720 tx=0x7f175406ef60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1750ff9640 1 -- 172.21.15.16:0/2103727595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744035040 con 0x55b3674b8b40 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1750ff9640 1 -- 172.21.15.16:0/2103727595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1744005510 con 0x55b3674b8b40 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.157+0000 7f1750ff9640 1 -- 172.21.15.16:0/2103727595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17440039f0 con 0x55b3674b8b40 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 -- 172.21.15.16:0/2103727595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 msgr2=0x55b3674b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x55b3674b8070 tx=0x7f17440095d0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 -- 172.21.15.16:0/2103727595 shutdown_connections 2024-09-12T01:03:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 --2- 172.21.15.16:0/2103727595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674b7090 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 -- 172.21.15.16:0/2103727595 >> 172.21.15.16:0/2103727595 conn(0x55b367480310 msgr2=0x55b3674b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 -- 172.21.15.16:0/2103727595 shutdown_connections 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.158+0000 7f176314e740 1 -- 172.21.15.16:0/2103727595 wait complete. 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 -- start start 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 Processor -- start 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674b75d0 0x55b3674c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b8b40 0x55b3674bf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b3674c5010 con 0x55b3674c2370 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b3673d8060 con 0x55b3674b8b40 2024-09-12T01:03:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b3674c5190 con 0x55b3674b75d0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f176314e740 10 client.0 _ll_register_callbacks cb 0x55b36712b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674b75d0 0x55b3674c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46158/0 (socket says 172.21.15.16:46158) 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 -- 172.21.15.16:0/3547820940 learned_addr learned my addr 172.21.15.16:0/3547820940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f175a59c640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b8b40 0x55b3674bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 -- 172.21.15.16:0/3547820940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674b75d0 msgr2=0x55b3674c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674b75d0 0x55b3674c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 -- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b8b40 msgr2=0x55b3674bf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.159+0000 7f1753fff640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b8b40 0x55b3674bf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1753fff640 1 -- 172.21.15.16:0/3547820940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36718e3c0 con 0x55b3674c2370 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1753fff640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f1744003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08bb1db846884d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744035040 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1744005510 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f1744003db0 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17440315b0 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 10 client.17475 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3674d0ae0 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1744005510 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1744004a50 con 0x55b3674c2370 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 10 client.17475 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 10 client.17475 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 20 client.17475 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b3674d1e30 0x55b3674d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 4 client.17475 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.160+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36742a860 con 0x55b3674d1e30 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.161+0000 7f175a59c640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b3674d1e30 0x55b3674d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.161+0000 7f175a59c640 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b3674d1e30 0x55b3674d4220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5a7d8f6069084e server_cookie=1d4cf339d8b55c79 in_seq=0 out_seq=0 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.161+0000 7f1741ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f1741ffb640 1 -- 172.21.15.16:0/3547820940 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b36742a860 con 0x55b3674d1e30 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f1741ffb640 10 client.17475 handle_command_reply: tid=0 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 client.17475 shutdown 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 msgr2=0x55b3674bf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f1744003690 comp rx=0 tx=0).stop 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 shutdown_connections 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b3674d1e30 0x55b3674d4220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674c2370 0x55b3674bf640 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b8b40 0x55b3674bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f1741ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 --2- 172.21.15.16:0/3547820940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674b75d0 0x55b3674c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 >> 172.21.15.16:0/3547820940 conn(0x55b367480310 msgr2=0x55b3674b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 shutdown_connections 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 1 -- 172.21.15.16:0/3547820940 wait complete. 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.162+0000 7f176314e740 20 client.17475 trim_cache size 0 max 16384 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 Processor -- start 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 -- start start 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1afa60 con 0x55b3674b8b40 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1b15a0 con 0x55b3674b75d0 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f176314e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1b0130 con 0x55b3674c2370 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f175a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f175a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46174/0 (socket says 172.21.15.16:46174) 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f175a59c640 1 -- 172.21.15.16:0/841216122 learned_addr learned my addr 172.21.15.16:0/841216122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f1753fff640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f1753fff640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.175+0000 7f175a59c640 1 -- 172.21.15.16:0/841216122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 msgr2=0x55b3674b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f175a59c640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f175a59c640 1 -- 172.21.15.16:0/841216122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 msgr2=0x55b3674ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f175a59c640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f175a59c640 1 -- 172.21.15.16:0/841216122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3674bf810 con 0x55b3674b8b40 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f1753fff640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f175a59c640 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f174c005fd0 tx=0x7f174c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b32d51bdfc7c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f1743fff640 1 -- 172.21.15.16:0/841216122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c022030 con 0x55b3674b8b40 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f1743fff640 1 -- 172.21.15.16:0/841216122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c01d5a0 con 0x55b3674b8b40 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f1743fff640 1 -- 172.21.15.16:0/841216122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0038d0 con 0x55b3674b8b40 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 -- 172.21.15.16:0/841216122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 msgr2=0x55b3674be010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f174c005fd0 tx=0x7f174c0118d0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 -- 172.21.15.16:0/841216122 shutdown_connections 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674be010 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 --2- 172.21.15.16:0/841216122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 -- 172.21.15.16:0/841216122 >> 172.21.15.16:0/841216122 conn(0x55b367480310 msgr2=0x55b3674bd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 -- 172.21.15.16:0/841216122 shutdown_connections 2024-09-12T01:03:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.176+0000 7f176314e740 1 -- 172.21.15.16:0/841216122 wait complete. 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 -- start start 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 Processor -- start 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1b0ed0 con 0x55b3674b8b40 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1b0800 con 0x55b3674b75d0 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1ada60 con 0x55b3674c2370 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f176314e740 10 client.0 _ll_register_callbacks cb 0x7f1744004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f175a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f175a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46180/0 (socket says 172.21.15.16:46180) 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f175a59c640 1 -- 172.21.15.16:0/3506090069 learned_addr learned my addr 172.21.15.16:0/3506090069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:19.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f1753fff640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.177+0000 7f1753fff640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 -- 172.21.15.16:0/3506090069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 msgr2=0x55b3674d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 -- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 msgr2=0x55b3674d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 -- 172.21.15.16:0/3506090069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3674d53b0 con 0x55b3674b8b40 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f175a59c640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f174c014040 tx=0x7f174c0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a417042d39106d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c02b070 con 0x55b3674b8b40 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174c01d5a0 con 0x55b3674b8b40 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f174c01d750 con 0x55b3674b8b40 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f174c037050 con 0x55b3674b8b40 2024-09-12T01:03:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f176314e740 10 client.17481 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b36753c3f0 con 0x55b3674b8b40 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c01d5a0 con 0x55b3674b8b40 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.178+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f174c027440 con 0x55b3674b8b40 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 10 client.17481 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 20 client.17481 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b36753ca50 0x55b36753ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 4 client.17481 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b367424d70 con 0x55b36753ca50 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1753fff640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b36753ca50 0x55b36753ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1753fff640 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b36753ca50 0x55b36753ee70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f86b0b4a95c25b9 server_cookie=2e9abc96273bf876 in_seq=0 out_seq=0 2024-09-12T01:03:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1741ffb640 10 client.17481 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1741ffb640 1 -- 172.21.15.16:0/3506090069 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b367424d70 con 0x55b36753ca50 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1741ffb640 10 client.17481 handle_command_reply: tid=0 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 client.17481 shutdown 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 msgr2=0x55b3674d7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f174c014040 tx=0x7f174c0040b0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 shutdown_connections 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b36753ca50 0x55b36753ee70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3674c2370 0x55b3674d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3674b8b40 0x55b3674d7b50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f1741ffb640 0 client.17481 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 --2- 172.21.15.16:0/3506090069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b3674b75d0 0x55b3674d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 >> 172.21.15.16:0/3506090069 conn(0x55b367480310 msgr2=0x55b3674be120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.179+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 shutdown_connections 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f176314e740 1 -- 172.21.15.16:0/3506090069 wait complete. 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f176314e740 20 client.17481 trim_cache size 0 max 16384 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 msgr2=0x7f17480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 0x7f17480a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17540cd720 tx=0x7f175406ef60 comp rx=0 tx=0).stop 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 msgr2=0x7f175c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f175400cce0 tx=0x7f175400a620 comp rx=0 tx=0).stop 2024-09-12T01:03:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 shutdown_connections 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17480a2cf0 0x7f17480a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c106f10 0x7f175c19ef70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f175c102790 0x7f175c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 --2- 172.21.15.16:0/1360230827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f175c101de0 0x7f175c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 >> 172.21.15.16:0/1360230827 conn(0x7f175c0fd8c0 msgr2=0x7f175c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 shutdown_connections 2024-09-12T01:03:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:19.180+0000 7f1760d75640 1 -- 172.21.15.16:0/1360230827 wait complete. 2024-09-12T01:03:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:19 smithi016 ceph-mon[30972]: pgmap v222: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:03:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:03:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:03:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:03:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:19 smithi099 ceph-mon[35654]: pgmap v222: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:03:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:03:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:03:19.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:03:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:19 smithi159 ceph-mon[35566]: pgmap v222: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-09-12T01:03:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:03:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:03:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:03:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:03:19.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:03:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:20.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:20 smithi099 ceph-mon[35654]: pgmap v223: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:20.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:20 smithi016 ceph-mon[30972]: pgmap v223: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:20.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:20 smithi159 ceph-mon[35566]: pgmap v223: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 -- 172.21.15.16:0/3491703485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c106f10 msgr2=0x7f7b6c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 --2- 172.21.15.16:0/3491703485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c106f10 0x7f7b6c10ea90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009ca0 tx=0x7f7b6402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 -- 172.21.15.16:0/3491703485 shutdown_connections 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 --2- 172.21.15.16:0/3491703485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c106f10 0x7f7b6c10ea90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 --2- 172.21.15.16:0/3491703485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 0x7f7b6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 --2- 172.21.15.16:0/3491703485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c101de0 0x7f7b6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 -- 172.21.15.16:0/3491703485 >> 172.21.15.16:0/3491703485 conn(0x7f7b6c0fd8c0 msgr2=0x7f7b6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 -- 172.21.15.16:0/3491703485 shutdown_connections 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.777+0000 7f7b73207640 1 -- 172.21.15.16:0/3491703485 wait complete. 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.778+0000 7f7b73207640 1 Processor -- start 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.778+0000 7f7b73207640 1 -- start start 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.778+0000 7f7b73207640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 0x7f7b6c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 0x7f7b6c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c111fc0 con 0x7f7b6c101de0 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c111e40 con 0x7f7b6c106f10 2024-09-12T01:03:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c112140 con 0x7f7b6c102790 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46186/0 (socket says 172.21.15.16:46186) 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 -- 172.21.15.16:0/1033555592 learned_addr learned my addr 172.21.15.16:0/1033555592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b717a4640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 0x7f7b6c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 0x7f7b6c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 -- 172.21.15.16:0/1033555592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 msgr2=0x7f7b6c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 0x7f7b6c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 -- 172.21.15.16:0/1033555592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 msgr2=0x7f7b6c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 0x7f7b6c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 -- 172.21.15.16:0/1033555592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c19d430 con 0x7f7b6c101de0 2024-09-12T01:03:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b717a4640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 0x7f7b6c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e9f0 tx=0x7f7b6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b69ffb640 1 -- 172.21.15.16:0/1033555592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000cd80 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b69ffb640 1 -- 172.21.15.16:0/1033555592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60010910 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c19d720 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.779+0000 7f7b69ffb640 1 -- 172.21.15.16:0/1033555592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60018990 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.780+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a5000 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.782+0000 7f7b69ffb640 1 -- 172.21.15.16:0/1033555592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b60010ac0 con 0x7f7b6c101de0 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 Processor -- start 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 -- start start 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 0x561a0a7b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a0a71d2f0 con 0x561a0a7b8b40 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a0a6c4cf0 con 0x561a0a7b75d0 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b755e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a0a7192b0 con 0x561a0a7c6780 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46200/0 (socket says 172.21.15.16:46200) 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 -- 172.21.15.16:0/1017479673 learned_addr learned my addr 172.21.15.16:0/1017479673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 0x561a0a7b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 -- 172.21.15.16:0/1017479673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 msgr2=0x561a0a7c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 -- 172.21.15.16:0/1017479673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 msgr2=0x561a0a7b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 0x561a0a7b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b717a4640 1 -- 172.21.15.16:0/1017479673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0a6a93a0 con 0x561a0a7b8b40 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.783+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 0x561a0a7b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b69ffb640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 0x7f7b5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b717a4640 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7b64035000 tx=0x7f7b64009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0e831de74db5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 0x7f7b5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b56ffd640 1 -- 172.21.15.16:0/1017479673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64031db0 con 0x561a0a7b8b40 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b56ffd640 1 -- 172.21.15.16:0/1017479673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64036040 con 0x561a0a7b8b40 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b56ffd640 1 -- 172.21.15.16:0/1017479673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004330 con 0x561a0a7b8b40 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.784+0000 7f7b69ffb640 1 -- 172.21.15.16:0/1033555592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7b600cf880 con 0x7f7b6c101de0 2024-09-12T01:03:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- 172.21.15.16:0/1017479673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 msgr2=0x561a0a7b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b70fa3640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 0x7f7b5c0a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561a0a7b4ac0 tx=0x7f7b60097950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7b64035000 tx=0x7f7b64009800 comp rx=0 tx=0).stop 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- 172.21.15.16:0/1017479673 shutdown_connections 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b75d0 0x561a0a7b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- 172.21.15.16:0/1017479673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7b7090 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- 172.21.15.16:0/1017479673 >> 172.21.15.16:0/1017479673 conn(0x561a0a780310 msgr2=0x561a0a7b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- 172.21.15.16:0/1017479673 shutdown_connections 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- 172.21.15.16:0/1017479673 wait complete. 2024-09-12T01:03:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- start start 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 Processor -- start 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7c6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a0a7c9420 con 0x561a0a7c6780 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a0a6d8060 con 0x561a0a7b8b40 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.785+0000 7f7b755e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a0a7c95a0 con 0x561a0a7b75d0 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b755e0740 10 client.0 _ll_register_callbacks cb 0x561a0a42b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46212/0 (socket says 172.21.15.16:46212) 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 -- 172.21.15.16:0/2885389424 learned_addr learned my addr 172.21.15.16:0/2885389424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 -- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 msgr2=0x561a0a7c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 -- 172.21.15.16:0/2885389424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 msgr2=0x561a0a7c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 -- 172.21.15.16:0/2885389424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0a48e3c0 con 0x561a0a7c6780 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7b64031940 tx=0x7f7b6403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc07935dfa9d255b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64003b70 con 0x561a0a7c6780 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7b64003d10 con 0x561a0a7c6780 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b64036040 con 0x561a0a7c6780 2024-09-12T01:03:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b64004b40 con 0x561a0a7c6780 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b755e0740 10 client.17493 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.786+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a0a7c9420 con 0x561a0a7c6780 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64036040 con 0x561a0a7c6780 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b6402f6a0 con 0x561a0a7c6780 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 10 client.17493 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 10 client.17493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 20 client.17493 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a7d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 4 client.17493 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0a72a860 con 0x561a0a7d5f60 2024-09-12T01:03:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.787+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a7d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a7d8350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53882a775318c953 server_cookie=57c3e84c1e816a97 in_seq=0 out_seq=0 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b4bfff640 10 client.17493 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2885389424 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a0a72a860 con 0x561a0a7d5f60 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b4bfff640 10 client.17493 handle_command_reply: tid=0 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 client.17493 shutdown 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 msgr2=0x561a0a7c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7b64031940 tx=0x7f7b6403e040 comp rx=0 tx=0).stop 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 shutdown_connections 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a7d8350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b4bfff640 0 client.17493 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7c6780 0x561a0a7c3970 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.788+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2885389424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.789+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 >> 172.21.15.16:0/2885389424 conn(0x561a0a780310 msgr2=0x561a0a7b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.789+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 shutdown_connections 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.789+0000 7f7b755e0740 1 -- 172.21.15.16:0/2885389424 wait complete. 2024-09-12T01:03:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.789+0000 7f7b755e0740 20 client.17493 trim_cache size 0 max 16384 2024-09-12T01:03:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.802+0000 7f7b755e0740 1 Processor -- start 2024-09-12T01:03:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 -- start start 2024-09-12T01:03:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1ab6c0 con 0x561a0a7b75d0 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1ad200 con 0x561a0a7b8b40 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b755e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1abd90 con 0x561a0a7c6780 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46222/0 (socket says 172.21.15.16:46222) 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 -- 172.21.15.16:0/719017825 learned_addr learned my addr 172.21.15.16:0/719017825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 -- 172.21.15.16:0/719017825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 msgr2=0x561a0a7dda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 -- 172.21.15.16:0/719017825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 msgr2=0x561a0a7c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 -- 172.21.15.16:0/719017825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0a7b3510 con 0x561a0a7b75d0 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.803+0000 7f7b717a4640 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ece400e728cc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b567fc640 1 -- 172.21.15.16:0/719017825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64031bb0 con 0x561a0a7b75d0 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b567fc640 1 -- 172.21.15.16:0/719017825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64036070 con 0x561a0a7b75d0 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b567fc640 1 -- 172.21.15.16:0/719017825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64030aa0 con 0x561a0a7b75d0 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 -- 172.21.15.16:0/719017825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 msgr2=0x561a0a7d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640093f0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 -- 172.21.15.16:0/719017825 shutdown_connections 2024-09-12T01:03:20.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7c6780 0x561a0a7dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7b8b40 0x561a0a7c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 --2- 172.21.15.16:0/719017825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b75d0 0x561a0a7d26c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 -- 172.21.15.16:0/719017825 >> 172.21.15.16:0/719017825 conn(0x561a0a780310 msgr2=0x561a0a7c1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 -- 172.21.15.16:0/719017825 shutdown_connections 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.804+0000 7f7b755e0740 1 -- 172.21.15.16:0/719017825 wait complete. 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 -- start start 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 Processor -- start 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 0x561a0a7c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1acb30 con 0x561a0a7b8b40 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1ac460 con 0x561a0a7c6780 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1a96c0 con 0x561a0a7b75d0 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b755e0740 10 client.0 _ll_register_callbacks cb 0x7f7b58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46238/0 (socket says 172.21.15.16:46238) 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b6bfff640 1 -- 172.21.15.16:0/2035136254 learned_addr learned my addr 172.21.15.16:0/2035136254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.805+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 0x561a0a7c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 -- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 msgr2=0x561a0a7cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 -- 172.21.15.16:0/2035136254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 msgr2=0x561a0a7c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 0x561a0a7c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 -- 172.21.15.16:0/2035136254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0a83f9d0 con 0x561a0a7b8b40 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 0x561a0a7c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b6bfff640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004700 tx=0x7f7b58012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51afe85ca0c1daa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58004460 con 0x561a0a7b8b40 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7b580035f0 con 0x561a0a7b8b40 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b64036070 con 0x561a0a7b8b40 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b5802e050 con 0x561a0a7b8b40 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b755e0740 10 client.17499 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a0a840a90 con 0x561a0a7b8b40 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.806+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64036070 con 0x561a0a7b8b40 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b5801ea00 con 0x561a0a7b8b40 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 10 client.17499 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 10 client.17499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 20 client.17499 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a843250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 4 client.17499 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a0a724d70 con 0x561a0a7d5f60 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.807+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a843250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b4bfff640 10 client.17499 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:20.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b717a4640 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a843250 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=714c9ae7f36b3b86 server_cookie=973730ee74009718 in_seq=0 out_seq=0 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b4bfff640 1 -- 172.21.15.16:0/2035136254 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a0a724d70 con 0x561a0a7d5f60 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b4bfff640 10 client.17499 handle_command_reply: tid=0 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 client.17499 shutdown 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 msgr2=0x561a0a7c0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004700 tx=0x7f7b58012820 comp rx=0 tx=0).stop 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 shutdown_connections 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a0a7c6780 0x561a0a7c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x561a0a7d5f60 0x561a0a843250 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a0a7b8b40 0x561a0a7c0270 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 --2- 172.21.15.16:0/2035136254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0a7b75d0 0x561a0a7cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 >> 172.21.15.16:0/2035136254 conn(0x561a0a780310 msgr2=0x561a0a7c24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b4bfff640 0 client.17499 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 shutdown_connections 2024-09-12T01:03:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 1 -- 172.21.15.16:0/2035136254 wait complete. 2024-09-12T01:03:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.808+0000 7f7b755e0740 20 client.17499 trim_cache size 0 max 16384 2024-09-12T01:03:20.816 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 msgr2=0x7f7b5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 0x7f7b5c0a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x561a0a7b4ac0 tx=0x7f7b60097950 comp rx=0 tx=0).stop 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 msgr2=0x7f7b6c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000e9f0 tx=0x7f7b6000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 shutdown_connections 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b5c0a2d70 0x7f7b5c0a5230 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b6c106f10 0x7f7b6c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c102790 0x7f7b6c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 --2- 172.21.15.16:0/1033555592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b6c101de0 0x7f7b6c198480 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 >> 172.21.15.16:0/1033555592 conn(0x7f7b6c0fd8c0 msgr2=0x7f7b6c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 shutdown_connections 2024-09-12T01:03:20.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:20.809+0000 7f7b73207640 1 -- 172.21.15.16:0/1033555592 wait complete. 2024-09-12T01:03:21.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:03:21.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 -- 172.21.15.16:0/1105271792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 msgr2=0x7fba90106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 --2- 172.21.15.16:0/1105271792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba90106340 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ca0 tx=0x7fba8802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 -- 172.21.15.16:0/1105271792 shutdown_connections 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 --2- 172.21.15.16:0/1105271792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba90106340 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 --2- 172.21.15.16:0/1105271792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 0x7fba90105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 --2- 172.21.15.16:0/1105271792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba90100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 -- 172.21.15.16:0/1105271792 >> 172.21.15.16:0/1105271792 conn(0x7fba900fc7d0 msgr2=0x7fba900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 -- 172.21.15.16:0/1105271792 shutdown_connections 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.427+0000 7fba97a1d640 1 -- 172.21.15.16:0/1105271792 wait complete. 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 Processor -- start 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 -- start start 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba901a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 0x7fba901a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba90118630 con 0x7fba90105ec0 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba901184b0 con 0x7fba901011e0 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba97a1d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba901187b0 con 0x7fba901008c0 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46244/0 (socket says 172.21.15.16:46244) 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba95fba640 1 -- 172.21.15.16:0/898791886 learned_addr learned my addr 172.21.15.16:0/898791886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.428+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba901a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba94fb8640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 0x7fba901a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 -- 172.21.15.16:0/898791886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 msgr2=0x7fba901a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 0x7fba901a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 -- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 msgr2=0x7fba9019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 -- 172.21.15.16:0/898791886 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9019f070 con 0x7fba901008c0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba95fba640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba901a4780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9f0 tx=0x7fba8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba867fc640 1 -- 172.21.15.16:0/898791886 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00ccc0 con 0x7fba901008c0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9019f360 con 0x7fba901008c0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba867fc640 1 -- 172.21.15.16:0/898791886 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba8c005020 con 0x7fba901008c0 2024-09-12T01:03:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba867fc640 1 -- 172.21.15.16:0/898791886 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0049e0 con 0x7fba901008c0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.429+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba901ab650 con 0x7fba901008c0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 Processor -- start 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 -- start start 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f37e852f0 con 0x560f37f20b40 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f37e2ccf0 con 0x560f37f1f5d0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba99df6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f37e812b0 con 0x560f37f2a370 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba867fc640 1 -- 172.21.15.16:0/898791886 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba8c005560 con 0x7fba901008c0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46256/0 (socket says 172.21.15.16:46256) 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba94fb8640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.432+0000 7fba95fba640 1 -- 172.21.15.16:0/2699055135 learned_addr learned my addr 172.21.15.16:0/2699055135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba957b9640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 -- 172.21.15.16:0/2699055135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 msgr2=0x560f37f2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 -- 172.21.15.16:0/2699055135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 msgr2=0x560f37f1fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f1fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 -- 172.21.15.16:0/2699055135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f37e113a0 con 0x560f37f20b40 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba957b9640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba95fba640 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba88033000 tx=0x7fba88005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682fa479fef1ac61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba867fc640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 0x7fba7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba7b7fe640 1 -- 172.21.15.16:0/2699055135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88003970 con 0x560f37f20b40 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba7b7fe640 1 -- 172.21.15.16:0/2699055135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba88036040 con 0x560f37f20b40 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba7b7fe640 1 -- 172.21.15.16:0/2699055135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880045d0 con 0x560f37f20b40 2024-09-12T01:03:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 0x7fba7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.433+0000 7fba867fc640 1 -- 172.21.15.16:0/898791886 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fba8c0cf3c0 con 0x7fba901008c0 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- 172.21.15.16:0/2699055135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 msgr2=0x560f37f1f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba957b9640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 0x7fba7c0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0ccbc0 tx=0x7fba8c0d1040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fba88033000 tx=0x7fba88005ef0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- 172.21.15.16:0/2699055135 shutdown_connections 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 --2- 172.21.15.16:0/2699055135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1f090 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- 172.21.15.16:0/2699055135 >> 172.21.15.16:0/2699055135 conn(0x560f37ee8310 msgr2=0x560f37f1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- 172.21.15.16:0/2699055135 shutdown_connections 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- 172.21.15.16:0/2699055135 wait complete. 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.434+0000 7fba99df6740 1 -- start start 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 Processor -- start 2024-09-12T01:03:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f26fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f37f2ce90 con 0x560f37f2a370 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f37e40060 con 0x560f37f1f5d0 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f37f2d010 con 0x560f37f20b40 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba99df6740 10 client.0 _ll_register_callbacks cb 0x560f37b938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46270/0 (socket says 172.21.15.16:46270) 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba95fba640 1 -- 172.21.15.16:0/3515469264 learned_addr learned my addr 172.21.15.16:0/3515469264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba94fb8640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f26fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.435+0000 7fba95fba640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 -- 172.21.15.16:0/3515469264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 msgr2=0x560f37f26fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f26fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 -- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 msgr2=0x560f37f29d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f29d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 -- 172.21.15.16:0/3515469264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f37bf63c0 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba95fba640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba88005e30 tx=0x7fba88004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca7b6879aee863 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88057060 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fba88031660 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba88036040 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba88031860 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba99df6740 10 client.17511 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f37f2ce90 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba88036040 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba88051040 con 0x560f37f2a370 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba99df6740 10 client.17511 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.436+0000 7fba99df6740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba99df6740 20 client.17511 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37f399b0 0x560f37f3bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba99df6740 4 client.17511 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f37e92860 con 0x560f37f399b0 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba94fb8640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37f399b0 0x560f37f3bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba94fb8640 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37f399b0 0x560f37f3bda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9de9f1845c6a47e server_cookie=fa5ea5ac4e26fc51 in_seq=0 out_seq=0 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.437+0000 7fba78ff9640 10 client.17511 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba78ff9640 1 -- 172.21.15.16:0/3515469264 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f37e92860 con 0x560f37f399b0 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba78ff9640 10 client.17511 handle_command_reply: tid=0 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 client.17511 shutdown 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 msgr2=0x560f37f27530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fba88005e30 tx=0x7fba88004290 comp rx=0 tx=0).stop 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 shutdown_connections 2024-09-12T01:03:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37f399b0 0x560f37f3bda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f2a370 0x560f37f27530 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f26fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba78ff9640 0 client.17511 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 --2- 172.21.15.16:0/3515469264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f29d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 >> 172.21.15.16:0/3515469264 conn(0x560f37ee8310 msgr2=0x560f37f21960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 shutdown_connections 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 1 -- 172.21.15.16:0/3515469264 wait complete. 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.438+0000 7fba99df6740 20 client.17511 trim_cache size 0 max 16384 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.451+0000 7fba99df6740 1 Processor -- start 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.451+0000 7fba99df6740 1 -- start start 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f1ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b3e40 con 0x560f37f20b40 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b5980 con 0x560f37f1f5d0 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba99df6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b4510 con 0x560f37f2a370 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46280/0 (socket says 172.21.15.16:46280) 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 -- 172.21.15.16:0/1196956901 learned_addr learned my addr 172.21.15.16:0/1196956901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba95fba640 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 -- 172.21.15.16:0/1196956901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 msgr2=0x560f37f1ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f1ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 -- 172.21.15.16:0/1196956901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 msgr2=0x560f37f28f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f28f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 -- 172.21.15.16:0/1196956901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f37f2a070 con 0x560f37f20b40 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba94fb8640 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fba80009dd0 tx=0x7fba80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba75aeb39e6d687c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba7affd640 1 -- 172.21.15.16:0/1196956901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80003e20 con 0x560f37f20b40 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba7affd640 1 -- 172.21.15.16:0/1196956901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba80027910 con 0x560f37f20b40 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.452+0000 7fba7affd640 1 -- 172.21.15.16:0/1196956901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba80010b20 con 0x560f37f20b40 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 -- 172.21.15.16:0/1196956901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 msgr2=0x560f37f1b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fba80009dd0 tx=0x7fba80005f90 comp rx=0 tx=0).stop 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 -- 172.21.15.16:0/1196956901 shutdown_connections 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f2a370 0x560f37f1ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f20b40 0x560f37f1b510 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 --2- 172.21.15.16:0/1196956901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f1f5d0 0x560f37f28f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 -- 172.21.15.16:0/1196956901 >> 172.21.15.16:0/1196956901 conn(0x560f37ee8310 msgr2=0x560f37f28810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 -- 172.21.15.16:0/1196956901 shutdown_connections 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.453+0000 7fba99df6740 1 -- 172.21.15.16:0/1196956901 wait complete. 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 -- start start 2024-09-12T01:03:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 Processor -- start 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f2a370 0x560f37f2df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b52b0 con 0x560f37f1f5d0 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b4be0 con 0x560f37f2a370 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba901b1e40 con 0x560f37f20b40 2024-09-12T01:03:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba99df6740 10 client.0 _ll_register_callbacks cb 0x7fba800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba95fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46284/0 (socket says 172.21.15.16:46284) 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba95fba640 1 -- 172.21.15.16:0/819623340 learned_addr learned my addr 172.21.15.16:0/819623340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.454+0000 7fba94fb8640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 -- 172.21.15.16:0/819623340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 msgr2=0x560f37f2da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f2da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 -- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f2a370 msgr2=0x560f37f2df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f2a370 0x560f37f2df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 -- 172.21.15.16:0/819623340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f37f2e480 con 0x560f37f1f5d0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba95fba640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fba88009a60 tx=0x7fba88004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af48db80f3653747 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88040020 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fba88002030 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba80027910 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba88057040 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba99df6740 10 client.17517 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f37fa4290 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.455+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba80027910 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba88044070 con 0x560f37f1f5d0 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 10 client.17517 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 20 client.17517 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37fa4900 0x560f37fa6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 4 client.17517 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f37e8cd70 con 0x560f37fa4900 2024-09-12T01:03:22.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba94fb8640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37fa4900 0x560f37fa6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba78ff9640 10 client.17517 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba94fb8640 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37fa4900 0x560f37fa6d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3dcea3029b2877d server_cookie=c0fa4d8b5e0be19d in_seq=0 out_seq=0 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba78ff9640 1 -- 172.21.15.16:0/819623340 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f37e8cd70 con 0x560f37fa4900 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba78ff9640 10 client.17517 handle_command_reply: tid=0 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.456+0000 7fba99df6740 1 client.17517 shutdown 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 msgr2=0x560f37f326d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fba88009a60 tx=0x7fba88004680 comp rx=0 tx=0).stop 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 shutdown_connections 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560f37fa4900 0x560f37fa6d20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f37f2a370 0x560f37f2df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f37f20b40 0x560f37f2da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba78ff9640 0 client.17517 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 --2- 172.21.15.16:0/819623340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f37f1f5d0 0x560f37f326d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 >> 172.21.15.16:0/819623340 conn(0x560f37ee8310 msgr2=0x560f37f29030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 shutdown_connections 2024-09-12T01:03:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 1 -- 172.21.15.16:0/819623340 wait complete. 2024-09-12T01:03:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba99df6740 20 client.17517 trim_cache size 0 max 16384 2024-09-12T01:03:22.471 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 msgr2=0x7fba7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 0x7fba7c0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0ccbc0 tx=0x7fba8c0d1040 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 msgr2=0x7fba901a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.457+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba901a4780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9f0 tx=0x7fba8c00eeb0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 shutdown_connections 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fba7c0a2cf0 0x7fba7c0a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba90105ec0 0x7fba9019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba901011e0 0x7fba901a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 --2- 172.21.15.16:0/898791886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba901008c0 0x7fba901a4780 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 >> 172.21.15.16:0/898791886 conn(0x7fba900fc7d0 msgr2=0x7fba90103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 shutdown_connections 2024-09-12T01:03:22.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:22.458+0000 7fba97a1d640 1 -- 172.21.15.16:0/898791886 wait complete. 2024-09-12T01:03:22.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:22 smithi016 ceph-mon[30972]: pgmap v224: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:22.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:22 smithi159 ceph-mon[35566]: pgmap v224: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:22 smithi099 ceph-mon[35654]: pgmap v224: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-12T01:03:23.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:03:23.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1289448402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 msgr2=0x7ff394106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1289448402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff394106e60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff388009930 tx=0x7ff38802a420 comp rx=0 tx=0).stop 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1289448402 shutdown_connections 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1289448402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 0x7ff39410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1289448402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 0x7ff394100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1289448402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff394106e60 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1289448402 >> 172.21.15.16:0/1289448402 conn(0x7ff3940fc7a0 msgr2=0x7ff3940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1289448402 shutdown_connections 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.952+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1289448402 wait complete. 2024-09-12T01:03:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 Processor -- start 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 -- start start 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 0x7ff3941984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 0x7ff394198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff394110ea0 con 0x7ff394106a80 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff394110d20 con 0x7ff394101420 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff39b1cc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff394111020 con 0x7ff394100a80 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff399769640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff399769640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46316/0 (socket says 172.21.15.16:46316) 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff399769640 1 -- 172.21.15.16:0/1007286045 learned_addr learned my addr 172.21.15.16:0/1007286045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff398f68640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 0x7ff3941984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.953+0000 7ff393fff640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 0x7ff394198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 -- 172.21.15.16:0/1007286045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 msgr2=0x7ff3941984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 0x7ff3941984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 -- 172.21.15.16:0/1007286045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 msgr2=0x7ff394198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 0x7ff394198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 -- 172.21.15.16:0/1007286045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39419d480 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff399769640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00eaa0 tx=0x7ff38c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff391ffb640 1 -- 172.21.15.16:0/1007286045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00cdb0 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff391ffb640 1 -- 172.21.15.16:0/1007286045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c005020 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff391ffb640 1 -- 172.21.15.16:0/1007286045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0116b0 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39419d770 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.954+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a5050 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.956+0000 7ff391ffb640 1 -- 172.21.15.16:0/1007286045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff38c005560 con 0x7ff394106a80 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 Processor -- start 2024-09-12T01:03:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 -- start start 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165836090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165847b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56316579c2f0 con 0x5631658365d0 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563165743cf0 con 0x563165845780 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5631657982b0 con 0x563165837b40 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff398f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff398f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46324/0 (socket says 172.21.15.16:46324) 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff398f68640 1 -- 172.21.15.16:0/447650452 learned_addr learned my addr 172.21.15.16:0/447650452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff391ffb640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 0x7ff3840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff393fff640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165836090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff399769640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 0x7ff3840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.957+0000 7ff393fff640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165847b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 -- 172.21.15.16:0/447650452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 msgr2=0x563165836090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165836090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 -- 172.21.15.16:0/447650452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 msgr2=0x563165847b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff391ffb640 1 -- 172.21.15.16:0/1007286045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff38c0cf6d0 con 0x7ff394106a80 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165847b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 -- 172.21.15.16:0/447650452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631657283a0 con 0x5631658365d0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff393fff640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165847b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff398f68640 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff388009930 tx=0x7ff388031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bb47345395dd60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff399769640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 0x7ff3840a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06ebd0 tx=0x7ff38c06db50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff37effd640 1 -- 172.21.15.16:0/447650452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388040070 con 0x5631658365d0 2024-09-12T01:03:23.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff37effd640 1 -- 172.21.15.16:0/447650452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff388005020 con 0x5631658365d0 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.958+0000 7ff37effd640 1 -- 172.21.15.16:0/447650452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388035040 con 0x5631658365d0 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- 172.21.15.16:0/447650452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 msgr2=0x563165836a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff388009930 tx=0x7ff388031630 comp rx=0 tx=0).stop 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- 172.21.15.16:0/447650452 shutdown_connections 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165847b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x563165836a10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/447650452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165836090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- 172.21.15.16:0/447650452 >> 172.21.15.16:0/447650452 conn(0x5631657ff310 msgr2=0x563165832c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- 172.21.15.16:0/447650452 shutdown_connections 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- 172.21.15.16:0/447650452 wait complete. 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 -- start start 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.959+0000 7ff39d5a5740 1 Processor -- start 2024-09-12T01:03:23.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165842350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165842890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563165848420 con 0x5631658365d0 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563165757060 con 0x563165845780 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5631658485a0 con 0x563165837b40 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff39d5a5740 10 client.0 _ll_register_callbacks cb 0x5631654aa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46330/0 (socket says 172.21.15.16:46330) 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 -- 172.21.15.16:0/1605021324 learned_addr learned my addr 172.21.15.16:0/1605021324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff398f68640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165842350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165842890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 -- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 msgr2=0x563165842350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165842350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 -- 172.21.15.16:0/1605021324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 msgr2=0x563165842890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165842890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 -- 172.21.15.16:0/1605021324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316550d3c0 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff393fff640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff38000afd0 tx=0x7ff3800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace3277c36e09c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.960+0000 7ff373fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38001d060 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff388005020 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff380011630 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff380011830 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 10 client.17529 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563165853e00 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff388005020 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff380018c40 con 0x5631658365d0 2024-09-12T01:03:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 10 client.17529 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 10 client.17529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 20 client.17529 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563165855210 0x563165857600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 4 client.17529 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.961+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631657a9860 con 0x563165855210 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.962+0000 7ff398f68640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563165855210 0x563165857600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.962+0000 7ff398f68640 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563165855210 0x563165857600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10e3f7e309bf2398 server_cookie=65a4b183a8734b04 in_seq=0 out_seq=0 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.962+0000 7ff373fff640 10 client.17529 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff373fff640 1 -- 172.21.15.16:0/1605021324 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631657a9860 con 0x563165855210 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff373fff640 10 client.17529 handle_command_reply: tid=0 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 client.17529 shutdown 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 msgr2=0x5631658450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff38000afd0 tx=0x7ff3800047d0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 shutdown_connections 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563165855210 0x563165857600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x563165842890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff373fff640 0 client.17529 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165842350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/1605021324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658450a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 >> 172.21.15.16:0/1605021324 conn(0x5631657ff310 msgr2=0x563165838960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 shutdown_connections 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 1 -- 172.21.15.16:0/1605021324 wait complete. 2024-09-12T01:03:23.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.963+0000 7ff39d5a5740 20 client.17529 trim_cache size 0 max 16384 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 Processor -- start 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 -- start start 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165846b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x5631658470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941ab6d0 con 0x5631658365d0 2024-09-12T01:03:23.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941ad210 con 0x563165845780 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.976+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941abda0 con 0x563165837b40 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46342/0 (socket says 172.21.15.16:46342) 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 -- 172.21.15.16:0/3317908985 learned_addr learned my addr 172.21.15.16:0/3317908985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff398f68640 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165846b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 -- 172.21.15.16:0/3317908985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 msgr2=0x563165846b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165846b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 -- 172.21.15.16:0/3317908985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 msgr2=0x5631658470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x5631658470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 -- 172.21.15.16:0/3317908985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56316583df40 con 0x5631658365d0 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff398f68640 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165846b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:23.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff393fff640 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff380006fd0 tx=0x7ff38000acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f4cec88a95bc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff37e7fc640 1 -- 172.21.15.16:0/3317908985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38000a4c0 con 0x5631658365d0 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff37e7fc640 1 -- 172.21.15.16:0/3317908985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff380017620 con 0x5631658365d0 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.977+0000 7ff37e7fc640 1 -- 172.21.15.16:0/3317908985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380011c30 con 0x5631658365d0 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- 172.21.15.16:0/3317908985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 msgr2=0x5631658516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff380006fd0 tx=0x7ff38000acc0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- 172.21.15.16:0/3317908985 shutdown_connections 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563165845780 0x5631658470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x563165846b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/3317908985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5631658365d0 0x5631658516c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- 172.21.15.16:0/3317908985 >> 172.21.15.16:0/3317908985 conn(0x5631657ff310 msgr2=0x563165842310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- 172.21.15.16:0/3317908985 shutdown_connections 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- 172.21.15.16:0/3317908985 wait complete. 2024-09-12T01:03:23.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.978+0000 7ff39d5a5740 1 -- start start 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 Processor -- start 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631658365d0 0x56316585c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x56316585cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941acb40 con 0x563165845780 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941ac470 con 0x5631658365d0 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941a96b0 con 0x563165837b40 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff39d5a5740 10 client.0 _ll_register_callbacks cb 0x7ff38000ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46348/0 (socket says 172.21.15.16:46348) 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 -- 172.21.15.16:0/2771630092 learned_addr learned my addr 172.21.15.16:0/2771630092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff398f68640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x56316585cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631658365d0 0x56316585c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 -- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 msgr2=0x56316585cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x56316585cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 -- 172.21.15.16:0/2771630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631658365d0 msgr2=0x56316585c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631658365d0 0x56316585c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.979+0000 7ff393fff640 1 -- 172.21.15.16:0/2771630092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56316584cf20 con 0x563165845780 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff393fff640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff3800165e0 tx=0x7ff380008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07b0b635a68ac03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380016f90 con 0x563165845780 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff380017aa0 con 0x563165845780 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff380017620 con 0x563165845780 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff38001d030 con 0x563165845780 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 10 client.17535 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56316584e190 con 0x563165845780 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff380017620 con 0x563165845780 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff380018770 con 0x563165845780 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 10 client.17535 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 20 client.17535 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5631658bffe0 0x5631658c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 4 client.17535 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.980+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631657a3d70 con 0x5631658bffe0 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff398f68640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5631658bffe0 0x5631658c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff398f68640 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5631658bffe0 0x5631658c2400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3382ad324267a03c server_cookie=4377e2705988102d in_seq=0 out_seq=0 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff373fff640 10 client.17535 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff373fff640 1 -- 172.21.15.16:0/2771630092 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631657a3d70 con 0x5631658bffe0 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff373fff640 10 client.17535 handle_command_reply: tid=0 2024-09-12T01:03:23.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.981+0000 7ff39d5a5740 1 client.17535 shutdown 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 msgr2=0x56316584c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff3800165e0 tx=0x7ff380008040 comp rx=0 tx=0).stop 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 shutdown_connections 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563165845780 0x56316584c830 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563165837b40 0x56316585cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5631658bffe0 0x5631658c2400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 --2- 172.21.15.16:0/2771630092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5631658365d0 0x56316585c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff373fff640 0 client.17535 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 >> 172.21.15.16:0/2771630092 conn(0x5631657ff310 msgr2=0x5631658407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 shutdown_connections 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 1 -- 172.21.15.16:0/2771630092 wait complete. 2024-09-12T01:03:23.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39d5a5740 20 client.17535 trim_cache size 0 max 16384 2024-09-12T01:03:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 msgr2=0x7ff3840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 0x7ff3840a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06ebd0 tx=0x7ff38c06db50 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 msgr2=0x7ff39419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.982+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00eaa0 tx=0x7ff38c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 shutdown_connections 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3840a2d70 0x7ff3840a5230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff394106a80 0x7ff39419cda0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff394101420 0x7ff394198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 --2- 172.21.15.16:0/1007286045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff394100a80 0x7ff3941984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 >> 172.21.15.16:0/1007286045 conn(0x7ff3940fc7a0 msgr2=0x7ff3941048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 shutdown_connections 2024-09-12T01:03:23.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:23.983+0000 7ff39b1cc640 1 -- 172.21.15.16:0/1007286045 wait complete. 2024-09-12T01:03:24.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:24 smithi016 ceph-mon[30972]: pgmap v225: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:03:24.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:03:24.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:24 smithi159 ceph-mon[35566]: pgmap v225: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:03:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:24 smithi099 ceph-mon[35654]: pgmap v225: 97 pgs: 97 active+clean; 8.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- 172.21.15.16:0/1204914785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4108160 msgr2=0x7fabe410eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/1204914785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4108160 0x7fabe410eb00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009cd0 tx=0x7fabdc02a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- 172.21.15.16:0/1204914785 shutdown_connections 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/1204914785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4108160 0x7fabe410eb00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/1204914785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4101290 0x7fabe4107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/1204914785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 0x7fabe4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- 172.21.15.16:0/1204914785 >> 172.21.15.16:0/1204914785 conn(0x7fabe4077210 msgr2=0x7fabe4077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- 172.21.15.16:0/1204914785 shutdown_connections 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- 172.21.15.16:0/1204914785 wait complete. 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 Processor -- start 2024-09-12T01:03:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- start start 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 0x7fabe419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4108160 0x7fabe419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.545+0000 7fabeaa7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4111ff0 con 0x7fabe4101290 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabeaa7e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4111e70 con 0x7fabe41008e0 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabeaa7e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabe4112170 con 0x7fabe4108160 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46368/0 (socket says 172.21.15.16:46368) 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 -- 172.21.15.16:0/2415756773 learned_addr learned my addr 172.21.15.16:0/2415756773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe881a640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 0x7fabe419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 -- 172.21.15.16:0/2415756773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4108160 msgr2=0x7fabe419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe901b640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4108160 0x7fabe419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4108160 0x7fabe419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 -- 172.21.15.16:0/2415756773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 msgr2=0x7fabe419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 0x7fabe419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 -- 172.21.15.16:0/2415756773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe419f610 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe3fff640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fabd800d720 tx=0x7fabd800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe1ffb640 1 -- 172.21.15.16:0/2415756773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8017070 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe1ffb640 1 -- 172.21.15.16:0/2415756773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd800de30 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabe1ffb640 1 -- 172.21.15.16:0/2415756773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8004c70 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.546+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe419f900 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.547+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe41a71e0 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabe1ffb640 1 -- 172.21.15.16:0/2415756773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fabd8004e50 con 0x7fabe4101290 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 Processor -- start 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 -- start start 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb61a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5ceac72f0 con 0x55a5ceb6c370 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5cea6ecf0 con 0x55a5ceb615d0 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabece57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5ceac32b0 con 0x55a5ceb62b40 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46376/0 (socket says 172.21.15.16:46376) 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabe3fff640 1 -- 172.21.15.16:0/3105234277 learned_addr learned my addr 172.21.15.16:0/3105234277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.550+0000 7fabe901b640 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe881a640 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb61a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe1ffb640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 0x7fabcc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 -- 172.21.15.16:0/3105234277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 msgr2=0x55a5ceb61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 -- 172.21.15.16:0/3105234277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 msgr2=0x55a5ceb61a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb61a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 -- 172.21.15.16:0/3105234277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cea533a0 con 0x55a5ceb6c370 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe901b640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 0x7fabcc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe1ffb640 1 -- 172.21.15.16:0/2415756773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fabd80cee40 con 0x7fabe4101290 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fabd80cc500 tx=0x7fabd8097430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676667a24aa38d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabcaffd640 1 -- 172.21.15.16:0/3105234277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd8020070 con 0x55a5ceb6c370 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabcaffd640 1 -- 172.21.15.16:0/3105234277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd809ba60 con 0x55a5ceb6c370 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.551+0000 7fabcaffd640 1 -- 172.21.15.16:0/3105234277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd80a0c20 con 0x55a5ceb6c370 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- 172.21.15.16:0/3105234277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 msgr2=0x55a5ceb6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabe901b640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 0x7fabcc0a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabdc02ae00 tx=0x7fabdc0095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fabd80cc500 tx=0x7fabd8097430 comp rx=0 tx=0).stop 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- 172.21.15.16:0/3105234277 shutdown_connections 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6e760 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb61a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 --2- 172.21.15.16:0/3105234277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- 172.21.15.16:0/3105234277 >> 172.21.15.16:0/3105234277 conn(0x55a5ceb2a310 msgr2=0x55a5ceb5dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- 172.21.15.16:0/3105234277 shutdown_connections 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- 172.21.15.16:0/3105234277 wait complete. 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 -- start start 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.552+0000 7fabece57740 1 Processor -- start 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 0x55a5ceb69520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5ceb6efb0 con 0x55a5ceb615d0 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5cea82060 con 0x55a5ceb62b40 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5ceb6f130 con 0x55a5ceb6c370 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabece57740 10 client.0 _ll_register_callbacks cb 0x55a5ce7d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46378/0 (socket says 172.21.15.16:46378) 2024-09-12T01:03:25.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 -- 172.21.15.16:0/3318923049 learned_addr learned my addr 172.21.15.16:0/3318923049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe901b640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 0x55a5ceb69520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 -- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 msgr2=0x55a5ceb69520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 0x55a5ceb69520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 -- 172.21.15.16:0/3318923049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 msgr2=0x55a5ceb68fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb68fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe881a640 1 -- 172.21.15.16:0/3318923049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ce8383c0 con 0x55a5ceb615d0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.553+0000 7fabe901b640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 0x55a5ceb69520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabe881a640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fabd400bed0 tx=0x7fabd40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f70b38cc2756fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004220 con 0x55a5ceb615d0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fabd40043c0 con 0x55a5ceb615d0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabd809ba60 con 0x55a5ceb615d0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabd40035f0 con 0x55a5ceb615d0 2024-09-12T01:03:25.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 10 client.17547 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5ceb7aa90 con 0x55a5ceb615d0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd809ba60 con 0x55a5ceb615d0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fabd4024a80 con 0x55a5ceb615d0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 10 client.17547 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 20 client.17547 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5ceb7bea0 0x55a5ceb7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 4 client.17547 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.554+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5cead4860 con 0x55a5ceb7bea0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.555+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5ceb7bea0 0x55a5ceb7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.555+0000 7fabc3fff640 10 client.17547 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.555+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5ceb7bea0 0x55a5ceb7e2b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8290e355e238457 server_cookie=89cc5017620d862f in_seq=0 out_seq=0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabc3fff640 1 -- 172.21.15.16:0/3318923049 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fabd8080e80 con 0x55a5ceb7bea0 2024-09-12T01:03:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabc3fff640 10 client.17547 handle_command_reply: tid=0 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 client.17547 shutdown 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 msgr2=0x55a5ceb6bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fabd400bed0 tx=0x7fabd40117b0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 shutdown_connections 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5ceb7bea0 0x55a5ceb7e2b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb6c370 0x55a5ceb69520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb68fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabc3fff640 0 client.17547 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 --2- 172.21.15.16:0/3318923049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb615d0 0x55a5ceb6bcf0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 >> 172.21.15.16:0/3318923049 conn(0x55a5ceb2a310 msgr2=0x55a5ceb63960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 shutdown_connections 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 1 -- 172.21.15.16:0/3318923049 wait complete. 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.556+0000 7fabece57740 20 client.17547 trim_cache size 0 max 16384 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 Processor -- start 2024-09-12T01:03:25.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 -- start start 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb615d0 0x55a5ceb78250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41afad0 con 0x55a5ceb6c370 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b1610 con 0x55a5ceb62b40 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabece57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b01a0 con 0x55a5ceb615d0 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabe901b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabe901b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46390/0 (socket says 172.21.15.16:46390) 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.569+0000 7fabe901b640 1 -- 172.21.15.16:0/1102413074 learned_addr learned my addr 172.21.15.16:0/1102413074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe881a640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb615d0 0x55a5ceb78250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe3fff640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 -- 172.21.15.16:0/1102413074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb615d0 msgr2=0x55a5ceb78250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb615d0 0x55a5ceb78250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 -- 172.21.15.16:0/1102413074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 msgr2=0x55a5ceb60d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb60d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 -- 172.21.15.16:0/1102413074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ceb5e730 con 0x55a5ceb6c370 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe3fff640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabe901b640 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009cd0 tx=0x7fabdc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2328fc4ca2f4bafa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabca7fc640 1 -- 172.21.15.16:0/1102413074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc004240 con 0x55a5ceb6c370 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabca7fc640 1 -- 172.21.15.16:0/1102413074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc036040 con 0x55a5ceb6c370 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.570+0000 7fabca7fc640 1 -- 172.21.15.16:0/1102413074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc03d420 con 0x55a5ceb6c370 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- 172.21.15.16:0/1102413074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 msgr2=0x55a5ceb6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009cd0 tx=0x7fabdc040000 comp rx=0 tx=0).stop 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- 172.21.15.16:0/1102413074 shutdown_connections 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb6d760 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb62b40 0x55a5ceb60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 --2- 172.21.15.16:0/1102413074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb615d0 0x55a5ceb78250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- 172.21.15.16:0/1102413074 >> 172.21.15.16:0/1102413074 conn(0x55a5ceb2a310 msgr2=0x55a5ceb67890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- 172.21.15.16:0/1102413074 shutdown_connections 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- 172.21.15.16:0/1102413074 wait complete. 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 -- start start 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.571+0000 7fabece57740 1 Processor -- start 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb74900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b0f40 con 0x55a5ceb6c370 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41b0870 con 0x55a5ceb615d0 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabe41adad0 con 0x55a5ceb62b40 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabece57740 10 client.0 _ll_register_callbacks cb 0x7fabd400b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46402/0 (socket says 172.21.15.16:46402) 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 -- 172.21.15.16:0/3516949387 learned_addr learned my addr 172.21.15.16:0/3516949387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe881a640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe3fff640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb74900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 -- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 msgr2=0x55a5ceb74900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb74900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 -- 172.21.15.16:0/3516949387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 msgr2=0x55a5ceb743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 -- 172.21.15.16:0/3516949387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ceb74f70 con 0x55a5ceb6c370 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabe901b640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0307d0 tx=0x7fabdc030800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a62f4bd23d9cf20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.572+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc004450 con 0x55a5ceb6c370 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fabdc0045f0 con 0x55a5ceb6c370 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabdc036040 con 0x55a5ceb6c370 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabdc0047f0 con 0x55a5ceb6c370 2024-09-12T01:03:25.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 10 client.17553 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5ceb66790 con 0x55a5ceb6c370 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc036040 con 0x55a5ceb6c370 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fabdc030070 con 0x55a5ceb6c370 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 10 client.17553 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 20 client.17553 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5cebe7b60 0x55a5cebe9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 4 client.17553 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.573+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5ceaced70 con 0x55a5cebe7b60 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabe881a640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5cebe7b60 0x55a5cebe9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabe881a640 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5cebe7b60 0x55a5cebe9f50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b921ecb6dff2dcfd server_cookie=d1990b02e6e31f5a in_seq=0 out_seq=0 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabc3fff640 10 client.17553 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabc3fff640 1 -- 172.21.15.16:0/3516949387 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5ceaced70 con 0x55a5cebe7b60 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabc3fff640 10 client.17553 handle_command_reply: tid=0 2024-09-12T01:03:25.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 client.17553 shutdown 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 msgr2=0x55a5ceb65230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0307d0 tx=0x7fabdc030800 comp rx=0 tx=0).stop 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 shutdown_connections 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a5cebe7b60 0x55a5cebe9f50 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5ceb6c370 0x55a5ceb65230 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5ceb62b40 0x55a5ceb74900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabc3fff640 0 client.17553 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 --2- 172.21.15.16:0/3516949387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5ceb615d0 0x55a5ceb743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.574+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 >> 172.21.15.16:0/3516949387 conn(0x55a5ceb2a310 msgr2=0x55a5ceb63110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 shutdown_connections 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabece57740 1 -- 172.21.15.16:0/3516949387 wait complete. 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabece57740 20 client.17553 trim_cache size 0 max 16384 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 msgr2=0x7fabcc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 0x7fabcc0a5230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fabdc02ae00 tx=0x7fabdc0095c0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 msgr2=0x7fabe419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fabd800d720 tx=0x7fabd800dbe0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 shutdown_connections 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fabcc0a2d70 0x7fabcc0a5230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabe4108160 0x7fabe419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe4101290 0x7fabe419aba0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 --2- 172.21.15.16:0/2415756773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabe41008e0 0x7fabe419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:25.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 >> 172.21.15.16:0/2415756773 conn(0x7fabe4077210 msgr2=0x7fabe4102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:25.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 shutdown_connections 2024-09-12T01:03:25.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:25.575+0000 7fabeaa7e640 1 -- 172.21.15.16:0/2415756773 wait complete. 2024-09-12T01:03:26.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:26.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:26 smithi016 ceph-mon[30972]: pgmap v226: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:03:26.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:26 smithi159 ceph-mon[35566]: pgmap v226: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:03:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:26 smithi099 ceph-mon[35654]: pgmap v226: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-12T01:03:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 -- 172.21.15.16:0/3043705612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 msgr2=0x7f4d78100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 --2- 172.21.15.16:0/3043705612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d78100ee0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009930 tx=0x7f4d6c02a420 comp rx=0 tx=0).stop 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 -- 172.21.15.16:0/3043705612 shutdown_connections 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 --2- 172.21.15.16:0/3043705612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78101420 0x7f4d7810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 --2- 172.21.15.16:0/3043705612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d78100ee0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 --2- 172.21.15.16:0/3043705612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78106a80 0x7f4d78106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 -- 172.21.15.16:0/3043705612 >> 172.21.15.16:0/3043705612 conn(0x7f4d780fc7a0 msgr2=0x7f4d780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 -- 172.21.15.16:0/3043705612 shutdown_connections 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.043+0000 7f4d7de76640 1 -- 172.21.15.16:0/3043705612 wait complete. 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.044+0000 7f4d7de76640 1 Processor -- start 2024-09-12T01:03:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.044+0000 7f4d7de76640 1 -- start start 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78101420 0x7f4d780fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78106a80 0x7f4d780fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78110ec0 con 0x7f4d78100a80 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78110d40 con 0x7f4d78106a80 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78111040 con 0x7f4d78101420 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46418/0 (socket says 172.21.15.16:46418) 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 -- 172.21.15.16:0/2910430536 learned_addr learned my addr 172.21.15.16:0/2910430536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d76ffd640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78101420 0x7f4d780fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:27.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d77fff640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78106a80 0x7f4d780fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 -- 172.21.15.16:0/2910430536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78101420 msgr2=0x7f4d780fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78101420 0x7f4d780fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 -- 172.21.15.16:0/2910430536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78106a80 msgr2=0x7f4d780fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78106a80 0x7f4d780fd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 -- 172.21.15.16:0/2910430536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d781a69b0 con 0x7f4d78100a80 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d777fe640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800d770 tx=0x7f4d6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68017070 con 0x7f4d78100a80 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d781a6ca0 con 0x7f4d78100a80 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d68005500 con 0x7f4d78100a80 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68004e00 con 0x7f4d78100a80 2024-09-12T01:03:27.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.045+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a7160 con 0x7f4d78100a80 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.046+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78100ee0 con 0x7f4d78100a80 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.048+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4d68004fa0 con 0x7f4d78100a80 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.048+0000 7f4d74ff9640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 0x7f4d600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.049+0000 7f4d76ffd640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 0x7f4d600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.049+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d680cf2e0 con 0x7f4d78100a80 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.049+0000 7f4d76ffd640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 0x7f4d600a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c000c00 tx=0x7f4d6c005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:27.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.052+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d6809bf00 con 0x7f4d78100a80 2024-09-12T01:03:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.266+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d780fe260 con 0x7f4d78100a80 2024-09-12T01:03:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.267+0000 7f4d74ff9640 1 -- 172.21.15.16:0/2910430536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f4d780fe260 con 0x7f4d78100a80 2024-09-12T01:03:27.267 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:27.267 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 msgr2=0x7f4d600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 0x7f4d600a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c000c00 tx=0x7f4d6c005a20 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 msgr2=0x7f4d780fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800d770 tx=0x7f4d6800dc30 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 shutdown_connections 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d600a2cf0 0x7f4d600a51b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d78106a80 0x7f4d780fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d78101420 0x7f4d780fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 --2- 172.21.15.16:0/2910430536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d78100a80 0x7f4d780fcd30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.269+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 >> 172.21.15.16:0/2910430536 conn(0x7f4d780fc7a0 msgr2=0x7f4d7806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.270+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 shutdown_connections 2024-09-12T01:03:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:27.270+0000 7f4d7de76640 1 -- 172.21.15.16:0/2910430536 wait complete. 2024-09-12T01:03:27.403 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:03:27.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:03:27.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:27 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2910430536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:27.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:27 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2910430536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:27.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:27 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2910430536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:28.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 4f7e6111 --release --await' 2024-09-12T01:03:28.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 4f7e6111 --release --await 2024-09-12T01:03:28.093 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 -- 172.21.15.16:0/3835817182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84810e3c0 msgr2=0x7ff848110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 --2- 172.21.15.16:0/3835817182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84810e3c0 0x7ff848110810 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ca0 tx=0x7ff84002a950 comp rx=0 tx=0).stop 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 -- 172.21.15.16:0/3835817182 shutdown_connections 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 --2- 172.21.15.16:0/3835817182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff84810e3c0 0x7ff848110810 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 --2- 172.21.15.16:0/3835817182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff848074f40 0x7ff84810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 --2- 172.21.15.16:0/3835817182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 0x7ff848074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 -- 172.21.15.16:0/3835817182 >> 172.21.15.16:0/3835817182 conn(0x7ff8480fd8c0 msgr2=0x7ff8480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 -- 172.21.15.16:0/3835817182 shutdown_connections 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.707+0000 7ff84fb35640 1 -- 172.21.15.16:0/3835817182 wait complete. 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 Processor -- start 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 -- start start 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 0x7ff84819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84810e3c0 0x7ff84819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff848114160 con 0x7ff848074f40 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113fe0 con 0x7ff84810e3c0 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84fb35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481142e0 con 0x7ff848074620 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84d0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84d0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50636/0 (socket says 172.21.15.16:50636) 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.708+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1425785526 learned_addr learned my addr 172.21.15.16:0/1425785526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 0x7ff84819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84810e3c0 0x7ff84819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1425785526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 msgr2=0x7ff84819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 0x7ff84819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1425785526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84810e3c0 msgr2=0x7ff84819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84810e3c0 0x7ff84819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1425785526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84819f5e0 con 0x7ff848074f40 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c890 tx=0x7ff83800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff83effd640 1 -- 172.21.15.16:0/1425785526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838004220 con 0x7ff848074f40 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff83effd640 1 -- 172.21.15.16:0/1425785526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8380043c0 con 0x7ff848074f40 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff83effd640 1 -- 172.21.15.16:0/1425785526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380049e0 con 0x7ff848074f40 2024-09-12T01:03:28.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84819f8d0 con 0x7ff848074f40 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.709+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a71b0 con 0x7ff848074f40 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.711+0000 7ff83effd640 1 -- 172.21.15.16:0/1425785526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff8380079b0 con 0x7ff848074f40 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 Processor -- start 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 -- start start 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd126ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd127d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bd11d5280 con 0x564bd1270480 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bd117cb10 con 0x564bd127ad40 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff851f0e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bd11d1240 con 0x564bd126fb60 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff84d8d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff84e0d2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd126ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff84d8d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50644/0 (socket says 172.21.15.16:50644) 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.712+0000 7ff84d8d1640 1 -- 172.21.15.16:0/1762007598 learned_addr learned my addr 172.21.15.16:0/1762007598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd127d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 -- 172.21.15.16:0/1762007598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 msgr2=0x564bd126ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd126ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 -- 172.21.15.16:0/1762007598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 msgr2=0x564bd127d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd127d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 -- 172.21.15.16:0/1762007598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd11891f0 con 0x564bd1270480 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff83effd640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 0x7ff8340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b760 tx=0x7ff84400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b11de3a97eb525 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 0x7ff8340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff83effd640 1 -- 172.21.15.16:0/1425785526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff8380cf3a0 con 0x7ff848074f40 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff833fff640 1 -- 172.21.15.16:0/1762007598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844003dd0 con 0x564bd1270480 2024-09-12T01:03:28.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff833fff640 1 -- 172.21.15.16:0/1762007598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff844015070 con 0x564bd1270480 2024-09-12T01:03:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.713+0000 7ff833fff640 1 -- 172.21.15.16:0/1762007598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844011440 con 0x564bd1270480 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.714+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 0x7ff8340a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff840005fe0 tx=0x7ff840005c50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.714+0000 7ff851f0e740 1 -- 172.21.15.16:0/1762007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 msgr2=0x564bd127a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.714+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b760 tx=0x7ff84400bc20 comp rx=0 tx=0).stop 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 -- 172.21.15.16:0/1762007598 shutdown_connections 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd127d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd127a800 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1762007598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd126ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 -- 172.21.15.16:0/1762007598 >> 172.21.15.16:0/1762007598 conn(0x564bd126c0f0 msgr2=0x564bd126c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 -- 172.21.15.16:0/1762007598 shutdown_connections 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 -- 172.21.15.16:0/1762007598 wait complete. 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 -- start start 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.715+0000 7ff851f0e740 1 Processor -- start 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd1270480 0x564bd1278150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd1278690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bd127d910 con 0x564bd126fb60 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bd118ff10 con 0x564bd127ad40 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bd127da90 con 0x564bd1270480 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff851f0e740 10 client.0 _ll_register_callbacks cb 0x564bd1158a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50650/0 (socket says 172.21.15.16:50650) 2024-09-12T01:03:28.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 -- 172.21.15.16:0/2616520018 learned_addr learned my addr 172.21.15.16:0/2616520018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd1278690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd1270480 0x564bd1278150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 -- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd1270480 msgr2=0x564bd1278150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd1270480 0x564bd1278150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 -- 172.21.15.16:0/2616520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 msgr2=0x564bd1278690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd1278690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.716+0000 7ff84d8d1640 1 -- 172.21.15.16:0/2616520018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd0f466a0 con 0x564bd126fb60 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b5e0 tx=0x7ff8440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720cad9a7481fb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844028060 con 0x564bd126fb60 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff844011440 con 0x564bd126fb60 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff844015070 con 0x564bd126fb60 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff844011640 con 0x564bd126fb60 2024-09-12T01:03:28.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff851f0e740 10 client.17571 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bd1289510 con 0x564bd126fb60 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff844011cc0 con 0x564bd126fb60 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.717+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff844022030 con 0x564bd126fb60 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 10 client.17571 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 20 client.17571 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd128a9d0 0x564bd128cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 4 client.17571 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff844015070 con 0x564bd128a9d0 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd128a9d0 0x564bd128cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd128a9d0 0x564bd128cdc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecbf3be44b05867 server_cookie=7f3b45e21be6e5ba in_seq=0 out_seq=0 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.718+0000 7ff8317fa640 10 client.17571 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.719+0000 7ff8317fa640 1 -- 172.21.15.16:0/2616520018 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff844015070 con 0x564bd128a9d0 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.719+0000 7ff8317fa640 10 client.17571 handle_command_reply: tid=0 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.719+0000 7ff851f0e740 1 client.17571 shutdown 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.719+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 msgr2=0x564bd1277c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.719+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff84400b5e0 tx=0x7ff8440048f0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 shutdown_connections 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd128a9d0 0x564bd128cdc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd127ad40 0x564bd1278690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd1270480 0x564bd1278150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 --2- 172.21.15.16:0/2616520018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd126fb60 0x564bd1277c10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff8317fa640 0 client.17571 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 >> 172.21.15.16:0/2616520018 conn(0x564bd126c0f0 msgr2=0x564bd1271dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 shutdown_connections 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 1 -- 172.21.15.16:0/2616520018 wait complete. 2024-09-12T01:03:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.720+0000 7ff851f0e740 20 client.17571 trim_cache size 0 max 16384 2024-09-12T01:03:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.739+0000 7ff851f0e740 1 Processor -- start 2024-09-12T01:03:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.739+0000 7ff851f0e740 1 -- start start 2024-09-12T01:03:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd126fb60 0x564bd1274e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd127ad40 0x564bd12763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481afb70 con 0x564bd1270480 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b16b0 con 0x564bd126fb60 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff851f0e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b0240 con 0x564bd127ad40 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50654/0 (socket says 172.21.15.16:50654) 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1925357200 learned_addr learned my addr 172.21.15.16:0/1925357200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd126fb60 0x564bd1274e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd127ad40 0x564bd12763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1925357200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd127ad40 msgr2=0x564bd12763b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd127ad40 0x564bd12763b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1925357200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd126fb60 msgr2=0x564bd1274e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd126fb60 0x564bd1274e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.740+0000 7ff84d0d0640 1 -- 172.21.15.16:0/1925357200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd1276160 con 0x564bd1270480 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.741+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ccba0 tx=0x7ff838086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62d0c4676510c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.741+0000 7ff8337fe640 1 -- 172.21.15.16:0/1925357200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83801d040 con 0x564bd1270480 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.741+0000 7ff8337fe640 1 -- 172.21.15.16:0/1925357200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83809b970 con 0x564bd1270480 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.741+0000 7ff8337fe640 1 -- 172.21.15.16:0/1925357200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380ccf00 con 0x564bd1270480 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- 172.21.15.16:0/1925357200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 msgr2=0x564bd12753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ccba0 tx=0x7ff838086040 comp rx=0 tx=0).stop 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- 172.21.15.16:0/1925357200 shutdown_connections 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd127ad40 0x564bd12763b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd1270480 0x564bd12753b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1925357200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd126fb60 0x564bd1274e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- 172.21.15.16:0/1925357200 >> 172.21.15.16:0/1925357200 conn(0x564bd126c0f0 msgr2=0x564bd1286500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- 172.21.15.16:0/1925357200 shutdown_connections 2024-09-12T01:03:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- 172.21.15.16:0/1925357200 wait complete. 2024-09-12T01:03:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 -- start start 2024-09-12T01:03:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.742+0000 7ff851f0e740 1 Processor -- start 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd128d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 0x564bd128df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b0fe0 con 0x564bd127ad40 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b0910 con 0x564bd1270480 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481adb70 con 0x564bd126fb60 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff851f0e740 10 client.0 _ll_register_callbacks cb 0x7ff840005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50662/0 (socket says 172.21.15.16:50662) 2024-09-12T01:03:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 -- 172.21.15.16:0/1864128589 learned_addr learned my addr 172.21.15.16:0/1864128589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd128d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 0x564bd128df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 -- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 msgr2=0x564bd128d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd128d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 -- 172.21.15.16:0/1864128589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 msgr2=0x564bd128df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.743+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 0x564bd128df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff84e0d2640 1 -- 172.21.15.16:0/1864128589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd12f4560 con 0x564bd127ad40 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd128d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff84d0d0640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 0x564bd128df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff84e0d2640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff840035000 tx=0x7ff8400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335ca96043760f61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff8317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840034040 con 0x564bd127ad40 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff840004710 con 0x564bd127ad40 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83809b970 con 0x564bd127ad40 2024-09-12T01:03:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff840061050 con 0x564bd127ad40 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff851f0e740 10 client.17577 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bd12f5860 con 0x564bd127ad40 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.744+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83809b970 con 0x564bd127ad40 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8400304d0 con 0x564bd127ad40 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 10 client.17577 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 20 client.17577 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd12f6030 0x564bd12f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 4 client.17577 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bd11e27f0 con 0x564bd12f6030 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd12f6030 0x564bd12f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff8317fa640 10 client.17577 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.745+0000 7ff84d8d1640 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd12f6030 0x564bd12f8450 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38c5efc3b316fda server_cookie=230a2215d05f81f4 in_seq=0 out_seq=0 2024-09-12T01:03:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff8317fa640 1 -- 172.21.15.16:0/1864128589 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bd11e27f0 con 0x564bd12f6030 2024-09-12T01:03:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff8317fa640 10 client.17577 handle_command_reply: tid=0 2024-09-12T01:03:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 client.17577 shutdown 2024-09-12T01:03:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 msgr2=0x564bd128e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff840035000 tx=0x7ff8400095c0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 shutdown_connections 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564bd12f6030 0x564bd12f8450 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bd127ad40 0x564bd128e440 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff8317fa640 0 client.17577 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bd1270480 0x564bd128df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 --2- 172.21.15.16:0/1864128589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564bd126fb60 0x564bd128d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.747+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 >> 172.21.15.16:0/1864128589 conn(0x564bd126c0f0 msgr2=0x564bd126aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 shutdown_connections 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff851f0e740 1 -- 172.21.15.16:0/1864128589 wait complete. 2024-09-12T01:03:28.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff851f0e740 20 client.17577 trim_cache size 0 max 16384 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "afe75efa-f5a6-4915-8be0-5d823d139741", 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 msgr2=0x7ff8340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 0x7ff8340a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff840005fe0 tx=0x7ff840005c50 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 msgr2=0x7ff84819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff83800c890 tx=0x7ff83800cd50 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 shutdown_connections 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff8340a2cf0 0x7ff8340a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84810e3c0 0x7ff84819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff848074f40 0x7ff84819ab70 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 --2- 172.21.15.16:0/1425785526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff848074620 0x7ff84819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.748+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 >> 172.21.15.16:0/1425785526 conn(0x7ff8480fd8c0 msgr2=0x7ff8480fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.749+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 shutdown_connections 2024-09-12T01:03:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:28.749+0000 7ff84fb35640 1 -- 172.21.15.16:0/1425785526 wait complete. 2024-09-12T01:03:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:28.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:28 smithi159 ceph-mon[35566]: pgmap v227: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:28.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:28 smithi016 ceph-mon[30972]: pgmap v227: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:28 smithi099 ceph-mon[35654]: pgmap v227: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:29.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:03:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:29 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:03:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:29 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:29 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:03:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:29 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:29.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:29 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:03:29.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:29 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:03:29.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:29 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:29.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:29 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:03:29.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:29 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:29.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:29 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:03:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:29 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:03:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:29 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:29 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:03:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:29 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:29 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:03:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 -- 172.21.15.16:0/626514687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 msgr2=0x7f4d241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/626514687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d241067e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a410 comp rx=0 tx=0).stop 2024-09-12T01:03:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 -- 172.21.15.16:0/626514687 shutdown_connections 2024-09-12T01:03:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/626514687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 0x7f4d2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/626514687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d241067e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/626514687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 0x7f4d241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 -- 172.21.15.16:0/626514687 >> 172.21.15.16:0/626514687 conn(0x7f4d240fd8c0 msgr2=0x7f4d240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 -- 172.21.15.16:0/626514687 shutdown_connections 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.162+0000 7f4d2c09a640 1 -- 172.21.15.16:0/626514687 wait complete. 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 Processor -- start 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 -- start start 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 0x7f4d241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 0x7f4d2419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24111f70 con 0x7f4d24102790 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24111df0 con 0x7f4d24101de0 2024-09-12T01:03:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2c09a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241120f0 con 0x7f4d24106f10 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50684/0 (socket says 172.21.15.16:50684) 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d29635640 1 -- 172.21.15.16:0/804237296 learned_addr learned my addr 172.21.15.16:0/804237296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d29e36640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 0x7f4d241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.163+0000 7f4d2a637640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 0x7f4d2419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 -- 172.21.15.16:0/804237296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 msgr2=0x7f4d2419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 0x7f4d2419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 -- 172.21.15.16:0/804237296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 msgr2=0x7f4d241984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 0x7f4d241984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 -- 172.21.15.16:0/804237296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2419d470 con 0x7f4d24102790 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d29635640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d22ffd640 1 -- 172.21.15.16:0/804237296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1803f030 con 0x7f4d24102790 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2419d700 con 0x7f4d24102790 2024-09-12T01:03:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d22ffd640 1 -- 172.21.15.16:0/804237296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18003ce0 con 0x7f4d24102790 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d22ffd640 1 -- 172.21.15.16:0/804237296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035070 con 0x7f4d24102790 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.164+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d241a5040 con 0x7f4d24102790 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.167+0000 7f4d22ffd640 1 -- 172.21.15.16:0/804237296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4d18042030 con 0x7f4d24102790 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.167+0000 7f4d2e473740 1 Processor -- start 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.167+0000 7f4d2e473740 1 -- start start 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 0x55dba374c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c980 0x55dba3756d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 0x55dba3759630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dba36b1620 con 0x55dba3757240 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dba3658e50 con 0x55dba374c980 2024-09-12T01:03:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2e473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dba36ad5e0 con 0x55dba374c060 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 0x55dba3759630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 0x55dba374c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d29e36640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c980 0x55dba3756d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 0x55dba3759630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50696/0 (socket says 172.21.15.16:50696) 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d29635640 1 -- 172.21.15.16:0/894551799 learned_addr learned my addr 172.21.15.16:0/894551799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d22ffd640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 -- 172.21.15.16:0/894551799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c980 msgr2=0x55dba3756d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c980 0x55dba3756d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 -- 172.21.15.16:0/894551799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 msgr2=0x55dba3759630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 0x55dba3759630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d2a637640 1 -- 172.21.15.16:0/894551799 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba3649d40 con 0x55dba374c060 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.168+0000 7f4d22ffd640 1 -- 172.21.15.16:0/804237296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d180efac0 con 0x7f4d24102790 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d2a637640 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 0x55dba374c440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005fd0 tx=0x7f4d1c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bd60ce314b12ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d2a637640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c01b000 tx=0x7f4d1c01d000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d0bfff640 1 -- 172.21.15.16:0/894551799 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c00b680 con 0x55dba374c060 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d0bfff640 1 -- 172.21.15.16:0/894551799 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c01d8e0 con 0x55dba374c060 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d0bfff640 1 -- 172.21.15.16:0/894551799 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c01c800 con 0x55dba374c060 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d2e473740 1 -- 172.21.15.16:0/894551799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 msgr2=0x55dba374c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.169+0000 7f4d2e473740 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 0x55dba374c440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005fd0 tx=0x7f4d1c0039c0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 -- 172.21.15.16:0/894551799 shutdown_connections 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba3757240 0x55dba3759630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c980 0x55dba3756d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- 172.21.15.16:0/894551799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c060 0x55dba374c440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 -- 172.21.15.16:0/894551799 >> 172.21.15.16:0/894551799 conn(0x55dba37485f0 msgr2=0x55dba37489d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 -- 172.21.15.16:0/894551799 shutdown_connections 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 -- 172.21.15.16:0/894551799 wait complete. 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 -- start start 2024-09-12T01:03:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 Processor -- start 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba37546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.170+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba3754c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dba3759e10 con 0x55dba374c060 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dba3639f10 con 0x55dba3757240 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dba3759f90 con 0x55dba374c980 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 10 client.0 _ll_register_callbacks cb 0x55dba365ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50698/0 (socket says 172.21.15.16:50698) 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 -- 172.21.15.16:0/4038151756 learned_addr learned my addr 172.21.15.16:0/4038151756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2a637640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba3754c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29635640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba37546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 -- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 msgr2=0x55dba37546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba37546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 -- 172.21.15.16:0/4038151756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 msgr2=0x55dba3754c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba3754c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 -- 172.21.15.16:0/4038151756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba33c0580 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d29e36640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d14003d10 tx=0x7f4d14004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53df073f5f59438 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004b20 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1c01d8e0 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d14004cc0 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d14011650 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.171+0000 7f4d2e473740 10 client.17589 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dba348a960 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1c01d8e0 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d1401bab0 con 0x55dba374c060 2024-09-12T01:03:30.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 10 client.17589 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 10 client.17589 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 20 client.17589 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba3766e90 0x55dba37692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 4 client.17589 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba36beb90 con 0x55dba3766e90 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.172+0000 7f4d29635640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba3766e90 0x55dba37692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d29635640 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba3766e90 0x55dba37692e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16de628468b2a0e9 server_cookie=228bbd137e2648df in_seq=0 out_seq=0 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d097fa640 10 client.17589 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d097fa640 1 -- 172.21.15.16:0/4038151756 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dba36beb90 con 0x55dba3766e90 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d097fa640 10 client.17589 handle_command_reply: tid=0 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d2e473740 1 client.17589 shutdown 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 msgr2=0x55dba37541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.173+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4d14003d10 tx=0x7f4d14004410 comp rx=0 tx=0).stop 2024-09-12T01:03:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 shutdown_connections 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba3766e90 0x55dba37692e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba3754c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba37546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d097fa640 0 client.17589 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 --2- 172.21.15.16:0/4038151756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba37541b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 >> 172.21.15.16:0/4038151756 conn(0x55dba37485f0 msgr2=0x55dba374e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 shutdown_connections 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 1 -- 172.21.15.16:0/4038151756 wait complete. 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.174+0000 7f4d2e473740 20 client.17589 trim_cache size 0 max 16384 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 Processor -- start 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 -- start start 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba374eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba376ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241ab690 con 0x55dba374c060 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241ad1d0 con 0x55dba3757240 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2e473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241abd60 con 0x55dba374c980 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50714/0 (socket says 172.21.15.16:50714) 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 -- 172.21.15.16:0/3527022664 learned_addr learned my addr 172.21.15.16:0/3527022664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29635640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba374eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 -- 172.21.15.16:0/3527022664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 msgr2=0x55dba374eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2a637640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba376ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba374eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 -- 172.21.15.16:0/3527022664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 msgr2=0x55dba376ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba376ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29e36640 1 -- 172.21.15.16:0/3527022664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba374d200 con 0x55dba374c060 2024-09-12T01:03:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d29635640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba374eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.191+0000 7f4d2a637640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba376ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d29e36640 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4d1401cfd0 tx=0x7f4d14009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2873a9aae543fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d0b7fe640 1 -- 172.21.15.16:0/3527022664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14028040 con 0x55dba374c060 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d0b7fe640 1 -- 172.21.15.16:0/3527022664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d14009890 con 0x55dba374c060 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d0b7fe640 1 -- 172.21.15.16:0/3527022664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140117f0 con 0x55dba374c060 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 -- 172.21.15.16:0/3527022664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 msgr2=0x55dba3747a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4d1401cfd0 tx=0x7f4d14009520 comp rx=0 tx=0).stop 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 -- 172.21.15.16:0/3527022664 shutdown_connections 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba3757240 0x55dba376ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba374c980 0x55dba374eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 --2- 172.21.15.16:0/3527022664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c060 0x55dba3747a10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 -- 172.21.15.16:0/3527022664 >> 172.21.15.16:0/3527022664 conn(0x55dba37485f0 msgr2=0x55dba3755a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 -- 172.21.15.16:0/3527022664 shutdown_connections 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.192+0000 7f4d2e473740 1 -- 172.21.15.16:0/3527022664 wait complete. 2024-09-12T01:03:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 -- start start 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 Processor -- start 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 0x55dba375f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba3757240 0x55dba375c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241acb00 con 0x55dba374c980 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241ac430 con 0x55dba374c060 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241a9670 con 0x55dba3757240 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d2e473740 10 client.0 _ll_register_callbacks cb 0x7f4d14003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d29635640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50720/0 (socket says 172.21.15.16:50720) 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.193+0000 7f4d29635640 1 -- 172.21.15.16:0/1905403167 learned_addr learned my addr 172.21.15.16:0/1905403167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29e36640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 0x55dba375f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d2a637640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba3757240 0x55dba375c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 -- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba3757240 msgr2=0x55dba375c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba3757240 0x55dba375c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 -- 172.21.15.16:0/1905403167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 msgr2=0x55dba375f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 0x55dba375f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 -- 172.21.15.16:0/1905403167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba375c7f0 con 0x55dba374c980 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29e36640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 0x55dba375f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d29635640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005ab0 tx=0x7f4d1802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc218f56ce23e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18049040 con 0x55dba374c980 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4d180c00b0 con 0x55dba374c980 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d14009890 con 0x55dba374c980 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d1803bcb0 con 0x55dba374c980 2024-09-12T01:03:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d2e473740 10 client.17595 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dba37d2510 con 0x55dba374c980 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.194+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d180d6020 con 0x55dba374c980 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d180b8760 con 0x55dba374c980 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 10 client.17595 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 10 client.17595 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 20 client.17595 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba37d2c70 0x55dba37d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 4 client.17595 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4f7e6111", "release": true, "await": true}] 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4f7e6111", "release": true, "await": true}) -- 0x55dba36b90a0 con 0x55dba37d2c70 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d29e36640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba37d2c70 0x55dba37d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d097fa640 10 client.17595 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.195+0000 7f4d29e36640 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba37d2c70 0x55dba37d5090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853ce8ed2eceefb8 server_cookie=1f3eec9dbb45a160 in_seq=0 out_seq=0 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.198+0000 7f4d097fa640 1 -- 172.21.15.16:0/1905403167 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dba36b90a0 con 0x55dba37d2c70 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.198+0000 7f4d097fa640 10 client.17595 handle_command_reply: tid=0 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.198+0000 7f4d2e473740 1 client.17595 shutdown 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 msgr2=0x55dba375f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005ab0 tx=0x7f4d1802a410 comp rx=0 tx=0).stop 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 shutdown_connections 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55dba37d2c70 0x55dba37d5090 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dba3757240 0x55dba375c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dba374c980 0x55dba375f720 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 --2- 172.21.15.16:0/1905403167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dba374c060 0x55dba375f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d097fa640 0 client.17595 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 >> 172.21.15.16:0/1905403167 conn(0x55dba37485f0 msgr2=0x55dba3762060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 shutdown_connections 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 1 -- 172.21.15.16:0/1905403167 wait complete. 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.199+0000 7f4d2e473740 20 client.17595 trim_cache size 0 max 16384 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 28, 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "4f7e6111": { 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 28, 2024-09-12T01:03:30.215 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 136.6, 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:03:30.216 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 msgr2=0x7f4d0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c01b000 tx=0x7f4d1c01d000 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 msgr2=0x7f4d24198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d180042f0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 shutdown_connections 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d24106f10 0x7f4d2419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d24102790 0x7f4d24198a00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 --2- 172.21.15.16:0/804237296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d24101de0 0x7f4d241984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 >> 172.21.15.16:0/804237296 conn(0x7f4d240fd8c0 msgr2=0x7f4d240fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 shutdown_connections 2024-09-12T01:03:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:30.200+0000 7f4d2c09a640 1 -- 172.21.15.16:0/804237296 wait complete. 2024-09-12T01:03:30.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:30 smithi159 ceph-mon[35566]: pgmap v228: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-12T01:03:30.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:30 smithi016 ceph-mon[30972]: pgmap v228: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-12T01:03:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:30 smithi099 ceph-mon[35654]: pgmap v228: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-12T01:03:31.029 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4f7e6111', total seconds elapsed: 21.3 2024-09-12T01:03:31.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.500+0000 7f49addac640 1 -- 172.21.15.16:0/2264488955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 msgr2=0x7f49a810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.500+0000 7f49addac640 1 --2- 172.21.15.16:0/2264488955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a810ec50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009cd0 tx=0x7f49a002a910 comp rx=0 tx=0).stop 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 -- 172.21.15.16:0/2264488955 shutdown_connections 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 --2- 172.21.15.16:0/2264488955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a810ec50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 --2- 172.21.15.16:0/2264488955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8103760 0x7f49a8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 --2- 172.21.15.16:0/2264488955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8102db0 0x7f49a8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 -- 172.21.15.16:0/2264488955 >> 172.21.15.16:0/2264488955 conn(0x7f49a80fc7d0 msgr2=0x7f49a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 -- 172.21.15.16:0/2264488955 shutdown_connections 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 -- 172.21.15.16:0/2264488955 wait complete. 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.501+0000 7f49addac640 1 Processor -- start 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 -- start start 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8102db0 0x7f49a81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8103760 0x7f49a8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112110 con 0x7f49a81082e0 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8111f90 con 0x7f49a8102db0 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49addac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112290 con 0x7f49a8103760 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50732/0 (socket says 172.21.15.16:50732) 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 -- 172.21.15.16:0/1967604335 learned_addr learned my addr 172.21.15.16:0/1967604335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a77fe640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8102db0 0x7f49a81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8103760 0x7f49a8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 -- 172.21.15.16:0/1967604335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8103760 msgr2=0x7f49a8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8103760 0x7f49a8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 -- 172.21.15.16:0/1967604335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8102db0 msgr2=0x7f49a81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8102db0 0x7f49a81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.502+0000 7f49a7fff640 1 -- 172.21.15.16:0/1967604335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a819d590 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49a7fff640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f49a0005cc0 tx=0x7f49a0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49a4ff9640 1 -- 172.21.15.16:0/1967604335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0031660 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a819d820 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49a4ff9640 1 -- 172.21.15.16:0/1967604335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49a0031800 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49a4ff9640 1 -- 172.21.15.16:0/1967604335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a002fb70 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.503+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a5160 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.505+0000 7f49a4ff9640 1 -- 172.21.15.16:0/1967604335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49a0004620 con 0x7f49a81082e0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 Processor -- start 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 -- start start 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bc3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bd0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab50b292e0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ab50ad0ce0 con 0x55ab50bce680 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49b0185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ab50b252a0 con 0x55ab50bc34a0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49a4ff9640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 0x7f49900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50740/0 (socket says 172.21.15.16:50740) 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.506+0000 7f49a77fe640 1 -- 172.21.15.16:0/698568412 learned_addr learned my addr 172.21.15.16:0/698568412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a7fff640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 0x7f49900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bc3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a4ff9640 1 -- 172.21.15.16:0/1967604335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f49a00f0720 con 0x7f49a81082e0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bd0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 -- 172.21.15.16:0/698568412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 msgr2=0x55ab50bc3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bc3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 -- 172.21.15.16:0/698568412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 msgr2=0x55ab50bd0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bd0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 -- 172.21.15.16:0/698568412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab50ab5440 con 0x55ab50bc3dc0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bd0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a77fe640 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f499800b760 tx=0x7f499800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bed5121e3585df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f49a7fff640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 0x7f49900a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49a00bc9f0 tx=0x7f49a002a910 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f498dffb640 1 -- 172.21.15.16:0/698568412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003dd0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f498dffb640 1 -- 172.21.15.16:0/698568412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4998015070 con 0x55ab50bc3dc0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.507+0000 7f498dffb640 1 -- 172.21.15.16:0/698568412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998004e10 con 0x55ab50bc3dc0 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 -- 172.21.15.16:0/698568412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 msgr2=0x55ab50bce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f499800b760 tx=0x7f499800bc20 comp rx=0 tx=0).stop 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 -- 172.21.15.16:0/698568412 shutdown_connections 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bd0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50bce140 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 --2- 172.21.15.16:0/698568412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bc3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 -- 172.21.15.16:0/698568412 >> 172.21.15.16:0/698568412 conn(0x55ab50bbfa30 msgr2=0x55ab50bbfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 -- 172.21.15.16:0/698568412 shutdown_connections 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.508+0000 7f49b0185740 1 -- 172.21.15.16:0/698568412 wait complete. 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 -- start start 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 Processor -- start 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc3dc0 0x55ab50bcb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bcb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab50bd1250 con 0x55ab50bc34a0 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ab50ae4050 con 0x55ab50bce680 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ab50bd13d0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49b0185740 10 client.0 _ll_register_callbacks cb 0x55ab50b24160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49a6ffd640 1 -- 172.21.15.16:0/3726518636 learned_addr learned my addr 172.21.15.16:0/3726518636 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc3dc0 0x55ab50bcb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.509+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bcb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 -- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc3dc0 msgr2=0x55ab50bcb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc3dc0 0x55ab50bcb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 -- 172.21.15.16:0/3726518636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 msgr2=0x55ab50bcb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bcb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 -- 172.21.15.16:0/3726518636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5089a3c0 con 0x55ab50bc34a0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f49a80697d0 tx=0x7f499c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5478523ffe740a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f4986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0146e0 con 0x55ab50bc34a0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f499c014880 con 0x55ab50bc34a0 2024-09-12T01:03:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4998015070 con 0x55ab50bc34a0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f499c026050 con 0x55ab50bc34a0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49b0185740 10 client.17607 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab50bdcbc0 con 0x55ab50bc34a0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.510+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4998015070 con 0x55ab50bc34a0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f499c015040 con 0x55ab50bc34a0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 10 client.17607 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 20 client.17607 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50bde080 0x55ab50be0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 4 client.17607 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab50b36850 con 0x55ab50bde080 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.511+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50bde080 0x55ab50be0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50bde080 0x55ab50be0470 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184f450f1740fdec server_cookie=b03a97f3ecb48a7d in_seq=0 out_seq=0 2024-09-12T01:03:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f4986ffd640 10 client.17607 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f4986ffd640 1 -- 172.21.15.16:0/3726518636 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab50b36850 con 0x55ab50bde080 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f4986ffd640 10 client.17607 handle_command_reply: tid=0 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f49b0185740 1 client.17607 shutdown 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.512+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 msgr2=0x55ab50bcdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f49a80697d0 tx=0x7f499c0047d0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 shutdown_connections 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50bde080 0x55ab50be0470 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bcb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc3dc0 0x55ab50bcb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 --2- 172.21.15.16:0/3726518636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc34a0 0x55ab50bcdf70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 >> 172.21.15.16:0/3726518636 conn(0x55ab50bbfa30 msgr2=0x55ab50bc5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f4986ffd640 0 client.17607 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 shutdown_connections 2024-09-12T01:03:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 1 -- 172.21.15.16:0/3726518636 wait complete. 2024-09-12T01:03:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.513+0000 7f49b0185740 20 client.17607 trim_cache size 0 max 16384 2024-09-12T01:03:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 Processor -- start 2024-09-12T01:03:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 -- start start 2024-09-12T01:03:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50be5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bc8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ad920 con 0x55ab50bc3dc0 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81af460 con 0x55ab50bce680 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49b0185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81adff0 con 0x55ab50bc34a0 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50768/0 (socket says 172.21.15.16:50768) 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.530+0000 7f49a77fe640 1 -- 172.21.15.16:0/3335711874 learned_addr learned my addr 172.21.15.16:0/3335711874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50be5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bc8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 -- 172.21.15.16:0/3335711874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 msgr2=0x55ab50be5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50be5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 -- 172.21.15.16:0/3335711874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 msgr2=0x55ab50bc8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bc8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 -- 172.21.15.16:0/3335711874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab50bca1d0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bc8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f49a77fe640 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4998004a20 tx=0x7f49980049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ccb7713931930 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f498d7fa640 1 -- 172.21.15.16:0/3335711874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003dd0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f498d7fa640 1 -- 172.21.15.16:0/3335711874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4998015040 con 0x55ab50bc3dc0 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.531+0000 7f498d7fa640 1 -- 172.21.15.16:0/3335711874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998019680 con 0x55ab50bc3dc0 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- 172.21.15.16:0/3335711874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 msgr2=0x55ab50be6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4998004a20 tx=0x7f49980049b0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- 172.21.15.16:0/3335711874 shutdown_connections 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bce680 0x55ab50bc8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bc3dc0 0x55ab50be6280 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 --2- 172.21.15.16:0/3335711874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50be5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- 172.21.15.16:0/3335711874 >> 172.21.15.16:0/3335711874 conn(0x55ab50bbfa30 msgr2=0x55ab50bbeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- 172.21.15.16:0/3335711874 shutdown_connections 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- 172.21.15.16:0/3335711874 wait complete. 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.532+0000 7f49b0185740 1 -- start start 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 Processor -- start 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bd42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 0x55ab50bd4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81aed90 con 0x55ab50bce680 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ae6c0 con 0x55ab50bc3dc0 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49a81ab920 con 0x55ab50bc34a0 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49b0185740 10 client.0 _ll_register_callbacks cb 0x7f499c00aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50774/0 (socket says 172.21.15.16:50774) 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 -- 172.21.15.16:0/1605275935 learned_addr learned my addr 172.21.15.16:0/1605275935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a77fe640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 0x55ab50bd4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 -- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 msgr2=0x55ab50bd42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bd42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 -- 172.21.15.16:0/1605275935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 msgr2=0x55ab50bd4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 0x55ab50bd4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a6ffd640 1 -- 172.21.15.16:0/1605275935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab50c47d20 con 0x55ab50bce680 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.533+0000 7f49a77fe640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 0x55ab50bd4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49a6ffd640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f499c00a6f0 tx=0x7f499c00a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff130bc47aa470f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00ab90 con 0x55ab50bce680 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4998015040 con 0x55ab50bce680 2024-09-12T01:03:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f499c018630 con 0x55ab50bce680 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f499c03b050 con 0x55ab50bce680 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49b0185740 10 client.17613 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab50c48ee0 con 0x55ab50bce680 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4998015040 con 0x55ab50bce680 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f499c0146e0 con 0x55ab50bce680 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49b0185740 10 client.17613 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.534+0000 7f49b0185740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49b0185740 20 client.17613 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50c495b0 0x55ab50c4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49b0185740 4 client.17613 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab50b30d60 con 0x55ab50c495b0 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49a77fe640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50c495b0 0x55ab50c4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f49a77fe640 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50c495b0 0x55ab50c4b9a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9c73658846fef3 server_cookie=673644ee9af7aa8e in_seq=0 out_seq=0 2024-09-12T01:03:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.535+0000 7f4986ffd640 10 client.17613 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f4986ffd640 1 -- 172.21.15.16:0/1605275935 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ab50b30d60 con 0x55ab50c495b0 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f4986ffd640 10 client.17613 handle_command_reply: tid=0 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 client.17613 shutdown 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 msgr2=0x55ab50bd4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f499c00a6f0 tx=0x7f499c00a0d0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 shutdown_connections 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ab50c495b0 0x55ab50c4b9a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab50bce680 0x55ab50bd4d60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ab50bc3dc0 0x55ab50bd4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 --2- 172.21.15.16:0/1605275935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab50bc34a0 0x55ab50bd42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 >> 172.21.15.16:0/1605275935 conn(0x55ab50bbfa30 msgr2=0x55ab50bcca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f4986ffd640 0 client.17613 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 shutdown_connections 2024-09-12T01:03:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 1 -- 172.21.15.16:0/1605275935 wait complete. 2024-09-12T01:03:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.536+0000 7f49b0185740 20 client.17613 trim_cache size 0 max 16384 2024-09-12T01:03:31.548 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:31.548 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:03:31.548 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:03:31.548 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 msgr2=0x7f49900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 0x7f49900a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49a00bc9f0 tx=0x7f49a002a910 comp rx=0 tx=0).stop 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 msgr2=0x7f49a819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f49a0005cc0 tx=0x7f49a0003f90 comp rx=0 tx=0).stop 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 shutdown_connections 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49900a2d70 0x7f49900a5230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49a81082e0 0x7f49a819ceb0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49a8103760 0x7f49a8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 --2- 172.21.15.16:0/1967604335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49a8102db0 0x7f49a81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 >> 172.21.15.16:0/1967604335 conn(0x7f49a80fc7d0 msgr2=0x7f49a8105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 shutdown_connections 2024-09-12T01:03:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:31.537+0000 7f49addac640 1 -- 172.21.15.16:0/1967604335 wait complete. 2024-09-12T01:03:32.332 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:03:32.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:32.600 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:32.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:32 smithi016 ceph-mon[30972]: pgmap v229: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:32 smithi159 ceph-mon[35566]: pgmap v229: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:32 smithi099 ceph-mon[35654]: pgmap v229: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:33.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 -- 172.21.15.16:0/2479896234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 msgr2=0x7f7c281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/2479896234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c281021c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009900 tx=0x7f7c1802a410 comp rx=0 tx=0).stop 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 -- 172.21.15.16:0/2479896234 shutdown_connections 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/2479896234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28106f10 0x7f7c2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/2479896234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28102790 0x7f7c281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/2479896234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c281021c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 -- 172.21.15.16:0/2479896234 >> 172.21.15.16:0/2479896234 conn(0x7f7c280fd8c0 msgr2=0x7f7c280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 -- 172.21.15.16:0/2479896234 shutdown_connections 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.057+0000 7f7c2d51e640 1 -- 172.21.15.16:0/2479896234 wait complete. 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 Processor -- start 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 -- start start 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28102790 0x7f7c281988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 0x7f7c2819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28111f30 con 0x7f7c28101de0 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c28111db0 con 0x7f7c28102790 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c2d51e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c281120b0 con 0x7f7c28106f10 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50786/0 (socket says 172.21.15.16:50786) 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.058+0000 7f7c26ffd640 1 -- 172.21.15.16:0/1267361042 learned_addr learned my addr 172.21.15.16:0/1267361042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c267fc640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28102790 0x7f7c281988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c277fe640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 0x7f7c2819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 -- 172.21.15.16:0/1267361042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 msgr2=0x7f7c2819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 0x7f7c2819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 -- 172.21.15.16:0/1267361042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28102790 msgr2=0x7f7c281988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28102790 0x7f7c281988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 -- 172.21.15.16:0/1267361042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2819d340 con 0x7f7c28101de0 2024-09-12T01:03:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c277fe640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 0x7f7c2819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c26ffd640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1803f030 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2819d5d0 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c18003ce0 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c281a4f10 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.059+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18035070 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.060+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c281021c0 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.061+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c18042030 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.062+0000 7f7c0ffff640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 0x7f7c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.062+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f7c180ef8a0 con 0x7f7c28101de0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.062+0000 7f7c267fc640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 0x7f7c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.063+0000 7f7c267fc640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 0x7f7c100a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c281088e0 tx=0x7f7c1c00a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.066+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c180bc4c0 con 0x7f7c28101de0 2024-09-12T01:03:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.279+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c2810a0f0 con 0x7f7c28101de0 2024-09-12T01:03:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.280+0000 7f7c0ffff640 1 -- 172.21.15.16:0/1267361042 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f7c180bbdc0 con 0x7f7c28101de0 2024-09-12T01:03:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:33.280 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 msgr2=0x7f7c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 0x7f7c100a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c281088e0 tx=0x7f7c1c00a2d0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 msgr2=0x7f7c28198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c180042f0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 shutdown_connections 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c100a2cf0 0x7f7c100a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c28106f10 0x7f7c2819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c28102790 0x7f7c281988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 --2- 172.21.15.16:0/1267361042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c28101de0 0x7f7c28198390 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 >> 172.21.15.16:0/1267361042 conn(0x7f7c280fd8c0 msgr2=0x7f7c280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:33.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 shutdown_connections 2024-09-12T01:03:33.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:33.282+0000 7f7c2d51e640 1 -- 172.21.15.16:0/1267361042 wait complete. 2024-09-12T01:03:33.843 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:33 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1267361042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:33.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:33 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1267361042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:33.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:03:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:33 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1267361042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:34.608 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:34 smithi016 ceph-mon[30972]: pgmap v230: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 -- 172.21.15.16:0/1247055061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 msgr2=0x7f20c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 --2- 172.21.15.16:0/1247055061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81021c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009900 tx=0x7f20c402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 -- 172.21.15.16:0/1247055061 shutdown_connections 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 --2- 172.21.15.16:0/1247055061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8106f10 0x7f20c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 --2- 172.21.15.16:0/1247055061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8102790 0x7f20c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 --2- 172.21.15.16:0/1247055061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81021c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 -- 172.21.15.16:0/1247055061 >> 172.21.15.16:0/1247055061 conn(0x7f20c80fd8c0 msgr2=0x7f20c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 -- 172.21.15.16:0/1247055061 shutdown_connections 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.692+0000 7f20cf609640 1 -- 172.21.15.16:0/1247055061 wait complete. 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 Processor -- start 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 -- start start 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 0x7f20c8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 0x7f20c819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8112010 con 0x7f20c8101de0 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8111e90 con 0x7f20c8106f10 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cf609640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8112190 con 0x7f20c8102790 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cd3a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cd3a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50812/0 (socket says 172.21.15.16:50812) 2024-09-12T01:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20cd3a5640 1 -- 172.21.15.16:0/466147693 learned_addr learned my addr 172.21.15.16:0/466147693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.693+0000 7f20ccba4640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 0x7f20c8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cdba6640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 0x7f20c819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 -- 172.21.15.16:0/466147693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 msgr2=0x7f20c8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 0x7f20c8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 -- 172.21.15.16:0/466147693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 msgr2=0x7f20c819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 0x7f20c819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 -- 172.21.15.16:0/466147693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c819d560 con 0x7f20c8101de0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20ccba4640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 0x7f20c8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cdba6640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 0x7f20c819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005ab0 tx=0x7f20c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20be7fc640 1 -- 172.21.15.16:0/466147693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c403f030 con 0x7f20c8101de0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20be7fc640 1 -- 172.21.15.16:0/466147693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4003ce0 con 0x7f20c8101de0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c819d7f0 con 0x7f20c8101de0 2024-09-12T01:03:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20be7fc640 1 -- 172.21.15.16:0/466147693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4035070 con 0x7f20c8101de0 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.694+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a5130 con 0x7f20c8101de0 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.696+0000 7f20be7fc640 1 -- 172.21.15.16:0/466147693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20c4042030 con 0x7f20c8101de0 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 Processor -- start 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 -- start start 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df32f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df32f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559df325b2f0 con 0x559df3304780 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559df3202cf0 con 0x559df32f55d0 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20d19e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559df32572b0 con 0x559df32f6b40 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.697+0000 7f20be7fc640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 0x7f20b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50824/0 (socket says 172.21.15.16:50824) 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 -- 172.21.15.16:0/2432086243 learned_addr learned my addr 172.21.15.16:0/2432086243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20cdba6640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 0x7f20b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20cdba6640 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df32f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20be7fc640 1 -- 172.21.15.16:0/466147693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20c40efe50 con 0x7f20c8101de0 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df32f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 -- 172.21.15.16:0/2432086243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 msgr2=0x559df32f5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df32f5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 -- 172.21.15.16:0/2432086243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 msgr2=0x559df32f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df32f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 -- 172.21.15.16:0/2432086243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df31e73a0 con 0x559df3304780 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20cdba6640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 0x7f20b40a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009440 tx=0x7f20c0009540 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20ccba4640 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f20b8007800 tx=0x7f20b8007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d077a5277eca54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20b37fe640 1 -- 172.21.15.16:0/2432086243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b800d950 con 0x559df3304780 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20b37fe640 1 -- 172.21.15.16:0/2432086243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b8018070 con 0x559df3304780 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.698+0000 7f20b37fe640 1 -- 172.21.15.16:0/2432086243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b80123e0 con 0x559df3304780 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 -- 172.21.15.16:0/2432086243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 msgr2=0x559df3306b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f20b8007800 tx=0x7f20b8007cc0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 -- 172.21.15.16:0/2432086243 shutdown_connections 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df3306b70 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df32f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 --2- 172.21.15.16:0/2432086243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df32f5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 -- 172.21.15.16:0/2432086243 >> 172.21.15.16:0/2432086243 conn(0x559df32be310 msgr2=0x559df32f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 -- 172.21.15.16:0/2432086243 shutdown_connections 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.699+0000 7f20d19e2740 1 -- 172.21.15.16:0/2432086243 wait complete. 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 -- start start 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 Processor -- start 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3304070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3301370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559df3307480 con 0x559df3304780 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559df3216060 con 0x559df32f55d0 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559df3307600 con 0x559df32f6b40 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20d19e2740 10 client.0 _ll_register_callbacks cb 0x559df2f698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50836/0 (socket says 172.21.15.16:50836) 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20cdba6640 1 -- 172.21.15.16:0/1962892492 learned_addr learned my addr 172.21.15.16:0/1962892492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3304070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.700+0000 7f20ccba4640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3301370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 -- 172.21.15.16:0/1962892492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 msgr2=0x559df3301370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3301370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 -- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 msgr2=0x559df3304070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3304070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 -- 172.21.15.16:0/1962892492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df2fcc3c0 con 0x559df3304780 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20cdba6640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f20c0016e50 tx=0x7f20c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93baa12f34aa3de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0029060 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b8018070 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20c001ce00 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20c0013040 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20d19e2740 10 client.17631 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559df3312e40 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.701+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20b8018070 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20c0024700 con 0x559df3304780 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 10 client.17631 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 20 client.17631 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df3314300 0x559df3316750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 4 client.17631 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559df3268860 con 0x559df3314300 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df3314300 0x559df3316750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20b0ff9640 10 client.17631 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.702+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df3314300 0x559df3316750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5549bf75644bd68c server_cookie=b073d98649d248c in_seq=0 out_seq=0 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20b0ff9640 1 -- 172.21.15.16:0/1962892492 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559df3268860 con 0x559df3314300 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20b0ff9640 10 client.17631 handle_command_reply: tid=0 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 client.17631 shutdown 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 msgr2=0x559df33018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f20c0016e50 tx=0x7f20c0004a40 comp rx=0 tx=0).stop 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 shutdown_connections 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df3314300 0x559df3316750 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33018b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3301370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 --2- 172.21.15.16:0/1962892492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3304070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 >> 172.21.15.16:0/1962892492 conn(0x559df32be310 msgr2=0x559df32f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20b0ff9640 0 client.17631 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.703+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 shutdown_connections 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.704+0000 7f20d19e2740 1 -- 172.21.15.16:0/1962892492 wait complete. 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.704+0000 7f20d19e2740 20 client.17631 trim_cache size 0 max 16384 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 Processor -- start 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 -- start start 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f55d0 0x559df3310720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 0x559df3316310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ab7b0 con 0x559df3304780 2024-09-12T01:03:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ad2f0 con 0x559df32f6b40 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20d19e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81abe80 con 0x559df32f55d0 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 -- 172.21.15.16:0/3355731890 learned_addr learned my addr 172.21.15.16:0/3355731890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f55d0 0x559df3310720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20ccba4640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 0x559df3316310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 -- 172.21.15.16:0/3355731890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f55d0 msgr2=0x559df3310720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f55d0 0x559df3310720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 -- 172.21.15.16:0/3355731890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 msgr2=0x559df3316310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 0x559df3316310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20cdba6640 1 -- 172.21.15.16:0/3355731890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df32f2960 con 0x559df3304780 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.717+0000 7f20ccba4640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 0x559df3316310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20cdba6640 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f20c0016e20 tx=0x7f20c0009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31a4ac699dfe55f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20b2ffd640 1 -- 172.21.15.16:0/3355731890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00243e0 con 0x559df3304780 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20b2ffd640 1 -- 172.21.15.16:0/3355731890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c001a660 con 0x559df3304780 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20b2ffd640 1 -- 172.21.15.16:0/3355731890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0036690 con 0x559df3304780 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 -- 172.21.15.16:0/3355731890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 msgr2=0x559df331c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f20c0016e20 tx=0x7f20c0009dc0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 -- 172.21.15.16:0/3355731890 shutdown_connections 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df331c020 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f6b40 0x559df3316310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 --2- 172.21.15.16:0/3355731890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f55d0 0x559df3310720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 -- 172.21.15.16:0/3355731890 >> 172.21.15.16:0/3355731890 conn(0x559df32be310 msgr2=0x559df32ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 -- 172.21.15.16:0/3355731890 shutdown_connections 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.718+0000 7f20d19e2740 1 -- 172.21.15.16:0/3355731890 wait complete. 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.719+0000 7f20d19e2740 1 -- start start 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.719+0000 7f20d19e2740 1 Processor -- start 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.719+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3308620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.719+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3308b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20d19e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20d19e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81acc20 con 0x559df3304780 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20d19e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81ac550 con 0x559df32f55d0 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20d19e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81a97b0 con 0x559df32f6b40 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20d19e2740 10 client.0 _ll_register_callbacks cb 0x7f20c0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50848/0 (socket says 172.21.15.16:50848) 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 -- 172.21.15.16:0/784096308 learned_addr learned my addr 172.21.15.16:0/784096308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20ccba4640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3308b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3308620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 -- 172.21.15.16:0/784096308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 msgr2=0x559df3308b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3308b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 -- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 msgr2=0x559df3308620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3308620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 -- 172.21.15.16:0/784096308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df337e040 con 0x559df3304780 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3308620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20cdba6640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f20c0024b20 tx=0x7f20c0024690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab05b90f5d70e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.720+0000 7f20b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c003b070 con 0x559df3304780 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f20c001a810 con 0x559df3304780 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c001a660 con 0x559df3304780 2024-09-12T01:03:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20c0047050 con 0x559df3304780 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 10 client.17637 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559df337f300 con 0x559df3304780 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c001a660 con 0x559df3304780 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20c0036bd0 con 0x559df3304780 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 10 client.17637 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 10 client.17637 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 20 client.17637 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df337f910 0x559df3381d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 4 client.17637 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.721+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559df3316ae0 con 0x559df337f910 2024-09-12T01:03:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.722+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df337f910 0x559df3381d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:34.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.722+0000 7f20b0ff9640 10 client.17637 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.722+0000 7f20cd3a5640 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df337f910 0x559df3381d00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50230b779900cfd7 server_cookie=673a441f27b54017 in_seq=0 out_seq=0 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.722+0000 7f20b0ff9640 1 -- 172.21.15.16:0/784096308 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559df3316ae0 con 0x559df337f910 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.722+0000 7f20b0ff9640 10 client.17637 handle_command_reply: tid=0 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 client.17637 shutdown 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 msgr2=0x559df33090a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f20c0024b20 tx=0x7f20c0024690 comp rx=0 tx=0).stop 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 shutdown_connections 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x559df337f910 0x559df3381d00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559df3304780 0x559df33090a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20b0ff9640 0 client.17637 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559df32f6b40 0x559df3308b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 --2- 172.21.15.16:0/784096308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559df32f55d0 0x559df3308620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 >> 172.21.15.16:0/784096308 conn(0x559df32be310 msgr2=0x559df3302bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 shutdown_connections 2024-09-12T01:03:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 1 -- 172.21.15.16:0/784096308 wait complete. 2024-09-12T01:03:34.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.723+0000 7f20d19e2740 20 client.17637 trim_cache size 0 max 16384 2024-09-12T01:03:34.739 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 msgr2=0x7f20b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 0x7f20b40a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009440 tx=0x7f20c0009540 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 msgr2=0x7f20c81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005ab0 tx=0x7f20c40042f0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 shutdown_connections 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f20b40a2cf0 0x7f20b40a51b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20c8106f10 0x7f20c819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c8102790 0x7f20c8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 --2- 172.21.15.16:0/466147693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c8101de0 0x7f20c81985b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:34.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 >> 172.21.15.16:0/466147693 conn(0x7f20c80fd8c0 msgr2=0x7f20c80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 shutdown_connections 2024-09-12T01:03:34.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:34.724+0000 7f20cf609640 1 -- 172.21.15.16:0/466147693 wait complete. 2024-09-12T01:03:34.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:34 smithi159 ceph-mon[35566]: pgmap v230: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:34 smithi099 ceph-mon[35654]: pgmap v230: 97 pgs: 97 active+clean; 8.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:03:35.234 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.234 INFO:tasks.workunit.client.0.smithi016.stdout:189/600: open fd = 3 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:190/600: open fd = 3 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:191/600: open fd = 3 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:192/600: open fd = 3 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.235 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:193/600: open fd = 3 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:194/600: open fd = 3 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.236 INFO:tasks.workunit.client.0.smithi016.stdout:195/600: open fd = 3 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:196/600: open fd = 3 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.237 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:197/600: open fd = 3 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:198/600: open fd = 3 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.238 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:199/600: open fd = 3 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:200/600: open fd = 3 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:201/600: open fd = 3 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:202/600: open fd = 3 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.239 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:203/600: open fd = 3 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:204/600: open fd = 3 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:205/600: open fd = 3 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.240 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:206/600: open fd = 3 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:207/600: open fd = 3 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:208/600: open fd = 3 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:209/600: open fd = 3 2024-09-12T01:03:35.241 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:210/600: open fd = 3 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:211/600: open fd = 3 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:212/600: open fd = 3 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.242 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:213/600: open fd = 3 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:214/600: open fd = 3 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:215/600: open fd = 3 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.243 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:216/600: open fd = 3 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:217/600: open fd = 3 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:218/600: open fd = 3 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.244 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:219/600: open fd = 3 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:220/600: open fd = 3 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:221/600: open fd = 3 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.245 INFO:tasks.workunit.client.0.smithi016.stdout:222/600: open fd = 3 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:223/600: open fd = 3 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:224/600: open fd = 3 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.246 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:225/600: open fd = 3 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:226/600: open fd = 3 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.247 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:227/600: open fd = 3 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:228/600: open fd = 3 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:229/600: open fd = 3 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.248 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:230/600: open fd = 3 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:231/600: open fd = 3 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.249 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:232/600: open fd = 3 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:233/600: open fd = 3 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:234/600: open fd = 3 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.250 INFO:tasks.workunit.client.0.smithi016.stdout:235/600: open fd = 3 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:236/600: open fd = 3 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:237/600: open fd = 3 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:238/600: open fd = 3 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.251 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:239/600: open fd = 3 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:240/600: open fd = 3 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:241/600: open fd = 3 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.252 INFO:tasks.workunit.client.0.smithi016.stdout:242/600: open fd = 3 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:243/600: open fd = 3 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:244/600: open fd = 3 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:245/600: open fd = 3 2024-09-12T01:03:35.253 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:246/600: open fd = 3 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:247/600: open fd = 3 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:248/600: open fd = 3 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:249/600: open fd = 3 2024-09-12T01:03:35.254 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:03:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:250/600: open fd = 3 2024-09-12T01:03:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.255 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:03:35.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:03:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 -- 172.21.15.16:0/2502678138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098102390 msgr2=0x7fe09810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 --2- 172.21.15.16:0/2502678138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098102390 0x7fe09810e9d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe090009930 tx=0x7fe09002a420 comp rx=0 tx=0).stop 2024-09-12T01:03:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 -- 172.21.15.16:0/2502678138 shutdown_connections 2024-09-12T01:03:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 --2- 172.21.15.16:0/2502678138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098102390 0x7fe09810e9d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 --2- 172.21.15.16:0/2502678138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0981019f0 0x7fe098101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 --2- 172.21.15.16:0/2502678138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe098107b00 0x7fe098107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 -- 172.21.15.16:0/2502678138 >> 172.21.15.16:0/2502678138 conn(0x7fe0980fd710 msgr2=0x7fe0980ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 -- 172.21.15.16:0/2502678138 shutdown_connections 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.277+0000 7fe09d801640 1 -- 172.21.15.16:0/2502678138 wait complete. 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 Processor -- start 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 -- start start 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0981019f0 0x7fe0981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe098102390 0x7fe0981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111f00 con 0x7fe098107b00 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111d80 con 0x7fe0981019f0 2024-09-12T01:03:36.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe09d801640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe098112080 con 0x7fe098102390 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe0977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe0977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50864/0 (socket says 172.21.15.16:50864) 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe0977fe640 1 -- 172.21.15.16:0/1030525869 learned_addr learned my addr 172.21.15.16:0/1030525869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.278+0000 7fe0967fc640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe098102390 0x7fe0981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe096ffd640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0981019f0 0x7fe0981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 -- 172.21.15.16:0/1030525869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe098102390 msgr2=0x7fe0981988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe098102390 0x7fe0981988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 -- 172.21.15.16:0/1030525869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0981019f0 msgr2=0x7fe0981983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0981019f0 0x7fe0981983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 -- 172.21.15.16:0/1030525869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819d350 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe0977fe640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe090005ab0 tx=0x7fe0900040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe07ffff640 1 -- 172.21.15.16:0/1030525869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09003f030 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe07ffff640 1 -- 172.21.15.16:0/1030525869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0900044c0 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09819d5e0 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe07ffff640 1 -- 172.21.15.16:0/1030525869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090035070 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.279+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a4f20 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.281+0000 7fe07ffff640 1 -- 172.21.15.16:0/1030525869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe090042030 con 0x7fe098107b00 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 Processor -- start 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 -- start start 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515948270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563515957960 0x563515959d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5635158ae2f0 con 0x5635159487b0 2024-09-12T01:03:36.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563515855cf0 con 0x563515957960 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe09fbda740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635158aa2b0 con 0x563515949d20 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50876/0 (socket says 172.21.15.16:50876) 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 -- 172.21.15.16:0/3931445601 learned_addr learned my addr 172.21.15.16:0/3931445601 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe07ffff640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 0x7fe0800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515948270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563515957960 0x563515959d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 -- 172.21.15.16:0/3931445601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 msgr2=0x563515948270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe0977fe640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 0x7fe0800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515948270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 -- 172.21.15.16:0/3931445601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563515957960 msgr2=0x563515959d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563515957960 0x563515959d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.282+0000 7fe096ffd640 1 -- 172.21.15.16:0/3931445601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351583a3a0 con 0x5635159487b0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515948270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe07ffff640 1 -- 172.21.15.16:0/1030525869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe0900ef980 con 0x7fe098107b00 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe088009fa0 tx=0x7fe088010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7223a6893cc3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe07d7fa640 1 -- 172.21.15.16:0/3931445601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088003b70 con 0x5635159487b0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe0977fe640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 0x7fe0800a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b82a0 tx=0x7fe0900b12d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe07d7fa640 1 -- 172.21.15.16:0/3931445601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe088003d10 con 0x5635159487b0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.283+0000 7fe07d7fa640 1 -- 172.21.15.16:0/3931445601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088011ac0 con 0x5635159487b0 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- 172.21.15.16:0/3931445601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 msgr2=0x563515948bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe088009fa0 tx=0x7fe088010860 comp rx=0 tx=0).stop 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- 172.21.15.16:0/3931445601 shutdown_connections 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563515957960 0x563515959d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5635159487b0 0x563515948bf0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3931445601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515948270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- 172.21.15.16:0/3931445601 >> 172.21.15.16:0/3931445601 conn(0x563515911310 msgr2=0x563515944e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- 172.21.15.16:0/3931445601 shutdown_connections 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- 172.21.15.16:0/3931445601 wait complete. 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 -- start start 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 Processor -- start 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x5635159572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.284+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515957960 0x563515954b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56351595a660 con 0x563515949d20 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563515869060 con 0x5635159487b0 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56351595a7e0 con 0x563515957960 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 10 client.0 _ll_register_callbacks cb 0x5635155bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50882/0 (socket says 172.21.15.16:50882) 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 -- 172.21.15.16:0/3750824567 learned_addr learned my addr 172.21.15.16:0/3750824567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515957960 0x563515954b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x5635159572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 -- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515957960 msgr2=0x563515954b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515957960 0x563515954b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 -- 172.21.15.16:0/3750824567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 msgr2=0x5635159572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x5635159572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 -- 172.21.15.16:0/3750824567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56351561f3c0 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x5635159572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe08800a010 tx=0x7fe0880115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258b0e60028df18f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088027020 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe088003ab0 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe088003d10 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe088032050 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 10 client.17649 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.285+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563515966020 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe088003d10 con 0x563515949d20 2024-09-12T01:03:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0880040e0 con 0x563515949d20 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 10 client.17649 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 10 client.17649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 20 client.17649 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x563515967420 0x563515969810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 4 client.17649 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635158bb860 con 0x563515967420 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.286+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x563515967420 0x563515969810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.287+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x563515967420 0x563515969810 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=526975fac9dce6b0 server_cookie=b4e5e6caef915e6d in_seq=0 out_seq=0 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.287+0000 7fe07aff5640 10 client.17649 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:36.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.287+0000 7fe07aff5640 1 -- 172.21.15.16:0/3750824567 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635158bb860 con 0x563515967420 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.287+0000 7fe07aff5640 10 client.17649 handle_command_reply: tid=0 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.287+0000 7fe09fbda740 1 client.17649 shutdown 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 msgr2=0x5635159545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe08800a010 tx=0x7fe0880115a0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 shutdown_connections 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515957960 0x563515954b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x563515967420 0x563515969810 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515949d20 0x5635159545c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3750824567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x5635159572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe07aff5640 0 client.17649 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 >> 172.21.15.16:0/3750824567 conn(0x563515911310 msgr2=0x56351594ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 shutdown_connections 2024-09-12T01:03:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 1 -- 172.21.15.16:0/3750824567 wait complete. 2024-09-12T01:03:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.288+0000 7fe09fbda740 20 client.17649 trim_cache size 0 max 16384 2024-09-12T01:03:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 Processor -- start 2024-09-12T01:03:36.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 -- start start 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515963900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515953e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ab500 con 0x563515957960 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ad040 con 0x5635159487b0 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe09fbda740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981abbd0 con 0x563515949d20 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe0967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe0967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50888/0 (socket says 172.21.15.16:50888) 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe0967fc640 1 -- 172.21.15.16:0/2910690414 learned_addr learned my addr 172.21.15.16:0/2910690414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe096ffd640 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515953e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.301+0000 7fe0967fc640 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515963900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 -- 172.21.15.16:0/2910690414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 msgr2=0x563515953e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515953e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 -- 172.21.15.16:0/2910690414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 msgr2=0x563515963900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515963900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 -- 172.21.15.16:0/2910690414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563515957630 con 0x563515957960 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe0967fc640 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe08c000c00 tx=0x7fe08c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0531bacbf6e6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe07cff9640 1 -- 172.21.15.16:0/2910690414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004230 con 0x563515957960 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe07cff9640 1 -- 172.21.15.16:0/2910690414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0043d0 con 0x563515957960 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.302+0000 7fe07cff9640 1 -- 172.21.15.16:0/2910690414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004d00 con 0x563515957960 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- 172.21.15.16:0/2910690414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 msgr2=0x563515958d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe08c000c00 tx=0x7fe08c014040 comp rx=0 tx=0).stop 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- 172.21.15.16:0/2910690414 shutdown_connections 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x563515958d50 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515953e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- 172.21.15.16:0/2910690414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515963900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- 172.21.15.16:0/2910690414 >> 172.21.15.16:0/2910690414 conn(0x563515911310 msgr2=0x56351594f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- 172.21.15.16:0/2910690414 shutdown_connections 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- 172.21.15.16:0/2910690414 wait complete. 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- start start 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 Processor -- start 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515958740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515958c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ac970 con 0x563515957960 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ac2a0 con 0x5635159487b0 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981a9520 con 0x563515949d20 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.303+0000 7fe09fbda740 10 client.0 _ll_register_callbacks cb 0x7fe088009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:36.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50904/0 (socket says 172.21.15.16:50904) 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 -- 172.21.15.16:0/3945688875 learned_addr learned my addr 172.21.15.16:0/3945688875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe096ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515958c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515958740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 -- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 msgr2=0x563515958c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515958c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 -- 172.21.15.16:0/3945688875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 msgr2=0x563515958740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515958740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 -- 172.21.15.16:0/3945688875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563515954ed0 con 0x563515957960 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515958c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe0967fc640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e780 tx=0x7fe08c0048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d180f9578efca46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:36.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe07aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c02d070 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe08c03b430 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe08c0043d0 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe08c034040 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe09fbda740 10 client.17655 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.304+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563515960040 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe08c0043d0 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe08c03d6f0 con 0x563515957960 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 10 client.17655 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:36.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 10 client.17655 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 20 client.17655 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5635159d1ed0 0x5635159d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 4 client.17655 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635158b5d70 con 0x5635159d1ed0 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.305+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5635159d1ed0 0x5635159d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe096ffd640 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5635159d1ed0 0x5635159d42c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f09c53df0c1f2467 server_cookie=b49cbb8569464b8b in_seq=0 out_seq=0 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe07aff5640 10 client.17655 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe07aff5640 1 -- 172.21.15.16:0/3945688875 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635158b5d70 con 0x5635159d1ed0 2024-09-12T01:03:36.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe07aff5640 10 client.17655 handle_command_reply: tid=0 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 client.17655 shutdown 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 msgr2=0x56351595ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00e780 tx=0x7fe08c0048e0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 shutdown_connections 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5635159d1ed0 0x5635159d42c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563515957960 0x56351595ea70 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563515949d20 0x563515958c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 --2- 172.21.15.16:0/3945688875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5635159487b0 0x563515958740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 >> 172.21.15.16:0/3945688875 conn(0x563515911310 msgr2=0x5635159536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe07aff5640 0 client.17655 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:36.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 shutdown_connections 2024-09-12T01:03:36.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 1 -- 172.21.15.16:0/3945688875 wait complete. 2024-09-12T01:03:36.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.306+0000 7fe09fbda740 20 client.17655 trim_cache size 0 max 16384 2024-09-12T01:03:36.316 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 msgr2=0x7fe0800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 0x7fe0800a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b82a0 tx=0x7fe0900b12d0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 msgr2=0x7fe09819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe090005ab0 tx=0x7fe0900040c0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 shutdown_connections 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe0800a2cf0 0x7fe0800a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe098107b00 0x7fe09819cc70 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe098102390 0x7fe0981988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 --2- 172.21.15.16:0/1030525869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0981019f0 0x7fe0981983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:36.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 >> 172.21.15.16:0/1030525869 conn(0x7fe0980fd710 msgr2=0x7fe0980fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 shutdown_connections 2024-09-12T01:03:36.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:36.307+0000 7fe09d801640 1 -- 172.21.15.16:0/1030525869 wait complete. 2024-09-12T01:03:36.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:36 smithi159 ceph-mon[35566]: pgmap v231: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:03:36.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:36 smithi016 ceph-mon[30972]: pgmap v231: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:03:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:36 smithi099 ceph-mon[35654]: pgmap v231: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-12T01:03:37.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:03:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:37.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- 172.21.15.16:0/2964746781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc101280 msgr2=0x7f28dc10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 --2- 172.21.15.16:0/2964746781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc101280 0x7f28dc10df40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009900 tx=0x7f28cc02a410 comp rx=0 tx=0).stop 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- 172.21.15.16:0/2964746781 shutdown_connections 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 --2- 172.21.15.16:0/2964746781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc10e480 0x7f28dc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 --2- 172.21.15.16:0/2964746781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc101280 0x7f28dc10df40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 --2- 172.21.15.16:0/2964746781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc1008d0 0x7f28dc100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- 172.21.15.16:0/2964746781 >> 172.21.15.16:0/2964746781 conn(0x7f28dc0fc7a0 msgr2=0x7f28dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- 172.21.15.16:0/2964746781 shutdown_connections 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- 172.21.15.16:0/2964746781 wait complete. 2024-09-12T01:03:37.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 Processor -- start 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.928+0000 7f28e41e2640 1 -- start start 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc1008d0 0x7f28dc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc101280 0x7f28dc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1141b0 con 0x7f28dc10e480 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc114030 con 0x7f28dc1008d0 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc114330 con 0x7f28dc101280 2024-09-12T01:03:37.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50914/0 (socket says 172.21.15.16:50914) 2024-09-12T01:03:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e1f7e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc1008d0 0x7f28dc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 -- 172.21.15.16:0/275177260 learned_addr learned my addr 172.21.15.16:0/275177260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e177d640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc101280 0x7f28dc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 -- 172.21.15.16:0/275177260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc101280 msgr2=0x7f28dc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc101280 0x7f28dc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 -- 172.21.15.16:0/275177260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc1008d0 msgr2=0x7f28dc19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc1008d0 0x7f28dc19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 -- 172.21.15.16:0/275177260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc19f660 con 0x7f28dc10e480 2024-09-12T01:03:37.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e277f640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009fd0 tx=0x7f28d400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28d2ffd640 1 -- 172.21.15.16:0/275177260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4004d20 con 0x7f28dc10e480 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28d2ffd640 1 -- 172.21.15.16:0/275177260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d4005020 con 0x7f28dc10e480 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.929+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc19f950 con 0x7f28dc10e480 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.930+0000 7f28d2ffd640 1 -- 172.21.15.16:0/275177260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40116d0 con 0x7f28dc10e480 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.930+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc1a7230 con 0x7f28dc10e480 2024-09-12T01:03:37.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28d2ffd640 1 -- 172.21.15.16:0/275177260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f28d40051d0 con 0x7f28dc10e480 2024-09-12T01:03:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 Processor -- start 2024-09-12T01:03:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 -- start start 2024-09-12T01:03:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557245a0d2f0 con 0x557245ab2370 2024-09-12T01:03:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5572459b4cf0 con 0x557245aa8b40 2024-09-12T01:03:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.932+0000 7f28e65bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557245a092b0 con 0x557245aa75d0 2024-09-12T01:03:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28d2ffd640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 0x7f28c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50930/0 (socket says 172.21.15.16:50930) 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e177d640 1 -- 172.21.15.16:0/1815565198 learned_addr learned my addr 172.21.15.16:0/1815565198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e277f640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 0x7f28c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e177d640 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28d2ffd640 1 -- 172.21.15.16:0/275177260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f28d40cf880 con 0x7f28dc10e480 2024-09-12T01:03:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1815565198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 msgr2=0x557245aa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1815565198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 msgr2=0x557245ab4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1815565198 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572459993a0 con 0x557245aa75d0 2024-09-12T01:03:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e177d640 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa7a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004630 tx=0x7f28d800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1889b1d60dce44ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.933+0000 7f28e277f640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 0x7f28c80a5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28d40982b0 tx=0x7f28d40980f0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28c7fff640 1 -- 172.21.15.16:0/1815565198 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8003bd0 con 0x557245aa75d0 2024-09-12T01:03:37.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28c7fff640 1 -- 172.21.15.16:0/1815565198 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x557245aa75d0 2024-09-12T01:03:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28c7fff640 1 -- 172.21.15.16:0/1815565198 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004d20 con 0x557245aa75d0 2024-09-12T01:03:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 -- 172.21.15.16:0/1815565198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 msgr2=0x557245aa7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa7a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004630 tx=0x7f28d800bb20 comp rx=0 tx=0).stop 2024-09-12T01:03:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 -- 172.21.15.16:0/1815565198 shutdown_connections 2024-09-12T01:03:37.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa7a10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1815565198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 -- 172.21.15.16:0/1815565198 >> 172.21.15.16:0/1815565198 conn(0x557245a70310 msgr2=0x557245aa3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 -- 172.21.15.16:0/1815565198 shutdown_connections 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.934+0000 7f28e65bb740 1 -- 172.21.15.16:0/1815565198 wait complete. 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 -- start start 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 Processor -- start 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aaf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245ab2370 0x557245aaf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557245ab5070 con 0x557245aa75d0 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5572459c8060 con 0x557245aa8b40 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.935+0000 7f28e65bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557245ab51f0 con 0x557245ab2370 2024-09-12T01:03:37.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e65bb740 10 client.0 _ll_register_callbacks cb 0x55724571b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50936/0 (socket says 172.21.15.16:50936) 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 -- 172.21.15.16:0/2435382464 learned_addr learned my addr 172.21.15.16:0/2435382464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aaf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 -- 172.21.15.16:0/2435382464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245ab2370 msgr2=0x557245aaf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245ab2370 0x557245aaf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 -- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 msgr2=0x557245aaf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aaf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 -- 172.21.15.16:0/2435382464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724577e3c0 con 0x557245aa75d0 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aaf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e177d640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009230 tx=0x7f28cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce711f358fc9340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc035070 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f28cc004300 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d8015070 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28cc0035f0 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e65bb740 10 client.17667 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.936+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557245ac0b90 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28cc0537e0 con 0x557245aa75d0 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 10 client.17667 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 10 client.17667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 20 client.17667 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245ac1fa0 0x557245ac4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 4 client.17667 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557245a1a860 con 0x557245ac1fa0 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.937+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245ac1fa0 0x557245ac4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.938+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245ac1fa0 0x557245ac4390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b313d82ceb5e48e3 server_cookie=440923ad57e72364 in_seq=0 out_seq=0 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.938+0000 7f28c57fa640 10 client.17667 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.938+0000 7f28c57fa640 1 -- 172.21.15.16:0/2435382464 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557245a1a860 con 0x557245ac1fa0 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.938+0000 7f28c57fa640 10 client.17667 handle_command_reply: tid=0 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 client.17667 shutdown 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 msgr2=0x557245ab1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009230 tx=0x7f28cc005850 comp rx=0 tx=0).stop 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 shutdown_connections 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245ac1fa0 0x557245ac4390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245ab2370 0x557245aaf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245aaf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28c57fa640 0 client.17667 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 --2- 172.21.15.16:0/2435382464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa75d0 0x557245ab1d80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 >> 172.21.15.16:0/2435382464 conn(0x557245a70310 msgr2=0x557245aa9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 shutdown_connections 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 1 -- 172.21.15.16:0/2435382464 wait complete. 2024-09-12T01:03:37.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.939+0000 7f28e65bb740 20 client.17667 trim_cache size 0 max 16384 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 Processor -- start 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 -- start start 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245ab2370 0x557245aaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1afa40 con 0x557245aa8b40 2024-09-12T01:03:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b1580 con 0x557245ab2370 2024-09-12T01:03:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.952+0000 7f28e65bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b0110 con 0x557245aa75d0 2024-09-12T01:03:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50944/0 (socket says 172.21.15.16:50944) 2024-09-12T01:03:37.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1766951031 learned_addr learned my addr 172.21.15.16:0/1766951031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e177d640 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1766951031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 msgr2=0x557245aa2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1766951031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245ab2370 msgr2=0x557245aaa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245ab2370 0x557245aaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 -- 172.21.15.16:0/1766951031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557245ac49c0 con 0x557245aa8b40 2024-09-12T01:03:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004580 tx=0x7f28d8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b0fe5a03524c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28c77fe640 1 -- 172.21.15.16:0/1766951031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8003bd0 con 0x557245aa8b40 2024-09-12T01:03:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28c77fe640 1 -- 172.21.15.16:0/1766951031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x557245aa8b40 2024-09-12T01:03:37.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.953+0000 7f28c77fe640 1 -- 172.21.15.16:0/1766951031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8027430 con 0x557245aa8b40 2024-09-12T01:03:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- 172.21.15.16:0/1766951031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 msgr2=0x557245aa3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f28d8004580 tx=0x7f28d8013040 comp rx=0 tx=0).stop 2024-09-12T01:03:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- 172.21.15.16:0/1766951031 shutdown_connections 2024-09-12T01:03:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245ab2370 0x557245aaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245aa8b40 0x557245aa3130 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1766951031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aa2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- 172.21.15.16:0/1766951031 >> 172.21.15.16:0/1766951031 conn(0x557245a70310 msgr2=0x557245aa24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- 172.21.15.16:0/1766951031 shutdown_connections 2024-09-12T01:03:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- 172.21.15.16:0/1766951031 wait complete. 2024-09-12T01:03:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.954+0000 7f28e65bb740 1 -- start start 2024-09-12T01:03:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 Processor -- start 2024-09-12T01:03:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245abad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b0eb0 con 0x557245ab2370 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1b07e0 con 0x557245aa8b40 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc1ada40 con 0x557245aa75d0 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e65bb740 10 client.0 _ll_register_callbacks cb 0x7f28cc005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50950/0 (socket says 172.21.15.16:50950) 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 -- 172.21.15.16:0/1128605574 learned_addr learned my addr 172.21.15.16:0/1128605574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245abad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 -- 172.21.15.16:0/1128605574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 msgr2=0x557245aba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 -- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 msgr2=0x557245abad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245abad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e177d640 1 -- 172.21.15.16:0/1128605574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557245ab65e0 con 0x557245ab2370 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.955+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245abad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e177d640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f28cc053d50 tx=0x7f28cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88eec06b417cb7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc035040 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f28cc004310 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d8015070 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28cc065050 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 10 client.17673 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557245b2c850 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f28cc003bd0 con 0x557245ab2370 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 10 client.17673 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 10 client.17673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 20 client.17673 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245b2cf40 0x557245b2f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 4 client.17673 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.956+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557245a14d70 con 0x557245b2cf40 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.957+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245b2cf40 0x557245b2f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.957+0000 7f28e1f7e640 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245b2cf40 0x557245b2f360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9608aabbcde1f3c9 server_cookie=e53a4f04194da06c in_seq=0 out_seq=0 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.957+0000 7f28c57fa640 10 client.17673 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28c57fa640 1 -- 172.21.15.16:0/1128605574 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557245a14d70 con 0x557245b2cf40 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28c57fa640 10 client.17673 handle_command_reply: tid=0 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 client.17673 shutdown 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 msgr2=0x557245ab5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f28cc053d50 tx=0x7f28cc000bc0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 shutdown_connections 2024-09-12T01:03:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557245b2cf40 0x557245b2f360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557245ab2370 0x557245ab5db0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557245aa8b40 0x557245abad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 --2- 172.21.15.16:0/1128605574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557245aa75d0 0x557245aba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 >> 172.21.15.16:0/1128605574 conn(0x557245a70310 msgr2=0x557245aa2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28c57fa640 0 client.17673 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 shutdown_connections 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 1 -- 172.21.15.16:0/1128605574 wait complete. 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.958+0000 7f28e65bb740 20 client.17673 trim_cache size 0 max 16384 2024-09-12T01:03:37.986 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 msgr2=0x7f28c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 0x7f28c80a5230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f28d40982b0 tx=0x7f28d40980f0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 msgr2=0x7f28dc19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009fd0 tx=0x7f28d400ee60 comp rx=0 tx=0).stop 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 shutdown_connections 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f28c80a2d70 0x7f28c80a5230 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc10e480 0x7f28dc19ef80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28dc101280 0x7f28dc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 --2- 172.21.15.16:0/275177260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28dc1008d0 0x7f28dc19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 >> 172.21.15.16:0/275177260 conn(0x7f28dc0fc7a0 msgr2=0x7f28dc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 shutdown_connections 2024-09-12T01:03:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:37.959+0000 7f28e41e2640 1 -- 172.21.15.16:0/275177260 wait complete. 2024-09-12T01:03:38.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:03:38.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:38 smithi159 ceph-mon[35566]: pgmap v232: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:03:38.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:38 smithi016 ceph-mon[30972]: pgmap v232: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:03:38.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:38 smithi099 ceph-mon[35654]: pgmap v232: 97 pgs: 97 active+clean; 8.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 -- 172.21.15.16:0/2097308098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e41036e0 msgr2=0x7f27e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 --2- 172.21.15.16:0/2097308098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e41036e0 0x7f27e4107b90 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009930 tx=0x7f27cc02a420 comp rx=0 tx=0).stop 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 -- 172.21.15.16:0/2097308098 shutdown_connections 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 --2- 172.21.15.16:0/2097308098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41082a0 0x7f27e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 --2- 172.21.15.16:0/2097308098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e41036e0 0x7f27e4107b90 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 --2- 172.21.15.16:0/2097308098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e4102d30 0x7f27e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 -- 172.21.15.16:0/2097308098 >> 172.21.15.16:0/2097308098 conn(0x7f27e40fc7d0 msgr2=0x7f27e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 -- 172.21.15.16:0/2097308098 shutdown_connections 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.394+0000 7f27ea09d640 1 -- 172.21.15.16:0/2097308098 wait complete. 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 Processor -- start 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 -- start start 2024-09-12T01:03:39.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41036e0 0x7f27e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e41082a0 0x7f27e419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41120b0 con 0x7f27e4102d30 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4111f30 con 0x7f27e41036e0 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27ea09d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112230 con 0x7f27e41082a0 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37564/0 (socket says 172.21.15.16:37564) 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 -- 172.21.15.16:0/1856985710 learned_addr learned my addr 172.21.15.16:0/1856985710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e3fff640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e41082a0 0x7f27e419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27dbfff640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41036e0 0x7f27e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 -- 172.21.15.16:0/1856985710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e41082a0 msgr2=0x7f27e419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e41082a0 0x7f27e419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 -- 172.21.15.16:0/1856985710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41036e0 msgr2=0x7f27e4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41036e0 0x7f27e4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.395+0000 7f27e37fe640 1 -- 172.21.15.16:0/1856985710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e419d560 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27e37fe640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c970 tx=0x7f27d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27e17fa640 1 -- 172.21.15.16:0/1856985710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4007bb0 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27e17fa640 1 -- 172.21.15.16:0/1856985710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d4007d50 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e419d850 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27e17fa640 1 -- 172.21.15.16:0/1856985710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d400f450 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.396+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a5130 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.398+0000 7f27e17fa640 1 -- 172.21.15.16:0/1856985710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f27d4005020 con 0x7f27e4102d30 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.398+0000 7f27ec476740 1 Processor -- start 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.398+0000 7f27ec476740 1 -- start start 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 0x558ba5c49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ba5ba22f0 con 0x558ba5c3db40 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ba5b49cf0 con 0x558ba5c47370 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27ec476740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ba5b9e2b0 con 0x558ba5c3c5d0 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37580/0 (socket says 172.21.15.16:37580) 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e3fff640 1 -- 172.21.15.16:0/30224878 learned_addr learned my addr 172.21.15.16:0/30224878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e17fa640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 0x7f27d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e37fe640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27dbfff640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 0x558ba5c49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e3fff640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 0x7f27d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e3fff640 1 -- 172.21.15.16:0/30224878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 msgr2=0x558ba5c3ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.399+0000 7f27e17fa640 1 -- 172.21.15.16:0/1856985710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f27d40cf1e0 con 0x7f27e4102d30 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e3fff640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c3ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e3fff640 1 -- 172.21.15.16:0/30224878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 msgr2=0x558ba5c49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e3fff640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 0x558ba5c49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e3fff640 1 -- 172.21.15.16:0/30224878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba5b2e3a0 con 0x558ba5c3db40 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27dbfff640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 0x558ba5c49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e37fe640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27e3fff640 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005f40 tx=0x7f27dc014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343916934f0eba01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27d9ffb640 1 -- 172.21.15.16:0/30224878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc01ee50 con 0x558ba5c3db40 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27d9ffb640 1 -- 172.21.15.16:0/30224878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc023070 con 0x558ba5c3db40 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.400+0000 7f27d9ffb640 1 -- 172.21.15.16:0/30224878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011ac0 con 0x558ba5c3db40 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- 172.21.15.16:0/30224878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 msgr2=0x558ba5c3c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27e3fff640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 0x7f27d00a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f27dc022000 tx=0x7f27dc020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005f40 tx=0x7f27dc014f10 comp rx=0 tx=0).stop 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- 172.21.15.16:0/30224878 shutdown_connections 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c47370 0x558ba5c49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 --2- 172.21.15.16:0/30224878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3db40 0x558ba5c3c090 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- 172.21.15.16:0/30224878 >> 172.21.15.16:0/30224878 conn(0x558ba5c05310 msgr2=0x558ba5c38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- 172.21.15.16:0/30224878 shutdown_connections 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- 172.21.15.16:0/30224878 wait complete. 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 -- start start 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.401+0000 7f27ec476740 1 Processor -- start 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 0x558ba5c46d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3db40 0x558ba5c43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ba5c4a070 con 0x558ba5c47370 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ba5b5d060 con 0x558ba5c3c5d0 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ba5c4a1f0 con 0x558ba5c3db40 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27ec476740 10 client.0 _ll_register_callbacks cb 0x558ba58b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37584/0 (socket says 172.21.15.16:37584) 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 -- 172.21.15.16:0/3987074799 learned_addr learned my addr 172.21.15.16:0/3987074799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 0x558ba5c46d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27dbfff640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3db40 0x558ba5c43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 -- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3db40 msgr2=0x558ba5c43fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3db40 0x558ba5c43fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 -- 172.21.15.16:0/3987074799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 msgr2=0x558ba5c46d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 0x558ba5c46d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 -- 172.21.15.16:0/3987074799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba59133c0 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 0x558ba5c46d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009cc0 tx=0x7f27dc0146f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb9ed328822c98f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.402+0000 7f27c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc011de0 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f27dc003970 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc023070 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27dc003b70 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 10 client.17685 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ba5c55a30 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc023070 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc00f040 con 0x558ba5c47370 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 10 client.17685 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 20 client.17685 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5c56e40 0x558ba5c59230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 4 client.17685 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.403+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ba5baf860 con 0x558ba5c56e40 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.404+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5c56e40 0x558ba5c59230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.404+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5c56e40 0x558ba5c59230 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43c0340af031b6a0 server_cookie=8f3c44f4beb76aa4 in_seq=0 out_seq=0 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.404+0000 7f27c2ffd640 10 client.17685 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27c2ffd640 1 -- 172.21.15.16:0/3987074799 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ba5baf860 con 0x558ba5c56e40 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27c2ffd640 10 client.17685 handle_command_reply: tid=0 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 client.17685 shutdown 2024-09-12T01:03:39.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 msgr2=0x558ba5c44570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009cc0 tx=0x7f27dc0146f0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 shutdown_connections 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c44570 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3db40 0x558ba5c43fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5c56e40 0x558ba5c59230 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 --2- 172.21.15.16:0/3987074799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3c5d0 0x558ba5c46d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27c2ffd640 0 client.17685 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 >> 172.21.15.16:0/3987074799 conn(0x558ba5c05310 msgr2=0x558ba5c3e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 shutdown_connections 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 1 -- 172.21.15.16:0/3987074799 wait complete. 2024-09-12T01:03:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.405+0000 7f27ec476740 20 client.17685 trim_cache size 0 max 16384 2024-09-12T01:03:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.419+0000 7f27ec476740 1 Processor -- start 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.419+0000 7f27ec476740 1 -- start start 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.419+0000 7f27ec476740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27ec476740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c38700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27ec476740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 0x558ba5c42fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27ec476740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41ad920 con 0x558ba5c3c5d0 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27ec476740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41af460 con 0x558ba5c3db40 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27ec476740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41adff0 con 0x558ba5c47370 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37596/0 (socket says 172.21.15.16:37596) 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 -- 172.21.15.16:0/3966995822 learned_addr learned my addr 172.21.15.16:0/3966995822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 0x558ba5c42fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 -- 172.21.15.16:0/3966995822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 msgr2=0x558ba5c42fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27dbfff640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c38700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 0x558ba5c42fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 -- 172.21.15.16:0/3966995822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 msgr2=0x558ba5c38700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c38700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 -- 172.21.15.16:0/3966995822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba5c49ec0 con 0x558ba5c3c5d0 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e3fff640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 0x558ba5c42fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27dbfff640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c38700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27e37fe640 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f27d409b6e0 tx=0x7f27d406e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab4f30370aabf36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27d97fa640 1 -- 172.21.15.16:0/3966995822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d401d040 con 0x558ba5c3c5d0 2024-09-12T01:03:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27d97fa640 1 -- 172.21.15.16:0/3966995822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d40a4310 con 0x558ba5c3c5d0 2024-09-12T01:03:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.420+0000 7f27d97fa640 1 -- 172.21.15.16:0/3966995822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4005310 con 0x558ba5c3c5d0 2024-09-12T01:03:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 -- 172.21.15.16:0/3966995822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 msgr2=0x558ba5c53310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f27d409b6e0 tx=0x7f27d406e7c0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 -- 172.21.15.16:0/3966995822 shutdown_connections 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c47370 0x558ba5c42fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c38700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 --2- 172.21.15.16:0/3966995822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c3c5d0 0x558ba5c53310 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 -- 172.21.15.16:0/3966995822 >> 172.21.15.16:0/3966995822 conn(0x558ba5c05310 msgr2=0x558ba5c428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 -- 172.21.15.16:0/3966995822 shutdown_connections 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.421+0000 7f27ec476740 1 -- 172.21.15.16:0/3966995822 wait complete. 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 -- start start 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 Processor -- start 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c4fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c4ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41aed90 con 0x558ba5c47370 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41ae6c0 con 0x558ba5c3db40 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41ab920 con 0x558ba5c3c5d0 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27ec476740 10 client.0 _ll_register_callbacks cb 0x7f27dc014880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37610/0 (socket says 172.21.15.16:37610) 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27e3fff640 1 -- 172.21.15.16:0/346597027 learned_addr learned my addr 172.21.15.16:0/346597027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27e37fe640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c4fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.422+0000 7f27dbfff640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c4ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 -- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 msgr2=0x558ba5c4fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c4fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 -- 172.21.15.16:0/346597027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 msgr2=0x558ba5c4ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c4ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 -- 172.21.15.16:0/346597027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba5c4b8b0 con 0x558ba5c47370 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27e3fff640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b2b2851935273 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc03f070 con 0x558ba5c47370 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f27dc003970 con 0x558ba5c47370 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d40a4310 con 0x558ba5c47370 2024-09-12T01:03:39.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27dc048050 con 0x558ba5c47370 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27ec476740 10 client.17691 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ba5cc59b0 con 0x558ba5c47370 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.423+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d40a4310 con 0x558ba5c47370 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc042020 con 0x558ba5c47370 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 10 client.17691 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 10 client.17691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 20 client.17691 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5cc6010 0x558ba5cc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 4 client.17691 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ba5ba9d70 con 0x558ba5cc6010 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27e37fe640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5cc6010 0x558ba5cc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:39.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27c2ffd640 10 client.17691 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.424+0000 7f27e37fe640 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5cc6010 0x558ba5cc8430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5d244b287f1ea61 server_cookie=31aff4fee73c04fa in_seq=0 out_seq=0 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27c2ffd640 1 -- 172.21.15.16:0/346597027 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ba5ba9d70 con 0x558ba5cc6010 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27c2ffd640 10 client.17691 handle_command_reply: tid=0 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 client.17691 shutdown 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 msgr2=0x558ba5c4b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc009ea0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 shutdown_connections 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ba5c47370 0x558ba5c4b270 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x558ba5cc6010 0x558ba5cc8430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ba5c3db40 0x558ba5c4ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 --2- 172.21.15.16:0/346597027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ba5c3c5d0 0x558ba5c4fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27c2ffd640 0 client.17691 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 >> 172.21.15.16:0/346597027 conn(0x558ba5c05310 msgr2=0x558ba5c430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 shutdown_connections 2024-09-12T01:03:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 1 -- 172.21.15.16:0/346597027 wait complete. 2024-09-12T01:03:39.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.425+0000 7f27ec476740 20 client.17691 trim_cache size 0 max 16384 2024-09-12T01:03:39.434 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 msgr2=0x7f27d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 0x7f27d00a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f27dc022000 tx=0x7f27dc020000 comp rx=0 tx=0).stop 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 msgr2=0x7f27e41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f27d400c970 tx=0x7f27d400ce30 comp rx=0 tx=0).stop 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 shutdown_connections 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f27d00a2cf0 0x7f27d00a51b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27e41082a0 0x7f27e419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27e41036e0 0x7f27e4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 --2- 172.21.15.16:0/1856985710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4102d30 0x7f27e41985b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 >> 172.21.15.16:0/1856985710 conn(0x7f27e40fc7d0 msgr2=0x7f27e4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 shutdown_connections 2024-09-12T01:03:39.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:39.426+0000 7f27ea09d640 1 -- 172.21.15.16:0/1856985710 wait complete. 2024-09-12T01:03:40.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:03:40.477 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:40.783 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:40 smithi016 ceph-mon[30972]: pgmap v233: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-12T01:03:40.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:40 smithi159 ceph-mon[35566]: pgmap v233: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 -- 172.21.15.16:0/2187532099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c102d30 msgr2=0x7fbc6c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 --2- 172.21.15.16:0/2187532099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c102d30 0x7fbc6c103110 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009930 tx=0x7fbc5c02a420 comp rx=0 tx=0).stop 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 -- 172.21.15.16:0/2187532099 shutdown_connections 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 --2- 172.21.15.16:0/2187532099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 0x7fbc6c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 --2- 172.21.15.16:0/2187532099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 --2- 172.21.15.16:0/2187532099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c102d30 0x7fbc6c103110 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 -- 172.21.15.16:0/2187532099 >> 172.21.15.16:0/2187532099 conn(0x7fbc6c0fc7d0 msgr2=0x7fbc6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 -- 172.21.15.16:0/2187532099 shutdown_connections 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 -- 172.21.15.16:0/2187532099 wait complete. 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.978+0000 7fbc71961640 1 Processor -- start 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 -- start start 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c102d30 0x7fbc6c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 0x7fbc6c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c112100 con 0x7fbc6c1036e0 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c111f80 con 0x7fbc6c102d30 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc71961640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c112280 con 0x7fbc6c1082a0 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37614/0 (socket says 172.21.15.16:37614) 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/3017865655 learned_addr learned my addr 172.21.15.16:0/3017865655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6affd640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c102d30 0x7fbc6c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 0x7fbc6c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/3017865655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 msgr2=0x7fbc6c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 0x7fbc6c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/3017865655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c102d30 msgr2=0x7fbc6c1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c102d30 0x7fbc6c1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.979+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/3017865655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c19af00 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000d740 tx=0x7fbc6000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc7095f640 1 -- 172.21.15.16:0/3017865655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60017070 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc7095f640 1 -- 172.21.15.16:0/3017865655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc6000de50 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c19b1f0 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc7095f640 1 -- 172.21.15.16:0/3017865655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6000f6b0 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.980+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1a7540 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.982+0000 7fbc7095f640 1 -- 172.21.15.16:0/3017865655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc6001a070 con 0x7fbc6c1036e0 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 Processor -- start 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 -- start start 2024-09-12T01:03:40.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da50e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da51b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629da4742f0 con 0x5629da50e5d0 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5629da41bcf0 con 0x5629da50fb40 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5629da4702b0 con 0x5629da519370 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc7095f640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 0x7fbc540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37628/0 (socket says 172.21.15.16:37628) 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6affd640 1 -- 172.21.15.16:0/1270180452 learned_addr learned my addr 172.21.15.16:0/1270180452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 0x7fbc540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da51b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da50e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.983+0000 7fbc7095f640 1 -- 172.21.15.16:0/3017865655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbc600ced80 con 0x7fbc6c1036e0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 -- 172.21.15.16:0/1270180452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 msgr2=0x5629da51b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da51b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 -- 172.21.15.16:0/1270180452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 msgr2=0x5629da50e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da50e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 -- 172.21.15.16:0/1270180452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629da4003a0 con 0x5629da50e5d0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 0x7fbc540a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009440 tx=0x7fbc6400bc30 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc6affd640 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009930 tx=0x7fbc5c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7114fe406e2e3c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc51ffb640 1 -- 172.21.15.16:0/1270180452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c040070 con 0x5629da50e5d0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc51ffb640 1 -- 172.21.15.16:0/1270180452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5c005020 con 0x5629da50e5d0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.984+0000 7fbc51ffb640 1 -- 172.21.15.16:0/1270180452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c035040 con 0x5629da50e5d0 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- 172.21.15.16:0/1270180452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 msgr2=0x5629da50ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009930 tx=0x7fbc5c031630 comp rx=0 tx=0).stop 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- 172.21.15.16:0/1270180452 shutdown_connections 2024-09-12T01:03:40.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da51b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da50ea10 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/1270180452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da50e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- 172.21.15.16:0/1270180452 >> 172.21.15.16:0/1270180452 conn(0x5629da4d7310 msgr2=0x5629da50ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- 172.21.15.16:0/1270180452 shutdown_connections 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- 172.21.15.16:0/1270180452 wait complete. 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 -- start start 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.985+0000 7fbc73d3a740 1 Processor -- start 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 0x5629da518c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da5163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5629da51c010 con 0x5629da50fb40 2024-09-12T01:03:40.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5629da42f060 con 0x5629da50e5d0 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5629da51c190 con 0x5629da519370 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc73d3a740 10 client.0 _ll_register_callbacks cb 0x5629da1828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37642/0 (socket says 172.21.15.16:37642) 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/776199248 learned_addr learned my addr 172.21.15.16:0/776199248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 0x5629da518c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da5163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 msgr2=0x5629da5163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da5163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/776199248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 msgr2=0x5629da518c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 0x5629da518c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/776199248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629da1e53c0 con 0x5629da50fb40 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da5163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6affd640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 0x5629da518c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbc6009b280 tx=0x7fbc60017e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecf518ddeab9b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.986+0000 7fbc4affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60021040 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbc600cc6f0 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5c005020 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc600cc8f0 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 10 client.17703 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629da527960 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5c005020 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc600a3a20 con 0x5629da50fb40 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 10 client.17703 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 20 client.17703 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da528d70 0x5629da52b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 4 client.17703 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.987+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629da481860 con 0x5629da528d70 2024-09-12T01:03:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.988+0000 7fbc6affd640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da528d70 0x5629da52b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.988+0000 7fbc6affd640 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da528d70 0x5629da52b180 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4058364dbfce0c server_cookie=21f534c04ae025a1 in_seq=0 out_seq=0 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.988+0000 7fbc4affd640 10 client.17703 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc4affd640 1 -- 172.21.15.16:0/776199248 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629da481860 con 0x5629da528d70 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc4affd640 10 client.17703 handle_command_reply: tid=0 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 client.17703 shutdown 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 msgr2=0x5629da515eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fbc6009b280 tx=0x7fbc60017e60 comp rx=0 tx=0).stop 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 shutdown_connections 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da528d70 0x5629da52b180 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da5163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da515eb0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc4affd640 0 client.17703 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/776199248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50e5d0 0x5629da518c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 >> 172.21.15.16:0/776199248 conn(0x5629da4d7310 msgr2=0x5629da510930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 shutdown_connections 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 1 -- 172.21.15.16:0/776199248 wait complete. 2024-09-12T01:03:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:40.989+0000 7fbc73d3a740 20 client.17703 trim_cache size 0 max 16384 2024-09-12T01:03:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.002+0000 7fbc73d3a740 1 Processor -- start 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.002+0000 7fbc73d3a740 1 -- start start 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.002+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da50e5d0 0x5629da5252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da519370 0x5629da51a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1afbc0 con 0x5629da50fb40 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b1700 con 0x5629da519370 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b0290 con 0x5629da50e5d0 2024-09-12T01:03:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37658/0 (socket says 172.21.15.16:37658) 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/382088031 learned_addr learned my addr 172.21.15.16:0/382088031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6affd640 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da50e5d0 0x5629da5252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/382088031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da50e5d0 msgr2=0x5629da5252b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da50e5d0 0x5629da5252b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/382088031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da519370 msgr2=0x5629da51a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da519370 0x5629da51a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da519370 0x5629da51a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 -- 172.21.15.16:0/382088031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629da50f780 con 0x5629da50fb40 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbc600a3980 tx=0x7fbc60005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101ba0534b0764d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc517fa640 1 -- 172.21.15.16:0/382088031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60095e60 con 0x5629da50fb40 2024-09-12T01:03:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc517fa640 1 -- 172.21.15.16:0/382088031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc6009b2f0 con 0x5629da50fb40 2024-09-12T01:03:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.003+0000 7fbc517fa640 1 -- 172.21.15.16:0/382088031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60016bb0 con 0x5629da50fb40 2024-09-12T01:03:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- 172.21.15.16:0/382088031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 msgr2=0x5629da50dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbc600a3980 tx=0x7fbc60005d40 comp rx=0 tx=0).stop 2024-09-12T01:03:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- 172.21.15.16:0/382088031 shutdown_connections 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da519370 0x5629da51a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50fb40 0x5629da50dc40 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/382088031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da50e5d0 0x5629da5252b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- 172.21.15.16:0/382088031 >> 172.21.15.16:0/382088031 conn(0x5629da4d7310 msgr2=0x5629da514780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- 172.21.15.16:0/382088031 shutdown_connections 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- 172.21.15.16:0/382088031 wait complete. 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.004+0000 7fbc73d3a740 1 -- start start 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 Processor -- start 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da530d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da531260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b1030 con 0x5629da50e5d0 2024-09-12T01:03:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1b0960 con 0x5629da50fb40 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1adbc0 con 0x5629da519370 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc73d3a740 10 client.0 _ll_register_callbacks cb 0x7fbc6009b0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37672/0 (socket says 172.21.15.16:37672) 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 -- 172.21.15.16:0/2704562815 learned_addr learned my addr 172.21.15.16:0/2704562815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da530d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 -- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 msgr2=0x5629da531260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da531260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da531260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 -- 172.21.15.16:0/2704562815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 msgr2=0x5629da530d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da530d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6affd640 1 -- 172.21.15.16:0/2704562815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629da51fde0 con 0x5629da50e5d0 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da530d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:41.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.005+0000 7fbc6b7fe640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da531260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc6affd640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009930 tx=0x7fbc5c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e389c0f34396ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c040030 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fbc5c005450 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc6009b2f0 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc5c05e050 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 10 client.17709 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629da593f20 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc6009b2f0 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc5c059030 con 0x5629da50e5d0 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 10 client.17709 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 20 client.17709 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:41.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da594260 0x5629da596650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 4 client.17709 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.006+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629da47bd70 con 0x5629da594260 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da594260 0x5629da596650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc6a7fc640 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da594260 0x5629da596650 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64c657a4a05cd18 server_cookie=f643358e93d9356e in_seq=0 out_seq=0 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc4affd640 10 client.17709 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc4affd640 1 -- 172.21.15.16:0/2704562815 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629da47bd70 con 0x5629da594260 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc4affd640 10 client.17709 handle_command_reply: tid=0 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.007+0000 7fbc73d3a740 1 client.17709 shutdown 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 msgr2=0x5629da5307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009930 tx=0x7fbc5c0045b0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 shutdown_connections 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5629da594260 0x5629da596650 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629da519370 0x5629da531260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5629da50fb40 0x5629da530d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 --2- 172.21.15.16:0/2704562815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629da50e5d0 0x5629da5307e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 >> 172.21.15.16:0/2704562815 conn(0x5629da4d7310 msgr2=0x5629da517f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc4affd640 0 client.17709 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 shutdown_connections 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 1 -- 172.21.15.16:0/2704562815 wait complete. 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc73d3a740 20 client.17709 trim_cache size 0 max 16384 2024-09-12T01:03:41.013 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 msgr2=0x7fbc540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 0x7fbc540a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009440 tx=0x7fbc6400bc30 comp rx=0 tx=0).stop 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 msgr2=0x7fbc6c1a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000d740 tx=0x7fbc6000dc00 comp rx=0 tx=0).stop 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 shutdown_connections 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.008+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbc540a2d70 0x7fbc540a5230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc6c1082a0 0x7fbc6c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c1036e0 0x7fbc6c1a0b00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 --2- 172.21.15.16:0/3017865655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc6c102d30 0x7fbc6c1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 >> 172.21.15.16:0/3017865655 conn(0x7fbc6c0fc7d0 msgr2=0x7fbc6c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 shutdown_connections 2024-09-12T01:03:41.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:41.009+0000 7fbc71961640 1 -- 172.21.15.16:0/3017865655 wait complete. 2024-09-12T01:03:41.023 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:40 smithi099 ceph-mon[35654]: pgmap v233: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-12T01:03:42.797 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:03:42.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:03:42.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:42 smithi016 ceph-mon[30972]: pgmap v234: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:42.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:42 smithi159 ceph-mon[35566]: pgmap v234: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:42 smithi099 ceph-mon[35654]: pgmap v234: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:43.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- 172.21.15.16:0/3761942749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc1082a0 msgr2=0x7ff9dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3761942749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc10ec40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009930 tx=0x7ff9d402a420 comp rx=0 tx=0).stop 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- 172.21.15.16:0/3761942749 shutdown_connections 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3761942749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc10ec40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3761942749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1036e0 0x7ff9dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3761942749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc102d30 0x7ff9dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- 172.21.15.16:0/3761942749 >> 172.21.15.16:0/3761942749 conn(0x7ff9dc0fc7d0 msgr2=0x7ff9dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- 172.21.15.16:0/3761942749 shutdown_connections 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- 172.21.15.16:0/3761942749 wait complete. 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 Processor -- start 2024-09-12T01:03:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.559+0000 7ff9e1561640 1 -- start start 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc1036e0 0x7ff9dc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1120a0 con 0x7ff9dc102d30 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc111f20 con 0x7ff9dc1082a0 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9e1561640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc112220 con 0x7ff9dc1036e0 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37692/0 (socket says 172.21.15.16:37692) 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 -- 172.21.15.16:0/3967967509 learned_addr learned my addr 172.21.15.16:0/3967967509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc1036e0 0x7ff9dc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 -- 172.21.15.16:0/3967967509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc1036e0 msgr2=0x7ff9dc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc1036e0 0x7ff9dc198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 -- 172.21.15.16:0/3967967509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 msgr2=0x7ff9dc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 -- 172.21.15.16:0/3967967509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc19d4f0 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9daffd640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00eab0 tx=0x7ff9cc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.560+0000 7ff9c3fff640 1 -- 172.21.15.16:0/3967967509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00ce40 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.561+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc19d7e0 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.561+0000 7ff9c3fff640 1 -- 172.21.15.16:0/3967967509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc011950 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.561+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc1a50c0 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.561+0000 7ff9c3fff640 1 -- 172.21.15.16:0/3967967509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc010a80 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9c3fff640 1 -- 172.21.15.16:0/3967967509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff9cc011b00 con 0x7ff9dc102d30 2024-09-12T01:03:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 Processor -- start 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 -- start start 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a81e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a91150 0x5606e5a93540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606e59e7280 con 0x5606e5a82360 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606e598eb10 con 0x5606e5a81a40 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.563+0000 7ff9e393a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606e59e3240 con 0x5606e5a91150 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37696/0 (socket says 172.21.15.16:37696) 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 -- 172.21.15.16:0/1052773720 learned_addr learned my addr 172.21.15.16:0/1052773720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a81e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9c3fff640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a91150 0x5606e5a93540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9c3fff640 1 -- 172.21.15.16:0/3967967509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff9cc0cfbe0 con 0x7ff9dc102d30 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 -- 172.21.15.16:0/1052773720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a91150 msgr2=0x5606e5a93540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a91150 0x5606e5a93540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 -- 172.21.15.16:0/1052773720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 msgr2=0x5606e5a81e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a81e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 -- 172.21.15.16:0/1052773720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e599b1f0 con 0x5606e5a82360 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0cd2a0 tx=0x7ff9cc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7529b564b5e2de9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9c17fa640 1 -- 172.21.15.16:0/1052773720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc09ba60 con 0x5606e5a82360 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9c17fa640 1 -- 172.21.15.16:0/1052773720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc09c800 con 0x5606e5a82360 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.564+0000 7ff9c17fa640 1 -- 172.21.15.16:0/1052773720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0cd5b0 con 0x5606e5a82360 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 -- 172.21.15.16:0/1052773720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 msgr2=0x5606e5a90c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0cd2a0 tx=0x7ff9cc000f40 comp rx=0 tx=0).stop 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40096c0 tx=0x7ff9d402a420 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 -- 172.21.15.16:0/1052773720 shutdown_connections 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a91150 0x5606e5a93540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a82360 0x5606e5a90c10 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1052773720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a81e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 -- 172.21.15.16:0/1052773720 >> 172.21.15.16:0/1052773720 conn(0x5606e5a7e0f0 msgr2=0x5606e5a7e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 -- 172.21.15.16:0/1052773720 shutdown_connections 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.565+0000 7ff9e393a740 1 -- 172.21.15.16:0/1052773720 wait complete. 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 -- start start 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 Processor -- start 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a8e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606e5a93d20 con 0x5606e5a91150 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606e59a1f10 con 0x5606e5a81a40 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606e5a93ea0 con 0x5606e5a82360 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9e393a740 10 client.0 _ll_register_callbacks cb 0x5606e596aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37708/0 (socket says 172.21.15.16:37708) 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 -- 172.21.15.16:0/2932242384 learned_addr learned my addr 172.21.15.16:0/2932242384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9daffd640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 -- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 msgr2=0x5606e5a8e630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a8e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 -- 172.21.15.16:0/2932242384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 msgr2=0x5606e5a8e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a8e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.566+0000 7ff9db7fe640 1 -- 172.21.15.16:0/2932242384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e57586a0 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40042c0 tx=0x7ff9d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd063bc154641984 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d404d020 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff9d40046c0 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc09c800 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9d4058050 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9e393a740 10 client.17721 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606e5a9fa10 con 0x5606e5a91150 2024-09-12T01:03:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc09c800 con 0x5606e5a91150 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9d403a040 con 0x5606e5a91150 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9e393a740 10 client.17721 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.567+0000 7ff9e393a740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9e393a740 20 client.17721 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5aa0f20 0x5606e5aa33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9e393a740 4 client.17721 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e59f47f0 con 0x5606e5aa0f20 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9daffd640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5aa0f20 0x5606e5aa33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9beff5640 10 client.17721 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.568+0000 7ff9daffd640 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5aa0f20 0x5606e5aa33e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392441c01220ff69 server_cookie=edf55c60d7361d43 in_seq=0 out_seq=0 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9beff5640 1 -- 172.21.15.16:0/2932242384 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606e59f47f0 con 0x5606e5aa0f20 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9beff5640 10 client.17721 handle_command_reply: tid=0 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 client.17721 shutdown 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 msgr2=0x5606e5a8eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40042c0 tx=0x7ff9d4009260 comp rx=0 tx=0).stop 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 shutdown_connections 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5aa0f20 0x5606e5aa33e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5a8eb70 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a8e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 --2- 172.21.15.16:0/2932242384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a81a40 0x5606e5a8e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9beff5640 0 client.17721 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 >> 172.21.15.16:0/2932242384 conn(0x5606e5a7e0f0 msgr2=0x5606e5a83e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.569+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 shutdown_connections 2024-09-12T01:03:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.570+0000 7ff9e393a740 1 -- 172.21.15.16:0/2932242384 wait complete. 2024-09-12T01:03:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.570+0000 7ff9e393a740 20 client.17721 trim_cache size 0 max 16384 2024-09-12T01:03:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.587+0000 7ff9e393a740 1 Processor -- start 2024-09-12T01:03:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.587+0000 7ff9e393a740 1 -- start start 2024-09-12T01:03:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.587+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 0x5606e5a8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.587+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 0x5606e5aa8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9e393a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1ab720 con 0x5606e5a91150 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9e393a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1ad260 con 0x5606e5a82360 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9e393a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1abdf0 con 0x5606e5a81a40 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 -- 172.21.15.16:0/1093156415 learned_addr learned my addr 172.21.15.16:0/1093156415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 0x5606e5a8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 0x5606e5aa8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 -- 172.21.15.16:0/1093156415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 msgr2=0x5606e5a8ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 0x5606e5a8ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 -- 172.21.15.16:0/1093156415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 msgr2=0x5606e5aa8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 0x5606e5aa8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 -- 172.21.15.16:0/1093156415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e5aa3a60 con 0x5606e5a91150 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 0x5606e5a8ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 0x5606e5aa8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402a940 tx=0x7ff9d40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15472b8691d8bfdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9c0ff9640 1 -- 172.21.15.16:0/1093156415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4061070 con 0x5606e5a91150 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9c0ff9640 1 -- 172.21.15.16:0/1093156415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4042030 con 0x5606e5a91150 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.588+0000 7ff9c0ff9640 1 -- 172.21.15.16:0/1093156415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d404d070 con 0x5606e5a91150 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 -- 172.21.15.16:0/1093156415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 msgr2=0x5606e5aa91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402a940 tx=0x7ff9d40058b0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 -- 172.21.15.16:0/1093156415 shutdown_connections 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a91150 0x5606e5aa91f0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a82360 0x5606e5aa8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1093156415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a81a40 0x5606e5a8ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 -- 172.21.15.16:0/1093156415 >> 172.21.15.16:0/1093156415 conn(0x5606e5a7e0f0 msgr2=0x5606e5a8f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 -- 172.21.15.16:0/1093156415 shutdown_connections 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.589+0000 7ff9e393a740 1 -- 172.21.15.16:0/1093156415 wait complete. 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 -- start start 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 Processor -- start 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a95320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a91150 0x5606e5a95860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1acb90 con 0x5606e5a81a40 2024-09-12T01:03:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1ac4c0 con 0x5606e5a91150 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a9720 con 0x5606e5a82360 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9e393a740 10 client.0 _ll_register_callbacks cb 0x7ff9d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37734/0 (socket says 172.21.15.16:37734) 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 -- 172.21.15.16:0/1357232267 learned_addr learned my addr 172.21.15.16:0/1357232267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9db7fe640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a91150 0x5606e5a95860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a95320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 -- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 msgr2=0x5606e5a95320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a95320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 -- 172.21.15.16:0/1357232267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a91150 msgr2=0x5606e5a95860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a91150 0x5606e5a95860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.590+0000 7ff9daffd640 1 -- 172.21.15.16:0/1357232267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e5b0ac80 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9daffd640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc098560 tx=0x7ff9cc0cffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75ab680eba09426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc09c800 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff9cc0cd5b0 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d4042030 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9cc09c100 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 10 client.17727 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606e5b0bf00 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4042030 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9cc00f040 con 0x5606e5a81a40 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 10 client.17727 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 20 client.17727 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5b0c1f0 0x5606e5b0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 4 client.17727 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.591+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606e59eed00 con 0x5606e5b0c1f0 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.592+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5b0c1f0 0x5606e5b0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.592+0000 7ff9beff5640 10 client.17727 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.592+0000 7ff9da7fc640 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5b0c1f0 0x5606e5b0e5e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a50264b2ec3b6b31 server_cookie=80232c4b89d7bf4e in_seq=0 out_seq=0 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9beff5640 1 -- 172.21.15.16:0/1357232267 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606e59eed00 con 0x5606e5b0c1f0 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9beff5640 10 client.17727 handle_command_reply: tid=0 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 client.17727 shutdown 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 msgr2=0x5606e5a94de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc098560 tx=0x7ff9cc0cffd0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 shutdown_connections 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5606e5b0c1f0 0x5606e5b0e5e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606e5a91150 0x5606e5a95860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9beff5640 0 client.17727 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606e5a82360 0x5606e5a95320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.593+0000 7ff9e393a740 1 --2- 172.21.15.16:0/1357232267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a81a40 0x5606e5a94de0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 >> 172.21.15.16:0/1357232267 conn(0x5606e5a7e0f0 msgr2=0x5606e5a7d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 shutdown_connections 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e393a740 1 -- 172.21.15.16:0/1357232267 wait complete. 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e393a740 20 client.17727 trim_cache size 0 max 16384 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:03:43.601 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "87eeb8d9-bbdb-41d0-8a74-3d7d78f15ddd", 2024-09-12T01:03:43.602 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:03:43.602 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:43.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 msgr2=0x7ff9c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40096c0 tx=0x7ff9d402a420 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 msgr2=0x7ff9dc198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00eab0 tx=0x7ff9cc00ef70 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 shutdown_connections 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9dc1082a0 0x7ff9dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9dc1036e0 0x7ff9dc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 --2- 172.21.15.16:0/3967967509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9dc102d30 0x7ff9dc198540 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.594+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 >> 172.21.15.16:0/3967967509 conn(0x7ff9dc0fc7d0 msgr2=0x7ff9dc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.595+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 shutdown_connections 2024-09-12T01:03:43.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:43.595+0000 7ff9e1561640 1 -- 172.21.15.16:0/3967967509 wait complete. 2024-09-12T01:03:44.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:03:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: pgmap v235: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:44.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:44 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: pgmap v235: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:44 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:03:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: pgmap v235: 97 pgs: 97 active+clean; 8.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-09-12T01:03:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:03:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:03:45.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:45.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:44 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:03:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 -- 172.21.15.16:0/872987195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 msgr2=0x7f3bec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 --2- 172.21.15.16:0/872987195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec10d950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009cd0 tx=0x7f3be402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 -- 172.21.15.16:0/872987195 shutdown_connections 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 --2- 172.21.15.16:0/872987195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec10d950 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 --2- 172.21.15.16:0/872987195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec100a80 0x7f3bec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 --2- 172.21.15.16:0/872987195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec106a80 0x7f3bec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 -- 172.21.15.16:0/872987195 >> 172.21.15.16:0/872987195 conn(0x7f3bec0fc7a0 msgr2=0x7f3bec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 -- 172.21.15.16:0/872987195 shutdown_connections 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.139+0000 7f3bf1825640 1 -- 172.21.15.16:0/872987195 wait complete. 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 Processor -- start 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 -- start start 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec100a80 0x7f3bec19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 0x7f3bec19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec110ec0 con 0x7f3bec101420 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec110d40 con 0x7f3bec106a80 2024-09-12T01:03:45.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3bf1825640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec111040 con 0x7f3bec100a80 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37762/0 (socket says 172.21.15.16:37762) 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3be3fff640 1 -- 172.21.15.16:0/4283173270 learned_addr learned my addr 172.21.15.16:0/4283173270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3beaffd640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec100a80 0x7f3bec19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.140+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 0x7f3bec19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 -- 172.21.15.16:0/4283173270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec100a80 msgr2=0x7f3bec19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec100a80 0x7f3bec19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 -- 172.21.15.16:0/4283173270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 msgr2=0x7f3bec19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 0x7f3bec19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 -- 172.21.15.16:0/4283173270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec19f6d0 con 0x7f3bec101420 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 0x7f3bec19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3be3fff640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b4a0 tx=0x7f3bd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3bf0823640 1 -- 172.21.15.16:0/4283173270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4014070 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3bf0823640 1 -- 172.21.15.16:0/4283173270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4010af0 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec19f9c0 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3bf0823640 1 -- 172.21.15.16:0/4283173270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4017d10 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.141+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1a72a0 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.143+0000 7f3bf0823640 1 -- 172.21.15.16:0/4283173270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3bd4017070 con 0x7f3bec101420 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.143+0000 7f3bf3bfe740 1 Processor -- start 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 -- start start 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a02c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a037680 0x55ea3a039a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea39f922e0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea39f39ce0 con 0x55ea3a037680 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ea39f8e2a0 con 0x55ea3a02c4a0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37774/0 (socket says 172.21.15.16:37774) 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 -- 172.21.15.16:0/2968174237 learned_addr learned my addr 172.21.15.16:0/2968174237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a02c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3be3fff640 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a037680 0x55ea3a039a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 -- 172.21.15.16:0/2968174237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 msgr2=0x55ea3a02c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a02c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 -- 172.21.15.16:0/2968174237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a037680 msgr2=0x55ea3a039a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a037680 0x55ea3a039a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.144+0000 7f3beaffd640 1 -- 172.21.15.16:0/2968174237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea39f1e440 con 0x55ea3a02cdc0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ac10 tx=0x7f3bdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a706fb72dab8e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3bf0823640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 0x7f3bd80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3be1ffb640 1 -- 172.21.15.16:0/2968174237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc01d070 con 0x55ea3a02cdc0 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3be1ffb640 1 -- 172.21.15.16:0/2968174237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc018b20 con 0x55ea3a02cdc0 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3be1ffb640 1 -- 172.21.15.16:0/2968174237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc010b30 con 0x55ea3a02cdc0 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 0x7f3bd80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3bf0823640 1 -- 172.21.15.16:0/4283173270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3bd40ce630 con 0x7f3bec101420 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2968174237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 msgr2=0x55ea3a037140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 0x7f3bd80a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009430 tx=0x7f3be4009360 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.145+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ac10 tx=0x7f3bdc00f790 comp rx=0 tx=0).stop 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2968174237 shutdown_connections 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a037680 0x55ea3a039a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a037140 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2968174237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a02c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2968174237 >> 172.21.15.16:0/2968174237 conn(0x55ea3a028a30 msgr2=0x55ea3a028e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2968174237 shutdown_connections 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2968174237 wait complete. 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- start start 2024-09-12T01:03:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 Processor -- start 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02c4a0 0x55ea3a036f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 0x55ea3a0341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea3a03a250 con 0x55ea3a037680 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea39f4d050 con 0x55ea3a02c4a0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.146+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ea3a03a3d0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bf3bfe740 10 client.0 _ll_register_callbacks cb 0x55ea39f8d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37776/0 (socket says 172.21.15.16:37776) 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 -- 172.21.15.16:0/2875327616 learned_addr learned my addr 172.21.15.16:0/2875327616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02c4a0 0x55ea3a036f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3be3fff640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 0x55ea3a0341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 -- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 msgr2=0x55ea3a0341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 0x55ea3a0341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 -- 172.21.15.16:0/2875327616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02c4a0 msgr2=0x55ea3a036f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02c4a0 0x55ea3a036f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 -- 172.21.15.16:0/2875327616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea39d033c0 con 0x55ea3a037680 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3be3fff640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 0x55ea3a0341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3be4031dc0 tx=0x7f3be402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133922ec4b63733c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4056060 con 0x55ea3a037680 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc018b20 con 0x55ea3a037680 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3be4041030 con 0x55ea3a037680 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.147+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3be4004250 con 0x55ea3a037680 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 10 client.17739 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea3a045bf0 con 0x55ea3a037680 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc018b20 con 0x55ea3a037680 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3be4030070 con 0x55ea3a037680 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 10 client.17739 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 20 client.17739 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a047000 0x55ea3a0493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 4 client.17739 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea39f9f850 con 0x55ea3a047000 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.148+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a047000 0x55ea3a0493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.149+0000 7f3beaffd640 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a047000 0x55ea3a0493f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab13faa12d099236 server_cookie=ace0e37f8d6ebfe0 in_seq=0 out_seq=0 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.149+0000 7f3bcaffd640 10 client.17739 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:45.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.149+0000 7f3bcaffd640 1 -- 172.21.15.16:0/2875327616 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea39f9f850 con 0x55ea3a047000 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.149+0000 7f3bcaffd640 10 client.17739 handle_command_reply: tid=0 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.149+0000 7f3bf3bfe740 1 client.17739 shutdown 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 msgr2=0x55ea3a034730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3be4031dc0 tx=0x7f3be402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 shutdown_connections 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a047000 0x55ea3a0493f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bcaffd640 0 client.17739 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a037680 0x55ea3a034730 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02cdc0 0x55ea3a0341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/2875327616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02c4a0 0x55ea3a036f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 >> 172.21.15.16:0/2875327616 conn(0x55ea3a028a30 msgr2=0x55ea3a02e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 shutdown_connections 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/2875327616 wait complete. 2024-09-12T01:03:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.150+0000 7f3bf3bfe740 20 client.17739 trim_cache size 0 max 16384 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.167+0000 7f3bf3bfe740 1 Processor -- start 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.167+0000 7f3bf3bfe740 1 -- start start 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.167+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.167+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a0341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.167+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 0x55ea3a02f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1afac0 con 0x55ea3a028a30 2024-09-12T01:03:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b1600 con 0x55ea3a02cdc0 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b0190 con 0x55ea3a02c4a0 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37778/0 (socket says 172.21.15.16:37778) 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 -- 172.21.15.16:0/3562920622 learned_addr learned my addr 172.21.15.16:0/3562920622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3be3fff640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a0341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 -- 172.21.15.16:0/3562920622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 msgr2=0x55ea3a0341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 0x55ea3a02f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a0341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 -- 172.21.15.16:0/3562920622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 msgr2=0x55ea3a02f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 0x55ea3a02f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 -- 172.21.15.16:0/3562920622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3a049760 con 0x55ea3a028a30 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3be3fff640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a0341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 0x55ea3a02f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3beaffd640 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ab30 tx=0x7f3bdc00a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5fdb1200d15106 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3be17fa640 1 -- 172.21.15.16:0/3562920622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc01d040 con 0x55ea3a028a30 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3be17fa640 1 -- 172.21.15.16:0/3562920622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc01e040 con 0x55ea3a028a30 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.168+0000 7f3be17fa640 1 -- 172.21.15.16:0/3562920622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0053a0 con 0x55ea3a028a30 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/3562920622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 msgr2=0x55ea3a0434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00ab30 tx=0x7f3bdc00a450 comp rx=0 tx=0).stop 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/3562920622 shutdown_connections 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a02cdc0 0x55ea3a02f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a0341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/3562920622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a028a30 0x55ea3a0434b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/3562920622 >> 172.21.15.16:0/3562920622 conn(0x55ea3a034890 msgr2=0x55ea3a02f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/3562920622 shutdown_connections 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.169+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/3562920622 wait complete. 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 -- start start 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 Processor -- start 2024-09-12T01:03:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a028a30 0x55ea3a03fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a03b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b0f30 con 0x55ea3a02cdc0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1b0860 con 0x55ea3a028a30 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1adac0 con 0x55ea3a02c4a0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3bf3bfe740 10 client.0 _ll_register_callbacks cb 0x7f3be4031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3beb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37786/0 (socket says 172.21.15.16:37786) 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3beb7fe640 1 -- 172.21.15.16:0/1018711110 learned_addr learned my addr 172.21.15.16:0/1018711110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3be3fff640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.170+0000 7f3beaffd640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a028a30 0x55ea3a03fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 -- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 msgr2=0x55ea3a03b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a03b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 -- 172.21.15.16:0/1018711110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a028a30 msgr2=0x55ea3a03fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a028a30 0x55ea3a03fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 -- 172.21.15.16:0/1018711110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3a03bec0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3beb7fe640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3be4042040 tx=0x7f3be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e24a25395a85944 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4056040 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f3be40305a0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bdc01e040 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3be4031bd0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bf3bfe740 10 client.17745 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea3a0b1ce0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.171+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bdc01e040 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3be40308a0 con 0x55ea3a02cdc0 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 10 client.17745 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 20 client.17745 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a037680 0x55ea3a0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 4 client.17745 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea39f99d60 con 0x55ea3a037680 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3beaffd640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a037680 0x55ea3a0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3beaffd640 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a037680 0x55ea3a0b4470 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47293e022ca8422b server_cookie=d91e720cd72d0df5 in_seq=0 out_seq=0 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.172+0000 7f3bcaffd640 10 client.17745 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bcaffd640 1 -- 172.21.15.16:0/1018711110 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea39f99d60 con 0x55ea3a037680 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bcaffd640 10 client.17745 handle_command_reply: tid=0 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 client.17745 shutdown 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 msgr2=0x55ea3a03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3be4042040 tx=0x7f3be4004a40 comp rx=0 tx=0).stop 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 shutdown_connections 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ea3a037680 0x55ea3a0b4470 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bcaffd640 0 client.17745 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea3a02cdc0 0x55ea3a03b690 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ea3a02c4a0 0x55ea3a03b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 --2- 172.21.15.16:0/1018711110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea3a028a30 0x55ea3a03fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 >> 172.21.15.16:0/1018711110 conn(0x55ea3a034890 msgr2=0x55ea3a02f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 shutdown_connections 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 1 -- 172.21.15.16:0/1018711110 wait complete. 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.173+0000 7f3bf3bfe740 20 client.17745 trim_cache size 0 max 16384 2024-09-12T01:03:45.182 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 msgr2=0x7f3bd80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 0x7f3bd80a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009430 tx=0x7f3be4009360 comp rx=0 tx=0).stop 2024-09-12T01:03:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 msgr2=0x7f3bec19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b4a0 tx=0x7f3bd400b960 comp rx=0 tx=0).stop 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 shutdown_connections 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3bd80a2d70 0x7f3bd80a5230 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bec106a80 0x7f3bec19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bec101420 0x7f3bec19ac60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 --2- 172.21.15.16:0/4283173270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bec100a80 0x7f3bec19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 >> 172.21.15.16:0/4283173270 conn(0x7f3bec0fc7a0 msgr2=0x7f3bec10c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 shutdown_connections 2024-09-12T01:03:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:45.174+0000 7f3bf1825640 1 -- 172.21.15.16:0/4283173270 wait complete. 2024-09-12T01:03:45.881 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:03:45.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:03:46.139 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 -- 172.21.15.16:0/2473062441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0106f10 msgr2=0x7f03f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 --2- 172.21.15.16:0/2473062441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0106f10 0x7f03f010ea90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e802a410 comp rx=0 tx=0).stop 2024-09-12T01:03:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 -- 172.21.15.16:0/2473062441 shutdown_connections 2024-09-12T01:03:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 --2- 172.21.15.16:0/2473062441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0106f10 0x7f03f010ea90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 --2- 172.21.15.16:0/2473062441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0102790 0x7f03f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 --2- 172.21.15.16:0/2473062441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 0x7f03f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 -- 172.21.15.16:0/2473062441 >> 172.21.15.16:0/2473062441 conn(0x7f03f00fd8c0 msgr2=0x7f03f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 -- 172.21.15.16:0/2473062441 shutdown_connections 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.596+0000 7f03f5135640 1 -- 172.21.15.16:0/2473062441 wait complete. 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 Processor -- start 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 -- start start 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 0x7f03f01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 0x7f03f019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0112010 con 0x7f03f0102790 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0111e90 con 0x7f03f0106f10 2024-09-12T01:03:46.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03f5135640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0112190 con 0x7f03f0101de0 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37802/0 (socket says 172.21.15.16:37802) 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03ee59c640 1 -- 172.21.15.16:0/387887980 learned_addr learned my addr 172.21.15.16:0/387887980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03eed9d640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 0x7f03f01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.597+0000 7f03ef59e640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 0x7f03f019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 -- 172.21.15.16:0/387887980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 msgr2=0x7f03f01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 0x7f03f01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 -- 172.21.15.16:0/387887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 msgr2=0x7f03f019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 0x7f03f019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 -- 172.21.15.16:0/387887980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f019d5a0 con 0x7f03f0102790 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ef59e640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 0x7f03f019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03ee59c640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b4a0 tx=0x7f03e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4014070 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f019d890 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e4010af0 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4017d10 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.598+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01a5170 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.599+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f0102240 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.600+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03e4017070 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.601+0000 7f03d7fff640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 0x7f03d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.601+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f03e40ceaa0 con 0x7f03f0102790 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.601+0000 7f03eed9d640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 0x7f03d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.602+0000 7f03eed9d640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 0x7f03d80a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03e000a400 tx=0x7f03e000d040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.604+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03e409b7d0 con 0x7f03f0102790 2024-09-12T01:03:46.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.819+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f010a150 con 0x7f03f0102790 2024-09-12T01:03:46.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:46 smithi016 ceph-mon[30972]: pgmap v236: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.819+0000 7f03d7fff640 1 -- 172.21.15.16:0/387887980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f03e409b0d0 con 0x7f03f0102790 2024-09-12T01:03:46.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:03:46.819 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:03:46.819 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.821+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 msgr2=0x7f03d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.821+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 0x7f03d80a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f03e000a400 tx=0x7f03e000d040 comp rx=0 tx=0).stop 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 msgr2=0x7f03f0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f03e400b4a0 tx=0x7f03e400b960 comp rx=0 tx=0).stop 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 shutdown_connections 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f03d80a2cf0 0x7f03d80a51b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f0106f10 0x7f03f019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03f0102790 0x7f03f0198b30 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 --2- 172.21.15.16:0/387887980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03f0101de0 0x7f03f01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 >> 172.21.15.16:0/387887980 conn(0x7f03f00fd8c0 msgr2=0x7f03f00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 shutdown_connections 2024-09-12T01:03:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:46.822+0000 7f03f5135640 1 -- 172.21.15.16:0/387887980 wait complete. 2024-09-12T01:03:46.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:46 smithi159 ceph-mon[35566]: pgmap v236: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:46 smithi099 ceph-mon[35654]: pgmap v236: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:47.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:03:47.809 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/387887980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:47.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/387887980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/387887980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 -- 172.21.15.16:0/3749829565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc108300 msgr2=0x7f00cc10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 --2- 172.21.15.16:0/3749829565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc108300 0x7f00cc10ec70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009cd0 tx=0x7f00c402a8d0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 -- 172.21.15.16:0/3749829565 shutdown_connections 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 --2- 172.21.15.16:0/3749829565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc108300 0x7f00cc10ec70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 --2- 172.21.15.16:0/3749829565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc103780 0x7f00cc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 --2- 172.21.15.16:0/3749829565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc102dd0 0x7f00cc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 -- 172.21.15.16:0/3749829565 >> 172.21.15.16:0/3749829565 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 -- 172.21.15.16:0/3749829565 shutdown_connections 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.248+0000 7f00d2272640 1 -- 172.21.15.16:0/3749829565 wait complete. 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 Processor -- start 2024-09-12T01:03:48.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 -- start start 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc103780 0x7f00cc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc108300 0x7f00cc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1120d0 con 0x7f00cc102dd0 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc111f50 con 0x7f00cc108300 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d2272640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc112250 con 0x7f00cc103780 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37820/0 (socket says 172.21.15.16:37820) 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 -- 172.21.15.16:0/3373744982 learned_addr learned my addr 172.21.15.16:0/3373744982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00c3fff640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc103780 0x7f00cc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00d080f640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc108300 0x7f00cc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 -- 172.21.15.16:0/3373744982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc103780 msgr2=0x7f00cc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc103780 0x7f00cc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 -- 172.21.15.16:0/3373744982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc108300 msgr2=0x7f00cc19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc108300 0x7f00cc19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.249+0000 7f00cbfff640 1 -- 172.21.15.16:0/3373744982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc19ad20 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d770 tx=0x7f00bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00c9ffb640 1 -- 172.21.15.16:0/3373744982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc017070 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00c9ffb640 1 -- 172.21.15.16:0/3373744982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00bc005500 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00c9ffb640 1 -- 172.21.15.16:0/3373744982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004e00 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc19b010 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.250+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a7300 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.252+0000 7f00c9ffb640 1 -- 172.21.15.16:0/3373744982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f00bc004fa0 con 0x7f00cc102dd0 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.252+0000 7f00d464b740 1 Processor -- start 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.252+0000 7f00d464b740 1 -- start start 2024-09-12T01:03:48.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b967ea72f0 con 0x55b967f4c370 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b967e4ecf0 con 0x55b967f415d0 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d464b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b967ea32b0 con 0x55b967f42b40 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c9ffb640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 0x7f00b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37824/0 (socket says 172.21.15.16:37824) 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 -- 172.21.15.16:0/562730580 learned_addr learned my addr 172.21.15.16:0/562730580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d080f640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 0x7f00b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00d080f640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c9ffb640 1 -- 172.21.15.16:0/3373744982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f00bc0cee80 con 0x7f00cc102dd0 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00cbfff640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 -- 172.21.15.16:0/562730580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 msgr2=0x55b967f41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 -- 172.21.15.16:0/562730580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 msgr2=0x55b967f41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00c3fff640 1 -- 172.21.15.16:0/562730580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b967e333a0 con 0x55b967f4c370 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.253+0000 7f00cbfff640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d080f640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 0x7f00b80a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00c4006010 tx=0x7f00c4009360 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00c3fff640 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f00b400bb70 tx=0x7f00b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae17a1e26ab458b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00c27fc640 1 -- 172.21.15.16:0/562730580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004030 con 0x55b967f4c370 2024-09-12T01:03:48.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00c27fc640 1 -- 172.21.15.16:0/562730580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40041d0 con 0x55b967f4c370 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00c27fc640 1 -- 172.21.15.16:0/562730580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4011680 con 0x55b967f4c370 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d464b740 1 -- 172.21.15.16:0/562730580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 msgr2=0x55b967f4e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d464b740 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f00b400bb70 tx=0x7f00b40107a0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d464b740 1 -- 172.21.15.16:0/562730580 shutdown_connections 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d464b740 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f4e760 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.254+0000 7f00d464b740 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 --2- 172.21.15.16:0/562730580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- 172.21.15.16:0/562730580 >> 172.21.15.16:0/562730580 conn(0x55b967f0a310 msgr2=0x55b967f3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- 172.21.15.16:0/562730580 shutdown_connections 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- 172.21.15.16:0/562730580 wait complete. 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- start start 2024-09-12T01:03:48.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 Processor -- start 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 0x55b967f4bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 0x55b967f49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b967f4f070 con 0x55b967f42b40 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b967e62060 con 0x55b967f4c370 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.255+0000 7f00d464b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b967f4f1f0 con 0x55b967f415d0 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00d464b740 10 client.0 _ll_register_callbacks cb 0x55b967bb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37828/0 (socket says 172.21.15.16:37828) 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 -- 172.21.15.16:0/2547506832 learned_addr learned my addr 172.21.15.16:0/2547506832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00d080f640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 0x55b967f49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00cbfff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 0x55b967f4bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 -- 172.21.15.16:0/2547506832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 msgr2=0x55b967f4bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 0x55b967f4bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 -- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 msgr2=0x55b967f49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 0x55b967f49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 -- 172.21.15.16:0/2547506832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b967c183c0 con 0x55b967f42b40 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00d080f640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 0x55b967f49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00cbfff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 0x55b967f4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00c3fff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f00b4003dc0 tx=0x7f00b400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fcd947e0490921 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b402a060 con 0x55b967f42b40 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f00b4004480 con 0x55b967f42b40 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.256+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b40041d0 con 0x55b967f42b40 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00b40035f0 con 0x55b967f42b40 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 10 client.17763 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:48.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b967f5aac0 con 0x55b967f42b40 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40041d0 con 0x55b967f42b40 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00b40048a0 con 0x55b967f42b40 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 10 client.17763 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 10 client.17763 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 20 client.17763 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967f5e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 4 client.17763 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b967eb4860 con 0x55b967f5bec0 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.257+0000 7f00cbfff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967f5e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.258+0000 7f00cbfff640 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967f5e2b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba67f18792a6f410 server_cookie=b9b317fbd30dae84 in_seq=0 out_seq=0 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.258+0000 7f00ab7fe640 10 client.17763 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00ab7fe640 1 -- 172.21.15.16:0/2547506832 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b967eb4860 con 0x55b967f5bec0 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00ab7fe640 10 client.17763 handle_command_reply: tid=0 2024-09-12T01:03:48.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 client.17763 shutdown 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 msgr2=0x55b967f48fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f00b4003dc0 tx=0x7f00b400b7d0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 shutdown_connections 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967f5e2b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00ab7fe640 0 client.17763 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f4c370 0x55b967f49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f42b40 0x55b967f48fd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 --2- 172.21.15.16:0/2547506832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f415d0 0x55b967f4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 >> 172.21.15.16:0/2547506832 conn(0x55b967f0a310 msgr2=0x55b967f43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 shutdown_connections 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 1 -- 172.21.15.16:0/2547506832 wait complete. 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.259+0000 7f00d464b740 20 client.17763 trim_cache size 0 max 16384 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.272+0000 7f00d464b740 1 Processor -- start 2024-09-12T01:03:48.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.272+0000 7f00d464b740 1 -- start start 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f4af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1afb20 con 0x55b967f4c370 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1b1660 con 0x55b967f415d0 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d464b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1b01f0 con 0x55b967f42b40 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37844/0 (socket says 172.21.15.16:37844) 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 -- 172.21.15.16:0/163633407 learned_addr learned my addr 172.21.15.16:0/163633407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00cbfff640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00c3fff640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f4af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 -- 172.21.15.16:0/163633407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 msgr2=0x55b967f4af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f4af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 -- 172.21.15.16:0/163633407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 msgr2=0x55b967f58310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f58310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00d080f640 1 -- 172.21.15.16:0/163633407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b967f3e960 con 0x55b967f4c370 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00cbfff640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.273+0000 7f00c3fff640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d080f640 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009cd0 tx=0x7f00c403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa65619b227503a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00c1ffb640 1 -- 172.21.15.16:0/163633407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4051070 con 0x55b967f4c370 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00c1ffb640 1 -- 172.21.15.16:0/163633407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c4003970 con 0x55b967f4c370 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00c1ffb640 1 -- 172.21.15.16:0/163633407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c40044b0 con 0x55b967f4c370 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 -- 172.21.15.16:0/163633407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 msgr2=0x55b967f63900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009cd0 tx=0x7f00c403f040 comp rx=0 tx=0).stop 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 -- 172.21.15.16:0/163633407 shutdown_connections 2024-09-12T01:03:48.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f63900 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f4af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 --2- 172.21.15.16:0/163633407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.274+0000 7f00d464b740 1 -- 172.21.15.16:0/163633407 >> 172.21.15.16:0/163633407 conn(0x55b967f0a310 msgr2=0x55b967f4a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.275+0000 7f00d464b740 1 -- 172.21.15.16:0/163633407 shutdown_connections 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.275+0000 7f00d464b740 1 -- 172.21.15.16:0/163633407 wait complete. 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.275+0000 7f00d464b740 1 -- start start 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.275+0000 7f00d464b740 1 Processor -- start 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f54660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f54ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1b0f90 con 0x55b967f4c370 2024-09-12T01:03:48.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1b08c0 con 0x55b967f415d0 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc1adb20 con 0x55b967f42b40 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 10 client.0 _ll_register_callbacks cb 0x7f00b400b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37848/0 (socket says 172.21.15.16:37848) 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 -- 172.21.15.16:0/3569231188 learned_addr learned my addr 172.21.15.16:0/3569231188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00c3fff640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f54ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f54660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 -- 172.21.15.16:0/3569231188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 msgr2=0x55b967f54ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f54ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 -- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 msgr2=0x55b967f54660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f54660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 -- 172.21.15.16:0/3569231188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b967f44120 con 0x55b967f4c370 2024-09-12T01:03:48.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d080f640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009ca0 tx=0x7f00c4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4863a9926c964cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4051070 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f00c40314e0 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.276+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c4003970 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00c40316e0 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 10 client.17769 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b967fc70d0 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c4003970 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00c402f990 con 0x55b967f4c370 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 10 client.17769 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 10 client.17769 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 20 client.17769 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:48.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967fc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 4 client.17769 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b967eaed70 con 0x55b967f5bec0 2024-09-12T01:03:48.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.277+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967fc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00cbfff640 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967fc9760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=197507563e9641a server_cookie=714022cd8e5af450 in_seq=0 out_seq=0 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00ab7fe640 10 client.17769 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00ab7fe640 1 -- 172.21.15.16:0/3569231188 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b967eaed70 con 0x55b967f5bec0 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00ab7fe640 10 client.17769 handle_command_reply: tid=0 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 client.17769 shutdown 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 msgr2=0x55b967f51450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009ca0 tx=0x7f00c4003d50 comp rx=0 tx=0).stop 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 shutdown_connections 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55b967f5bec0 0x55b967fc9760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b967f4c370 0x55b967f51450 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b967f42b40 0x55b967f54ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00ab7fe640 0 client.17769 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 --2- 172.21.15.16:0/3569231188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b967f415d0 0x55b967f54660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 >> 172.21.15.16:0/3569231188 conn(0x55b967f0a310 msgr2=0x55b967f43900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 shutdown_connections 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.278+0000 7f00d464b740 1 -- 172.21.15.16:0/3569231188 wait complete. 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d464b740 20 client.17769 trim_cache size 0 max 16384 2024-09-12T01:03:48.293 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 msgr2=0x7f00b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 0x7f00b80a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f00c4006010 tx=0x7f00c4009360 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 msgr2=0x7f00cc1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d770 tx=0x7f00bc00dc30 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 shutdown_connections 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f00b80a2cf0 0x7f00b80a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00cc108300 0x7f00cc19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00cc103780 0x7f00cc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 --2- 172.21.15.16:0/3373744982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00cc102dd0 0x7f00cc1a0430 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:48.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 >> 172.21.15.16:0/3373744982 conn(0x7f00cc0fc7d0 msgr2=0x7f00cc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 shutdown_connections 2024-09-12T01:03:48.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:48.279+0000 7f00d2272640 1 -- 172.21.15.16:0/3373744982 wait complete. 2024-09-12T01:03:48.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:48 smithi159 ceph-mon[35566]: pgmap v237: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:48 smithi016 ceph-mon[30972]: pgmap v237: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:48 smithi099 ceph-mon[35654]: pgmap v237: 97 pgs: 97 active+clean; 8.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:49.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:03:49.389 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- 172.21.15.16:0/3086280765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c102570 msgr2=0x7f4f1c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 --2- 172.21.15.16:0/3086280765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c102570 0x7f4f1c1029b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009900 tx=0x7f4f0402a480 comp rx=0 tx=0).stop 2024-09-12T01:03:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- 172.21.15.16:0/3086280765 shutdown_connections 2024-09-12T01:03:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 --2- 172.21.15.16:0/3086280765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 0x7f4f1c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 --2- 172.21.15.16:0/3086280765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c102570 0x7f4f1c1029b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 --2- 172.21.15.16:0/3086280765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c100e20 0x7f4f1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- 172.21.15.16:0/3086280765 >> 172.21.15.16:0/3086280765 conn(0x7f4f1c0fc7d0 msgr2=0x7f4f1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- 172.21.15.16:0/3086280765 shutdown_connections 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- 172.21.15.16:0/3086280765 wait complete. 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 Processor -- start 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.899+0000 7f4f231fa640 1 -- start start 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c102570 0x7f4f1c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 0x7f4f1c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c110f80 con 0x7f4f1c100e20 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c110e00 con 0x7f4f1c102ef0 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c111100 con 0x7f4f1c102570 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60840/0 (socket says 172.21.15.16:60840) 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 -- 172.21.15.16:0/667755715 learned_addr learned my addr 172.21.15.16:0/667755715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f21797640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 0x7f4f1c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c102570 0x7f4f1c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 -- 172.21.15.16:0/667755715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c102570 msgr2=0x7f4f1c19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c102570 0x7f4f1c19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 -- 172.21.15.16:0/667755715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 msgr2=0x7f4f1c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 0x7f4f1c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 -- 172.21.15.16:0/667755715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19f6c0 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f20f96640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d770 tx=0x7f4f0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f19ffb640 1 -- 172.21.15.16:0/667755715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c017070 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f19ffb640 1 -- 172.21.15.16:0/667755715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c005500 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f19ffb640 1 -- 172.21.15.16:0/667755715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004e00 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.900+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19f9b0 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.901+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a7290 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f19ffb640 1 -- 172.21.15.16:0/667755715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4f0c004fa0 con 0x7f4f1c100e20 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 Processor -- start 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 -- start start 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be5b3202f0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be5b2c7cf0 con 0x55be5b3bbb40 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f255d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be5b31c2b0 con 0x55be5b3c5370 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60852/0 (socket says 172.21.15.16:60852) 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f20f96640 1 -- 172.21.15.16:0/467864723 learned_addr learned my addr 172.21.15.16:0/467864723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.904+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f19ffb640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 0x7f4f080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f21797640 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 -- 172.21.15.16:0/467864723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 msgr2=0x55be5b3c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 -- 172.21.15.16:0/467864723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 msgr2=0x55be5b3ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 -- 172.21.15.16:0/467864723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5b2ac3a0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f21797640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 0x7f4f080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f19ffb640 1 -- 172.21.15.16:0/667755715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4f0c0ced70 con 0x7f4f1c100e20 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f20f96640 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0cc430 tx=0x7f4f0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b27ef490912051 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f12ffd640 1 -- 172.21.15.16:0/467864723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c020030 con 0x55be5b3ba5d0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f12ffd640 1 -- 172.21.15.16:0/467864723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c0046c0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f21797640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 0x7f4f080a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005fd0 tx=0x7f4f14014040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.905+0000 7f4f12ffd640 1 -- 172.21.15.16:0/467864723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0a3f00 con 0x55be5b3ba5d0 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- 172.21.15.16:0/467864723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 msgr2=0x55be5b3baa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0cc430 tx=0x7f4f0c009c30 comp rx=0 tx=0).stop 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- 172.21.15.16:0/467864723 shutdown_connections 2024-09-12T01:03:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3baa10 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 --2- 172.21.15.16:0/467864723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- 172.21.15.16:0/467864723 >> 172.21.15.16:0/467864723 conn(0x55be5b383310 msgr2=0x55be5b3b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- 172.21.15.16:0/467864723 shutdown_connections 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- 172.21.15.16:0/467864723 wait complete. 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 -- start start 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.906+0000 7f4f255d3740 1 Processor -- start 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 0x55be5b3c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be5b3c7fb0 con 0x55be5b3bbb40 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be5b2db060 con 0x55be5b3c5370 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be5b3c8130 con 0x55be5b3ba5d0 2024-09-12T01:03:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f255d3740 10 client.0 _ll_register_callbacks cb 0x55be5b02e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60860/0 (socket says 172.21.15.16:60860) 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 -- 172.21.15.16:0/2646715705 learned_addr learned my addr 172.21.15.16:0/2646715705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f21797640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 0x55be5b3c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f20f96640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 -- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 msgr2=0x55be5b3c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 -- 172.21.15.16:0/2646715705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 msgr2=0x55be5b3c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 0x55be5b3c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 -- 172.21.15.16:0/2646715705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5b0913c0 con 0x55be5b3bbb40 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f21797640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 0x55be5b3c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f20f96640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4f04000c00 tx=0x7f4f04004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f13858dd8fd9cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.907+0000 7f4efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04035040 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4f04003ac0 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be5b32d860 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f04003cc0 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 10 client.17781 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be5b3d3a00 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55be5b32d860 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f0404f6e0 con 0x55be5b3bbb40 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 10 client.17781 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 10 client.17781 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 20 client.17781 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b3d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 4 client.17781 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.908+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be5b327d70 con 0x55be5b3d4e60 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.909+0000 7f4f20f96640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b3d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.909+0000 7f4f20f96640 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b3d72e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1bfd7e2dced861a9 server_cookie=d3170e39d0f5b72 in_seq=0 out_seq=0 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.909+0000 7f4efbfff640 10 client.17781 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4efbfff640 1 -- 172.21.15.16:0/2646715705 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be5b327d70 con 0x55be5b3d4e60 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4efbfff640 10 client.17781 handle_command_reply: tid=0 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 client.17781 shutdown 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 msgr2=0x55be5b3c1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f4f04000c00 tx=0x7f4f04004440 comp rx=0 tx=0).stop 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 shutdown_connections 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3c5370 0x55be5b3c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b3d72e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4efbfff640 0 client.17781 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3bbb40 0x55be5b3c1fd0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 --2- 172.21.15.16:0/2646715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 >> 172.21.15.16:0/2646715705 conn(0x55be5b383310 msgr2=0x55be5b3bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 shutdown_connections 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 1 -- 172.21.15.16:0/2646715705 wait complete. 2024-09-12T01:03:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.910+0000 7f4f255d3740 20 client.17781 trim_cache size 0 max 16384 2024-09-12T01:03:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.924+0000 7f4f255d3740 1 Processor -- start 2024-09-12T01:03:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 -- start start 2024-09-12T01:03:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3bbb40 0x55be5b3bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1afa10 con 0x55be5b3c5370 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b1550 con 0x55be5b3ba5d0 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f255d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b00e0 con 0x55be5b3bbb40 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f21797640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f21797640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60876/0 (socket says 172.21.15.16:60876) 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f21797640 1 -- 172.21.15.16:0/3749382459 learned_addr learned my addr 172.21.15.16:0/3749382459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3bbb40 0x55be5b3bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 -- 172.21.15.16:0/3749382459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3bbb40 msgr2=0x55be5b3bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3bbb40 0x55be5b3bd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 -- 172.21.15.16:0/3749382459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 msgr2=0x55be5b3bd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f20f96640 1 -- 172.21.15.16:0/3749382459 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be5b3bdba0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.925+0000 7f4f21797640 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f20f96640 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c3f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c097d10 tx=0x7f4f0c0048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbb1d29242f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f127fc640 1 -- 172.21.15.16:0/3749382459 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c020040 con 0x55be5b3ba5d0 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f127fc640 1 -- 172.21.15.16:0/3749382459 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c0a3f00 con 0x55be5b3ba5d0 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f127fc640 1 -- 172.21.15.16:0/3749382459 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0979b0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f255d3740 1 -- 172.21.15.16:0/3749382459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 msgr2=0x55be5b3c3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f255d3740 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c3f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c097d10 tx=0x7f4f0c0048a0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.926+0000 7f4f255d3740 1 -- 172.21.15.16:0/3749382459 shutdown_connections 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3c5370 0x55be5b3bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3bbb40 0x55be5b3bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 --2- 172.21.15.16:0/3749382459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c3f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 -- 172.21.15.16:0/3749382459 >> 172.21.15.16:0/3749382459 conn(0x55be5b383310 msgr2=0x55be5b3c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 -- 172.21.15.16:0/3749382459 shutdown_connections 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 -- 172.21.15.16:0/3749382459 wait complete. 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 -- start start 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.927+0000 7f4f255d3740 1 Processor -- start 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b0e80 con 0x55be5b3ba5d0 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b07b0 con 0x55be5b3bbb40 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1ada10 con 0x55be5b3c5370 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 10 client.0 _ll_register_callbacks cb 0x7f4f04009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 -- 172.21.15.16:0/4195549908 learned_addr learned my addr 172.21.15.16:0/4195549908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f21797640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 -- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 msgr2=0x55be5b3c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 -- 172.21.15.16:0/4195549908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 msgr2=0x55be5b3c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 -- 172.21.15.16:0/4195549908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5b43edd0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f21797640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f20f96640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009610 tx=0x7f4f0c0d2ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4750e673835107a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c020040 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f4f0c0979b0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.928+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c0a3f00 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c0b4280 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 10 client.17787 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be5b4400d0 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c0a3f00 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f0c097090 con 0x55be5b3ba5d0 2024-09-12T01:03:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 10 client.17787 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 10 client.17787 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 20 client.17787 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b442360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 4 client.17787 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be5b2a4170 con 0x55be5b3d4e60 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.929+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b442360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f1bfff640 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b442360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5af913eac13266d8 server_cookie=1b689c51743b41d3 in_seq=0 out_seq=0 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4efbfff640 10 client.17787 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4efbfff640 1 -- 172.21.15.16:0/4195549908 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be5b2a4170 con 0x55be5b3d4e60 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4efbfff640 10 client.17787 handle_command_reply: tid=0 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 client.17787 shutdown 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 msgr2=0x55be5b3c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009610 tx=0x7f4f0c0d2ac0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 shutdown_connections 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be5b3c5370 0x55be5b3c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be5b3bbb40 0x55be5b3c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55be5b3d4e60 0x55be5b442360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4efbfff640 0 client.17787 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 --2- 172.21.15.16:0/4195549908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be5b3ba5d0 0x55be5b3c93d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 >> 172.21.15.16:0/4195549908 conn(0x55be5b383310 msgr2=0x55be5b3c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 shutdown_connections 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 1 -- 172.21.15.16:0/4195549908 wait complete. 2024-09-12T01:03:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.930+0000 7f4f255d3740 20 client.17787 trim_cache size 0 max 16384 2024-09-12T01:03:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 msgr2=0x7f4f080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 0x7f4f080a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f14005fd0 tx=0x7f4f14014040 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 msgr2=0x7f4f1c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00d770 tx=0x7f4f0c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 shutdown_connections 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4f080a2d70 0x7f4f080a5230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f1c102ef0 0x7f4f1c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f1c102570 0x7f4f1c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 --2- 172.21.15.16:0/667755715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f1c100e20 0x7f4f1c19a710 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 >> 172.21.15.16:0/667755715 conn(0x7f4f1c0fc7d0 msgr2=0x7f4f1c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 shutdown_connections 2024-09-12T01:03:49.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:49.931+0000 7f4f231fa640 1 -- 172.21.15.16:0/667755715 wait complete. 2024-09-12T01:03:50.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:03:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:50 smithi159 ceph-mon[35566]: pgmap v238: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:50.892 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:50 smithi016 ceph-mon[30972]: pgmap v238: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:50 smithi099 ceph-mon[35654]: pgmap v238: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-09-12T01:03:51.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 -- 172.21.15.16:0/2761585165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066010c170 msgr2=0x7f066010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 --2- 172.21.15.16:0/2761585165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066010c170 0x7f066010e610 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0658009ca0 tx=0x7f065802a950 comp rx=0 tx=0).stop 2024-09-12T01:03:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 -- 172.21.15.16:0/2761585165 shutdown_connections 2024-09-12T01:03:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 --2- 172.21.15.16:0/2761585165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f066010c170 0x7f066010e610 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 --2- 172.21.15.16:0/2761585165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0660103660 0x7f066010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.499+0000 7f0664b06640 1 --2- 172.21.15.16:0/2761585165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102d40 0x7f0660103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.500+0000 7f0664b06640 1 -- 172.21.15.16:0/2761585165 >> 172.21.15.16:0/2761585165 conn(0x7f06600fc7a0 msgr2=0x7f06600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.500+0000 7f0664b06640 1 -- 172.21.15.16:0/2761585165 shutdown_connections 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.500+0000 7f0664b06640 1 -- 172.21.15.16:0/2761585165 wait complete. 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.500+0000 7f0664b06640 1 Processor -- start 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.500+0000 7f0664b06640 1 -- start start 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660103660 0x7f06601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066010c170 0x7f066019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111f80 con 0x7f0660102d40 2024-09-12T01:03:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111e00 con 0x7f0660103660 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f0664b06640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0660112100 con 0x7f066010c170 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60892/0 (socket says 172.21.15.16:60892) 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 -- 172.21.15.16:0/2891431499 learned_addr learned my addr 172.21.15.16:0/2891431499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065dd9b640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660103660 0x7f06601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066010c170 0x7f066019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 -- 172.21.15.16:0/2891431499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066010c170 msgr2=0x7f066019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066010c170 0x7f066019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 -- 172.21.15.16:0/2891431499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660103660 msgr2=0x7f06601988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660103660 0x7f06601988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 -- 172.21.15.16:0/2891431499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019d310 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f065e59c640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f065000c920 tx=0x7f065000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.501+0000 7f064f7fe640 1 -- 172.21.15.16:0/2891431499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650007c70 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.502+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066019d600 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.502+0000 7f064f7fe640 1 -- 172.21.15.16:0/2891431499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650007e10 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.502+0000 7f064f7fe640 1 -- 172.21.15.16:0/2891431499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650018890 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.502+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06601a4ee0 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.504+0000 7f064f7fe640 1 -- 172.21.15.16:0/2891431499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0650018a70 con 0x7f0660102d40 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.504+0000 7f0666edf740 1 Processor -- start 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 -- start start 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae80a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae8b370 0x55c24ae8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c24ade62f0 con 0x55c24ae81b40 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c24ad8dcf0 con 0x55c24ae805d0 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f0666edf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c24ade22b0 con 0x55c24ae8b370 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60908/0 (socket says 172.21.15.16:60908) 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 -- 172.21.15.16:0/2346293706 learned_addr learned my addr 172.21.15.16:0/2346293706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065e59c640 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065dd9b640 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae8b370 0x55c24ae8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f064f7fe640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 0x7f06480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 -- 172.21.15.16:0/2346293706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae8b370 msgr2=0x55c24ae8d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae8b370 0x55c24ae8d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 -- 172.21.15.16:0/2346293706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 msgr2=0x55c24ae80a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae80a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.505+0000 7f065ed9d640 1 -- 172.21.15.16:0/2346293706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24ad723a0 con 0x55c24ae81b40 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 0x7f06480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f064f7fe640 1 -- 172.21.15.16:0/2891431499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f06500cf550 con 0x7f0660102d40 2024-09-12T01:03:51.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f0658009ca0 tx=0x7f065802f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6131bbfc8f56e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f064cff9640 1 -- 172.21.15.16:0/2346293706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658034040 con 0x55c24ae81b40 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f064cff9640 1 -- 172.21.15.16:0/2346293706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658003cf0 con 0x55c24ae81b40 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.506+0000 7f064cff9640 1 -- 172.21.15.16:0/2346293706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658046750 con 0x55c24ae81b40 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- 172.21.15.16:0/2346293706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 msgr2=0x55c24ae80090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 0x7f06480a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f065804d000 tx=0x7f065804a000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f0658009ca0 tx=0x7f065802f720 comp rx=0 tx=0).stop 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- 172.21.15.16:0/2346293706 shutdown_connections 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae8b370 0x55c24ae8d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae80a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- 172.21.15.16:0/2346293706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae80090 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- 172.21.15.16:0/2346293706 >> 172.21.15.16:0/2346293706 conn(0x55c24ae49310 msgr2=0x55c24ae7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- 172.21.15.16:0/2346293706 shutdown_connections 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- 172.21.15.16:0/2346293706 wait complete. 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- start start 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 Processor -- start 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c24ae8e070 con 0x55c24ae8b370 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c24ada1060 con 0x55c24ae805d0 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c24ae8e1f0 con 0x55c24ae81b40 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.507+0000 7f0666edf740 10 client.0 _ll_register_callbacks cb 0x55c24aaf48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60924/0 (socket says 172.21.15.16:60924) 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 -- 172.21.15.16:0/3476872019 learned_addr learned my addr 172.21.15.16:0/3476872019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065dd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065e59c640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae8ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 -- 172.21.15.16:0/3476872019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 msgr2=0x55c24ae87fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae87fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 -- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 msgr2=0x55c24ae8ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae8ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 -- 172.21.15.16:0/3476872019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24ab573c0 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f065ed9d640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f06580468a0 tx=0x7f0658009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318baddd4c5950a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f0645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065805a060 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0658003cf0 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.508+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f0658004040 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0658004240 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 10 client.17799 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c24ae8e070 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658003cf0 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0658030a40 con 0x55c24ae8b370 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 10 client.17799 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 10 client.17799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:51.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 20 client.17799 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24ae9ac50 0x55c24ae9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 4 client.17799 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c24adf3860 con 0x55c24ae9ac50 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.509+0000 7f065e59c640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24ae9ac50 0x55c24ae9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f065e59c640 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24ae9ac50 0x55c24ae9d040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83d262364314585e server_cookie=38868b3c3024f245 in_seq=0 out_seq=0 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0645ffb640 10 client.17799 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0645ffb640 1 -- 172.21.15.16:0/3476872019 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c24adf3860 con 0x55c24ae9ac50 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0645ffb640 10 client.17799 handle_command_reply: tid=0 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0666edf740 1 client.17799 shutdown 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 msgr2=0x55c24ae885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.510+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f06580468a0 tx=0x7f0658009800 comp rx=0 tx=0).stop 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 shutdown_connections 2024-09-12T01:03:51.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24ae9ac50 0x55c24ae9d040 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae885d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae87fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 --2- 172.21.15.16:0/3476872019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0645ffb640 0 client.17799 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 >> 172.21.15.16:0/3476872019 conn(0x55c24ae49310 msgr2=0x55c24ae82960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 shutdown_connections 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 1 -- 172.21.15.16:0/3476872019 wait complete. 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.511+0000 7f0666edf740 20 client.17799 trim_cache size 0 max 16384 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 Processor -- start 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 -- start start 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06601ad800 con 0x55c24ae8b370 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06601af340 con 0x55c24ae805d0 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f0666edf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06601aded0 con 0x55c24ae81b40 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60936/0 (socket says 172.21.15.16:60936) 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 -- 172.21.15.16:0/2415853392 learned_addr learned my addr 172.21.15.16:0/2415853392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065dd9b640 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 -- 172.21.15.16:0/2415853392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 msgr2=0x55c24ae83120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae83120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 -- 172.21.15.16:0/2415853392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 msgr2=0x55c24ae868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 -- 172.21.15.16:0/2415853392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24ae8b0a0 con 0x55c24ae8b370 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.524+0000 7f065ed9d640 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0658030de0 tx=0x7f065802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5e55d0253252b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0647fff640 1 -- 172.21.15.16:0/2415853392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065805a070 con 0x55c24ae8b370 2024-09-12T01:03:51.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0647fff640 1 -- 172.21.15.16:0/2415853392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06580098e0 con 0x55c24ae8b370 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0647fff640 1 -- 172.21.15.16:0/2415853392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658051740 con 0x55c24ae8b370 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- 172.21.15.16:0/2415853392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 msgr2=0x55c24ae83660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0658030de0 tx=0x7f065802a950 comp rx=0 tx=0).stop 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- 172.21.15.16:0/2415853392 shutdown_connections 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae8b370 0x55c24ae83660 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae81b40 0x55c24ae83120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 --2- 172.21.15.16:0/2415853392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae805d0 0x55c24ae868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- 172.21.15.16:0/2415853392 >> 172.21.15.16:0/2415853392 conn(0x55c24ae49310 msgr2=0x55c24ae8a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- 172.21.15.16:0/2415853392 shutdown_connections 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- 172.21.15.16:0/2415853392 wait complete. 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.525+0000 7f0666edf740 1 -- start start 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 Processor -- start 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 0x55c24ae8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 0x55c24af04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06601aec70 con 0x55c24ae81b40 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06601ae5a0 con 0x55c24ae8b370 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06601ab800 con 0x55c24ae805d0 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0666edf740 10 client.0 _ll_register_callbacks cb 0x7f0650097d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60952/0 (socket says 172.21.15.16:60952) 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 -- 172.21.15.16:0/797377733 learned_addr learned my addr 172.21.15.16:0/797377733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065ed9d640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 0x55c24af04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065e59c640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 0x55c24ae8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 -- 172.21.15.16:0/797377733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 msgr2=0x55c24ae8f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 0x55c24ae8f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 -- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 msgr2=0x55c24af04730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 0x55c24af04730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 -- 172.21.15.16:0/797377733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24af04c70 con 0x55c24ae81b40 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065ed9d640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 0x55c24af04730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065e59c640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 0x55c24ae8f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f065dd9b640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f066010d420 tx=0x7f065400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ac21c04f0a09a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.526+0000 7f0645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065401c470 con 0x55c24ae81b40 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f065401c610 con 0x55c24ae81b40 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06580098e0 con 0x55c24ae81b40 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f065402a050 con 0x55c24ae81b40 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 10 client.17805 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:51.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c24af05f60 con 0x55c24ae81b40 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06580098e0 con 0x55c24ae81b40 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0654011a70 con 0x55c24ae81b40 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 10 client.17805 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 20 client.17805 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24af06790 0x55c24af08bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 4 client.17805 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.527+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c24adedd70 con 0x55c24af06790 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f065e59c640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24af06790 0x55c24af08bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0645ffb640 10 client.17805 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f065e59c640 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24af06790 0x55c24af08bb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2095733de52d320 server_cookie=7cbe9a3b1806cfe7 in_seq=0 out_seq=0 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0645ffb640 1 -- 172.21.15.16:0/797377733 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c24adedd70 con 0x55c24af06790 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0645ffb640 10 client.17805 handle_command_reply: tid=0 2024-09-12T01:03:51.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 client.17805 shutdown 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 msgr2=0x55c24ae8fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f066010d420 tx=0x7f065400be60 comp rx=0 tx=0).stop 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 shutdown_connections 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55c24af06790 0x55c24af08bb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c24ae8b370 0x55c24af04730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c24ae81b40 0x55c24ae8fdc0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 --2- 172.21.15.16:0/797377733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c24ae805d0 0x55c24ae8f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 >> 172.21.15.16:0/797377733 conn(0x55c24ae49310 msgr2=0x55c24ae7b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0645ffb640 0 client.17805 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 shutdown_connections 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 1 -- 172.21.15.16:0/797377733 wait complete. 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.528+0000 7f0666edf740 20 client.17805 trim_cache size 0 max 16384 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 msgr2=0x7f06480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 0x7f06480a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f065804d000 tx=0x7f065804a000 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 msgr2=0x7f0660198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f065000c920 tx=0x7f065000cde0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 shutdown_connections 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f06480a2cf0 0x7f06480a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066010c170 0x7f066019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660103660 0x7f06601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 --2- 172.21.15.16:0/2891431499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0660102d40 0x7f0660198360 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 >> 172.21.15.16:0/2891431499 conn(0x7f06600fc7a0 msgr2=0x7f06600fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 shutdown_connections 2024-09-12T01:03:51.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:51.529+0000 7f0664b06640 1 -- 172.21.15.16:0/2891431499 wait complete. 2024-09-12T01:03:52.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:03:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:52.841 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:52 smithi016 ceph-mon[30972]: pgmap v239: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:52 smithi159 ceph-mon[35566]: pgmap v239: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:52 smithi099 ceph-mon[35654]: pgmap v239: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/351246013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102790 msgr2=0x7ff6a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/351246013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102790 0x7ff6a01067e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff688009900 tx=0x7ff68802a410 comp rx=0 tx=0).stop 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/351246013 shutdown_connections 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/351246013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 0x7ff6a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/351246013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0102790 0x7ff6a01067e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/351246013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0101de0 0x7ff6a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.065+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/351246013 >> 172.21.15.16:0/351246013 conn(0x7ff6a00fd8c0 msgr2=0x7ff6a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/351246013 shutdown_connections 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/351246013 wait complete. 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 Processor -- start 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- start start 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102790 0x7ff6a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 0x7ff6a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0111fc0 con 0x7ff6a0101de0 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0111e40 con 0x7ff6a0106f10 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.066+0000 7ff6a6ad4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a0112140 con 0x7ff6a0102790 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60978/0 (socket says 172.21.15.16:60978) 2024-09-12T01:03:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 -- 172.21.15.16:0/3868574665 learned_addr learned my addr 172.21.15.16:0/3868574665 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff69ffff640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102790 0x7ff6a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a5071640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 0x7ff6a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 -- 172.21.15.16:0/3868574665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102790 msgr2=0x7ff6a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102790 0x7ff6a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 -- 172.21.15.16:0/3868574665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 msgr2=0x7ff6a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 0x7ff6a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 -- 172.21.15.16:0/3868574665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01a68c0 con 0x7ff6a0101de0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d770 tx=0x7ff69000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff69dffb640 1 -- 172.21.15.16:0/3868574665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690017070 con 0x7ff6a0101de0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff69dffb640 1 -- 172.21.15.16:0/3868574665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff690005500 con 0x7ff6a0101de0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a01a6bb0 con 0x7ff6a0101de0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.067+0000 7ff69dffb640 1 -- 172.21.15.16:0/3868574665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690004e00 con 0x7ff6a0101de0 2024-09-12T01:03:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.068+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a01a7070 con 0x7ff6a0101de0 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff69dffb640 1 -- 172.21.15.16:0/3868574665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff690004fa0 con 0x7ff6a0101de0 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 Processor -- start 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 -- start start 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da1352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.070+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555da12b82f0 con 0x555da13525d0 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555da125fcf0 con 0x555da1353b40 2024-09-12T01:03:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555da12b42b0 con 0x555da135d370 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60986/0 (socket says 172.21.15.16:60986) 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 -- 172.21.15.16:0/3097685654 learned_addr learned my addr 172.21.15.16:0/3097685654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a5071640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da1352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff69ffff640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff69dffb640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 0x7ff68c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 -- 172.21.15.16:0/3097685654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 msgr2=0x555da135f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 -- 172.21.15.16:0/3097685654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 msgr2=0x555da1352090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da1352090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 -- 172.21.15.16:0/3097685654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da12443a0 con 0x555da13525d0 2024-09-12T01:03:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff69ffff640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a5071640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da1352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a5071640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 0x7ff68c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff69dffb640 1 -- 172.21.15.16:0/3868574665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff6900cef30 con 0x7ff6a0101de0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.071+0000 7ff6a4870640 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cc560 tx=0x7ff690022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1329fafc0593a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff696ffd640 1 -- 172.21.15.16:0/3097685654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69009af30 con 0x555da13525d0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a5071640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 0x7ff68c0a51b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff698007e90 tx=0x7ff698004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff696ffd640 1 -- 172.21.15.16:0/3097685654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6900046c0 con 0x555da13525d0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff696ffd640 1 -- 172.21.15.16:0/3097685654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69009ad50 con 0x555da13525d0 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 -- 172.21.15.16:0/3097685654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 msgr2=0x555da1352a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cc560 tx=0x7ff690022000 comp rx=0 tx=0).stop 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 -- 172.21.15.16:0/3097685654 shutdown_connections 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da1352a10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/3097685654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da1352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 -- 172.21.15.16:0/3097685654 >> 172.21.15.16:0/3097685654 conn(0x555da131b310 msgr2=0x555da134ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 -- 172.21.15.16:0/3097685654 shutdown_connections 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.072+0000 7ff6a8ead740 1 -- 172.21.15.16:0/3097685654 wait complete. 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 -- start start 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 Processor -- start 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da135a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555da135ffb0 con 0x555da13525d0 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555da1273060 con 0x555da1353b40 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555da1360130 con 0x555da135d370 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a8ead740 10 client.0 _ll_register_callbacks cb 0x555da0fc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a4870640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60998/0 (socket says 172.21.15.16:60998) 2024-09-12T01:03:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.073+0000 7ff6a4870640 1 -- 172.21.15.16:0/1250562100 learned_addr learned my addr 172.21.15.16:0/1250562100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff69ffff640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da135a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a5071640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 -- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 msgr2=0x555da135a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 -- 172.21.15.16:0/1250562100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 msgr2=0x555da135a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da135a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 -- 172.21.15.16:0/1250562100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da10293c0 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a5071640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a4870640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff69007cdd0 tx=0x7ff6900048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ebc4d583e27f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69009af30 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff6900996f0 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555da12c5860 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff69001e120 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a8ead740 10 client.17817 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555da136bac0 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.074+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555da12c5860 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6900cc8a0 con 0x555da13525d0 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 10 client.17817 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 10 client.17817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 20 client.17817 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da136ce10 0x555da136f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 4 client.17817 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555da12bfd70 con 0x555da136ce10 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.075+0000 7ff69ffff640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da136ce10 0x555da136f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff69ffff640 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da136ce10 0x555da136f200 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc81ac9558d038dc server_cookie=d1b31fc7c70def67 in_seq=0 out_seq=0 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff67ffff640 10 client.17817 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff67ffff640 1 -- 172.21.15.16:0/1250562100 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555da12bfd70 con 0x555da136ce10 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff67ffff640 10 client.17817 handle_command_reply: tid=0 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff6a8ead740 1 client.17817 shutdown 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 msgr2=0x555da135cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff69007cdd0 tx=0x7ff6900048a0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 shutdown_connections 2024-09-12T01:03:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.076+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da135d370 0x555da135a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da136ce10 0x555da136f200 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da1353b40 0x555da135a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/1250562100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da13525d0 0x555da135cdc0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 >> 172.21.15.16:0/1250562100 conn(0x555da131b310 msgr2=0x555da1354990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff67ffff640 0 client.17817 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 shutdown_connections 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 1 -- 172.21.15.16:0/1250562100 wait complete. 2024-09-12T01:03:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.077+0000 7ff6a8ead740 20 client.17817 trim_cache size 0 max 16384 2024-09-12T01:03:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 Processor -- start 2024-09-12T01:03:53.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 -- start start 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da13525d0 0x555da135a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da135d370 0x555da1355660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01af9e0 con 0x555da1353b40 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01b1520 con 0x555da135d370 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01b00b0 con 0x555da13525d0 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff69ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32774/0 (socket says 172.21.15.16:32774) 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.090+0000 7ff69ffff640 1 -- 172.21.15.16:0/2093007802 learned_addr learned my addr 172.21.15.16:0/2093007802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff6a4870640 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da13525d0 0x555da135a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff6a5071640 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da135d370 0x555da1355660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 -- 172.21.15.16:0/2093007802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da13525d0 msgr2=0x555da135a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da13525d0 0x555da135a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 -- 172.21.15.16:0/2093007802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da135d370 msgr2=0x555da1355660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da135d370 0x555da1355660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 -- 172.21.15.16:0/2093007802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da135ab70 con 0x555da1353b40 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff69ffff640 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff688031550 tx=0x7ff68802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e936ba7e6b44656c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff6967fc640 1 -- 172.21.15.16:0/2093007802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688035040 con 0x555da1353b40 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff6967fc640 1 -- 172.21.15.16:0/2093007802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68804a070 con 0x555da1353b40 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.091+0000 7ff6967fc640 1 -- 172.21.15.16:0/2093007802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688031810 con 0x555da1353b40 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2093007802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 msgr2=0x555da1355120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff688031550 tx=0x7ff68802ab90 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2093007802 shutdown_connections 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da135d370 0x555da1355660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da1353b40 0x555da1355120 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2093007802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da13525d0 0x555da135a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2093007802 >> 172.21.15.16:0/2093007802 conn(0x555da131b310 msgr2=0x555da1359930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2093007802 shutdown_connections 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2093007802 wait complete. 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 -- start start 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.092+0000 7ff6a8ead740 1 Processor -- start 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da13525d0 0x555da1365710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da1353b40 0x555da1365c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01b0e50 con 0x555da135d370 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01b0780 con 0x555da13525d0 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a01ad9e0 con 0x555da1353b40 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a8ead740 10 client.0 _ll_register_callbacks cb 0x7ff690097550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32782/0 (socket says 172.21.15.16:32782) 2024-09-12T01:03:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 -- 172.21.15.16:0/2738620979 learned_addr learned my addr 172.21.15.16:0/2738620979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a4870640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da13525d0 0x555da1365710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff69ffff640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da1353b40 0x555da1365c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 -- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da1353b40 msgr2=0x555da1365c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da1353b40 0x555da1365c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 -- 172.21.15.16:0/2738620979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da13525d0 msgr2=0x555da1365710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da13525d0 0x555da1365710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 -- 172.21.15.16:0/2738620979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da1361340 con 0x555da135d370 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff6a5071640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff698007530 tx=0x7ff698007470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ba760f7450745 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.093+0000 7ff67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980128d0 con 0x555da135d370 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7ff698012a70 con 0x555da135d370 2024-09-12T01:03:53.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff68804a070 con 0x555da135d370 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff698012c70 con 0x555da135d370 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff6a8ead740 10 client.17823 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555da13d73d0 con 0x555da135d370 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68804a070 con 0x555da135d370 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff69801b550 con 0x555da135d370 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff6a8ead740 10 client.17823 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.094+0000 7ff6a8ead740 10 client.17823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a8ead740 20 client.17823 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da13d7b00 0x555da13d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a8ead740 4 client.17823 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555da123c170 con 0x555da13d7b00 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a4870640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da13d7b00 0x555da13d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff67ffff640 10 client.17823 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a4870640 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da13d7b00 0x555da13d9f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd1b895a4142d96c server_cookie=82a74d30e0de7314 in_seq=0 out_seq=0 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff67ffff640 1 -- 172.21.15.16:0/2738620979 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555da123c170 con 0x555da13d7b00 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff67ffff640 10 client.17823 handle_command_reply: tid=0 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.095+0000 7ff6a8ead740 1 client.17823 shutdown 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 msgr2=0x555da1360b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff698007530 tx=0x7ff698007470 comp rx=0 tx=0).stop 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 shutdown_connections 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555da135d370 0x555da1360b10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555da1353b40 0x555da1365c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x555da13d7b00 0x555da13d9f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 --2- 172.21.15.16:0/2738620979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555da13525d0 0x555da1365710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 >> 172.21.15.16:0/2738620979 conn(0x555da131b310 msgr2=0x555da1368240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff67ffff640 0 client.17823 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 shutdown_connections 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 1 -- 172.21.15.16:0/2738620979 wait complete. 2024-09-12T01:03:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.096+0000 7ff6a8ead740 20 client.17823 trim_cache size 0 max 16384 2024-09-12T01:03:53.104 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 msgr2=0x7ff68c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 0x7ff68c0a51b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff698007e90 tx=0x7ff698004a40 comp rx=0 tx=0).stop 2024-09-12T01:03:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 msgr2=0x7ff6a00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff69000d770 tx=0x7ff69000dc30 comp rx=0 tx=0).stop 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 shutdown_connections 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff68c0a2cf0 0x7ff68c0a51b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6a0106f10 0x7ff6a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6a0102790 0x7ff6a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 --2- 172.21.15.16:0/3868574665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a0101de0 0x7ff6a00744f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 >> 172.21.15.16:0/3868574665 conn(0x7ff6a00fd8c0 msgr2=0x7ff6a00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 shutdown_connections 2024-09-12T01:03:53.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:53.097+0000 7ff6a6ad4640 1 -- 172.21.15.16:0/3868574665 wait complete. 2024-09-12T01:03:53.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:03:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 -- 172.21.15.16:0/3431732988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c101420 msgr2=0x7fe31c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 --2- 172.21.15.16:0/3431732988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c101420 0x7fe31c10d950 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe314009cd0 tx=0x7fe31402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 -- 172.21.15.16:0/3431732988 shutdown_connections 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 --2- 172.21.15.16:0/3431732988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c101420 0x7fe31c10d950 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 --2- 172.21.15.16:0/3431732988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 0x7fe31c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 --2- 172.21.15.16:0/3431732988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c106a80 0x7fe31c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 -- 172.21.15.16:0/3431732988 >> 172.21.15.16:0/3431732988 conn(0x7fe31c0fc7a0 msgr2=0x7fe31c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 -- 172.21.15.16:0/3431732988 shutdown_connections 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.645+0000 7fe322a8b640 1 -- 172.21.15.16:0/3431732988 wait complete. 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 Processor -- start 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 -- start start 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 0x7fe31c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101420 0x7fe31c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c110e80 con 0x7fe31c106a80 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c110d00 con 0x7fe31c100a80 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe322a8b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c111000 con 0x7fe31c101420 2024-09-12T01:03:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe321028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe321028640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe321028640 1 -- 172.21.15.16:0/1660875789 learned_addr learned my addr 172.21.15.16:0/1660875789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe31bfff640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101420 0x7fe31c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.646+0000 7fe320827640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 0x7fe31c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 -- 172.21.15.16:0/1660875789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101420 msgr2=0x7fe31c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101420 0x7fe31c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 -- 172.21.15.16:0/1660875789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 msgr2=0x7fe31c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 0x7fe31c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 -- 172.21.15.16:0/1660875789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c19d3a0 con 0x7fe31c106a80 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe321028640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe314005cc0 tx=0x7fe314003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe319ffb640 1 -- 172.21.15.16:0/1660875789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004040 con 0x7fe31c106a80 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe319ffb640 1 -- 172.21.15.16:0/1660875789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3140046c0 con 0x7fe31c106a80 2024-09-12T01:03:54.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c19d630 con 0x7fe31c106a80 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe319ffb640 1 -- 172.21.15.16:0/1660875789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314046480 con 0x7fe31c106a80 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.647+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c1a4f70 con 0x7fe31c106a80 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.650+0000 7fe319ffb640 1 -- 172.21.15.16:0/1660875789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe3140305a0 con 0x7fe31c106a80 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.650+0000 7fe324e64740 1 Processor -- start 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.650+0000 7fe324e64740 1 -- start start 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e1633faa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe319ffb640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 0x7fe30c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163407760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e1633602f0 con 0x55e1633fbb40 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e163307cf0 con 0x55e163405370 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe324e64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e16335c2b0 con 0x55e1633fa5d0 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32808/0 (socket says 172.21.15.16:32808) 2024-09-12T01:03:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 -- 172.21.15.16:0/3716661318 learned_addr learned my addr 172.21.15.16:0/3716661318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 0x7fe30c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe319ffb640 1 -- 172.21.15.16:0/1660875789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe31402e030 con 0x7fe31c106a80 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe321028640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163407760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe320827640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e1633faa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 -- 172.21.15.16:0/3716661318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 msgr2=0x55e1633faa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e1633faa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 -- 172.21.15.16:0/3716661318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 msgr2=0x55e163407760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163407760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe31bfff640 1 -- 172.21.15.16:0/3716661318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1632ec3a0 con 0x55e1633fbb40 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.651+0000 7fe321028640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163407760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe308012ab0 tx=0x7fe308012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f68917736ababeb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe31bfff640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 0x7fe30c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3080058a0 tx=0x7fe308017000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe306ffd640 1 -- 172.21.15.16:0/3716661318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308023070 con 0x55e1633fbb40 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe306ffd640 1 -- 172.21.15.16:0/3716661318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe308012590 con 0x55e1633fbb40 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe306ffd640 1 -- 172.21.15.16:0/3716661318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30801f6e0 con 0x55e1633fbb40 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe324e64740 1 -- 172.21.15.16:0/3716661318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 msgr2=0x55e1633fa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.652+0000 7fe324e64740 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe308012ab0 tx=0x7fe308012f70 comp rx=0 tx=0).stop 2024-09-12T01:03:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 -- 172.21.15.16:0/3716661318 shutdown_connections 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163407760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e1633faa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 --2- 172.21.15.16:0/3716661318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fa090 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 -- 172.21.15.16:0/3716661318 >> 172.21.15.16:0/3716661318 conn(0x55e1633c3310 msgr2=0x55e1633f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 -- 172.21.15.16:0/3716661318 shutdown_connections 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 -- 172.21.15.16:0/3716661318 wait complete. 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 -- start start 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 Processor -- start 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.653+0000 7fe324e64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e163404dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163402800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e163408010 con 0x55e1633fbb40 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e16331b060 con 0x55e163405370 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e163408190 con 0x55e1633fa5d0 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 10 client.0 _ll_register_callbacks cb 0x55e16306e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32816/0 (socket says 172.21.15.16:32816) 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 -- 172.21.15.16:0/833158361 learned_addr learned my addr 172.21.15.16:0/833158361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe320827640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e163404dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe321028640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163402800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 -- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 msgr2=0x55e163404dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e163404dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 -- 172.21.15.16:0/833158361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 msgr2=0x55e163402800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163402800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 -- 172.21.15.16:0/833158361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1630d13c0 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe31bfff640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe308005cd0 tx=0x7fe30801f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d0f50ae9ceed46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30802c020 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe30801fc20 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe308012590 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe308037050 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 10 client.17835 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.654+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e163413b70 con 0x55e1633fbb40 2024-09-12T01:03:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe308012590 con 0x55e1633fbb40 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe30800dbf0 con 0x55e1633fbb40 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 10 client.17835 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 20 client.17835 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e163415030 0x55e163417420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 4 client.17835 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e16336d860 con 0x55e163415030 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.655+0000 7fe320827640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e163415030 0x55e163417420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe2fbfff640 10 client.17835 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe320827640 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e163415030 0x55e163417420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec58ff025c3bc2cb server_cookie=ae5f2d839bd03290 in_seq=0 out_seq=0 2024-09-12T01:03:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe2fbfff640 1 -- 172.21.15.16:0/833158361 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e16336d860 con 0x55e163415030 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe2fbfff640 10 client.17835 handle_command_reply: tid=0 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 client.17835 shutdown 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 msgr2=0x55e163402070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe308005cd0 tx=0x7fe30801f6b0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 shutdown_connections 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e163415030 0x55e163417420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e163405370 0x55e163402800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e163402070 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 --2- 172.21.15.16:0/833158361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1633fa5d0 0x55e163404dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 >> 172.21.15.16:0/833158361 conn(0x55e1633c3310 msgr2=0x55e1633fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.656+0000 7fe2fbfff640 0 client.17835 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.657+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 shutdown_connections 2024-09-12T01:03:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.657+0000 7fe324e64740 1 -- 172.21.15.16:0/833158361 wait complete. 2024-09-12T01:03:54.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.657+0000 7fe324e64740 20 client.17835 trim_cache size 0 max 16384 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 Processor -- start 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 -- start start 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fa5d0 0x55e1633fd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e16341cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1ad750 con 0x55e1633fbb40 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1af290 con 0x55e1633fa5d0 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe324e64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1ade20 con 0x55e163405370 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe31bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32824/0 (socket says 172.21.15.16:32824) 2024-09-12T01:03:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe31bfff640 1 -- 172.21.15.16:0/3409023199 learned_addr learned my addr 172.21.15.16:0/3409023199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.670+0000 7fe321028640 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e16341cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe320827640 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fa5d0 0x55e1633fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 -- 172.21.15.16:0/3409023199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 msgr2=0x55e16341cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e16341cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 -- 172.21.15.16:0/3409023199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fa5d0 msgr2=0x55e1633fd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fa5d0 0x55e1633fd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 -- 172.21.15.16:0/3409023199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e163402b70 con 0x55e1633fbb40 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe308005d00 tx=0x7fe308012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23104cecfa142ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe3067fc640 1 -- 172.21.15.16:0/3409023199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308040070 con 0x55e1633fbb40 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe3067fc640 1 -- 172.21.15.16:0/3409023199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30800bc50 con 0x55e1633fbb40 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.671+0000 7fe3067fc640 1 -- 172.21.15.16:0/3409023199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30802c040 con 0x55e1633fbb40 2024-09-12T01:03:54.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- 172.21.15.16:0/3409023199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 msgr2=0x55e1633fd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe308005d00 tx=0x7fe308012f90 comp rx=0 tx=0).stop 2024-09-12T01:03:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- 172.21.15.16:0/3409023199 shutdown_connections 2024-09-12T01:03:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e16341cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fbb40 0x55e1633fd660 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 --2- 172.21.15.16:0/3409023199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fa5d0 0x55e1633fd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- 172.21.15.16:0/3409023199 >> 172.21.15.16:0/3409023199 conn(0x55e1633c3310 msgr2=0x55e1633f5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- 172.21.15.16:0/3409023199 shutdown_connections 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- 172.21.15.16:0/3409023199 wait complete. 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 -- start start 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.672+0000 7fe324e64740 1 Processor -- start 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fbb40 0x55e16340dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e1633fec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1aebc0 con 0x55e1633fa5d0 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1ae4f0 con 0x55e1633fbb40 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c1ab750 con 0x55e163405370 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 10 client.0 _ll_register_callbacks cb 0x7fe308012880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32834/0 (socket says 172.21.15.16:32834) 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 -- 172.21.15.16:0/3602518118 learned_addr learned my addr 172.21.15.16:0/3602518118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fbb40 0x55e16340dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe321028640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e1633fec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 -- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 msgr2=0x55e1633fec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e1633fec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 -- 172.21.15.16:0/3602518118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fbb40 msgr2=0x55e16340dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fbb40 0x55e16340dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 -- 172.21.15.16:0/3602518118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1633ff450 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe320827640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe31000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba12f54b7d035661 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe2fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310003d30 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fe310003ed0 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.673+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe30800bc50 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe310024470 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 10 client.17841 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e16347f760 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30800bc50 con 0x55e1633fa5d0 2024-09-12T01:03:54.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe310012880 con 0x55e1633fa5d0 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 10 client.17841 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 20 client.17841 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e16347fe80 0x55e163482270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 4 client.17841 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e163367d70 con 0x55e16347fe80 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.674+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e16347fe80 0x55e163482270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe2fbfff640 10 client.17841 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe31bfff640 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e16347fe80 0x55e163482270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d1d8799c722962 server_cookie=60e2bb8987833ddf in_seq=0 out_seq=0 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe2fbfff640 1 -- 172.21.15.16:0/3602518118 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e163367d70 con 0x55e16347fe80 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe2fbfff640 10 client.17841 handle_command_reply: tid=0 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 client.17841 shutdown 2024-09-12T01:03:54.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 msgr2=0x55e16340d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe31000bbb0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 shutdown_connections 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e16347fe80 0x55e163482270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e163405370 0x55e1633fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e1633fbb40 0x55e16340dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 --2- 172.21.15.16:0/3602518118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1633fa5d0 0x55e16340d790 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 >> 172.21.15.16:0/3602518118 conn(0x55e1633c3310 msgr2=0x55e163400850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe2fbfff640 0 client.17841 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 shutdown_connections 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 1 -- 172.21.15.16:0/3602518118 wait complete. 2024-09-12T01:03:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.675+0000 7fe324e64740 20 client.17841 trim_cache size 0 max 16384 2024-09-12T01:03:54.689 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:03:54.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 msgr2=0x7fe30c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 0x7fe30c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3080058a0 tx=0x7fe308017000 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 msgr2=0x7fe31c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe314005cc0 tx=0x7fe314003970 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 shutdown_connections 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe30c0a2cf0 0x7fe30c0a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe31c106a80 0x7fe31c19ccc0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe31c101420 0x7fe31c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 --2- 172.21.15.16:0/1660875789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe31c100a80 0x7fe31c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 >> 172.21.15.16:0/1660875789 conn(0x7fe31c0fc7a0 msgr2=0x7fe31c06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 shutdown_connections 2024-09-12T01:03:54.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:54.676+0000 7fe322a8b640 1 -- 172.21.15.16:0/1660875789 wait complete. 2024-09-12T01:03:54.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:54 smithi159 ceph-mon[35566]: pgmap v240: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:54.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:54 smithi016 ceph-mon[30972]: pgmap v240: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:54 smithi099 ceph-mon[35654]: pgmap v240: 97 pgs: 97 active+clean; 9.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-09-12T01:03:56.338 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:03:56.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:03:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:56.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:56 smithi016 ceph-mon[30972]: pgmap v241: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-12T01:03:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:56 smithi159 ceph-mon[35566]: pgmap v241: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-12T01:03:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:56 smithi099 ceph-mon[35654]: pgmap v241: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 -- 172.21.15.16:0/1468673915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102570 msgr2=0x7f11c81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 --2- 172.21.15.16:0/1468673915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102570 0x7f11c81029b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b802a480 comp rx=0 tx=0).stop 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 -- 172.21.15.16:0/1468673915 shutdown_connections 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 --2- 172.21.15.16:0/1468673915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8102ef0 0x7f11c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 --2- 172.21.15.16:0/1468673915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102570 0x7f11c81029b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 --2- 172.21.15.16:0/1468673915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8100e20 0x7f11c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 -- 172.21.15.16:0/1468673915 >> 172.21.15.16:0/1468673915 conn(0x7f11c80fc7d0 msgr2=0x7f11c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 -- 172.21.15.16:0/1468673915 shutdown_connections 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.111+0000 7f11d00b2640 1 -- 172.21.15.16:0/1468673915 wait complete. 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 Processor -- start 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 -- start start 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 0x7f11c8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 0x7f11c81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8110fa0 con 0x7f11c8102ef0 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8110e20 con 0x7f11c8100e20 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11d00b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8111120 con 0x7f11c8102570 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11ce64f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11ce64f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32848/0 (socket says 172.21.15.16:32848) 2024-09-12T01:03:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.112+0000 7f11ce64f640 1 -- 172.21.15.16:0/353349831 learned_addr learned my addr 172.21.15.16:0/353349831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11cd64d640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 0x7f11c81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11cde4e640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 0x7f11c8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 -- 172.21.15.16:0/353349831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 msgr2=0x7f11c81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 0x7f11c81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 -- 172.21.15.16:0/353349831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 msgr2=0x7f11c8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 0x7f11c8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 -- 172.21.15.16:0/353349831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81a4950 con 0x7f11c8102ef0 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11cd64d640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 0x7f11c81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11cde4e640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 0x7f11c8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11ce64f640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b970 tx=0x7f11c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11beffd640 1 -- 172.21.15.16:0/353349831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0019070 con 0x7f11c8102ef0 2024-09-12T01:03:57.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11beffd640 1 -- 172.21.15.16:0/353349831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c000cb40 con 0x7f11c8102ef0 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11beffd640 1 -- 172.21.15.16:0/353349831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0007650 con 0x7f11c8102ef0 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81a4b20 con 0x7f11c8102ef0 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.113+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c81a5030 con 0x7f11c8102ef0 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11beffd640 1 -- 172.21.15.16:0/353349831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f11c001c070 con 0x7f11c8102ef0 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 Processor -- start 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 -- start start 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f6af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b88ed0280 con 0x555b88f75d40 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b88e77b10 con 0x555b88f6b480 2024-09-12T01:03:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11d248b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555b88ecc240 con 0x555b88f6ab60 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11beffd640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 0x7f11b40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32862/0 (socket says 172.21.15.16:32862) 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.116+0000 7f11cd64d640 1 -- 172.21.15.16:0/474874379 learned_addr learned my addr 172.21.15.16:0/474874379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cde4e640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f6af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11ce64f640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 0x7f11b40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11beffd640 1 -- 172.21.15.16:0/353349831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f11c00ce940 con 0x7f11c8102ef0 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 -- 172.21.15.16:0/474874379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 msgr2=0x555b88f6af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f6af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 -- 172.21.15.16:0/474874379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 msgr2=0x555b88f75800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f75800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 -- 172.21.15.16:0/474874379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b88e841f0 con 0x555b88f75d40 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cde4e640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11ce64f640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 0x7f11b40a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11c00a36b0 tx=0x7f11c000e040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11cd64d640 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f11b80314e0 tx=0x7f11b8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2151ff44998af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11b3fff640 1 -- 172.21.15.16:0/474874379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8035050 con 0x555b88f75d40 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11b3fff640 1 -- 172.21.15.16:0/474874379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b80435f0 con 0x555b88f75d40 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.117+0000 7f11b3fff640 1 -- 172.21.15.16:0/474874379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8041cf0 con 0x555b88f75d40 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 -- 172.21.15.16:0/474874379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 msgr2=0x555b88f78130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f11b80314e0 tx=0x7f11b8005b50 comp rx=0 tx=0).stop 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 -- 172.21.15.16:0/474874379 shutdown_connections 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f78130 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 --2- 172.21.15.16:0/474874379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f6af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 -- 172.21.15.16:0/474874379 >> 172.21.15.16:0/474874379 conn(0x555b88f670f0 msgr2=0x555b88f674d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 -- 172.21.15.16:0/474874379 shutdown_connections 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.118+0000 7f11d248b740 1 -- 172.21.15.16:0/474874379 wait complete. 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 -- start start 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 Processor -- start 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f73150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555b88f78910 con 0x555b88f75d40 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b88e8af10 con 0x555b88f6b480 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555b88f78a90 con 0x555b88f6ab60 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11d248b740 10 client.0 _ll_register_callbacks cb 0x555b88e53a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32878/0 (socket says 172.21.15.16:32878) 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11cd64d640 1 -- 172.21.15.16:0/3347729664 learned_addr learned my addr 172.21.15.16:0/3347729664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11cd64d640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.119+0000 7f11cde4e640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f73150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 -- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 msgr2=0x555b88f72c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f72c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 -- 172.21.15.16:0/3347729664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 msgr2=0x555b88f73150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f73150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 -- 172.21.15.16:0/3347729664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b88c416a0 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11cd64d640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f11b80096c0 tx=0x7f11b8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a263f03d94d59e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8057060 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f11b8043310 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b80435f0 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11b8035070 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11d248b740 10 client.17853 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b88f84540 con 0x555b88f75d40 2024-09-12T01:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b80435f0 con 0x555b88f75d40 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.120+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f11b8031660 con 0x555b88f75d40 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 10 client.17853 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 20 client.17853 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88f85890 0x555b88f87c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 4 client.17853 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b88edd7f0 con 0x555b88f85890 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11cde4e640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88f85890 0x555b88f87c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11cde4e640 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88f85890 0x555b88f87c80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2692005bdc6397cf server_cookie=d07c1d4bc6e1c250 in_seq=0 out_seq=0 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.121+0000 7f11b17fa640 10 client.17853 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11b17fa640 1 -- 172.21.15.16:0/3347729664 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b88edd7f0 con 0x555b88f85890 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11b17fa640 10 client.17853 handle_command_reply: tid=0 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 client.17853 shutdown 2024-09-12T01:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 msgr2=0x555b88f73690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f11b80096c0 tx=0x7f11b8005c90 comp rx=0 tx=0).stop 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 shutdown_connections 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88f85890 0x555b88f87c80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f73690 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11b17fa640 0 client.17853 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f73150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 --2- 172.21.15.16:0/3347729664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.122+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 >> 172.21.15.16:0/3347729664 conn(0x555b88f670f0 msgr2=0x555b88f6cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.123+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 shutdown_connections 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.123+0000 7f11d248b740 1 -- 172.21.15.16:0/3347729664 wait complete. 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.123+0000 7f11d248b740 20 client.17853 trim_cache size 0 max 16384 2024-09-12T01:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 Processor -- start 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 -- start start 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6b480 0x555b88f703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f75d40 0x555b88f6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81ab690 con 0x555b88f6ab60 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81ad1d0 con 0x555b88f75d40 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11d248b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81abd60 con 0x555b88f6b480 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32894/0 (socket says 172.21.15.16:32894) 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 -- 172.21.15.16:0/2885676335 learned_addr learned my addr 172.21.15.16:0/2885676335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cde4e640 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6b480 0x555b88f703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 -- 172.21.15.16:0/2885676335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6b480 msgr2=0x555b88f703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6b480 0x555b88f703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 -- 172.21.15.16:0/2885676335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f75d40 msgr2=0x555b88f6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f75d40 0x555b88f6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 -- 172.21.15.16:0/2885676335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b88f754e0 con 0x555b88f6ab60 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11cd64d640 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c308f4cbf86d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.141+0000 7f11b37fe640 1 -- 172.21.15.16:0/2885676335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8057070 con 0x555b88f6ab60 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11b37fe640 1 -- 172.21.15.16:0/2885676335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8031c00 con 0x555b88f6ab60 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11b37fe640 1 -- 172.21.15.16:0/2885676335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8005020 con 0x555b88f6ab60 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 -- 172.21.15.16:0/2885676335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 msgr2=0x555b88f6fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 -- 172.21.15.16:0/2885676335 shutdown_connections 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f75d40 0x555b88f6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6b480 0x555b88f703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 --2- 172.21.15.16:0/2885676335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f6ab60 0x555b88f6fe70 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 -- 172.21.15.16:0/2885676335 >> 172.21.15.16:0/2885676335 conn(0x555b88f670f0 msgr2=0x555b88f74b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 -- 172.21.15.16:0/2885676335 shutdown_connections 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.142+0000 7f11d248b740 1 -- 172.21.15.16:0/2885676335 wait complete. 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 -- start start 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 Processor -- start 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f88bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81acb00 con 0x555b88f75d40 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81ac430 con 0x555b88f6b480 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c81a9690 con 0x555b88f6ab60 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11d248b740 10 client.0 _ll_register_callbacks cb 0x7f11c400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.143+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32898/0 (socket says 172.21.15.16:32898) 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 -- 172.21.15.16:0/1125663267 learned_addr learned my addr 172.21.15.16:0/1125663267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f88bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cde4e640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 -- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 msgr2=0x555b88f88bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f88bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 -- 172.21.15.16:0/1125663267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 msgr2=0x555b88f890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 -- 172.21.15.16:0/1125663267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b88fef370 con 0x555b88f75d40 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11cd64d640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f11b80314e0 tx=0x7f11b8043aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2015d1d470b0517 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8057070 con 0x555b88f75d40 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f11b8031540 con 0x555b88f75d40 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b8031c00 con 0x555b88f75d40 2024-09-12T01:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11b8043e20 con 0x555b88f75d40 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11d248b740 10 client.17859 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.144+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b88ff0670 con 0x555b88f75d40 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8031c00 con 0x555b88f75d40 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f11b8004830 con 0x555b88f75d40 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 10 client.17859 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 20 client.17859 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88ff0ae0 0x555b88ff2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 4 client.17859 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555b88ed7d00 con 0x555b88ff0ae0 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.145+0000 7f11cde4e640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88ff0ae0 0x555b88ff2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.146+0000 7f11b17fa640 10 client.17859 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.146+0000 7f11cde4e640 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88ff0ae0 0x555b88ff2ed0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b602a1c9f58fea47 server_cookie=4f56869eb5e75d4b in_seq=0 out_seq=0 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11b17fa640 1 -- 172.21.15.16:0/1125663267 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555b88ed7d00 con 0x555b88ff0ae0 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11b17fa640 10 client.17859 handle_command_reply: tid=0 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 client.17859 shutdown 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 msgr2=0x555b88f89630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f11b80314e0 tx=0x7f11b8043aa0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 shutdown_connections 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x555b88ff0ae0 0x555b88ff2ed0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555b88f75d40 0x555b88f89630 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b88f6b480 0x555b88f890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 --2- 172.21.15.16:0/1125663267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f6ab60 0x555b88f88bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11b17fa640 0 client.17859 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 >> 172.21.15.16:0/1125663267 conn(0x555b88f670f0 msgr2=0x555b88f65a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 shutdown_connections 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 1 -- 172.21.15.16:0/1125663267 wait complete. 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.147+0000 7f11d248b740 20 client.17859 trim_cache size 0 max 16384 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c94849ba-4696-4edf-860b-8b8eae5f7776", 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:03:57.166 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 msgr2=0x7f11b40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 0x7f11b40a5230 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11c00a36b0 tx=0x7f11c000e040 comp rx=0 tx=0).stop 2024-09-12T01:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 msgr2=0x7f11c81a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f11c000b970 tx=0x7f11c000be30 comp rx=0 tx=0).stop 2024-09-12T01:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 shutdown_connections 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f11b40a2d70 0x7f11b40a5230 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c8102ef0 0x7f11c81a4410 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c8102570 0x7f11c81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 --2- 172.21.15.16:0/353349831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c8100e20 0x7f11c8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 >> 172.21.15.16:0/353349831 conn(0x7f11c80fc7d0 msgr2=0x7f11c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 shutdown_connections 2024-09-12T01:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:57.148+0000 7f11d00b2640 1 -- 172.21.15.16:0/353349831 wait complete. 2024-09-12T01:03:57.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:03:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:57 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:03:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:57 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:57 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:03:57.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:57 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:57.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:57 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:03:57.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:57 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:57.874 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:57 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:03:57.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:57 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:57.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:57 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:03:57.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:57 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:57.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:57 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:03:57.925 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:57 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:03:58.107 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 -- 172.21.15.16:0/859887371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c101de0 msgr2=0x7fb93c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 --2- 172.21.15.16:0/859887371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c101de0 0x7fb93c1021c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb938009900 tx=0x7fb93802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 -- 172.21.15.16:0/859887371 shutdown_connections 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 --2- 172.21.15.16:0/859887371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c106f10 0x7fb93c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 --2- 172.21.15.16:0/859887371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c102790 0x7fb93c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 --2- 172.21.15.16:0/859887371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c101de0 0x7fb93c1021c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 -- 172.21.15.16:0/859887371 >> 172.21.15.16:0/859887371 conn(0x7fb93c0fd8c0 msgr2=0x7fb93c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 -- 172.21.15.16:0/859887371 shutdown_connections 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.606+0000 7fb943751640 1 -- 172.21.15.16:0/859887371 wait complete. 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 Processor -- start 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 -- start start 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c101de0 0x7fb93c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c102790 0x7fb93c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c111f30 con 0x7fb93c106f10 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c111db0 con 0x7fb93c101de0 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb943751640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1120b0 con 0x7fb93c102790 2024-09-12T01:03:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb941cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb941cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49706/0 (socket says 172.21.15.16:49706) 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.607+0000 7fb941cee640 1 -- 172.21.15.16:0/2610152927 learned_addr learned my addr 172.21.15.16:0/2610152927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb9414ed640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c101de0 0x7fb93c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb940cec640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c102790 0x7fb93c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 -- 172.21.15.16:0/2610152927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c102790 msgr2=0x7fb93c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c102790 0x7fb93c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 -- 172.21.15.16:0/2610152927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c101de0 msgr2=0x7fb93c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c101de0 0x7fb93c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 -- 172.21.15.16:0/2610152927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c19d3e0 con 0x7fb93c106f10 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb941cee640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cad0 tx=0x7fb93400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb9327fc640 1 -- 172.21.15.16:0/2610152927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93400ad60 con 0x7fb93c106f10 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb9327fc640 1 -- 172.21.15.16:0/2610152927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb934005020 con 0x7fb93c106f10 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c19d6d0 con 0x7fb93c106f10 2024-09-12T01:03:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb9327fc640 1 -- 172.21.15.16:0/2610152927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93400f6b0 con 0x7fb93c106f10 2024-09-12T01:03:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.608+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c1a4fb0 con 0x7fb93c106f10 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.611+0000 7fb9327fc640 1 -- 172.21.15.16:0/2610152927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb934005560 con 0x7fb93c106f10 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.611+0000 7fb945b2a740 1 Processor -- start 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.611+0000 7fb945b2a740 1 -- start start 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fa8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7f0e2e0 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7eb5ce0 con 0x55b1b7fb3680 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb945b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7f0a2a0 con 0x55b1b7fa84a0 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9327fc640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 0x7fb9280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49714/0 (socket says 172.21.15.16:49714) 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 -- 172.21.15.16:0/3061047263 learned_addr learned my addr 172.21.15.16:0/3061047263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb940cec640 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9327fc640 1 -- 172.21.15.16:0/2610152927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb9340cf390 con 0x7fb93c106f10 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb940cec640 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fa8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb941cee640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 0x7fb9280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 -- 172.21.15.16:0/3061047263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 msgr2=0x55b1b7fa8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fa8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 -- 172.21.15.16:0/3061047263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fb3680 msgr2=0x55b1b7fb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.612+0000 7fb9414ed640 1 -- 172.21.15.16:0/3061047263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b7e9a440 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.613+0000 7fb941cee640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 0x7fb9280a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb9340a3f90 tx=0x7fb93400a7d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.613+0000 7fb9414ed640 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb938009900 tx=0x7fb9380316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4caaef89a5bc40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.613+0000 7fb9277fe640 1 -- 172.21.15.16:0/3061047263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938040070 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.613+0000 7fb9277fe640 1 -- 172.21.15.16:0/3061047263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb938005020 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.613+0000 7fb9277fe640 1 -- 172.21.15.16:0/3061047263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938035040 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- 172.21.15.16:0/3061047263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 msgr2=0x55b1b7fb3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb938009900 tx=0x7fb9380316e0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- 172.21.15.16:0/3061047263 shutdown_connections 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb3140 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 --2- 172.21.15.16:0/3061047263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fa8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- 172.21.15.16:0/3061047263 >> 172.21.15.16:0/3061047263 conn(0x55b1b7fa4a30 msgr2=0x55b1b7fa4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- 172.21.15.16:0/3061047263 shutdown_connections 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- 172.21.15.16:0/3061047263 wait complete. 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 -- start start 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.614+0000 7fb945b2a740 1 Processor -- start 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7fb6250 con 0x55b1b7fb3680 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7ec9050 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1b7fb63d0 con 0x55b1b7fa84a0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb945b2a740 10 client.0 _ll_register_callbacks cb 0x55b1b7f09160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49728/0 (socket says 172.21.15.16:49728) 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 -- 172.21.15.16:0/839490718 learned_addr learned my addr 172.21.15.16:0/839490718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb9414ed640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 -- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 msgr2=0x55b1b7fb2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 -- 172.21.15.16:0/839490718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 msgr2=0x55b1b7fb01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 -- 172.21.15.16:0/839490718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b7c7f3c0 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb940cec640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005bd0 tx=0x7fb92c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a9d4ca430f974 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.615+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c005020 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb92c0051c0 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb938005020 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb92c0053c0 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 10 client.17871 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b7fc1ba0 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb938005020 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb92c01d050 con 0x55b1b7fb3680 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 10 client.17871 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 20 client.17871 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fc3060 0x55b1b7fc54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 4 client.17871 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:03:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.616+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1b7f1b850 con 0x55b1b7fc3060 2024-09-12T01:03:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.617+0000 7fb9414ed640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fc3060 0x55b1b7fc54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.617+0000 7fb924ff9640 10 client.17871 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.617+0000 7fb9414ed640 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fc3060 0x55b1b7fc54b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f59e4672ff85bc server_cookie=cb7a31d5d8d64b11 in_seq=0 out_seq=0 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb924ff9640 1 -- 172.21.15.16:0/839490718 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1b7f1b850 con 0x55b1b7fc3060 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb924ff9640 10 client.17871 handle_command_reply: tid=0 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 client.17871 shutdown 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 msgr2=0x55b1b7fb0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005bd0 tx=0x7fb92c004730 comp rx=0 tx=0).stop 2024-09-12T01:03:58.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 shutdown_connections 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fc3060 0x55b1b7fc54b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fb3680 0x55b1b7fb0730 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 --2- 172.21.15.16:0/839490718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 >> 172.21.15.16:0/839490718 conn(0x55b1b7fa4a30 msgr2=0x55b1b7faa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb924ff9640 0 client.17871 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 shutdown_connections 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 1 -- 172.21.15.16:0/839490718 wait complete. 2024-09-12T01:03:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.618+0000 7fb945b2a740 20 client.17871 trim_cache size 0 max 16384 2024-09-12T01:03:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 Processor -- start 2024-09-12T01:03:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 -- start start 2024-09-12T01:03:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fb2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fcb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1ad870 con 0x55b1b7fa84a0 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1af3b0 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb945b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1adf40 con 0x55b1b7fa4a30 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49740/0 (socket says 172.21.15.16:49740) 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 -- 172.21.15.16:0/305954662 learned_addr learned my addr 172.21.15.16:0/305954662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb940cec640 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fcb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb940cec640 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fb2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 -- 172.21.15.16:0/305954662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa4a30 msgr2=0x55b1b7fb2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fb2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 -- 172.21.15.16:0/305954662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 msgr2=0x55b1b7fcb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fcb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 -- 172.21.15.16:0/305954662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b7fa44d0 con 0x55b1b7fa84a0 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.636+0000 7fb9414ed640 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x55b1b7fa5890 tx=0x7fb938005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef1125dae1ca6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb926ffd640 1 -- 172.21.15.16:0/305954662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938055070 con 0x55b1b7fa84a0 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb926ffd640 1 -- 172.21.15.16:0/305954662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9380044a0 con 0x55b1b7fa84a0 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb926ffd640 1 -- 172.21.15.16:0/305954662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938040040 con 0x55b1b7fa84a0 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 -- 172.21.15.16:0/305954662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 msgr2=0x55b1b7fcad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x55b1b7fa5890 tx=0x7fb938005fd0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 -- 172.21.15.16:0/305954662 shutdown_connections 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fcb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fcad80 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 --2- 172.21.15.16:0/305954662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fb2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 -- 172.21.15.16:0/305954662 >> 172.21.15.16:0/305954662 conn(0x55b1b7fb0890 msgr2=0x55b1b7fb1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.637+0000 7fb945b2a740 1 -- 172.21.15.16:0/305954662 shutdown_connections 2024-09-12T01:03:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 -- 172.21.15.16:0/305954662 wait complete. 2024-09-12T01:03:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 -- start start 2024-09-12T01:03:58.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 Processor -- start 2024-09-12T01:03:58.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1aece0 con 0x55b1b7fa4a30 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1ae610 con 0x55b1b7fa84a0 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1ab870 con 0x55b1b7fa8dc0 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb945b2a740 10 client.0 _ll_register_callbacks cb 0x7fb92c00acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb940cec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb940cec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.638+0000 7fb940cec640 1 -- 172.21.15.16:0/289294615 learned_addr learned my addr 172.21.15.16:0/289294615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb9414ed640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 -- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa8dc0 msgr2=0x55b1b7fb92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 -- 172.21.15.16:0/289294615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa84a0 msgr2=0x55b1b7fb8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 -- 172.21.15.16:0/289294615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b7fb9820 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb940cec640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55b1b7fa56f0 tx=0x7fb92c00a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd30838d035b039a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c020420 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7fb92c00a2d0 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9380044a0 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb92c020930 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb945b2a740 10 client.17877 fetch_fsmap learned FSMap version 28 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.639+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b802da60 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9380044a0 con 0x55b1b7fa4a30 2024-09-12T01:03:58.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb92c01d040 con 0x55b1b7fa4a30 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 10 client.17877 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 20 client.17877 populate_metadata read hostname 'smithi016' 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fb3680 0x55b1b8030130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 4 client.17877 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1b7f15d60 con 0x55b1b7fb3680 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.640+0000 7fb9414ed640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fb3680 0x55b1b8030130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb924ff9640 10 client.17877 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb9414ed640 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fb3680 0x55b1b8030130 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37d3e4c7918f382 server_cookie=5103d12edf26de49 in_seq=0 out_seq=0 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb924ff9640 1 -- 172.21.15.16:0/289294615 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1b7f15d60 con 0x55b1b7fb3680 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb924ff9640 10 client.17877 handle_command_reply: tid=0 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 client.17877 shutdown 2024-09-12T01:03:58.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 msgr2=0x55b1b7fbba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x55b1b7fa56f0 tx=0x7fb92c00a6c0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 shutdown_connections 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1b7fb3680 0x55b1b8030130 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1b7fa8dc0 0x55b1b7fb92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1b7fa84a0 0x55b1b7fb8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 --2- 172.21.15.16:0/289294615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1b7fa4a30 0x55b1b7fbba50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 >> 172.21.15.16:0/289294615 conn(0x55b1b7fb0890 msgr2=0x55b1b7fa3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb924ff9640 0 client.17877 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 shutdown_connections 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 1 -- 172.21.15.16:0/289294615 wait complete. 2024-09-12T01:03:58.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.641+0000 7fb945b2a740 20 client.17877 trim_cache size 0 max 16384 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "c94849ba-4696-4edf-860b-8b8eae5f7776": { 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c94849ba-4696-4edf-860b-8b8eae5f7776", 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:03:58.647 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 msgr2=0x7fb9280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 0x7fb9280a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb9340a3f90 tx=0x7fb93400a7d0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 msgr2=0x7fb93c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cad0 tx=0x7fb93400ced0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 shutdown_connections 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9280a2d70 0x7fb9280a5230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb93c106f10 0x7fb93c19cd00 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb93c102790 0x7fb93c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 --2- 172.21.15.16:0/2610152927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb93c101de0 0x7fb93c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:03:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 >> 172.21.15.16:0/2610152927 conn(0x7fb93c0fd8c0 msgr2=0x7fb93c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:03:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 shutdown_connections 2024-09-12T01:03:58.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:03:58.642+0000 7fb943751640 1 -- 172.21.15.16:0/2610152927 wait complete. 2024-09-12T01:03:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:58 smithi159 ceph-mon[35566]: pgmap v242: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-09-12T01:03:58.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:58 smithi016 ceph-mon[30972]: pgmap v242: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-09-12T01:03:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:58 smithi099 ceph-mon[35654]: pgmap v242: 97 pgs: 97 active+clean; 9.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-09-12T01:03:59.532 INFO:tasks.cephfs.filesystem:scrub status for tag:c94849ba-4696-4edf-860b-8b8eae5f7776 - {'path': '/', 'tag': 'c94849ba-4696-4edf-860b-8b8eae5f7776', 'options': 'recursive,force'} 2024-09-12T01:03:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:03:59 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:03:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:03:59 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:04:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:03:59 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:04:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:00 smithi159 ceph-mon[35566]: pgmap v243: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-12T01:04:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:00 smithi016 ceph-mon[30972]: pgmap v243: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-12T01:04:01.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:00 smithi099 ceph-mon[35654]: pgmap v243: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-09-12T01:04:01.859 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:04:01.859 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T01:04:01.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- 172.21.15.16:0/2740911603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 msgr2=0x7f9ef41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 --2- 172.21.15.16:0/2740911603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef41067e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009900 tx=0x7f9ee002a410 comp rx=0 tx=0).stop 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- 172.21.15.16:0/2740911603 shutdown_connections 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 --2- 172.21.15.16:0/2740911603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 --2- 172.21.15.16:0/2740911603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef41067e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 --2- 172.21.15.16:0/2740911603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 0x7f9ef41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- 172.21.15.16:0/2740911603 >> 172.21.15.16:0/2740911603 conn(0x7f9ef40fd880 msgr2=0x7f9ef40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- 172.21.15.16:0/2740911603 shutdown_connections 2024-09-12T01:04:02.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- 172.21.15.16:0/2740911603 wait complete. 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 Processor -- start 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.617+0000 7f9efac5b640 1 -- start start 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 0x7f9ef4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4111f70 con 0x7f9ef4102790 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4111df0 con 0x7f9ef4106f10 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9efac5b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41120f0 con 0x7f9ef4101de0 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef89f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 0x7f9ef4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49760/0 (socket says 172.21.15.16:49760) 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 -- 172.21.15.16:0/1869381765 learned_addr learned my addr 172.21.15.16:0/1869381765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef91f8640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 -- 172.21.15.16:0/1869381765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 msgr2=0x7f9ef4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 0x7f9ef4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 -- 172.21.15.16:0/1869381765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 msgr2=0x7f9ef419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 -- 172.21.15.16:0/1869381765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef419d3c0 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef91f8640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef3fff640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0006010 tx=0x7f9ee0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee003f030 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.618+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0004440 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.619+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef419d650 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.619+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0035070 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.619+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef41a4f90 con 0x7f9ef4102790 2024-09-12T01:04:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.620+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef41021c0 con 0x7f9ef4102790 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.621+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ee0042030 con 0x7f9ef4102790 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.621+0000 7f9ef1ffb640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 0x7f9ee40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.622+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f9ee00ef7a0 con 0x7f9ef4102790 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.622+0000 7f9ef89f7640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 0x7f9ee40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.622+0000 7f9ef89f7640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 0x7f9ee40a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009bd0 tx=0x7f9ee8005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.626+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ee00bbd70 con 0x7f9ef4102790 2024-09-12T01:04:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.836+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ef410a0f0 con 0x7f9ef4102790 2024-09-12T01:04:02.835 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:02 smithi016 ceph-mon[30972]: pgmap v244: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.837+0000 7f9ef1ffb640 1 -- 172.21.15.16:0/1869381765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f9ee00c0070 con 0x7f9ef4102790 2024-09-12T01:04:02.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:02.836 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 msgr2=0x7f9ee40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 0x7f9ee40a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009bd0 tx=0x7f9ee8005f90 comp rx=0 tx=0).stop 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 msgr2=0x7f9ef4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0006010 tx=0x7f9ee0004040 comp rx=0 tx=0).stop 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 shutdown_connections 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9ee40a2cf0 0x7f9ee40a51b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9ef4106f10 0x7f9ef419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ef4102790 0x7f9ef4198950 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 --2- 172.21.15.16:0/1869381765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ef4101de0 0x7f9ef4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:02.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 >> 172.21.15.16:0/1869381765 conn(0x7f9ef40fd880 msgr2=0x7f9ef40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:02.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.840+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 shutdown_connections 2024-09-12T01:04:02.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:02.841+0000 7f9efac5b640 1 -- 172.21.15.16:0/1869381765 wait complete. 2024-09-12T01:04:02.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:02 smithi159 ceph-mon[35566]: pgmap v244: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:02 smithi099 ceph-mon[35654]: pgmap v244: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:03.564 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T01:04:03.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T01:04:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:03 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1869381765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:04.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:03 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1869381765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:04.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:03 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1869381765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 -- 172.21.15.16:0/2627439926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849810c1a0 msgr2=0x7f849810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 --2- 172.21.15.16:0/2627439926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849810c1a0 0x7f849810e610 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8490009ca0 tx=0x7f849002a950 comp rx=0 tx=0).stop 2024-09-12T01:04:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 -- 172.21.15.16:0/2627439926 shutdown_connections 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 --2- 172.21.15.16:0/2627439926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f849810c1a0 0x7f849810e610 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 --2- 172.21.15.16:0/2627439926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84981036e0 0x7f849810bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 --2- 172.21.15.16:0/2627439926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8498102dc0 0x7f84981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 -- 172.21.15.16:0/2627439926 >> 172.21.15.16:0/2627439926 conn(0x7f84980fc7a0 msgr2=0x7f84980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 -- 172.21.15.16:0/2627439926 shutdown_connections 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.295+0000 7f84a014f640 1 -- 172.21.15.16:0/2627439926 wait complete. 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 Processor -- start 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 -- start start 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498102dc0 0x7f8498100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f849810c1a0 0x7f84981014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8498111f70 con 0x7f84981036e0 2024-09-12T01:04:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8498111df0 con 0x7f849810c1a0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f84a014f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84981120f0 con 0x7f8498102dc0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49784/0 (socket says 172.21.15.16:49784) 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f849d6ea640 1 -- 172.21.15.16:0/2791068024 learned_addr learned my addr 172.21.15.16:0/2791068024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f849deeb640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498102dc0 0x7f8498100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.296+0000 7f849e6ec640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f849810c1a0 0x7f84981014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 -- 172.21.15.16:0/2791068024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498102dc0 msgr2=0x7f8498100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498102dc0 0x7f8498100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 -- 172.21.15.16:0/2791068024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f849810c1a0 msgr2=0x7f84981014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f849810c1a0 0x7f84981014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 -- 172.21.15.16:0/2791068024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84981a6890 con 0x7f84981036e0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f849d6ea640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f848800d950 tx=0x7f848800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f848effd640 1 -- 172.21.15.16:0/2791068024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488017070 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f848effd640 1 -- 172.21.15.16:0/2791068024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8488009c30 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f848effd640 1 -- 172.21.15.16:0/2791068024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848800f6b0 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84981a6b80 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.297+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a7110 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.299+0000 7f848effd640 1 -- 172.21.15.16:0/2791068024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84880109d0 con 0x7f84981036e0 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 Processor -- start 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 -- start start 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 0x563aee61db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aee5832a0 con 0x563aee61d760 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563aee52ab10 con 0x563aee628940 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f84a2528740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563aee57f260 con 0x563aee61e080 2024-09-12T01:04:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f849e6ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 0x563aee61db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f848effd640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 0x7f84840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f849deeb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f849deeb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:49642/0 (socket says 172.21.15.16:49642) 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f849deeb640 1 -- 172.21.15.16:0/1283669479 learned_addr learned my addr 172.21.15.16:0/1283669479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.300+0000 7f849e6ec640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 0x7f84840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f848effd640 1 -- 172.21.15.16:0/2791068024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f84880ceec0 con 0x7f84981036e0 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 -- 172.21.15.16:0/1283669479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 msgr2=0x563aee62ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 -- 172.21.15.16:0/1283669479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 msgr2=0x563aee61db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 0x563aee61db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 -- 172.21.15.16:0/1283669479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aee2f4b90 con 0x563aee61e080 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 0x563aee61db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849deeb640 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f849400bd10 tx=0x7f84940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dade43a400dfea66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f849e6ec640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 0x7f84840a5230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fe0 tx=0x7f849002a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f8483fff640 1 -- 172.21.15.16:0/1283669479 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494004030 con 0x563aee61e080 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f8483fff640 1 -- 172.21.15.16:0/1283669479 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84940041d0 con 0x563aee61e080 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.301+0000 7f8483fff640 1 -- 172.21.15.16:0/1283669479 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84940117e0 con 0x563aee61e080 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- 172.21.15.16:0/1283669479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 msgr2=0x563aee628400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f849400bd10 tx=0x7f84940107a0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- 172.21.15.16:0/1283669479 shutdown_connections 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee628400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 --2- 172.21.15.16:0/1283669479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61d760 0x563aee61db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- 172.21.15.16:0/1283669479 >> 172.21.15.16:0/1283669479 conn(0x563aee619cf0 msgr2=0x563aee61a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- 172.21.15.16:0/1283669479 shutdown_connections 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- 172.21.15.16:0/1283669479 wait complete. 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.302+0000 7f84a2528740 1 -- start start 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 Processor -- start 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee6257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 0x563aee626230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563aee62b510 con 0x563aee61e080 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563aee50b6e0 con 0x563aee61d760 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563aee62b690 con 0x563aee628940 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f84a2528740 10 client.0 _ll_register_callbacks cb 0x563aee52fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49804/0 (socket says 172.21.15.16:49804) 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 -- 172.21.15.16:0/1488745046 learned_addr learned my addr 172.21.15.16:0/1488745046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 0x563aee626230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849deeb640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee6257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 -- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 msgr2=0x563aee626230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 0x563aee626230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 -- 172.21.15.16:0/1488745046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 msgr2=0x563aee6257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee6257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849d6ea640 1 -- 172.21.15.16:0/1488745046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aee634980 con 0x563aee61e080 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.303+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 0x563aee626230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f84880cc670 tx=0x7f8488005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c10f34f496861ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488020040 con 0x563aee61e080 2024-09-12T01:04:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f84880b42b0 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84940041d0 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f848809b490 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 10 client.17895 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563aee637330 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84940041d0 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f848809a8e0 con 0x563aee61e080 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 10 client.17895 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 10 client.17895 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 20 client.17895 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee638790 0x563aee63aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 4 client.17895 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.304+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aee5909d0 con 0x563aee638790 2024-09-12T01:04:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.305+0000 7f849deeb640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee638790 0x563aee63aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.305+0000 7f849deeb640 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee638790 0x563aee63aba0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45251d567abf0c07 server_cookie=a4075c44ae5a6ebe in_seq=0 out_seq=0 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.305+0000 7f84817fa640 10 client.17895 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84817fa640 1 -- 172.21.15.16:0/1488745046 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563aee5909d0 con 0x563aee638790 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84817fa640 10 client.17895 handle_command_reply: tid=0 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 client.17895 shutdown 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 msgr2=0x563aee625cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f84880cc670 tx=0x7f8488005880 comp rx=0 tx=0).stop 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 shutdown_connections 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee638790 0x563aee63aba0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee628940 0x563aee626230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee625cf0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 --2- 172.21.15.16:0/1488745046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee6257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 >> 172.21.15.16:0/1488745046 conn(0x563aee619cf0 msgr2=0x563aee61fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84817fa640 0 client.17895 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 shutdown_connections 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.306+0000 7f84a2528740 1 -- 172.21.15.16:0/1488745046 wait complete. 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.307+0000 7f84a2528740 20 client.17895 trim_cache size 0 max 16384 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.323+0000 7f84a2528740 1 Processor -- start 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.323+0000 7f84a2528740 1 -- start start 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee618d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee640420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84981af970 con 0x563aee628940 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84981b14b0 con 0x563aee61d760 2024-09-12T01:04:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f84a2528740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84981b0040 con 0x563aee61e080 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49818/0 (socket says 172.21.15.16:49818) 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 -- 172.21.15.16:0/1225386498 learned_addr learned my addr 172.21.15.16:0/1225386498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee640420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849deeb640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee618d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 -- 172.21.15.16:0/1225386498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 msgr2=0x563aee640420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee640420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 -- 172.21.15.16:0/1225386498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 msgr2=0x563aee618d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee618d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 -- 172.21.15.16:0/1225386498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aee619b20 con 0x563aee628940 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee640420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849deeb640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee618d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.324+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f8490035000 tx=0x7f849003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80241b09847bc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84837fe640 1 -- 172.21.15.16:0/1225386498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490034040 con 0x563aee628940 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84837fe640 1 -- 172.21.15.16:0/1225386498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490004520 con 0x563aee628940 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84837fe640 1 -- 172.21.15.16:0/1225386498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490030a10 con 0x563aee628940 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- 172.21.15.16:0/1225386498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 msgr2=0x563aee640960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f8490035000 tx=0x7f849003b000 comp rx=0 tx=0).stop 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- 172.21.15.16:0/1225386498 shutdown_connections 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee628940 0x563aee640960 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61e080 0x563aee640420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 --2- 172.21.15.16:0/1225386498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee61d760 0x563aee618d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- 172.21.15.16:0/1225386498 >> 172.21.15.16:0/1225386498 conn(0x563aee619cf0 msgr2=0x563aee618650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- 172.21.15.16:0/1225386498 shutdown_connections 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- 172.21.15.16:0/1225386498 wait complete. 2024-09-12T01:04:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.325+0000 7f84a2528740 1 -- start start 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 Processor -- start 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 0x563aee62d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84981b0de0 con 0x563aee61e080 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84981b0710 con 0x563aee628940 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84981ad970 con 0x563aee61d760 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f84a2528740 10 client.0 _ll_register_callbacks cb 0x7f848809afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49822/0 (socket says 172.21.15.16:49822) 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 -- 172.21.15.16:0/1839763910 learned_addr learned my addr 172.21.15.16:0/1839763910 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849e6ec640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849deeb640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 0x563aee62d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 -- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 msgr2=0x563aee62d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 0x563aee62d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 -- 172.21.15.16:0/1839763910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 msgr2=0x563aee62e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849d6ea640 1 -- 172.21.15.16:0/1839763910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aee6205d0 con 0x563aee61e080 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.326+0000 7f849deeb640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 0x563aee62d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f849d6ea640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8488010fd0 tx=0x7f84880b44e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6227e902ce325b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488020070 con 0x563aee61e080 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8490004520 con 0x563aee61e080 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8521+0+0 (secure 0 0 0) 0x7f84880971e0 con 0x563aee61e080 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8488082050 con 0x563aee61e080 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 10 client.17901 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563aee6a44e0 con 0x563aee61e080 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 30, 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "7468c7b9": { 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 30, 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:04:04.366 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:04.367 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490004520 con 0x563aee61e080 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84880a0920 con 0x563aee61e080 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 10 client.17901 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 10 client.17901 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 20 client.17901 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee6a4c70 0x563aee6a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 4 client.17901 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.327+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x563aee58ad20 con 0x563aee6a4c70 2024-09-12T01:04:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.328+0000 7f849deeb640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee6a4c70 0x563aee6a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.328+0000 7f849deeb640 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee6a4c70 0x563aee6a7090 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaafe470d8c097f0 server_cookie=600254aef20370aa in_seq=0 out_seq=0 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.328+0000 7f84817fa640 10 client.17901 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84817fa640 1 -- 172.21.15.16:0/1839763910 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563aee58ad20 con 0x563aee6a4c70 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84817fa640 10 client.17901 handle_command_reply: tid=0 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 client.17901 shutdown 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 msgr2=0x563aee62dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8488010fd0 tx=0x7f84880b44e0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 shutdown_connections 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563aee6a4c70 0x563aee6a7090 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563aee628940 0x563aee62e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563aee61e080 0x563aee62dc10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 --2- 172.21.15.16:0/1839763910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563aee61d760 0x563aee62d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84817fa640 0 client.17901 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 >> 172.21.15.16:0/1839763910 conn(0x563aee619cf0 msgr2=0x563aee625f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 shutdown_connections 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 1 -- 172.21.15.16:0/1839763910 wait complete. 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.340+0000 7f84a2528740 20 client.17901 trim_cache size 0 max 16384 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 msgr2=0x7f84840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 0x7f84840a5230 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fe0 tx=0x7f849002a950 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 msgr2=0x7f8498100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f848800d950 tx=0x7f848800de10 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 shutdown_connections 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f84840a2d70 0x7f84840a5230 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f849810c1a0 0x7f84981014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84981036e0 0x7f8498100f60 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 --2- 172.21.15.16:0/2791068024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8498102dc0 0x7f8498100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 >> 172.21.15.16:0/2791068024 conn(0x7f84980fc7a0 msgr2=0x7f84980fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 shutdown_connections 2024-09-12T01:04:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:04.341+0000 7f84a014f640 1 -- 172.21.15.16:0/2791068024 wait complete. 2024-09-12T01:04:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:04 smithi016 ceph-mon[30972]: pgmap v245: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:04 smithi099 ceph-mon[35654]: pgmap v245: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:04 smithi159 ceph-mon[35566]: pgmap v245: 97 pgs: 97 active+clean; 9.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-09-12T01:04:05.126 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7468c7b9', quiesce took 3.3 seconds. Will release after: 16.099999999999998 2024-09-12T01:04:05.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:05 smithi016 ceph-mon[30972]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:04:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:05 smithi099 ceph-mon[35654]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:04:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:05 smithi159 ceph-mon[35566]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T01:04:06.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:06 smithi016 ceph-mon[30972]: pgmap v247: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:06 smithi099 ceph-mon[35654]: pgmap v247: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:06 smithi159 ceph-mon[35566]: pgmap v247: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:08.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:08 smithi016 ceph-mon[30972]: pgmap v248: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:08 smithi099 ceph-mon[35654]: pgmap v248: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:08 smithi159 ceph-mon[35566]: pgmap v248: 97 pgs: 97 active+clean; 9.7 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-09-12T01:04:10.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:10 smithi016 ceph-mon[30972]: pgmap v249: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:10 smithi099 ceph-mon[35654]: pgmap v249: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:10 smithi159 ceph-mon[35566]: pgmap v249: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:12 smithi016 ceph-mon[30972]: pgmap v250: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:12 smithi016 ceph-mon[30972]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:04:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:12 smithi099 ceph-mon[35654]: pgmap v250: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:12 smithi099 ceph-mon[35654]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:04:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:12 smithi159 ceph-mon[35566]: pgmap v250: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:04:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:12 smithi159 ceph-mon[35566]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T01:04:14.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:14 smithi016 ceph-mon[30972]: pgmap v252: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 5 op/s 2024-09-12T01:04:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:14 smithi099 ceph-mon[35654]: pgmap v252: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 5 op/s 2024-09-12T01:04:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:14 smithi159 ceph-mon[35566]: pgmap v252: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 115 B/s wr, 5 op/s 2024-09-12T01:04:17.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:16 smithi099 ceph-mon[35654]: pgmap v253: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:16 smithi159 ceph-mon[35566]: pgmap v253: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:16 smithi016 ceph-mon[30972]: pgmap v253: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:18 smithi099 ceph-mon[35654]: pgmap v254: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:18 smithi159 ceph-mon[35566]: pgmap v254: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:19.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:18 smithi016 ceph-mon[30972]: pgmap v254: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-12T01:04:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:04:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:04:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:04:20.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:04:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:04:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:04:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:04:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:04:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:04:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:04:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:04:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:04:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:20 smithi099 ceph-mon[35654]: pgmap v255: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:20 smithi159 ceph-mon[35566]: pgmap v255: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:20 smithi016 ceph-mon[30972]: pgmap v255: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:21.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 -- 172.21.15.16:0/410194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 msgr2=0x7f3294107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 --2- 172.21.15.16:0/410194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294107b90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f327c009930 tx=0x7f327c02a420 comp rx=0 tx=0).stop 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 -- 172.21.15.16:0/410194537 shutdown_connections 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 --2- 172.21.15.16:0/410194537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 0x7f329410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 --2- 172.21.15.16:0/410194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294107b90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 --2- 172.21.15.16:0/410194537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 -- 172.21.15.16:0/410194537 >> 172.21.15.16:0/410194537 conn(0x7f32940fc7d0 msgr2=0x7f32940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:22.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 -- 172.21.15.16:0/410194537 shutdown_connections 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.016+0000 7f3298b0d640 1 -- 172.21.15.16:0/410194537 wait complete. 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 Processor -- start 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 -- start start 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 0x7f329419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32941120a0 con 0x7f32941036e0 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3294111f20 con 0x7f32941082a0 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3298b0d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3294112220 con 0x7f3294102d30 2024-09-12T01:04:22.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3291d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3291d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50104/0 (socket says 172.21.15.16:50104) 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.017+0000 7f3291d9b640 1 -- 172.21.15.16:0/1153031559 learned_addr learned my addr 172.21.15.16:0/1153031559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f329259c640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3292d9d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 0x7f329419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 -- 172.21.15.16:0/1153031559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 msgr2=0x7f3294198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 -- 172.21.15.16:0/1153031559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 msgr2=0x7f329419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 0x7f329419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 -- 172.21.15.16:0/1153031559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329419d510 con 0x7f32941036e0 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f329259c640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3291d9b640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f327c005940 tx=0x7f327c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c03f030 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329419d7a0 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f327c003ce0 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c035070 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.018+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a50e0 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.019+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329406a850 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.020+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f327c042030 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.021+0000 7f328b7fe640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 0x7f32800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.021+0000 7f329259c640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 0x7f32800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.021+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f327c0efa10 con 0x7f32941036e0 2024-09-12T01:04:22.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.022+0000 7f329259c640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 0x7f32800a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f328c005fd0 tx=0x7f328c009520 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:22.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.025+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f327c0bc5d0 con 0x7f32941036e0 2024-09-12T01:04:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.236+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f329410a2b0 con 0x7f32941036e0 2024-09-12T01:04:22.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.237+0000 7f328b7fe640 1 -- 172.21.15.16:0/1153031559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f327c0bbed0 con 0x7f32941036e0 2024-09-12T01:04:22.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:22.237 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 msgr2=0x7f32800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 0x7f32800a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f328c005fd0 tx=0x7f328c009520 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 msgr2=0x7f3294198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f327c005940 tx=0x7f327c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 shutdown_connections 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32800a2cf0 0x7f32800a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32941082a0 0x7f329419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32941036e0 0x7f3294198aa0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 --2- 172.21.15.16:0/1153031559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3294102d30 0x7f3294198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 >> 172.21.15.16:0/1153031559 conn(0x7f32940fc7d0 msgr2=0x7f3294105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 shutdown_connections 2024-09-12T01:04:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:22.240+0000 7f3298b0d640 1 -- 172.21.15.16:0/1153031559 wait complete. 2024-09-12T01:04:23.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:22 smithi016 ceph-mon[30972]: pgmap v256: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:23.006 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:22 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1153031559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:22 smithi099 ceph-mon[35654]: pgmap v256: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:22 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1153031559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:23.039 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 7468c7b9 --release --await' 2024-09-12T01:04:23.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 7468c7b9 --release --await 2024-09-12T01:04:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:22 smithi159 ceph-mon[35566]: pgmap v256: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-09-12T01:04:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:22 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1153031559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:23.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- 172.21.15.16:0/2358013813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 msgr2=0x7f6f8010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 --2- 172.21.15.16:0/2358013813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f8010bb50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009900 tx=0x7f6f7002a480 comp rx=0 tx=0).stop 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- 172.21.15.16:0/2358013813 shutdown_connections 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 --2- 172.21.15.16:0/2358013813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f8010c170 0x7f6f8010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 --2- 172.21.15.16:0/2358013813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f8010bb50 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 --2- 172.21.15.16:0/2358013813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f80102d40 0x7f6f80103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- 172.21.15.16:0/2358013813 >> 172.21.15.16:0/2358013813 conn(0x7f6f800fc7a0 msgr2=0x7f6f800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- 172.21.15.16:0/2358013813 shutdown_connections 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- 172.21.15.16:0/2358013813 wait complete. 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 Processor -- start 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.807+0000 7f6f88389640 1 -- start start 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f80102d40 0x7f6f80198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f8010c170 0x7f6f8019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80111f70 con 0x7f6f80103660 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80111df0 con 0x7f6f80102d40 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f88389640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801120f0 con 0x7f6f8010c170 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50120/0 (socket says 172.21.15.16:50120) 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 -- 172.21.15.16:0/1337810536 learned_addr learned my addr 172.21.15.16:0/1337810536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f86125640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f80102d40 0x7f6f80198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f86926640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f8010c170 0x7f6f8019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 -- 172.21.15.16:0/1337810536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f8010c170 msgr2=0x7f6f8019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f8010c170 0x7f6f8019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 -- 172.21.15.16:0/1337810536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f80102d40 msgr2=0x7f6f80198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f80102d40 0x7f6f80198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 -- 172.21.15.16:0/1337810536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019d3c0 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f85924640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005ab0 tx=0x7f6f700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.808+0000 7f6f777fe640 1 -- 172.21.15.16:0/1337810536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7003f030 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.809+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019d650 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.809+0000 7f6f777fe640 1 -- 172.21.15.16:0/1337810536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f70003ce0 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.809+0000 7f6f777fe640 1 -- 172.21.15.16:0/1337810536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70035070 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.809+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801a4f90 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f777fe640 1 -- 172.21.15.16:0/1337810536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f70042030 con 0x7f6f80103660 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 Processor -- start 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 -- start start 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80395d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c80398630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c802f0620 con 0x563c8038b060 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c80297e50 con 0x563c8038b980 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.811+0000 7f6f8a762740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c802ec5e0 con 0x563c80396240 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50132/0 (socket says 172.21.15.16:50132) 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f85924640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c80398630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 -- 172.21.15.16:0/2045485382 learned_addr learned my addr 172.21.15.16:0/2045485382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86125640 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80395d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f777fe640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 0x7f6f6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 -- 172.21.15.16:0/2045485382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 msgr2=0x563c80398630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c80398630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 -- 172.21.15.16:0/2045485382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 msgr2=0x563c80395d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80395d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 -- 172.21.15.16:0/2045485382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c80288d40 con 0x563c8038b060 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f85924640 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c80398630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f777fe640 1 -- 172.21.15.16:0/1337810536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f6f700ef900 con 0x7f6f80103660 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 0x7f6f6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.812+0000 7f6f86926640 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6f78008ec0 tx=0x7f6f7800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bfd8f5937e28c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f74dea640 1 -- 172.21.15.16:0/2045485382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78022070 con 0x563c8038b060 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f74dea640 1 -- 172.21.15.16:0/2045485382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7801c470 con 0x563c8038b060 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f74dea640 1 -- 172.21.15.16:0/2045485382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7801a600 con 0x563c8038b060 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f86926640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 0x7f6f6c0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801f000 tx=0x7f6f78020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 -- 172.21.15.16:0/2045485382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 msgr2=0x563c8038b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6f78008ec0 tx=0x7f6f7800e720 comp rx=0 tx=0).stop 2024-09-12T01:04:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 -- 172.21.15.16:0/2045485382 shutdown_connections 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c80398630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80395d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2045485382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038b440 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.813+0000 7f6f8a762740 1 -- 172.21.15.16:0/2045485382 >> 172.21.15.16:0/2045485382 conn(0x563c803875f0 msgr2=0x563c803879d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- 172.21.15.16:0/2045485382 shutdown_connections 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- 172.21.15.16:0/2045485382 wait complete. 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- start start 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 Processor -- start 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 0x563c80393100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b980 0x563c80393640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563c80398e10 con 0x563c80396240 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c80278f10 con 0x563c8038b060 2024-09-12T01:04:23.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.814+0000 7f6f8a762740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c80398f90 con 0x563c8038b980 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f8a762740 10 client.0 _ll_register_callbacks cb 0x563c8029cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 0x563c80393100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50134/0 (socket says 172.21.15.16:50134) 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86926640 1 -- 172.21.15.16:0/3388103618 learned_addr learned my addr 172.21.15.16:0/3388103618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f85924640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b980 0x563c80393640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 -- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b980 msgr2=0x563c80393640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b980 0x563c80393640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 -- 172.21.15.16:0/3388103618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 msgr2=0x563c80393b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 -- 172.21.15.16:0/3388103618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7ffff580 con 0x563c8038b060 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86926640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:23.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f86125640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 0x563c80393100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c010840 tx=0x7f6f7c010c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e9ae4893bed6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.815+0000 7f6f69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c01e060 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7801c470 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f6f7c00fb90 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f7c00fd90 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 10 client.26903 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c800c9960 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7801c470 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f7c0215c0 con 0x563c8038b060 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 10 client.26903 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 10 client.26903 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 20 client.26903 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c803a5d80 0x563c803a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 4 client.26903 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.816+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c802fdb90 con 0x563c803a5d80 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.817+0000 7f6f85924640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c803a5d80 0x563c803a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.817+0000 7f6f85924640 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c803a5d80 0x563c803a8170 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9bd75fb84268880 server_cookie=65f33a85ef432d8e in_seq=0 out_seq=0 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.817+0000 7f6f69ffb640 10 client.26903 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f69ffb640 1 -- 172.21.15.16:0/3388103618 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c802fdb90 con 0x563c803a5d80 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f69ffb640 10 client.26903 handle_command_reply: tid=0 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 client.26903 shutdown 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 msgr2=0x563c80393100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 0x563c80393100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c010840 tx=0x7f6f7c010c40 comp rx=0 tx=0).stop 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 shutdown_connections 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c803a5d80 0x563c803a8170 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f69ffb640 0 client.26903 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b980 0x563c80393640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 --2- 172.21.15.16:0/3388103618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b060 0x563c80393100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 >> 172.21.15.16:0/3388103618 conn(0x563c803875f0 msgr2=0x563c8038d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 shutdown_connections 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 1 -- 172.21.15.16:0/3388103618 wait complete. 2024-09-12T01:04:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.818+0000 7f6f8a762740 20 client.26903 trim_cache size 0 max 16384 2024-09-12T01:04:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 Processor -- start 2024-09-12T01:04:23.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 -- start start 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80386640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c803ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801ad890 con 0x563c8038b060 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801af3d0 con 0x563c8038b980 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f8a762740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801adf60 con 0x563c80396240 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f86125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f86125640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50136/0 (socket says 172.21.15.16:50136) 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.835+0000 7f6f86125640 1 -- 172.21.15.16:0/1234734275 learned_addr learned my addr 172.21.15.16:0/1234734275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86926640 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c803ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f85924640 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80386640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 -- 172.21.15.16:0/1234734275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 msgr2=0x563c803ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c803ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 -- 172.21.15.16:0/1234734275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 msgr2=0x563c80386640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80386640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 -- 172.21.15.16:0/1234734275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8038b7b0 con 0x563c8038b060 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f86125640 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0219e0 tx=0x7f6f7c00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05cf09769fe88ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f6bfff640 1 -- 172.21.15.16:0/1234734275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c021c50 con 0x563c8038b060 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f6bfff640 1 -- 172.21.15.16:0/1234734275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c021df0 con 0x563c8038b060 2024-09-12T01:04:23.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.836+0000 7f6f6bfff640 1 -- 172.21.15.16:0/1234734275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c022e10 con 0x563c8038b060 2024-09-12T01:04:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- 172.21.15.16:0/1234734275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 msgr2=0x563c8038db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0219e0 tx=0x7f6f7c00f8b0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- 172.21.15.16:0/1234734275 shutdown_connections 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c80396240 0x563c803ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c80386640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 --2- 172.21.15.16:0/1234734275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c8038b060 0x563c8038db10 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- 172.21.15.16:0/1234734275 >> 172.21.15.16:0/1234734275 conn(0x563c803875f0 msgr2=0x563c80385f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- 172.21.15.16:0/1234734275 shutdown_connections 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- 172.21.15.16:0/1234734275 wait complete. 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 -- start start 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.837+0000 7f6f8a762740 1 Processor -- start 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b060 0x563c80390190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c803906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801aed00 con 0x563c80396240 2024-09-12T01:04:23.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801ae630 con 0x563c8038b980 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f801ab890 con 0x563c8038b060 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 10 client.0 _ll_register_callbacks cb 0x7f6f7c00acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50152/0 (socket says 172.21.15.16:50152) 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 -- 172.21.15.16:0/2379736701 learned_addr learned my addr 172.21.15.16:0/2379736701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f85924640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c803906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86125640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b060 0x563c80390190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 -- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b060 msgr2=0x563c80390190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b060 0x563c80390190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 -- 172.21.15.16:0/2379736701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 msgr2=0x563c803906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c803906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 -- 172.21.15.16:0/2379736701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8040fa40 con 0x563c80396240 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f86926640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801cdc0 tx=0x7f6f78011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df563a4baec5338c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:23.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78022040 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f6f78004a00 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.838+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7c021df0 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f7801c470 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 10 client.17925 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c80410d80 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f7c021df0 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6f78019040 con 0x563c80396240 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 10 client.17925 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 10 client.17925 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 20 client.17925 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c80411280 0x563c804136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 4 client.17925 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7468c7b9", "release": true, "await": true}] 2024-09-12T01:04:23.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7468c7b9", "release": true, "await": true}) -- 0x563c802f80a0 con 0x563c80411280 2024-09-12T01:04:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.839+0000 7f6f86125640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c80411280 0x563c804136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.840+0000 7f6f86125640 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c80411280 0x563c804136a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7faa53b11013de4 server_cookie=6b1c8b019163a90c in_seq=0 out_seq=0 2024-09-12T01:04:23.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.840+0000 7f6f69ffb640 10 client.17925 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f69ffb640 1 -- 172.21.15.16:0/2379736701 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563c802f80a0 con 0x563c80411280 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f69ffb640 10 client.17925 handle_command_reply: tid=0 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 client.17925 shutdown 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 msgr2=0x563c80390c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801cdc0 tx=0x7f6f78011a60 comp rx=0 tx=0).stop 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 shutdown_connections 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x563c80411280 0x563c804136a0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c80396240 0x563c80390c10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c8038b980 0x563c803906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 --2- 172.21.15.16:0/2379736701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c8038b060 0x563c80390190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 >> 172.21.15.16:0/2379736701 conn(0x563c803875f0 msgr2=0x563c803930c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f69ffb640 0 client.17925 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 shutdown_connections 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 1 -- 172.21.15.16:0/2379736701 wait complete. 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.843+0000 7f6f8a762740 20 client.17925 trim_cache size 0 max 16384 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:04:23.845 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 32, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "7468c7b9": { 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 32, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:04:23.846 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:23.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 msgr2=0x7f6f6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 0x7f6f6c0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f7801f000 tx=0x7f6f78020000 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 msgr2=0x7f6f80198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005ab0 tx=0x7f6f700042f0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 shutdown_connections 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6f6c0a2cf0 0x7f6f6c0a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f8010c170 0x7f6f8019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f80103660 0x7f6f80198950 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 --2- 172.21.15.16:0/1337810536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f80102d40 0x7f6f80198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 >> 172.21.15.16:0/1337810536 conn(0x7f6f800fc7a0 msgr2=0x7f6f800fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 shutdown_connections 2024-09-12T01:04:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:23.844+0000 7f6f88389640 1 -- 172.21.15.16:0/1337810536 wait complete. 2024-09-12T01:04:24.706 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7468c7b9', total seconds elapsed: 22.8 2024-09-12T01:04:24.707 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T01:04:24.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:24.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:24 smithi159 ceph-mon[35566]: pgmap v257: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:04:24.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:24 smithi099 ceph-mon[35654]: pgmap v257: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:04:25.086 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:24 smithi016 ceph-mon[30972]: pgmap v257: 97 pgs: 97 active+clean; 9.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-12T01:04:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 -- 172.21.15.16:0/1298213549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704010c2f0 msgr2=0x7f704010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 --2- 172.21.15.16:0/1298213549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704010c2f0 0x7f704010e790 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7038009ca0 tx=0x7f703802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 -- 172.21.15.16:0/1298213549 shutdown_connections 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 --2- 172.21.15.16:0/1298213549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704010c2f0 0x7f704010e790 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 --2- 172.21.15.16:0/1298213549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70401037a0 0x7f704010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 --2- 172.21.15.16:0/1298213549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 0x7f70401031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 -- 172.21.15.16:0/1298213549 >> 172.21.15.16:0/1298213549 conn(0x7f70400fc7d0 msgr2=0x7f70400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 -- 172.21.15.16:0/1298213549 shutdown_connections 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.435+0000 7f7045f97640 1 -- 172.21.15.16:0/1298213549 wait complete. 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 Processor -- start 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 -- start start 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 0x7f7040198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 0x7f704019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f70401120f0 con 0x7f70401037a0 2024-09-12T01:04:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7040111f70 con 0x7f704010c2f0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f7045f97640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7040112270 con 0x7f7040102df0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f703effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f703effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50170/0 (socket says 172.21.15.16:50170) 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f703effd640 1 -- 172.21.15.16:0/460629307 learned_addr learned my addr 172.21.15.16:0/460629307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.436+0000 7f703f7fe640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 0x7f7040198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703ffff640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 0x7f704019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 -- 172.21.15.16:0/460629307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 msgr2=0x7f7040198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 0x7f7040198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 -- 172.21.15.16:0/460629307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 msgr2=0x7f704019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 0x7f704019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 -- 172.21.15.16:0/460629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704019d540 con 0x7f70401037a0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703ffff640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 0x7f704019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703effd640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f703400d950 tx=0x7f703400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034017070 con 0x7f70401037a0 2024-09-12T01:04:25.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704019d830 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7034009c30 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340047a0 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.437+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a5110 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.438+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70401031d0 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.439+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7034004020 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.440+0000 7f703cff9640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 0x7f70280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.440+0000 7f703f7fe640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 0x7f70280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.440+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f70340cf090 con 0x7f70401037a0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.441+0000 7f703f7fe640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 0x7f70280a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f703000bbe0 tx=0x7f703000b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:25.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.445+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7034020020 con 0x7f70401037a0 2024-09-12T01:04:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.659+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70401072c0 con 0x7f70401037a0 2024-09-12T01:04:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.660+0000 7f703cff9640 1 -- 172.21.15.16:0/460629307 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f703409bc50 con 0x7f70401037a0 2024-09-12T01:04:25.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:25.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 msgr2=0x7f70280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 0x7f70280a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f703000bbe0 tx=0x7f703000b390 comp rx=0 tx=0).stop 2024-09-12T01:04:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 msgr2=0x7f7040198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:25.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f703400d950 tx=0x7f703400de10 comp rx=0 tx=0).stop 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 shutdown_connections 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f70280a2cf0 0x7f70280a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010c2f0 0x7f704019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70401037a0 0x7f7040198ad0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 --2- 172.21.15.16:0/460629307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7040102df0 0x7f7040198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 >> 172.21.15.16:0/460629307 conn(0x7f70400fc7d0 msgr2=0x7f70400fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 shutdown_connections 2024-09-12T01:04:25.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:25.663+0000 7f7045f97640 1 -- 172.21.15.16:0/460629307 wait complete. 2024-09-12T01:04:26.456 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T01:04:26.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T01:04:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:26.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:26 smithi016 ceph-mon[30972]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:04:26.839 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:26 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/460629307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:26 smithi099 ceph-mon[35654]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:04:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:26 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/460629307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:26 smithi159 ceph-mon[35566]: pgmap v258: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:04:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:26 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/460629307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 -- 172.21.15.16:0/289316699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150101420 msgr2=0x7f215010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 --2- 172.21.15.16:0/289316699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150101420 0x7f215010d950 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2148009cd0 tx=0x7f214802a8f0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 -- 172.21.15.16:0/289316699 shutdown_connections 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 --2- 172.21.15.16:0/289316699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150101420 0x7f215010d950 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 --2- 172.21.15.16:0/289316699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150100a80 0x7f2150100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 --2- 172.21.15.16:0/289316699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150106a80 0x7f2150106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 -- 172.21.15.16:0/289316699 >> 172.21.15.16:0/289316699 conn(0x7f21500fc7a0 msgr2=0x7f21500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 -- 172.21.15.16:0/289316699 shutdown_connections 2024-09-12T01:04:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.221+0000 7f215628f640 1 -- 172.21.15.16:0/289316699 wait complete. 2024-09-12T01:04:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 Processor -- start 2024-09-12T01:04:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 -- start start 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150101420 0x7f2150198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 0x7f215019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2150110e80 con 0x7f2150100a80 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2150110d00 con 0x7f2150101420 2024-09-12T01:04:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215628f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2150111000 con 0x7f2150106a80 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f214ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f214ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50184/0 (socket says 172.21.15.16:50184) 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f214ffff640 1 -- 172.21.15.16:0/2165190149 learned_addr learned my addr 172.21.15.16:0/2165190149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.222+0000 7f215482c640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 0x7f215019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214f7fe640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150101420 0x7f2150198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 -- 172.21.15.16:0/2165190149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 msgr2=0x7f215019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 0x7f215019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 -- 172.21.15.16:0/2165190149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150101420 msgr2=0x7f2150198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150101420 0x7f2150198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 -- 172.21.15.16:0/2165190149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215019d2c0 con 0x7f2150100a80 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f215482c640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 0x7f215019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214ffff640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f214000c920 tx=0x7f214000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214d7fa640 1 -- 172.21.15.16:0/2165190149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140007c70 con 0x7f2150100a80 2024-09-12T01:04:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215019d5b0 con 0x7f2150100a80 2024-09-12T01:04:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214d7fa640 1 -- 172.21.15.16:0/2165190149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2140007e10 con 0x7f2150100a80 2024-09-12T01:04:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f214d7fa640 1 -- 172.21.15.16:0/2165190149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140018890 con 0x7f2150100a80 2024-09-12T01:04:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.223+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21501a4e90 con 0x7f2150100a80 2024-09-12T01:04:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.225+0000 7f214d7fa640 1 -- 172.21.15.16:0/2165190149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2140018a70 con 0x7f2150100a80 2024-09-12T01:04:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 Processor -- start 2024-09-12T01:04:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 -- start start 2024-09-12T01:04:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaacf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567aaa2a2a0 con 0x5567aaac4760 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5567aa9d1b10 con 0x5567aaac5080 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f2158668740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567aaa26260 con 0x5567aaacf940 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50188/0 (socket says 172.21.15.16:50188) 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 -- 172.21.15.16:0/3495827792 learned_addr learned my addr 172.21.15.16:0/3495827792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f214d7fa640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 0x7f21380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f214ffff640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaacf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f214f7fe640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 -- 172.21.15.16:0/3495827792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 msgr2=0x5567aaad1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 -- 172.21.15.16:0/3495827792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 msgr2=0x5567aaacf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaacf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 -- 172.21.15.16:0/3495827792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aa79bb90 con 0x5567aaac4760 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f214ffff640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaacf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f214d7fa640 1 -- 172.21.15.16:0/2165190149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f21400cf5d0 con 0x7f2150100a80 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.226+0000 7f215482c640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 0x7f21380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.227+0000 7f215482c640 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2148035000 tx=0x7f21480304f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bda07415ebe64b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.227+0000 7f21367fc640 1 -- 172.21.15.16:0/3495827792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148003970 con 0x5567aaac4760 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.227+0000 7f21367fc640 1 -- 172.21.15.16:0/3495827792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148036040 con 0x5567aaac4760 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.227+0000 7f21367fc640 1 -- 172.21.15.16:0/3495827792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148048440 con 0x5567aaac4760 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- 172.21.15.16:0/3495827792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 msgr2=0x5567aaac4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2148035000 tx=0x7f21480304f0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f215482c640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 0x7f21380a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f214804c000 tx=0x7f2148049000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- 172.21.15.16:0/3495827792 shutdown_connections 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaacf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 --2- 172.21.15.16:0/3495827792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaac4b40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- 172.21.15.16:0/3495827792 >> 172.21.15.16:0/3495827792 conn(0x5567aaac0cf0 msgr2=0x5567aaac10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- 172.21.15.16:0/3495827792 shutdown_connections 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- 172.21.15.16:0/3495827792 wait complete. 2024-09-12T01:04:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 -- start start 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.228+0000 7f2158668740 1 Processor -- start 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaacc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaacd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567aaad2510 con 0x5567aaac5080 2024-09-12T01:04:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5567aa9b26e0 con 0x5567aaac4760 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567aaad2690 con 0x5567aaacf940 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f2158668740 10 client.0 _ll_register_callbacks cb 0x5567aa9d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50192/0 (socket says 172.21.15.16:50192) 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 -- 172.21.15.16:0/145318124 learned_addr learned my addr 172.21.15.16:0/145318124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214ffff640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaacc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f215482c640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaacd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 -- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 msgr2=0x5567aaacd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaacd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 -- 172.21.15.16:0/145318124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 msgr2=0x5567aaacc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaacc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 -- 172.21.15.16:0/145318124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aaadb9d0 con 0x5567aaac5080 2024-09-12T01:04:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f214f7fe640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f21440088d0 tx=0x7f2144008cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14a94506977da69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.229+0000 7f212f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214401e060 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2148036040 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2144007bb0 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2144007db0 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 10 client.17943 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567aaade3f0 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148036040 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2144018030 con 0x5567aaac5080 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 10 client.17943 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 10 client.17943 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 20 client.17943 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aaadf800 0x5567aaae1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 4 client.17943 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.230+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567aaa379d0 con 0x5567aaadf800 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.231+0000 7f214ffff640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aaadf800 0x5567aaae1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.231+0000 7f212f7fe640 10 client.17943 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.231+0000 7f214ffff640 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aaadf800 0x5567aaae1c30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56add8b58426e479 server_cookie=6963a216f2c053ff in_seq=0 out_seq=0 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f212f7fe640 1 -- 172.21.15.16:0/145318124 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567aaa379d0 con 0x5567aaadf800 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f212f7fe640 10 client.17943 handle_command_reply: tid=0 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 client.17943 shutdown 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 msgr2=0x5567aaaccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f21440088d0 tx=0x7f2144008cd0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 shutdown_connections 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aaadf800 0x5567aaae1c30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaacd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac5080 0x5567aaaccd20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 --2- 172.21.15.16:0/145318124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaacc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 >> 172.21.15.16:0/145318124 conn(0x5567aaac0cf0 msgr2=0x5567aaabfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f212f7fe640 0 client.17943 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 shutdown_connections 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 1 -- 172.21.15.16:0/145318124 wait complete. 2024-09-12T01:04:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.232+0000 7f2158668740 20 client.17943 trim_cache size 0 max 16384 2024-09-12T01:04:27.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 Processor -- start 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 -- start start 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaae7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaac5080 0x5567aaae7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21501ab5e0 con 0x5567aaacf940 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21501ad120 con 0x5567aaac4760 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.249+0000 7f2158668740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21501abcb0 con 0x5567aaac5080 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50206/0 (socket says 172.21.15.16:50206) 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 -- 172.21.15.16:0/2442474948 learned_addr learned my addr 172.21.15.16:0/2442474948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f214ffff640 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaae7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f214f7fe640 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaac5080 0x5567aaae7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 -- 172.21.15.16:0/2442474948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaac5080 msgr2=0x5567aaae7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaac5080 0x5567aaae7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 -- 172.21.15.16:0/2442474948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 msgr2=0x5567aaae7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaae7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 -- 172.21.15.16:0/2442474948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567aaac75d0 con 0x5567aaacf940 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f215482c640 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f21480095f0 tx=0x7f2148009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87cb40ea4338a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f2135ffb640 1 -- 172.21.15.16:0/2442474948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148030e70 con 0x5567aaacf940 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f2135ffb640 1 -- 172.21.15.16:0/2442474948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148036030 con 0x5567aaacf940 2024-09-12T01:04:27.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.250+0000 7f2135ffb640 1 -- 172.21.15.16:0/2442474948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148004040 con 0x5567aaacf940 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- 172.21.15.16:0/2442474948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 msgr2=0x5567aaac9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f21480095f0 tx=0x7f2148009b20 comp rx=0 tx=0).stop 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- 172.21.15.16:0/2442474948 shutdown_connections 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaacf940 0x5567aaac9a40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaac5080 0x5567aaae7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 --2- 172.21.15.16:0/2442474948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac4760 0x5567aaae7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- 172.21.15.16:0/2442474948 >> 172.21.15.16:0/2442474948 conn(0x5567aaac0cf0 msgr2=0x5567aaace860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- 172.21.15.16:0/2442474948 shutdown_connections 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- 172.21.15.16:0/2442474948 wait complete. 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.251+0000 7f2158668740 1 -- start start 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 Processor -- start 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaad3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21501aca50 con 0x5567aaac4760 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21501ac380 con 0x5567aaac5080 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21501a95e0 con 0x5567aaacf940 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f2158668740 10 client.0 _ll_register_callbacks cb 0x7f2144008460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50218/0 (socket says 172.21.15.16:50218) 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 -- 172.21.15.16:0/358026980 learned_addr learned my addr 172.21.15.16:0/358026980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214f7fe640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaad3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f215482c640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 -- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 msgr2=0x5567aaad41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 -- 172.21.15.16:0/358026980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 msgr2=0x5567aaad3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaad3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214ffff640 1 -- 172.21.15.16:0/358026980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567aab49560 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f215482c640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.252+0000 7f214f7fe640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaad3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f214ffff640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21400ccdd0 tx=0x7f214000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba53e0a4c8722cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140018d60 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f214000fb30 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2148036030 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21400ccfa0 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 10 client.17949 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5567aab4a880 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148036030 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21400ba390 con 0x5567aaac4760 2024-09-12T01:04:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 10 client.17949 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 10 client.17949 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 20 client.17949 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aab4af70 0x5567aab4d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 4 client.17949 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.253+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x5567aaa31d20 con 0x5567aab4af70 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.254+0000 7f214f7fe640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aab4af70 0x5567aab4d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.254+0000 7f214f7fe640 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aab4af70 0x5567aab4d360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23adb29c329c1cfd server_cookie=c639fdecf9b70954 in_seq=0 out_seq=0 2024-09-12T01:04:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.254+0000 7f212f7fe640 10 client.17949 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f212f7fe640 1 -- 172.21.15.16:0/358026980 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5567aaa31d20 con 0x5567aab4af70 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f212f7fe640 10 client.17949 handle_command_reply: tid=0 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 client.17949 shutdown 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 msgr2=0x5567aaad3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21400ccdd0 tx=0x7f214000ff30 comp rx=0 tx=0).stop 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 shutdown_connections 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5567aab4af70 0x5567aab4d360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567aaacf940 0x5567aaad41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567aaac5080 0x5567aaad3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 --2- 172.21.15.16:0/358026980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5567aaac4760 0x5567aaad3720 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 >> 172.21.15.16:0/358026980 conn(0x5567aaac0cf0 msgr2=0x5567aaacc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f212f7fe640 0 client.17949 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 shutdown_connections 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.266+0000 7f2158668740 1 -- 172.21.15.16:0/358026980 wait complete. 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f2158668740 20 client.17949 trim_cache size 0 max 16384 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 34, 2024-09-12T01:04:27.267 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "54f31edd": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 34, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:27.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 msgr2=0x7f21380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 0x7f21380a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f214804c000 tx=0x7f2148049000 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 msgr2=0x7f2150198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f214000c920 tx=0x7f214000cde0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 shutdown_connections 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21380a2cf0 0x7f21380a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2150106a80 0x7f215019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2150101420 0x7f2150198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 --2- 172.21.15.16:0/2165190149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2150100a80 0x7f2150198310 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 >> 172.21.15.16:0/2165190149 conn(0x7f21500fc7a0 msgr2=0x7f2150104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 shutdown_connections 2024-09-12T01:04:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:27.267+0000 7f215628f640 1 -- 172.21.15.16:0/2165190149 wait complete. 2024-09-12T01:04:28.097 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '54f31edd', quiesce took 3.4 seconds. Will release after: 15.999999999999998 2024-09-12T01:04:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:28 smithi099 ceph-mon[35654]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-12T01:04:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:28 smithi159 ceph-mon[35566]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-12T01:04:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:28 smithi016 ceph-mon[30972]: pgmap v259: 97 pgs: 97 active+clean; 9.8 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-09-12T01:04:29.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:04:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 -- 172.21.15.16:0/292109520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64102790 msgr2=0x7fee641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 --2- 172.21.15.16:0/292109520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64102790 0x7fee641067e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fee58009900 tx=0x7fee5802a410 comp rx=0 tx=0).stop 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 -- 172.21.15.16:0/292109520 shutdown_connections 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 --2- 172.21.15.16:0/292109520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64106f10 0x7fee6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 --2- 172.21.15.16:0/292109520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64102790 0x7fee641067e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 --2- 172.21.15.16:0/292109520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64101de0 0x7fee641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 -- 172.21.15.16:0/292109520 >> 172.21.15.16:0/292109520 conn(0x7fee640fd8c0 msgr2=0x7fee640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 -- 172.21.15.16:0/292109520 shutdown_connections 2024-09-12T01:04:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.288+0000 7fee6a018640 1 -- 172.21.15.16:0/292109520 wait complete. 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 Processor -- start 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 -- start start 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64102790 0x7fee6419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64106f10 0x7fee6419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee64111f90 con 0x7fee64101de0 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee64111e10 con 0x7fee64106f10 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee6a018640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee64112110 con 0x7fee64102790 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37074/0 (socket says 172.21.15.16:37074) 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.289+0000 7fee637fe640 1 -- 172.21.15.16:0/3000812041 learned_addr learned my addr 172.21.15.16:0/3000812041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee63fff640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64106f10 0x7fee6419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee62ffd640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64102790 0x7fee6419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 -- 172.21.15.16:0/3000812041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64102790 msgr2=0x7fee6419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64102790 0x7fee6419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 -- 172.21.15.16:0/3000812041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64106f10 msgr2=0x7fee6419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64106f10 0x7fee6419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 -- 172.21.15.16:0/3000812041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6419f700 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee637fe640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c890 tx=0x7fee5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee60ff9640 1 -- 172.21.15.16:0/3000812041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54004220 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee60ff9640 1 -- 172.21.15.16:0/3000812041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee540043c0 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6419f9f0 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee60ff9640 1 -- 172.21.15.16:0/3000812041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540049e0 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.290+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641a72d0 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.292+0000 7fee60ff9640 1 -- 172.21.15.16:0/3000812041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee540079b0 con 0x7fee64101de0 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.292+0000 7fee6c3f1740 1 Processor -- start 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 -- start start 2024-09-12T01:04:30.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270edc0 0x556c82719140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c82719680 0x556c8271ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c826742e0 con 0x556c8270edc0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c8261bce0 con 0x556c82719680 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c826702a0 con 0x556c8270e4a0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee60ff9640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 0x7fee4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270edc0 0x556c82719140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:36744/0 (socket says 172.21.15.16:36744) 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 -- 172.21.15.16:0/4239541764 learned_addr learned my addr 172.21.15.16:0/4239541764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 0x7fee4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee62ffd640 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c82719680 0x556c8271ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee60ff9640 1 -- 172.21.15.16:0/3000812041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fee540cf560 con 0x7fee64101de0 2024-09-12T01:04:30.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 -- 172.21.15.16:0/4239541764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c82719680 msgr2=0x556c8271ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c82719680 0x556c8271ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 -- 172.21.15.16:0/4239541764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270edc0 msgr2=0x556c82719140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270edc0 0x556c82719140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.293+0000 7fee63fff640 1 -- 172.21.15.16:0/4239541764 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c82600440 con 0x556c8270e4a0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.294+0000 7fee63fff640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 0x7fee4c0a51b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009440 tx=0x7fee5c00c000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.294+0000 7fee63fff640 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00af20 tx=0x7fee5c016700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc7401b0569be6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.294+0000 7fee49ffb640 1 -- 172.21.15.16:0/4239541764 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c01d070 con 0x556c8270e4a0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.294+0000 7fee49ffb640 1 -- 172.21.15.16:0/4239541764 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c009c80 con 0x556c8270e4a0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.294+0000 7fee49ffb640 1 -- 172.21.15.16:0/4239541764 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c013040 con 0x556c8270e4a0 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- 172.21.15.16:0/4239541764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 msgr2=0x556c8270e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00af20 tx=0x7fee5c016700 comp rx=0 tx=0).stop 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- 172.21.15.16:0/4239541764 shutdown_connections 2024-09-12T01:04:30.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c82719680 0x556c8271ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270edc0 0x556c82719140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/4239541764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8270e880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- 172.21.15.16:0/4239541764 >> 172.21.15.16:0/4239541764 conn(0x556c8270aa30 msgr2=0x556c8270ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- 172.21.15.16:0/4239541764 shutdown_connections 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- 172.21.15.16:0/4239541764 wait complete. 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 -- start start 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 Processor -- start 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c82719030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c827162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.295+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c8271c1f0 con 0x556c82719680 2024-09-12T01:04:30.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c8262f050 con 0x556c8270edc0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c8271c370 con 0x556c8270e4a0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee6c3f1740 10 client.0 _ll_register_callbacks cb 0x556c8266f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37096/0 (socket says 172.21.15.16:37096) 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 -- 172.21.15.16:0/2748142468 learned_addr learned my addr 172.21.15.16:0/2748142468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee637fe640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c82719030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee62ffd640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c827162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 -- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 msgr2=0x556c82719030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c82719030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 -- 172.21.15.16:0/2748142468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 msgr2=0x556c827162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c827162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 -- 172.21.15.16:0/2748142468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c823e53c0 con 0x556c82719680 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee637fe640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c82719030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee63fff640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fee5c016fd0 tx=0x7fee5c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a06982f85eead0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c02f020 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fee5c00a870 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.296+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c82681850 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee5c03a050 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 10 client.17961 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c82727c00 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556c82681850 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee5c02a7a0 con 0x556c82719680 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 10 client.17961 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 20 client.17961 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c8272b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 4 client.17961 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8267bd60 con 0x556c827290c0 2024-09-12T01:04:30.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.297+0000 7fee637fe640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c8272b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.298+0000 7fee42ffd640 10 client.17961 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.298+0000 7fee637fe640 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c8272b4b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a019b65d6fe8700 server_cookie=1b8ccb2f02ac267f in_seq=0 out_seq=0 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee42ffd640 1 -- 172.21.15.16:0/2748142468 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c8267bd60 con 0x556c827290c0 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee42ffd640 10 client.17961 handle_command_reply: tid=0 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 client.17961 shutdown 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 msgr2=0x556c82716880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fee5c016fd0 tx=0x7fee5c007000 comp rx=0 tx=0).stop 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 shutdown_connections 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c8272b4b0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee42ffd640 0 client.17961 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c82716880 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c827162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2748142468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c82719030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 >> 172.21.15.16:0/2748142468 conn(0x556c8270aa30 msgr2=0x556c82709b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 shutdown_connections 2024-09-12T01:04:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2748142468 wait complete. 2024-09-12T01:04:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.299+0000 7fee6c3f1740 20 client.17961 trim_cache size 0 max 16384 2024-09-12T01:04:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 Processor -- start 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 -- start start 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c82730ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c82719680 0x556c82731020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee641afbe0 con 0x556c8270e4a0 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee641b1720 con 0x556c8270edc0 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee641b02b0 con 0x556c82719680 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37104/0 (socket says 172.21.15.16:37104) 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.316+0000 7fee637fe640 1 -- 172.21.15.16:0/2990412396 learned_addr learned my addr 172.21.15.16:0/2990412396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee63fff640 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c82719680 0x556c82731020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee62ffd640 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c82730ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 -- 172.21.15.16:0/2990412396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c82719680 msgr2=0x556c82731020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c82719680 0x556c82731020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 -- 172.21.15.16:0/2990412396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 msgr2=0x556c82730ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c82730ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 -- 172.21.15.16:0/2990412396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8270e2d0 con 0x556c8270e4a0 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee637fe640 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fee54016000 tx=0x7fee540d1a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcadab23e90679d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee497fa640 1 -- 172.21.15.16:0/2990412396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5401d040 con 0x556c8270e4a0 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee497fa640 1 -- 172.21.15.16:0/2990412396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5409fe30 con 0x556c8270e4a0 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.317+0000 7fee497fa640 1 -- 172.21.15.16:0/2990412396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540a4260 con 0x556c8270e4a0 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2990412396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 msgr2=0x556c82725450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fee54016000 tx=0x7fee540d1a70 comp rx=0 tx=0).stop 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2990412396 shutdown_connections 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c82719680 0x556c82731020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c82730ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/2990412396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c8270e4a0 0x556c82725450 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2990412396 >> 172.21.15.16:0/2990412396 conn(0x556c8270aa30 msgr2=0x556c827153d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2990412396 shutdown_connections 2024-09-12T01:04:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- 172.21.15.16:0/2990412396 wait complete. 2024-09-12T01:04:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 -- start start 2024-09-12T01:04:30.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.318+0000 7fee6c3f1740 1 Processor -- start 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8271cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c8271d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c8278ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee641b1050 con 0x556c82719680 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee641b0980 con 0x556c8270edc0 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee641adbe0 con 0x556c8270e4a0 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee6c3f1740 10 client.0 _ll_register_callbacks cb 0x7fee54097df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c8278ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c8278ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37120/0 (socket says 172.21.15.16:37120) 2024-09-12T01:04:30.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee63fff640 1 -- 172.21.15.16:0/3525140497 learned_addr learned my addr 172.21.15.16:0/3525140497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c8271d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee637fe640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8271cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 -- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 msgr2=0x556c8271cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8271cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 -- 172.21.15.16:0/3525140497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 msgr2=0x556c8278ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c8278ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.319+0000 7fee62ffd640 1 -- 172.21.15.16:0/3525140497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8278c060 con 0x556c8270edc0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee62ffd640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c8271d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee58006010 tx=0x7fee58031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ae0c7436b3023e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5804a020 con 0x556c8270edc0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5409fe30 con 0x556c8270edc0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fee58005220 con 0x556c8270edc0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee58035040 con 0x556c8270edc0 2024-09-12T01:04:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee6c3f1740 10 client.26939 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c82793420 con 0x556c8270edc0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5409fe30 con 0x556c8270edc0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee580037b0 con 0x556c8270edc0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee6c3f1740 10 client.26939 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.320+0000 7fee6c3f1740 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee6c3f1740 20 client.26939 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c82795bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee6c3f1740 4 client.26939 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c825f8210 con 0x556c827290c0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee637fe640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c82795bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee637fe640 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c82795bc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cdbb800e6ac649a server_cookie=d6a1c61ed3288e9c in_seq=0 out_seq=0 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee42ffd640 10 client.26939 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:30.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee42ffd640 1 -- 172.21.15.16:0/3525140497 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556c825f8210 con 0x556c827290c0 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee42ffd640 10 client.26939 handle_command_reply: tid=0 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.321+0000 7fee6c3f1740 1 client.26939 shutdown 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 msgr2=0x556c8271d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c8271d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fee58006010 tx=0x7fee58031b60 comp rx=0 tx=0).stop 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 shutdown_connections 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556c827290c0 0x556c82795bc0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c82719680 0x556c8278ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c8270edc0 0x556c8271d260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee42ffd640 0 client.26939 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 --2- 172.21.15.16:0/3525140497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c8270e4a0 0x556c8271cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 >> 172.21.15.16:0/3525140497 conn(0x556c8270aa30 msgr2=0x556c82709350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 shutdown_connections 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 1 -- 172.21.15.16:0/3525140497 wait complete. 2024-09-12T01:04:30.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6c3f1740 20 client.26939 trim_cache size 0 max 16384 2024-09-12T01:04:30.327 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:30.327 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:04:30.327 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:04:30.327 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 msgr2=0x7fee4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 0x7fee4c0a51b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009440 tx=0x7fee5c00c000 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 msgr2=0x7fee6419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c890 tx=0x7fee5400cd50 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 shutdown_connections 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee4c0a2cf0 0x7fee4c0a51b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee64106f10 0x7fee6419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee64102790 0x7fee6419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 --2- 172.21.15.16:0/3000812041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee64101de0 0x7fee6419a750 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.322+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 >> 172.21.15.16:0/3000812041 conn(0x7fee640fd8c0 msgr2=0x7fee640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.323+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 shutdown_connections 2024-09-12T01:04:30.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:30.323+0000 7fee6a018640 1 -- 172.21.15.16:0/3000812041 wait complete. 2024-09-12T01:04:30.954 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:30 smithi016 ceph-mon[30972]: pgmap v260: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T01:04:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:30 smithi099 ceph-mon[35654]: pgmap v260: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T01:04:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:30 smithi159 ceph-mon[35566]: pgmap v260: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-12T01:04:31.148 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:04:31.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:31.402 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:31.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 -- 172.21.15.16:0/4233429005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c102d30 msgr2=0x7fe98c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:31.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 --2- 172.21.15.16:0/4233429005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c102d30 0x7fe98c103110 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009930 tx=0x7fe97c02a420 comp rx=0 tx=0).stop 2024-09-12T01:04:31.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 -- 172.21.15.16:0/4233429005 shutdown_connections 2024-09-12T01:04:31.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 --2- 172.21.15.16:0/4233429005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1082a0 0x7fe98c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:31.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 --2- 172.21.15.16:0/4233429005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c1036e0 0x7fe98c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 --2- 172.21.15.16:0/4233429005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c102d30 0x7fe98c103110 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 -- 172.21.15.16:0/4233429005 >> 172.21.15.16:0/4233429005 conn(0x7fe98c0fc7d0 msgr2=0x7fe98c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 -- 172.21.15.16:0/4233429005 shutdown_connections 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.896+0000 7fe99263e640 1 -- 172.21.15.16:0/4233429005 wait complete. 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 Processor -- start 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 -- start start 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 0x7fe98c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1036e0 0x7fe98c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1120a0 con 0x7fe98c1082a0 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c111f20 con 0x7fe98c102d30 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe99263e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c112220 con 0x7fe98c1036e0 2024-09-12T01:04:31.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37138/0 (socket says 172.21.15.16:37138) 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 -- 172.21.15.16:0/575485588 learned_addr learned my addr 172.21.15.16:0/575485588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe98b7fe640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1036e0 0x7fe98c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe98bfff640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 0x7fe98c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 -- 172.21.15.16:0/575485588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1036e0 msgr2=0x7fe98c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1036e0 0x7fe98c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 -- 172.21.15.16:0/575485588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 msgr2=0x7fe98c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 0x7fe98c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe990bdb640 1 -- 172.21.15.16:0/575485588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c19d570 con 0x7fe98c1082a0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.897+0000 7fe98bfff640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 0x7fe98c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe990bdb640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe98400eaa0 tx=0x7fe98400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400cdb0 con 0x7fe98c1082a0 2024-09-12T01:04:31.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe984005020 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9840116b0 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c19d860 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.898+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a5140 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.899+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c06a850 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.900+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe984005560 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.901+0000 7fe9897fa640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 0x7fe9740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.901+0000 7fe98bfff640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 0x7fe9740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.901+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe9840cf660 con 0x7fe98c1082a0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.902+0000 7fe98bfff640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 0x7fe9740a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:31.906+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe984016030 con 0x7fe98c1082a0 2024-09-12T01:04:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.117+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c10a2b0 con 0x7fe98c1082a0 2024-09-12T01:04:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.118+0000 7fe9897fa640 1 -- 172.21.15.16:0/575485588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fe98409c220 con 0x7fe98c1082a0 2024-09-12T01:04:32.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:32.118 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:32.118 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 msgr2=0x7fe9740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 0x7fe9740a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe97c000c00 tx=0x7fe97c005a20 comp rx=0 tx=0).stop 2024-09-12T01:04:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 msgr2=0x7fe98c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:32.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe98400eaa0 tx=0x7fe98400ef60 comp rx=0 tx=0).stop 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 shutdown_connections 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9740a2cf0 0x7fe9740a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe98c1082a0 0x7fe98c19ce90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe98c1036e0 0x7fe98c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 --2- 172.21.15.16:0/575485588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe98c102d30 0x7fe98c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 >> 172.21.15.16:0/575485588 conn(0x7fe98c0fc7d0 msgr2=0x7fe98c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 shutdown_connections 2024-09-12T01:04:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:32.121+0000 7fe99263e640 1 -- 172.21.15.16:0/575485588 wait complete. 2024-09-12T01:04:32.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:32 smithi016 ceph-mon[30972]: pgmap v261: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:32.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/575485588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:32.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:04:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:32 smithi099 ceph-mon[35654]: pgmap v261: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:32 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/575485588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:32 smithi159 ceph-mon[35566]: pgmap v261: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:32 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/575485588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- 172.21.15.16:0/780438468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 msgr2=0x7f3c7c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 --2- 172.21.15.16:0/780438468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c1067e0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009900 tx=0x7f3c6402a410 comp rx=0 tx=0).stop 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- 172.21.15.16:0/780438468 shutdown_connections 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 --2- 172.21.15.16:0/780438468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 --2- 172.21.15.16:0/780438468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c1067e0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 --2- 172.21.15.16:0/780438468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- 172.21.15.16:0/780438468 >> 172.21.15.16:0/780438468 conn(0x7f3c7c0fd8c0 msgr2=0x7f3c7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- 172.21.15.16:0/780438468 shutdown_connections 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- 172.21.15.16:0/780438468 wait complete. 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 Processor -- start 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.682+0000 7f3c82880640 1 -- start start 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c112000 con 0x7f3c7c102790 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c111e80 con 0x7f3c7c101de0 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c112180 con 0x7f3c7c106f10 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37164/0 (socket says 172.21.15.16:37164) 2024-09-12T01:04:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 -- 172.21.15.16:0/2220022198 learned_addr learned my addr 172.21.15.16:0/2220022198 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 -- 172.21.15.16:0/2220022198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 msgr2=0x7f3c7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 -- 172.21.15.16:0/2220022198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 msgr2=0x7f3c7c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 -- 172.21.15.16:0/2220022198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7c1a69f0 con 0x7f3c7c102790 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c73fff640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c64004770 tx=0x7f3c64003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c79ffb640 1 -- 172.21.15.16:0/2220022198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6403f070 con 0x7f3c7c102790 2024-09-12T01:04:33.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c79ffb640 1 -- 172.21.15.16:0/2220022198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c64031ca0 con 0x7f3c7c102790 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7c1a6c80 con 0x7f3c7c102790 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c79ffb640 1 -- 172.21.15.16:0/2220022198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64035030 con 0x7f3c7c102790 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.683+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7c1a7160 con 0x7f3c7c102790 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.686+0000 7f3c79ffb640 1 -- 172.21.15.16:0/2220022198 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c64042070 con 0x7f3c7c102790 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.686+0000 7f3c84c59740 1 Processor -- start 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.686+0000 7f3c84c59740 1 -- start start 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b635d0 0x55d9b0b63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 0x55d9b0b70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0ac92f0 con 0x55d9b0b6e370 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0a70cf0 con 0x55d9b0b64b40 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c84c59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0ac52b0 con 0x55d9b0b635d0 2024-09-12T01:04:33.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c80e1d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c79ffb640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 0x7f3c680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c80e1d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:48724/0 (socket says 172.21.15.16:48724) 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c80e1d640 1 -- 172.21.15.16:0/313825979 learned_addr learned my addr 172.21.15.16:0/313825979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c73fff640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 0x55d9b0b70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 0x7f3c680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.687+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b635d0 0x55d9b0b63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c79ffb640 1 -- 172.21.15.16:0/2220022198 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f3c640efcd0 con 0x7f3c7c102790 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 -- 172.21.15.16:0/313825979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b635d0 msgr2=0x55d9b0b63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b635d0 0x55d9b0b63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 -- 172.21.15.16:0/313825979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 msgr2=0x55d9b0b70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 0x55d9b0b70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 -- 172.21.15.16:0/313825979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b0a553a0 con 0x55d9b0b64b40 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c73fff640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 0x55d9b0b70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 0x7f3c680a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c74014bc0 tx=0x7f3c740099c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c74011dc0 tx=0x7f3c7401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b16ff30113b8847c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c727fc640 1 -- 172.21.15.16:0/313825979 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004b20 con 0x55d9b0b64b40 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c727fc640 1 -- 172.21.15.16:0/313825979 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74004cc0 con 0x55d9b0b64b40 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.688+0000 7f3c727fc640 1 -- 172.21.15.16:0/313825979 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005670 con 0x55d9b0b64b40 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- 172.21.15.16:0/313825979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 msgr2=0x55d9b0b63090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3c74011dc0 tx=0x7f3c7401b000 comp rx=0 tx=0).stop 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- 172.21.15.16:0/313825979 shutdown_connections 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b6e370 0x55d9b0b70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b635d0 0x55d9b0b63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 --2- 172.21.15.16:0/313825979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b63090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- 172.21.15.16:0/313825979 >> 172.21.15.16:0/313825979 conn(0x55d9b0b2c310 msgr2=0x55d9b0b5fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- 172.21.15.16:0/313825979 shutdown_connections 2024-09-12T01:04:33.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- 172.21.15.16:0/313825979 wait complete. 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.689+0000 7f3c84c59740 1 -- start start 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 Processor -- start 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b635d0 0x55d9b0b6dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b6b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0b71010 con 0x55d9b0b64b40 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0a84060 con 0x55d9b0b635d0 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9b0b71190 con 0x55d9b0b6e370 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c84c59740 10 client.0 _ll_register_callbacks cb 0x55d9b07d78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37184/0 (socket says 172.21.15.16:37184) 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 -- 172.21.15.16:0/3096316657 learned_addr learned my addr 172.21.15.16:0/3096316657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b635d0 0x55d9b0b6dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 -- 172.21.15.16:0/3096316657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 msgr2=0x55d9b0b6b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b6b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 -- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b635d0 msgr2=0x55d9b0b6dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b635d0 0x55d9b0b6dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.690+0000 7f3c73fff640 1 -- 172.21.15.16:0/3096316657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b083a3c0 con 0x55d9b0b64b40 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c73fff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3c64031550 tx=0x7f3c6408c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0e0c53ea615d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64049030 con 0x55d9b0b64b40 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f3c6403b5f0 con 0x55d9b0b64b40 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c74004cc0 con 0x55d9b0b64b40 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c64087340 con 0x55d9b0b64b40 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 10 client.17985 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:33.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9b0b7cab0 con 0x55d9b0b64b40 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74004cc0 con 0x55d9b0b64b40 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c640dc0e0 con 0x55d9b0b64b40 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 10 client.17985 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 10 client.17985 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 20 client.17985 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0b7de00 0x55d9b0b80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 4 client.17985 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.691+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b0ad6860 con 0x55d9b0b7de00 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.692+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0b7de00 0x55d9b0b80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.692+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0b7de00 0x55d9b0b80210 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd5011f032561066 server_cookie=4f03df09d241ee8a in_seq=0 out_seq=0 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.692+0000 7f3c5b7fe640 10 client.17985 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/3096316657 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9b0ad6860 con 0x55d9b0b7de00 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c5b7fe640 10 client.17985 handle_command_reply: tid=0 2024-09-12T01:04:33.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 client.17985 shutdown 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 msgr2=0x55d9b0b6b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3c64031550 tx=0x7f3c6408c110 comp rx=0 tx=0).stop 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 shutdown_connections 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0b7de00 0x55d9b0b80210 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b6b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6b010 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3096316657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b635d0 0x55d9b0b6dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 >> 172.21.15.16:0/3096316657 conn(0x55d9b0b2c310 msgr2=0x55d9b0b65960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c5b7fe640 0 client.17985 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 shutdown_connections 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 1 -- 172.21.15.16:0/3096316657 wait complete. 2024-09-12T01:04:33.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.693+0000 7f3c84c59740 20 client.17985 trim_cache size 0 max 16384 2024-09-12T01:04:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 Processor -- start 2024-09-12T01:04:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 -- start start 2024-09-12T01:04:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1afad0 con 0x55d9b0b635d0 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1b1610 con 0x55d9b0b64b40 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c84c59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1b01a0 con 0x55d9b0b6e370 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37186/0 (socket says 172.21.15.16:37186) 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 -- 172.21.15.16:0/3643638197 learned_addr learned my addr 172.21.15.16:0/3643638197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c73fff640 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 -- 172.21.15.16:0/3643638197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 msgr2=0x55d9b0b85ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b85ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 -- 172.21.15.16:0/3643638197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 msgr2=0x55d9b0b6cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.707+0000 7f3c7bfff640 1 -- 172.21.15.16:0/3643638197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b0b5fed0 con 0x55d9b0b635d0 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c007880 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca723e29fad8d64a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c71ffb640 1 -- 172.21.15.16:0/3643638197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c016a90 con 0x55d9b0b635d0 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c71ffb640 1 -- 172.21.15.16:0/3643638197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c020030 con 0x55d9b0b635d0 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c71ffb640 1 -- 172.21.15.16:0/3643638197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c024d80 con 0x55d9b0b635d0 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 -- 172.21.15.16:0/3643638197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 msgr2=0x55d9b0b7a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c007880 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 -- 172.21.15.16:0/3643638197 shutdown_connections 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b6e370 0x55d9b0b85ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b64b40 0x55d9b0b6cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.708+0000 7f3c84c59740 1 --2- 172.21.15.16:0/3643638197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b7a2b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- 172.21.15.16:0/3643638197 >> 172.21.15.16:0/3643638197 conn(0x55d9b0b2c310 msgr2=0x55d9b0b6c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- 172.21.15.16:0/3643638197 shutdown_connections 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- 172.21.15.16:0/3643638197 wait complete. 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- start start 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 Processor -- start 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 0x55d9b0b71f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b6e370 0x55d9b0b72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1b0f40 con 0x55d9b0b635d0 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1b0870 con 0x55d9b0b6e370 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7c1adad0 con 0x55d9b0b64b40 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.709+0000 7f3c84c59740 10 client.0 _ll_register_callbacks cb 0x7f3c64005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37190/0 (socket says 172.21.15.16:37190) 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 -- 172.21.15.16:0/586252054 learned_addr learned my addr 172.21.15.16:0/586252054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c73fff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 0x55d9b0b71f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c80e1d640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b6e370 0x55d9b0b72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 -- 172.21.15.16:0/586252054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 msgr2=0x55d9b0b71f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 0x55d9b0b71f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 -- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b6e370 msgr2=0x55d9b0b72440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b6e370 0x55d9b0b72440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 -- 172.21.15.16:0/586252054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b0b72c70 con 0x55d9b0b635d0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c73fff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 0x55d9b0b71f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c7bfff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01f040 tx=0x7f3c6c024f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15035b9acad92303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c029070 con 0x55d9b0b635d0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f3c6c0245d0 con 0x55d9b0b635d0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c020030 con 0x55d9b0b635d0 2024-09-12T01:04:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0247d0 con 0x55d9b0b635d0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c84c59740 10 client.17991 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.710+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9b0be8b10 con 0x55d9b0b635d0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c020030 con 0x55d9b0b635d0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6c0143b0 con 0x55d9b0b635d0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 10 client.17991 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 10 client.17991 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 20 client.17991 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0be9170 0x55d9b0beb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 4 client.17991 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9b0ad0d70 con 0x55d9b0be9170 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.711+0000 7f3c73fff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0be9170 0x55d9b0beb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c73fff640 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0be9170 0x55d9b0beb560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3296d886d0686337 server_cookie=16c6ab163efad46 in_seq=0 out_seq=0 2024-09-12T01:04:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c5b7fe640 10 client.17991 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c5b7fe640 1 -- 172.21.15.16:0/586252054 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9b0ad0d70 con 0x55d9b0be9170 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c5b7fe640 10 client.17991 handle_command_reply: tid=0 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 client.17991 shutdown 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 msgr2=0x55d9b0b76bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01f040 tx=0x7f3c6c024f20 comp rx=0 tx=0).stop 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 shutdown_connections 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55d9b0be9170 0x55d9b0beb560 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9b0b6e370 0x55d9b0b72440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b0b64b40 0x55d9b0b71f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 --2- 172.21.15.16:0/586252054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9b0b635d0 0x55d9b0b76bd0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 >> 172.21.15.16:0/586252054 conn(0x55d9b0b2c310 msgr2=0x55d9b0b6d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c5b7fe640 0 client.17991 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 shutdown_connections 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 1 -- 172.21.15.16:0/586252054 wait complete. 2024-09-12T01:04:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.712+0000 7f3c84c59740 20 client.17991 trim_cache size 0 max 16384 2024-09-12T01:04:33.719 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 msgr2=0x7f3c680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 0x7f3c680a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c74014bc0 tx=0x7f3c740099c0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 msgr2=0x7f3c7c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f3c64004770 tx=0x7f3c64003c10 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 shutdown_connections 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3c680a2cf0 0x7f3c680a51b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c7c106f10 0x7f3c7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c102790 0x7f3c7c074a30 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 --2- 172.21.15.16:0/2220022198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c7c101de0 0x7f3c7c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 >> 172.21.15.16:0/2220022198 conn(0x7f3c7c0fd8c0 msgr2=0x7f3c7c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 shutdown_connections 2024-09-12T01:04:33.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:33.713+0000 7f3c82880640 1 -- 172.21.15.16:0/2220022198 wait complete. 2024-09-12T01:04:34.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:04:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:34 smithi016 ceph-mon[30972]: pgmap v262: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:34 smithi099 ceph-mon[35654]: pgmap v262: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:34 smithi159 ceph-mon[35566]: pgmap v262: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 10 op/s 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 -- 172.21.15.16:0/2610732741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c1082a0 msgr2=0x7fe33c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 --2- 172.21.15.16:0/2610732741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c1082a0 0x7fe33c10ec40 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fe334009cd0 tx=0x7fe33402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 -- 172.21.15.16:0/2610732741 shutdown_connections 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 --2- 172.21.15.16:0/2610732741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c1082a0 0x7fe33c10ec40 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 --2- 172.21.15.16:0/2610732741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 --2- 172.21.15.16:0/2610732741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c102d30 0x7fe33c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 -- 172.21.15.16:0/2610732741 >> 172.21.15.16:0/2610732741 conn(0x7fe33c0fc7d0 msgr2=0x7fe33c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 -- 172.21.15.16:0/2610732741 shutdown_connections 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.334+0000 7fe341db8640 1 -- 172.21.15.16:0/2610732741 wait complete. 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 Processor -- start 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 -- start start 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 0x7fe33c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1120a0 con 0x7fe33c102d30 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c111f20 con 0x7fe33c1036e0 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe341db8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c112220 con 0x7fe33c1082a0 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe33b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe33b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37200/0 (socket says 172.21.15.16:37200) 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe33b7fe640 1 -- 172.21.15.16:0/1276351326 learned_addr learned my addr 172.21.15.16:0/1276351326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe33bfff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 0x7fe33c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.335+0000 7fe333fff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 -- 172.21.15.16:0/1276351326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 msgr2=0x7fe33c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 0x7fe33c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 -- 172.21.15.16:0/1276351326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 msgr2=0x7fe33c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 -- 172.21.15.16:0/1276351326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c1a69d0 con 0x7fe33c102d30 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33bfff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 0x7fe33c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:35.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe333fff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00d950 tx=0x7fe32c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe3397fa640 1 -- 172.21.15.16:0/1276351326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017030 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe3397fa640 1 -- 172.21.15.16:0/1276351326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c004020 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1a6cc0 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe3397fa640 1 -- 172.21.15.16:0/1276351326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0047a0 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.336+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1a71b0 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.338+0000 7fe3397fa640 1 -- 172.21.15.16:0/1276351326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe32c0041d0 con 0x7fe33c102d30 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 Processor -- start 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 -- start start 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27673090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27673a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555d275d92f0 con 0x555d2767e370 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555d27580cf0 con 0x555d276735d0 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe344191740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d275d52b0 con 0x555d27674b40 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37202/0 (socket says 172.21.15.16:37202) 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe3397fa640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 0x7fe3280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe333fff640 1 -- 172.21.15.16:0/2976178431 learned_addr learned my addr 172.21.15.16:0/2976178431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe33bfff640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27673090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27673a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.339+0000 7fe33bfff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 0x7fe3280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe3397fa640 1 -- 172.21.15.16:0/1276351326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe32c0cef50 con 0x7fe33c102d30 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 -- 172.21.15.16:0/2976178431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 msgr2=0x555d27673090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27673090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 -- 172.21.15.16:0/2976178431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 msgr2=0x555d27673a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27673a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 -- 172.21.15.16:0/2976178431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d275653a0 con 0x555d2767e370 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe33bfff640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27673090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe333fff640 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe32400cb90 tx=0x7fe32400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32131c1440e55217 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe33bfff640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 0x7fe3280a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe3340095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe331ffb640 1 -- 172.21.15.16:0/2976178431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324005220 con 0x555d2767e370 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe331ffb640 1 -- 172.21.15.16:0/2976178431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004c80 con 0x555d2767e370 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.340+0000 7fe331ffb640 1 -- 172.21.15.16:0/2976178431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3240107c0 con 0x555d2767e370 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- 172.21.15.16:0/2976178431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 msgr2=0x555d27680760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe32400cb90 tx=0x7fe32400e720 comp rx=0 tx=0).stop 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- 172.21.15.16:0/2976178431 shutdown_connections 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27680760 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27673a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 --2- 172.21.15.16:0/2976178431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27673090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- 172.21.15.16:0/2976178431 >> 172.21.15.16:0/2976178431 conn(0x555d2763c310 msgr2=0x555d2766fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- 172.21.15.16:0/2976178431 shutdown_connections 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- 172.21.15.16:0/2976178431 wait complete. 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 -- start start 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.341+0000 7fe344191740 1 Processor -- start 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2767dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d2767b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555d27681070 con 0x555d27674b40 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555d27594060 con 0x555d2767e370 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d276811f0 con 0x555d276735d0 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe344191740 10 client.0 _ll_register_callbacks cb 0x555d272e78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37214/0 (socket says 172.21.15.16:37214) 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 -- 172.21.15.16:0/1523876401 learned_addr learned my addr 172.21.15.16:0/1523876401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe33bfff640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d2767b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2767dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 -- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 msgr2=0x555d2767dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2767dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 -- 172.21.15.16:0/1523876401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 msgr2=0x555d2767b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d2767b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 -- 172.21.15.16:0/1523876401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2734a3c0 con 0x555d27674b40 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe333fff640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe3240058c0 tx=0x7fe3240104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32501c6b89b3f07b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe31affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.342+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324029060 con 0x555d27674b40 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe324004c80 con 0x555d27674b40 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe3240048d0 con 0x555d27674b40 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe32400ce90 con 0x555d27674b40 2024-09-12T01:04:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 10 client.18003 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d2768cac0 con 0x555d27674b40 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004c80 con 0x555d27674b40 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe3240244a0 con 0x555d27674b40 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 10 client.18003 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 10 client.18003 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 20 client.18003 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d2768de10 0x555d27690200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 4 client.18003 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d275e6860 con 0x555d2768de10 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.343+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d2768de10 0x555d27690200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.344+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d2768de10 0x555d27690200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc6cc1b41e9330bd server_cookie=5807634f39f5c460 in_seq=0 out_seq=0 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.344+0000 7fe31affd640 10 client.18003 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.344+0000 7fe31affd640 1 -- 172.21.15.16:0/1523876401 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d275e6860 con 0x555d2768de10 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.344+0000 7fe31affd640 10 client.18003 handle_command_reply: tid=0 2024-09-12T01:04:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 client.18003 shutdown 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 msgr2=0x555d2767af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe3240058c0 tx=0x7fe3240104c0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 shutdown_connections 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d2767b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d2768de10 0x555d27690200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d2767af50 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 --2- 172.21.15.16:0/1523876401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2767dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe31affd640 0 client.18003 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 >> 172.21.15.16:0/1523876401 conn(0x555d2763c310 msgr2=0x555d27675960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 shutdown_connections 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 1 -- 172.21.15.16:0/1523876401 wait complete. 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.345+0000 7fe344191740 20 client.18003 trim_cache size 0 max 16384 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 Processor -- start 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 -- start start 2024-09-12T01:04:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2768a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d27676660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1afac0 con 0x555d27674b40 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1b1600 con 0x555d2767e370 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe344191740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1b0190 con 0x555d276735d0 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.358+0000 7fe333fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37228/0 (socket says 172.21.15.16:37228) 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 -- 172.21.15.16:0/3069571740 learned_addr learned my addr 172.21.15.16:0/3069571740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2768a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe33bfff640 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d27676660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 -- 172.21.15.16:0/3069571740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 msgr2=0x555d2768a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2768a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 -- 172.21.15.16:0/3069571740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 msgr2=0x555d27676660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d27676660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 -- 172.21.15.16:0/3069571740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d27676ba0 con 0x555d27674b40 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe333fff640 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe32400f010 tx=0x7fe324005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da59d1b59449ef6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.359+0000 7fe3317fa640 1 -- 172.21.15.16:0/3069571740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32401e020 con 0x555d27674b40 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe3317fa640 1 -- 172.21.15.16:0/3069571740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004db0 con 0x555d27674b40 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe3317fa640 1 -- 172.21.15.16:0/3069571740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324024640 con 0x555d27674b40 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 -- 172.21.15.16:0/3069571740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 msgr2=0x555d27676120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe32400f010 tx=0x7fe324005a10 comp rx=0 tx=0).stop 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 -- 172.21.15.16:0/3069571740 shutdown_connections 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d2767e370 0x555d27676660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d27674b40 0x555d27676120 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 --2- 172.21.15.16:0/3069571740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d276735d0 0x555d2768a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 -- 172.21.15.16:0/3069571740 >> 172.21.15.16:0/3069571740 conn(0x555d2763c310 msgr2=0x555d2767a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 -- 172.21.15.16:0/3069571740 shutdown_connections 2024-09-12T01:04:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.360+0000 7fe344191740 1 -- 172.21.15.16:0/3069571740 wait complete. 2024-09-12T01:04:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 -- start start 2024-09-12T01:04:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 Processor -- start 2024-09-12T01:04:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27686a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27677aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1b0f30 con 0x555d2767e370 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1b0860 con 0x555d276735d0 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1adac0 con 0x555d27674b40 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe344191740 10 client.0 _ll_register_callbacks cb 0x7fe324014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe33bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.361+0000 7fe33bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37240/0 (socket says 172.21.15.16:37240) 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 -- 172.21.15.16:0/3250825545 learned_addr learned my addr 172.21.15.16:0/3250825545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe333fff640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27677aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27686a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 -- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 msgr2=0x555d27677aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27677aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 -- 172.21.15.16:0/3250825545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 msgr2=0x555d27686a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27686a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 -- 172.21.15.16:0/3250825545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d276785d0 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe33bfff640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe334009cd0 tx=0x7fe334005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2542cf0f75457280 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe31affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334058060 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe334004240 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe324004db0 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe334043070 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.362+0000 7fe344191740 10 client.18009 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d276f85b0 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe324004db0 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe334052070 con 0x555d2767e370 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 10 client.18009 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 10 client.18009 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 20 client.18009 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d276f8d90 0x555d276fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 4 client.18009 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d275e0d70 con 0x555d276f8d90 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.363+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d276f8d90 0x555d276fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe33b7fe640 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d276f8d90 0x555d276fb1b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a76ca1ba7c2be856 server_cookie=933f4fa9f61367b9 in_seq=0 out_seq=0 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe31affd640 10 client.18009 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe31affd640 1 -- 172.21.15.16:0/3250825545 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d275e0d70 con 0x555d276f8d90 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe31affd640 10 client.18009 handle_command_reply: tid=0 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 client.18009 shutdown 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 msgr2=0x555d27677fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe334009cd0 tx=0x7fe334005ef0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 shutdown_connections 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555d276f8d90 0x555d276fb1b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555d2767e370 0x555d27677fe0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe31affd640 0 client.18009 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d27674b40 0x555d27677aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 --2- 172.21.15.16:0/3250825545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555d276735d0 0x555d27686a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 >> 172.21.15.16:0/3250825545 conn(0x555d2763c310 msgr2=0x555d2766e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 shutdown_connections 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.364+0000 7fe344191740 1 -- 172.21.15.16:0/3250825545 wait complete. 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe344191740 20 client.18009 trim_cache size 0 max 16384 2024-09-12T01:04:35.378 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 msgr2=0x7fe3280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 0x7fe3280a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe334006010 tx=0x7fe3340095c0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 msgr2=0x7fe33c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00d950 tx=0x7fe32c00de10 comp rx=0 tx=0).stop 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 shutdown_connections 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe3280a2cf0 0x7fe3280a51b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe33c1082a0 0x7fe33c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe33c1036e0 0x7fe33c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 --2- 172.21.15.16:0/1276351326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe33c102d30 0x7fe33c100990 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 >> 172.21.15.16:0/1276351326 conn(0x7fe33c0fc7d0 msgr2=0x7fe33c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 shutdown_connections 2024-09-12T01:04:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:35.365+0000 7fe341db8640 1 -- 172.21.15.16:0/1276351326 wait complete. 2024-09-12T01:04:36.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:04:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:36.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:36 smithi016 ceph-mon[30972]: pgmap v263: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 -- 172.21.15.16:0/1920267664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4100e20 msgr2=0x7f5be4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 --2- 172.21.15.16:0/1920267664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4100e20 0x7f5be4101200 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009900 tx=0x7f5bd402a480 comp rx=0 tx=0).stop 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 -- 172.21.15.16:0/1920267664 shutdown_connections 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 --2- 172.21.15.16:0/1920267664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102ef0 0x7f5be410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 --2- 172.21.15.16:0/1920267664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4102570 0x7f5be41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 --2- 172.21.15.16:0/1920267664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4100e20 0x7f5be4101200 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 -- 172.21.15.16:0/1920267664 >> 172.21.15.16:0/1920267664 conn(0x7f5be40fc7d0 msgr2=0x7f5be40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 -- 172.21.15.16:0/1920267664 shutdown_connections 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 -- 172.21.15.16:0/1920267664 wait complete. 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.835+0000 7f5bea79c640 1 Processor -- start 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 -- start start 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 0x7f5be4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102570 0x7f5be4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110fa0 con 0x7f5be4102ef0 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4110e20 con 0x7f5be4100e20 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5bea79c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4111120 con 0x7f5be4102570 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37266/0 (socket says 172.21.15.16:37266) 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 -- 172.21.15.16:0/3105719125 learned_addr learned my addr 172.21.15.16:0/3105719125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be3fff640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 0x7f5be4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be37fe640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102570 0x7f5be4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 -- 172.21.15.16:0/3105719125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102570 msgr2=0x7f5be4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102570 0x7f5be4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 -- 172.21.15.16:0/3105719125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 msgr2=0x7f5be4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 0x7f5be4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be8d39640 1 -- 172.21.15.16:0/3105719125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be419d4c0 con 0x7f5be4102ef0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.836+0000 7f5be3fff640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 0x7f5be4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5be8d39640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00b970 tx=0x7f5bdc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5be17fa640 1 -- 172.21.15.16:0/3105719125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc019070 con 0x7f5be4102ef0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5be17fa640 1 -- 172.21.15.16:0/3105719125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc00cb40 con 0x7f5be4102ef0 2024-09-12T01:04:36.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be419d7b0 con 0x7f5be4102ef0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5be17fa640 1 -- 172.21.15.16:0/3105719125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc007650 con 0x7f5be4102ef0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.837+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a5090 con 0x7f5be4102ef0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.839+0000 7f5be17fa640 1 -- 172.21.15.16:0/3105719125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5bdc01c070 con 0x7f5be4102ef0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.839+0000 7f5becb75740 1 Processor -- start 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.839+0000 7f5becb75740 1 -- start start 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d6b40 0x561b6a6d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d55d0 0x561b6a6d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561b6a63b2f0 con 0x561b6a6e0370 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561b6a5e2cf0 con 0x561b6a6d6b40 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5becb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561b6a6372b0 con 0x561b6a6d55d0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37280/0 (socket says 172.21.15.16:37280) 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 -- 172.21.15.16:0/2857359476 learned_addr learned my addr 172.21.15.16:0/2857359476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d6b40 0x561b6a6d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be3fff640 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d55d0 0x561b6a6d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be17fa640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 0x7f5bcc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 -- 172.21.15.16:0/2857359476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d55d0 msgr2=0x561b6a6d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d55d0 0x561b6a6d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 -- 172.21.15.16:0/2857359476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d6b40 msgr2=0x561b6a6d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d6b40 0x561b6a6d5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be37fe640 1 -- 172.21.15.16:0/2857359476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6a5c73a0 con 0x561b6a6e0370 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.840+0000 7f5be8d39640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 0x7f5bcc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5be37fe640 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8013930 tx=0x7f5bd800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a72fd631d220e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5be17fa640 1 -- 172.21.15.16:0/3105719125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f5bdc0ce9e0 con 0x7f5be4102ef0 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5bca7fc640 1 -- 172.21.15.16:0/2857359476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800e950 con 0x561b6a6e0370 2024-09-12T01:04:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5bca7fc640 1 -- 172.21.15.16:0/2857359476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8011440 con 0x561b6a6e0370 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5bca7fc640 1 -- 172.21.15.16:0/2857359476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8010690 con 0x561b6a6e0370 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5becb75740 1 -- 172.21.15.16:0/2857359476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 msgr2=0x561b6a6e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5be8d39640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 0x7f5bcc0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0cc2f0 tx=0x7f5bdc0bb040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.841+0000 7f5becb75740 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8013930 tx=0x7f5bd800b460 comp rx=0 tx=0).stop 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- 172.21.15.16:0/2857359476 shutdown_connections 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6e2760 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d55d0 0x561b6a6d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- 172.21.15.16:0/2857359476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d6b40 0x561b6a6d5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- 172.21.15.16:0/2857359476 >> 172.21.15.16:0/2857359476 conn(0x561b6a69e310 msgr2=0x561b6a6d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- 172.21.15.16:0/2857359476 shutdown_connections 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- 172.21.15.16:0/2857359476 wait complete. 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- start start 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 Processor -- start 2024-09-12T01:04:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d55d0 0x561b6a6dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561b6a6e3070 con 0x561b6a6e0370 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561b6a5f6060 con 0x561b6a6d55d0 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.842+0000 7f5becb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561b6a6e31f0 con 0x561b6a6d6b40 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5becb75740 10 client.0 _ll_register_callbacks cb 0x561b6a3498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37290/0 (socket says 172.21.15.16:37290) 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 -- 172.21.15.16:0/907843232 learned_addr learned my addr 172.21.15.16:0/907843232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be3fff640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d55d0 0x561b6a6dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 -- 172.21.15.16:0/907843232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 msgr2=0x561b6a6dd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 -- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d55d0 msgr2=0x561b6a6dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d55d0 0x561b6a6dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 -- 172.21.15.16:0/907843232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6a3ac3c0 con 0x561b6a6e0370 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be3fff640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5be37fe640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80057c0 tx=0x7f5bd8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51a94a2b3c1cb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5bc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80115f0 con 0x561b6a6e0370 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f5bd8011790 con 0x561b6a6e0370 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.843+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd8011440 con 0x561b6a6e0370 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bd8032050 con 0x561b6a6e0370 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 10 client.18021 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b6a6eeb40 con 0x561b6a6e0370 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8011440 con 0x561b6a6e0370 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5bd800ea50 con 0x561b6a6e0370 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 10 client.18021 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 10 client.18021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 20 client.18021 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a6efef0 0x561b6a6f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 4 client.18021 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b6a648860 con 0x561b6a6efef0 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.844+0000 7f5be3fff640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a6efef0 0x561b6a6f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.845+0000 7f5be3fff640 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a6efef0 0x561b6a6f22e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3899126830ab8910 server_cookie=3774e231578972fe in_seq=0 out_seq=0 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.845+0000 7f5bc37fe640 10 client.18021 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.845+0000 7f5bc37fe640 1 -- 172.21.15.16:0/907843232 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b6a648860 con 0x561b6a6efef0 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.845+0000 7f5bc37fe640 10 client.18021 handle_command_reply: tid=0 2024-09-12T01:04:36.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.845+0000 7f5becb75740 1 client.18021 shutdown 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 msgr2=0x561b6a6dd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80057c0 tx=0x7f5bd8011e50 comp rx=0 tx=0).stop 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 shutdown_connections 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a6efef0 0x561b6a6f22e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6e0370 0x561b6a6dd640 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 --2- 172.21.15.16:0/907843232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6d55d0 0x561b6a6dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5bc37fe640 0 client.18021 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 >> 172.21.15.16:0/907843232 conn(0x561b6a69e310 msgr2=0x561b6a6d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 shutdown_connections 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 1 -- 172.21.15.16:0/907843232 wait complete. 2024-09-12T01:04:36.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.846+0000 7f5becb75740 20 client.18021 trim_cache size 0 max 16384 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 Processor -- start 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 -- start start 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ad8a0 con 0x561b6a6d55d0 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41af3e0 con 0x561b6a6e0370 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5becb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41adf70 con 0x561b6a6d6b40 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37300/0 (socket says 172.21.15.16:37300) 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 -- 172.21.15.16:0/610547573 learned_addr learned my addr 172.21.15.16:0/610547573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be3fff640 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 -- 172.21.15.16:0/610547573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 msgr2=0x561b6a6dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 -- 172.21.15.16:0/610547573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 msgr2=0x561b6a6f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be37fe640 1 -- 172.21.15.16:0/610547573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6a6d8120 con 0x561b6a6d55d0 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.859+0000 7f5be3fff640 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5be37fe640 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8006010 tx=0x7f5bd80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c94bb9cc2a303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5bc9ffb640 1 -- 172.21.15.16:0/610547573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800b900 con 0x561b6a6d55d0 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5bc9ffb640 1 -- 172.21.15.16:0/610547573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8013300 con 0x561b6a6d55d0 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5bc9ffb640 1 -- 172.21.15.16:0/610547573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8011440 con 0x561b6a6d55d0 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 -- 172.21.15.16:0/610547573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 msgr2=0x561b6a6ec310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8006010 tx=0x7f5bd80048d0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 -- 172.21.15.16:0/610547573 shutdown_connections 2024-09-12T01:04:36.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 --2- 172.21.15.16:0/610547573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6ec310 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.860+0000 7f5becb75740 1 -- 172.21.15.16:0/610547573 >> 172.21.15.16:0/610547573 conn(0x561b6a69e310 msgr2=0x561b6a6db920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- 172.21.15.16:0/610547573 shutdown_connections 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- 172.21.15.16:0/610547573 wait complete. 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- start start 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 Processor -- start 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41aed10 con 0x561b6a6d55d0 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ae640 con 0x561b6a6e0370 2024-09-12T01:04:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ab8a0 con 0x561b6a6d6b40 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5becb75740 10 client.0 _ll_register_callbacks cb 0x7f5bd800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37314/0 (socket says 172.21.15.16:37314) 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.861+0000 7f5be37fe640 1 -- 172.21.15.16:0/168424224 learned_addr learned my addr 172.21.15.16:0/168424224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be3fff640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 -- 172.21.15.16:0/168424224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 msgr2=0x561b6a6d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6d9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 -- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 msgr2=0x561b6a6da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 -- 172.21.15.16:0/168424224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6a6da610 con 0x561b6a6d55d0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5be37fe640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b810 tx=0x7f5bd800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce1056ebed7349d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5bc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800b900 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f5bd803c070 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd8013300 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bd800e930 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5becb75740 10 client.18027 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.862+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b6a75eae0 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd8013300 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5bd8047030 con 0x561b6a6d55d0 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 10 client.18027 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 10 client.18027 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 20 client.18027 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a75f110 0x561b6a761530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 4 client.18027 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b6a642d70 con 0x561b6a75f110 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5be3fff640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a75f110 0x561b6a761530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5be3fff640 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a75f110 0x561b6a761530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41173002ae467196 server_cookie=a43198fcd7a1e28 in_seq=0 out_seq=0 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.863+0000 7f5bc37fe640 10 client.18027 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5bc37fe640 1 -- 172.21.15.16:0/168424224 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b6a642d70 con 0x561b6a75f110 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5bc37fe640 10 client.18027 handle_command_reply: tid=0 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 client.18027 shutdown 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 msgr2=0x561b6a6e8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b810 tx=0x7f5bd800b5a0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 shutdown_connections 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x561b6a75f110 0x561b6a761530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5bc37fe640 0 client.18027 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561b6a6e0370 0x561b6a6da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561b6a6d6b40 0x561b6a6d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 --2- 172.21.15.16:0/168424224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b6a6d55d0 0x561b6a6e8950 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 >> 172.21.15.16:0/168424224 conn(0x561b6a69e310 msgr2=0x561b6a6dc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 shutdown_connections 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 1 -- 172.21.15.16:0/168424224 wait complete. 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.864+0000 7f5becb75740 20 client.18027 trim_cache size 0 max 16384 2024-09-12T01:04:36.881 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 msgr2=0x7f5bcc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 0x7f5bcc0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0cc2f0 tx=0x7f5bdc0bb040 comp rx=0 tx=0).stop 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 msgr2=0x7f5be419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00b970 tx=0x7f5bdc00be30 comp rx=0 tx=0).stop 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 shutdown_connections 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5bcc0a2cf0 0x7f5bcc0a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5be4102ef0 0x7f5be419cde0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5be4102570 0x7f5be4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 --2- 172.21.15.16:0/3105719125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5be4100e20 0x7f5be4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 >> 172.21.15.16:0/3105719125 conn(0x7f5be40fc7d0 msgr2=0x7f5be406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 shutdown_connections 2024-09-12T01:04:36.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:36.865+0000 7f5bea79c640 1 -- 172.21.15.16:0/3105719125 wait complete. 2024-09-12T01:04:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:36 smithi099 ceph-mon[35654]: pgmap v263: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:04:37.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:36 smithi159 ceph-mon[35566]: pgmap v263: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:04:37.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:04:38.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.545+0000 7f603ac90640 1 -- 172.21.15.16:0/1974956953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102ef0 msgr2=0x7f603410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.545+0000 7f603ac90640 1 --2- 172.21.15.16:0/1974956953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102ef0 0x7f603410da40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f602c009ca0 tx=0x7f602c02a950 comp rx=0 tx=0).stop 2024-09-12T01:04:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- 172.21.15.16:0/1974956953 shutdown_connections 2024-09-12T01:04:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- 172.21.15.16:0/1974956953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102ef0 0x7f603410da40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- 172.21.15.16:0/1974956953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102570 0x7f60341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- 172.21.15.16:0/1974956953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034100e20 0x7f6034101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- 172.21.15.16:0/1974956953 >> 172.21.15.16:0/1974956953 conn(0x7f60340fc7d0 msgr2=0x7f60340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- 172.21.15.16:0/1974956953 shutdown_connections 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- 172.21.15.16:0/1974956953 wait complete. 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 Processor -- start 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- start start 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034102570 0x7f60341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102ef0 0x7f6034199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6034110f80 con 0x7f6034100e20 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6034110e00 con 0x7f6034102ef0 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.546+0000 7f603ac90640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111100 con 0x7f6034102570 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51330/0 (socket says 172.21.15.16:51330) 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 -- 172.21.15.16:0/3229061300 learned_addr learned my addr 172.21.15.16:0/3229061300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f603922d640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102ef0 0x7f6034199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6033fff640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034102570 0x7f60341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 -- 172.21.15.16:0/3229061300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034102570 msgr2=0x7f60341998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034102570 0x7f60341998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 -- 172.21.15.16:0/3229061300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102ef0 msgr2=0x7f6034199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102ef0 0x7f6034199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 -- 172.21.15.16:0/3229061300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60341a47b0 con 0x7f6034100e20 2024-09-12T01:04:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6038a2c640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f602400d950 tx=0x7f602400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6031ffb640 1 -- 172.21.15.16:0/3229061300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017070 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60341a4aa0 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6031ffb640 1 -- 172.21.15.16:0/3229061300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6024009c30 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.547+0000 7f6031ffb640 1 -- 172.21.15.16:0/3229061300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400f6b0 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.548+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a4fe0 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.550+0000 7f6031ffb640 1 -- 172.21.15.16:0/3229061300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f60240109d0 con 0x7f6034100e20 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.550+0000 7f603d069740 1 Processor -- start 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.550+0000 7f603d069740 1 -- start start 2024-09-12T01:04:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6031ffb640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 0x7f601c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 0x5596d3e40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5596d3da62f0 con 0x5596d3e41b40 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596d3d4dcf0 con 0x5596d3e405d0 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603d069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596d3da22b0 con 0x5596d3e4b370 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51334/0 (socket says 172.21.15.16:51334) 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603922d640 1 -- 172.21.15.16:0/2045288845 learned_addr learned my addr 172.21.15.16:0/2045288845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6038a2c640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 0x7f601c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 0x5596d3e40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6031ffb640 1 -- 172.21.15.16:0/3229061300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f60240cf090 con 0x7f6034100e20 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603922d640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 -- 172.21.15.16:0/2045288845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 msgr2=0x5596d3e4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 -- 172.21.15.16:0/2045288845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 msgr2=0x5596d3e40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6033fff640 1 -- 172.21.15.16:0/2045288845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d3d323a0 con 0x5596d3e405d0 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f603922d640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.551+0000 7f6038a2c640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 0x7f601c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f602409b300 tx=0x7f6024005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f6033fff640 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 0x5596d3e40a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f602800bb40 tx=0x7f60280103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed93ad983e88bbc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f601affd640 1 -- 172.21.15.16:0/2045288845 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028004030 con 0x5596d3e405d0 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f601affd640 1 -- 172.21.15.16:0/2045288845 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280041d0 con 0x5596d3e405d0 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f601affd640 1 -- 172.21.15.16:0/2045288845 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280116a0 con 0x5596d3e405d0 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f603d069740 1 -- 172.21.15.16:0/2045288845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 msgr2=0x5596d3e40a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.552+0000 7f603d069740 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 0x5596d3e40a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f602800bb40 tx=0x7f60280103d0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- 172.21.15.16:0/2045288845 shutdown_connections 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e405d0 0x5596d3e40a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- 172.21.15.16:0/2045288845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e41b40 0x5596d3e40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- 172.21.15.16:0/2045288845 >> 172.21.15.16:0/2045288845 conn(0x5596d3e09310 msgr2=0x5596d3e3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- 172.21.15.16:0/2045288845 shutdown_connections 2024-09-12T01:04:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- 172.21.15.16:0/2045288845 wait complete. 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- start start 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 Processor -- start 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e405d0 0x5596d3e4ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e47ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5596d3e4e070 con 0x5596d3e4b370 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596d3d61060 con 0x5596d3e41b40 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.553+0000 7f603d069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596d3e4e1f0 con 0x5596d3e405d0 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603d069740 10 client.0 _ll_register_callbacks cb 0x5596d3ab48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51346/0 (socket says 172.21.15.16:51346) 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 -- 172.21.15.16:0/3095203423 learned_addr learned my addr 172.21.15.16:0/3095203423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f6033fff640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e47ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e405d0 0x5596d3e4ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 -- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e405d0 msgr2=0x5596d3e4ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e405d0 0x5596d3e4ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 -- 172.21.15.16:0/3095203423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 msgr2=0x5596d3e47ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e47ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 -- 172.21.15.16:0/3095203423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d3b173c0 con 0x5596d3e4b370 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603922d640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f602c005e30 tx=0x7f602c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a34cd7e5c39d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f6013fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c043040 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f602c030700 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60280041d0 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f602c034040 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603d069740 10 client.18039 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.554+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596d3e59b10 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60280041d0 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f602c0539a0 con 0x5596d3e4b370 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 10 client.18039 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 10 client.18039 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:04:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 20 client.18039 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3e5af20 0x5596d3e5d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 4 client.18039 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d3db3860 con 0x5596d3e5af20 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.555+0000 7f6033fff640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3e5af20 0x5596d3e5d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.556+0000 7f6033fff640 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3e5af20 0x5596d3e5d310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=919766f70f768948 server_cookie=3a945698d732129d in_seq=0 out_seq=0 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.556+0000 7f6013fff640 10 client.18039 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.556+0000 7f6013fff640 1 -- 172.21.15.16:0/3095203423 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596d3db3860 con 0x5596d3e5af20 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.556+0000 7f6013fff640 10 client.18039 handle_command_reply: tid=0 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.556+0000 7f603d069740 1 client.18039 shutdown 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 msgr2=0x5596d3e485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f602c005e30 tx=0x7f602c005c50 comp rx=0 tx=0).stop 2024-09-12T01:04:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 shutdown_connections 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e4b370 0x5596d3e485e0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3e5af20 0x5596d3e5d310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e47ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 --2- 172.21.15.16:0/3095203423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e405d0 0x5596d3e4ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 >> 172.21.15.16:0/3095203423 conn(0x5596d3e09310 msgr2=0x5596d3e42930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f6013fff640 0 client.18039 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 shutdown_connections 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 1 -- 172.21.15.16:0/3095203423 wait complete. 2024-09-12T01:04:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.557+0000 7f603d069740 20 client.18039 trim_cache size 0 max 16384 2024-09-12T01:04:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 Processor -- start 2024-09-12T01:04:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 -- start start 2024-09-12T01:04:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ad8f0 con 0x5596d3e405d0 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60341af430 con 0x5596d3e41b40 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.570+0000 7f603d069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60341adfc0 con 0x5596d3e4b370 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51360/0 (socket says 172.21.15.16:51360) 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 -- 172.21.15.16:0/3543868019 learned_addr learned my addr 172.21.15.16:0/3543868019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f6033fff640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 -- 172.21.15.16:0/3543868019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 msgr2=0x5596d3e43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 -- 172.21.15.16:0/3543868019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 msgr2=0x5596d3e43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 -- 172.21.15.16:0/3543868019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d3e4b0b0 con 0x5596d3e405d0 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f6033fff640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f603922d640 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f602c005de0 tx=0x7f602c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df26d3addf8eafaf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f601a7fc640 1 -- 172.21.15.16:0/3543868019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c04d0d0 con 0x5596d3e405d0 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f601a7fc640 1 -- 172.21.15.16:0/3543868019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c044040 con 0x5596d3e405d0 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.571+0000 7f601a7fc640 1 -- 172.21.15.16:0/3543868019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c04ddf0 con 0x5596d3e405d0 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- 172.21.15.16:0/3543868019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 msgr2=0x5596d3e49f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f602c005de0 tx=0x7f602c02a950 comp rx=0 tx=0).stop 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- 172.21.15.16:0/3543868019 shutdown_connections 2024-09-12T01:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e4b370 0x5596d3e43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e41b40 0x5596d3e43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 --2- 172.21.15.16:0/3543868019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e49f60 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- 172.21.15.16:0/3543868019 >> 172.21.15.16:0/3543868019 conn(0x5596d3e09310 msgr2=0x5596d3e49850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- 172.21.15.16:0/3543868019 shutdown_connections 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- 172.21.15.16:0/3543868019 wait complete. 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 -- start start 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.572+0000 7f603d069740 1 Processor -- start 2024-09-12T01:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 0x5596d3e45480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e4b370 0x5596d3e459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aed60 con 0x5596d3e405d0 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ae690 con 0x5596d3e4b370 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ab8f0 con 0x5596d3e41b40 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603d069740 10 client.0 _ll_register_callbacks cb 0x7f602c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51374/0 (socket says 172.21.15.16:51374) 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 -- 172.21.15.16:0/2723710554 learned_addr learned my addr 172.21.15.16:0/2723710554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f6033fff640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 0x5596d3e45480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 -- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 msgr2=0x5596d3e45480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 0x5596d3e45480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 -- 172.21.15.16:0/2723710554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e4b370 msgr2=0x5596d3e459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e4b370 0x5596d3e459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f603922d640 1 -- 172.21.15.16:0/2723710554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d3ec4b90 con 0x5596d3e405d0 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.573+0000 7f6033fff640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 0x5596d3e45480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603922d640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f602c053e40 tx=0x7f602c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aa435f5bb3c97a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c04d4e0 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f602c04d680 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c044040 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f602c004b40 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 10 client.18045 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596d3ec5dd0 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f602c044040 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f602c030500 con 0x5596d3e405d0 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 10 client.18045 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:38.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 10 client.18045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 20 client.18045 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3ec6480 0x5596d3ec8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 4 client.18045 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.574+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596d3dadd70 con 0x5596d3ec6480 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f6033fff640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3ec6480 0x5596d3ec8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f6033fff640 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3ec6480 0x5596d3ec8870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da3533efccf843b1 server_cookie=664630a8fb323d01 in_seq=0 out_seq=0 2024-09-12T01:04:38.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f6013fff640 10 client.18045 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f6013fff640 1 -- 172.21.15.16:0/2723710554 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596d3dadd70 con 0x5596d3ec6480 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f6013fff640 10 client.18045 handle_command_reply: tid=0 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f603d069740 1 client.18045 shutdown 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 msgr2=0x5596d3e44f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.575+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f602c053e40 tx=0x7f602c005c40 comp rx=0 tx=0).stop 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 shutdown_connections 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5596d3ec6480 0x5596d3ec8870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596d3e4b370 0x5596d3e459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596d3e41b40 0x5596d3e45480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 --2- 172.21.15.16:0/2723710554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5596d3e405d0 0x5596d3e44f40 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f6013fff640 0 client.18045 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 >> 172.21.15.16:0/2723710554 conn(0x5596d3e09310 msgr2=0x5596d3e3bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 shutdown_connections 2024-09-12T01:04:38.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 1 -- 172.21.15.16:0/2723710554 wait complete. 2024-09-12T01:04:38.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603d069740 20 client.18045 trim_cache size 0 max 16384 2024-09-12T01:04:38.594 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:38.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 msgr2=0x7f601c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 0x7f601c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f602409b300 tx=0x7f6024005f90 comp rx=0 tx=0).stop 2024-09-12T01:04:38.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 msgr2=0x7f60341993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:38.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f602400d950 tx=0x7f602400de10 comp rx=0 tx=0).stop 2024-09-12T01:04:38.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 shutdown_connections 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f601c0a2cf0 0x7f601c0a51b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6034102ef0 0x7f6034199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6034102570 0x7f60341998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 --2- 172.21.15.16:0/3229061300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f60341993b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.576+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 >> 172.21.15.16:0/3229061300 conn(0x7f60340fc7d0 msgr2=0x7f60341068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.577+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 shutdown_connections 2024-09-12T01:04:38.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:38.577+0000 7f603ac90640 1 -- 172.21.15.16:0/3229061300 wait complete. 2024-09-12T01:04:38.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:38 smithi016 ceph-mon[30972]: pgmap v264: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-12T01:04:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:38 smithi099 ceph-mon[35654]: pgmap v264: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-12T01:04:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:38 smithi159 ceph-mon[35566]: pgmap v264: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 9 op/s 2024-09-12T01:04:39.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:04:39.758 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:40.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 -- 172.21.15.16:0/1829074048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 msgr2=0x7fd880100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 --2- 172.21.15.16:0/1829074048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd880100ee0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd870009930 tx=0x7fd87002a420 comp rx=0 tx=0).stop 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 -- 172.21.15.16:0/1829074048 shutdown_connections 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 --2- 172.21.15.16:0/1829074048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880101420 0x7fd88010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 --2- 172.21.15.16:0/1829074048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd880100ee0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 --2- 172.21.15.16:0/1829074048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880106a80 0x7fd880106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 -- 172.21.15.16:0/1829074048 >> 172.21.15.16:0/1829074048 conn(0x7fd8800fc7a0 msgr2=0x7fd8800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 -- 172.21.15.16:0/1829074048 shutdown_connections 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.203+0000 7fd886f80640 1 -- 172.21.15.16:0/1829074048 wait complete. 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 Processor -- start 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 -- start start 2024-09-12T01:04:40.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880101420 0x7fd8800fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880106a80 0x7fd8800fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd880110ec0 con 0x7fd880100a80 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd880110d40 con 0x7fd880101420 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd886f80640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd880111040 con 0x7fd880106a80 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51388/0 (socket says 172.21.15.16:51388) 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd884d1c640 1 -- 172.21.15.16:0/98687888 learned_addr learned my addr 172.21.15.16:0/98687888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd88551d640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880106a80 0x7fd8800fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.204+0000 7fd87ffff640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880101420 0x7fd8800fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 -- 172.21.15.16:0/98687888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880106a80 msgr2=0x7fd8800fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880106a80 0x7fd8800fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 -- 172.21.15.16:0/98687888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880101420 msgr2=0x7fd8800fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880101420 0x7fd8800fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 -- 172.21.15.16:0/98687888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a6970 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd884d1c640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b4a0 tx=0x7fd86800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd87dffb640 1 -- 172.21.15.16:0/98687888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868014070 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd87dffb640 1 -- 172.21.15.16:0/98687888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd868010af0 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a6c60 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd87dffb640 1 -- 172.21.15.16:0/98687888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017d10 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.205+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a71c0 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd87dffb640 1 -- 172.21.15.16:0/98687888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd868017070 con 0x7fd880100a80 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 Processor -- start 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 -- start start 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65764090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd87dffb640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 0x7fd86c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65771760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a656ca2f0 con 0x559a657645d0 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559a65671cf0 con 0x559a65765b40 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd889359740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559a656c62b0 con 0x559a6576f370 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51392/0 (socket says 172.21.15.16:51392) 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd884d1c640 1 -- 172.21.15.16:0/4262004874 learned_addr learned my addr 172.21.15.16:0/4262004874 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.208+0000 7fd87ffff640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 0x7fd86c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd88551d640 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65764090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd88551d640 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65771760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd87dffb640 1 -- 172.21.15.16:0/98687888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd8680ce630 con 0x7fd880100a80 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 -- 172.21.15.16:0/4262004874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 msgr2=0x559a65771760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65771760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 -- 172.21.15.16:0/4262004874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 msgr2=0x559a65764090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65764090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 -- 172.21.15.16:0/4262004874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a656563a0 con 0x559a657645d0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd884d1c640 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd8680a33a0 tx=0x7fd868065d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0188a0255b99bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd87ffff640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 0x7fd86c0a5230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd876ffd640 1 -- 172.21.15.16:0/4262004874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801e070 con 0x559a657645d0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd876ffd640 1 -- 172.21.15.16:0/4262004874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86809ac60 con 0x559a657645d0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.209+0000 7fd876ffd640 1 -- 172.21.15.16:0/4262004874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017430 con 0x559a657645d0 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- 172.21.15.16:0/4262004874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 msgr2=0x559a65764a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd8680a33a0 tx=0x7fd868065d30 comp rx=0 tx=0).stop 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- 172.21.15.16:0/4262004874 shutdown_connections 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65771760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65764a10 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 --2- 172.21.15.16:0/4262004874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65764090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- 172.21.15.16:0/4262004874 >> 172.21.15.16:0/4262004874 conn(0x559a6572d310 msgr2=0x559a65760c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- 172.21.15.16:0/4262004874 shutdown_connections 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- 172.21.15.16:0/4262004874 wait complete. 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.210+0000 7fd889359740 1 -- start start 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 Processor -- start 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a6576c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a6576c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559a65772070 con 0x559a657645d0 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559a65685060 con 0x559a65765b40 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559a657721f0 con 0x559a6576f370 2024-09-12T01:04:40.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd889359740 10 client.0 _ll_register_callbacks cb 0x559a653d88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 -- 172.21.15.16:0/1607941333 learned_addr learned my addr 172.21.15.16:0/1607941333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd88551d640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a6576c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 -- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 msgr2=0x559a6576c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a6576c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a6576c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 -- 172.21.15.16:0/1607941333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 msgr2=0x559a6576c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a6576c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.211+0000 7fd884d1c640 1 -- 172.21.15.16:0/1607941333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a6543b3c0 con 0x559a657645d0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd868009420 tx=0x7fd8680047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe6e1a41e383fae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:40.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801e070 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd8680cc000 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86809ac60 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd86809f010 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 10 client.18057 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a6577dbb0 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86809ac60 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd8680a3440 con 0x559a657645d0 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 10 client.18057 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 20 client.18057 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a6577ef00 0x559a657812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 4 client.18057 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.212+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a656d7860 con 0x559a6577ef00 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.213+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a6577ef00 0x559a657812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.213+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a6577ef00 0x559a657812f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eaeb9882773909e server_cookie=d207a928a1ca0722 in_seq=0 out_seq=0 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.213+0000 7fd85ffff640 10 client.18057 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd85ffff640 1 -- 172.21.15.16:0/1607941333 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a656d7860 con 0x559a6577ef00 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd85ffff640 10 client.18057 handle_command_reply: tid=0 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 client.18057 shutdown 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 msgr2=0x559a6576edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd868009420 tx=0x7fd8680047d0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 shutdown_connections 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a6577ef00 0x559a657812f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a6576c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a6576c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd85ffff640 0 client.18057 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 --2- 172.21.15.16:0/1607941333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a6576edc0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 >> 172.21.15.16:0/1607941333 conn(0x559a6572d310 msgr2=0x559a65766990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 shutdown_connections 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 1 -- 172.21.15.16:0/1607941333 wait complete. 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.214+0000 7fd889359740 20 client.18057 trim_cache size 0 max 16384 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 Processor -- start 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 -- start start 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a657645d0 0x559a6577abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a6576f370 0x559a65767660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801afae0 con 0x559a65765b40 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b1620 con 0x559a6576f370 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd889359740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b01b0 con 0x559a657645d0 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.227+0000 7fd87ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51404/0 (socket says 172.21.15.16:51404) 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 -- 172.21.15.16:0/3608884220 learned_addr learned my addr 172.21.15.16:0/3608884220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd88551d640 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a6576f370 0x559a65767660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 -- 172.21.15.16:0/3608884220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a657645d0 msgr2=0x559a6577abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a657645d0 0x559a6577abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 -- 172.21.15.16:0/3608884220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a6576f370 msgr2=0x559a65767660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a6576f370 0x559a65767660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 -- 172.21.15.16:0/3608884220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a6576cb70 con 0x559a65765b40 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd87ffff640 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd870031740 tx=0x7fd87003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f6e70d479dcb48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd8767fc640 1 -- 172.21.15.16:0/3608884220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870035050 con 0x559a65765b40 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd8767fc640 1 -- 172.21.15.16:0/3608884220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd870054070 con 0x559a65765b40 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.228+0000 7fd8767fc640 1 -- 172.21.15.16:0/3608884220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700048e0 con 0x559a65765b40 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- 172.21.15.16:0/3608884220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 msgr2=0x559a65767120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd870031740 tx=0x7fd87003c000 comp rx=0 tx=0).stop 2024-09-12T01:04:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- 172.21.15.16:0/3608884220 shutdown_connections 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a6576f370 0x559a65767660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a65765b40 0x559a65767120 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 --2- 172.21.15.16:0/3608884220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a657645d0 0x559a6577abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- 172.21.15.16:0/3608884220 >> 172.21.15.16:0/3608884220 conn(0x559a6572d310 msgr2=0x559a6577a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- 172.21.15.16:0/3608884220 shutdown_connections 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- 172.21.15.16:0/3608884220 wait complete. 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 -- start start 2024-09-12T01:04:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.229+0000 7fd889359740 1 Processor -- start 2024-09-12T01:04:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65782550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65782a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b0f50 con 0x559a657645d0 2024-09-12T01:04:40.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b0880 con 0x559a65765b40 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801adae0 con 0x559a6576f370 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd889359740 10 client.0 _ll_register_callbacks cb 0x7fd878009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51418/0 (socket says 172.21.15.16:51418) 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 -- 172.21.15.16:0/1134803209 learned_addr learned my addr 172.21.15.16:0/1134803209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65782550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd88551d640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65782a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 -- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 msgr2=0x559a65782a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65782a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 -- 172.21.15.16:0/1134803209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 msgr2=0x559a65782550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65782550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 -- 172.21.15.16:0/1134803209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a657e8840 con 0x559a657645d0 2024-09-12T01:04:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd88551d640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65782a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd884d1c640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd8680970c0 tx=0x7fd86809a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28f46604e702baf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.230+0000 7fd85ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680b3e60 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd8680972d0 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd870054070 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd86801e070 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 10 client.18063 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a657e8840 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd870054070 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd86809a0b0 con 0x559a657645d0 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 10 client.18063 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 20 client.18063 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a657e9df0 0x559a657ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 4 client.18063 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a656d1d70 con 0x559a657e9df0 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.231+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a657e9df0 0x559a657ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd87ffff640 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a657e9df0 0x559a657ec1e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe93a319234d807 server_cookie=4c629b0489d571c0 in_seq=0 out_seq=0 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd85ffff640 10 client.18063 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd85ffff640 1 -- 172.21.15.16:0/1134803209 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a656d1d70 con 0x559a657e9df0 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd85ffff640 10 client.18063 handle_command_reply: tid=0 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 client.18063 shutdown 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 msgr2=0x559a65782010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd8680970c0 tx=0x7fd86809a430 comp rx=0 tx=0).stop 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 shutdown_connections 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559a657e9df0 0x559a657ec1e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd85ffff640 0 client.18063 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a6576f370 0x559a65782a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559a65765b40 0x559a65782550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 --2- 172.21.15.16:0/1134803209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559a657645d0 0x559a65782010 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 >> 172.21.15.16:0/1134803209 conn(0x559a6572d310 msgr2=0x559a6577acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 shutdown_connections 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.232+0000 7fd889359740 1 -- 172.21.15.16:0/1134803209 wait complete. 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd889359740 20 client.18063 trim_cache size 0 max 16384 2024-09-12T01:04:40.247 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 msgr2=0x7fd86c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 0x7fd86c0a5230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870005a20 comp rx=0 tx=0).stop 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 msgr2=0x7fd8800fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b4a0 tx=0x7fd86800b960 comp rx=0 tx=0).stop 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 shutdown_connections 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd86c0a2d70 0x7fd86c0a5230 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd880106a80 0x7fd8800fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd880101420 0x7fd8800fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 --2- 172.21.15.16:0/98687888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd880100a80 0x7fd8800fcdc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 >> 172.21.15.16:0/98687888 conn(0x7fd8800fc7a0 msgr2=0x7fd88006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 shutdown_connections 2024-09-12T01:04:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:40.233+0000 7fd886f80640 1 -- 172.21.15.16:0/98687888 wait complete. 2024-09-12T01:04:40.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:40 smithi016 ceph-mon[30972]: pgmap v265: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T01:04:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:40 smithi099 ceph-mon[35654]: pgmap v265: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T01:04:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:40 smithi159 ceph-mon[35566]: pgmap v265: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-09-12T01:04:42.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:04:42.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:04:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- 172.21.15.16:0/4002766042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8100a60 msgr2=0x7fe9a8100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- 172.21.15.16:0/4002766042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8100a60 0x7fe9a8100ec0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe994009930 tx=0x7fe99402a420 comp rx=0 tx=0).stop 2024-09-12T01:04:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- 172.21.15.16:0/4002766042 shutdown_connections 2024-09-12T01:04:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- 172.21.15.16:0/4002766042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8101400 0x7fe9a810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- 172.21.15.16:0/4002766042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8100a60 0x7fe9a8100ec0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- 172.21.15.16:0/4002766042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a8106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- 172.21.15.16:0/4002766042 >> 172.21.15.16:0/4002766042 conn(0x7fe9a80fc780 msgr2=0x7fe9a80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- 172.21.15.16:0/4002766042 shutdown_connections 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- 172.21.15.16:0/4002766042 wait complete. 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 Processor -- start 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 -- start start 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 0x7fe9a80fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.840+0000 7fe9af13f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9af13f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a80fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9af13f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110e60 con 0x7fe9a8101400 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9af13f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110ce0 con 0x7fe9a8106a60 2024-09-12T01:04:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9af13f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8110fe0 con 0x7fe9a8100a60 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51432/0 (socket says 172.21.15.16:51432) 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 -- 172.21.15.16:0/3896236750 learned_addr learned my addr 172.21.15.16:0/3896236750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 0x7fe9a80fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a80fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 -- 172.21.15.16:0/3896236750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 msgr2=0x7fe9a80fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 0x7fe9a80fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 -- 172.21.15.16:0/3896236750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 msgr2=0x7fe9a80fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a80fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 -- 172.21.15.16:0/3896236750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81a6830 con 0x7fe9a8101400 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a80fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 0x7fe9a80fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a50 tx=0x7fe994004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a5ffb640 1 -- 172.21.15.16:0/3896236750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99403f030 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a5ffb640 1 -- 172.21.15.16:0/3896236750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994003ce0 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a6ac0 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.841+0000 7fe9a5ffb640 1 -- 172.21.15.16:0/3896236750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994035070 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.842+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a6ff0 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9a5ffb640 1 -- 172.21.15.16:0/3896236750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe994042030 con 0x7fe9a8101400 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 Processor -- start 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 -- start start 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39ba8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39ba8d40 0x559e39bab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e39b03280 con 0x559e39b9db60 2024-09-12T01:04:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e39aaab10 con 0x559e39ba8d40 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9b1518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e39aff240 con 0x559e39b9e480 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51444/0 (socket says 172.21.15.16:51444) 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.845+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3454337123 learned_addr learned my addr 172.21.15.16:0/3454337123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39ba8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39ba8d40 0x559e39bab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9a5ffb640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 0x7fe9980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3454337123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 msgr2=0x559e39ba8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39ba8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3454337123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39ba8d40 msgr2=0x559e39bab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39ba8d40 0x559e39bab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3454337123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e39ab71f0 con 0x559e39b9db60 2024-09-12T01:04:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9a5ffb640 1 -- 172.21.15.16:0/3896236750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe9940efa60 con 0x7fe9a8101400 2024-09-12T01:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 0x7fe9980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0008e60 tx=0x7fe9a000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256a2f2526d2395f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.846+0000 7fe992ffd640 1 -- 172.21.15.16:0/3454337123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a001f070 con 0x559e39b9db60 2024-09-12T01:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe992ffd640 1 -- 172.21.15.16:0/3454337123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a000e3b0 con 0x559e39b9db60 2024-09-12T01:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe992ffd640 1 -- 172.21.15.16:0/3454337123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000b600 con 0x559e39b9db60 2024-09-12T01:04:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 -- 172.21.15.16:0/3454337123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 msgr2=0x559e39b9df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 0x7fe9980a5230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0008fd0 tx=0x7fe9a001f000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0008e60 tx=0x7fe9a000d710 comp rx=0 tx=0).stop 2024-09-12T01:04:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 -- 172.21.15.16:0/3454337123 shutdown_connections 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39ba8d40 0x559e39bab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39ba8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3454337123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39b9db60 0x559e39b9df40 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 -- 172.21.15.16:0/3454337123 >> 172.21.15.16:0/3454337123 conn(0x559e39b9a0f0 msgr2=0x559e39b9a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 -- 172.21.15.16:0/3454337123 shutdown_connections 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.847+0000 7fe9b1518740 1 -- 172.21.15.16:0/3454337123 wait complete. 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 -- start start 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 Processor -- start 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39ba6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e39bab910 con 0x559e39ba8d40 2024-09-12T01:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559e39abdf10 con 0x559e39b9e480 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e39baba90 con 0x559e39b9db60 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.848+0000 7fe9b1518740 10 client.0 _ll_register_callbacks cb 0x559e39a86a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51454/0 (socket says 172.21.15.16:51454) 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3661446670 learned_addr learned my addr 172.21.15.16:0/3661446670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39ba6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 msgr2=0x559e39ba5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3661446670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 msgr2=0x559e39ba6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39ba6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/3661446670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e398746a0 con 0x559e39ba8d40 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39ba6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000dc10 tx=0x7fe9a000e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2306ce849bd38ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe987fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0019e10 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe9a0004a00 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.849+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a000e3b0 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0004c00 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 10 client.18075 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e39bb7540 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a000e3b0 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9a002c070 con 0x559e39ba8d40 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 10 client.18075 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 10 client.18075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 20 client.18075 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39bb88f0 0x559e39bbace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 4 client.18075 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e39b107f0 con 0x559e39bb88f0 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.850+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39bb88f0 0x559e39bbace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.851+0000 7fe9acedb640 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39bb88f0 0x559e39bbace0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d783d64cc65dce server_cookie=3b1ad6e5d1ba9069 in_seq=0 out_seq=0 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.851+0000 7fe987fff640 10 client.18075 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe987fff640 1 -- 172.21.15.16:0/3661446670 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e39b107f0 con 0x559e39bb88f0 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe987fff640 10 client.18075 handle_command_reply: tid=0 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 client.18075 shutdown 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 msgr2=0x559e39ba66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000dc10 tx=0x7fe9a000e790 comp rx=0 tx=0).stop 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 shutdown_connections 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39bb88f0 0x559e39bbace0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39ba66d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39ba6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 --2- 172.21.15.16:0/3661446670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 >> 172.21.15.16:0/3661446670 conn(0x559e39b9a0f0 msgr2=0x559e39b99270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe987fff640 0 client.18075 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 shutdown_connections 2024-09-12T01:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 1 -- 172.21.15.16:0/3661446670 wait complete. 2024-09-12T01:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.852+0000 7fe9b1518740 20 client.18075 trim_cache size 0 max 16384 2024-09-12T01:04:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.869+0000 7fe9b1518740 1 Processor -- start 2024-09-12T01:04:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.869+0000 7fe9b1518740 1 -- start start 2024-09-12T01:04:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39bc0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81afa00 con 0x559e39ba8d40 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b1540 con 0x559e39b9e480 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9b1518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b00d0 con 0x559e39b9db60 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51464/0 (socket says 172.21.15.16:51464) 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/540413340 learned_addr learned my addr 172.21.15.16:0/540413340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39bc0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9acedb640 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/540413340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 msgr2=0x559e39ba05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/540413340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 msgr2=0x559e39bc0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39bc0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/540413340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e39b9e2b0 con 0x559e39ba8d40 2024-09-12T01:04:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000fb10 tx=0x7fe9a000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8525a069bdf8af1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.870+0000 7fe9927fc640 1 -- 172.21.15.16:0/540413340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000e3b0 con 0x559e39ba8d40 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9927fc640 1 -- 172.21.15.16:0/540413340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a000e8b0 con 0x559e39ba8d40 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9927fc640 1 -- 172.21.15.16:0/540413340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0019c10 con 0x559e39ba8d40 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 -- 172.21.15.16:0/540413340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 msgr2=0x559e39bc0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000fb10 tx=0x7fe9a000d710 comp rx=0 tx=0).stop 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 -- 172.21.15.16:0/540413340 shutdown_connections 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39bc0990 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9e480 0x559e39bc0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 --2- 172.21.15.16:0/540413340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9db60 0x559e39ba05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 -- 172.21.15.16:0/540413340 >> 172.21.15.16:0/540413340 conn(0x559e39b9a0f0 msgr2=0x559e39ba7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 -- 172.21.15.16:0/540413340 shutdown_connections 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.871+0000 7fe9b1518740 1 -- 172.21.15.16:0/540413340 wait complete. 2024-09-12T01:04:42.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 -- start start 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 Processor -- start 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 0x559e39bb0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39bb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b0e70 con 0x559e39ba8d40 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b07a0 con 0x559e39b9db60 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81ada00 con 0x559e39b9e480 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9b1518740 10 client.0 _ll_register_callbacks cb 0x7fe9a001d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9ad6dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51478/0 (socket says 172.21.15.16:51478) 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/2862898113 learned_addr learned my addr 172.21.15.16:0/2862898113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39bb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.872+0000 7fe9acedb640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 0x559e39bb0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 msgr2=0x559e39bb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39bb1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/2862898113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 msgr2=0x559e39bb0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 0x559e39bb0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 -- 172.21.15.16:0/2862898113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e39bae2f0 con 0x559e39ba8d40 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9a7fff640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39bb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9acedb640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 0x559e39bb0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9ad6dc640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000dc10 tx=0x7fe9a00056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb78390eb62acc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe987fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000e3b0 con 0x559e39ba8d40 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe9a000edc0 con 0x559e39ba8d40 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a000e8b0 con 0x559e39ba8d40 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0004da0 con 0x559e39ba8d40 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9b1518740 10 client.18081 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e39c23d90 con 0x559e39ba8d40 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.873+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a000e8b0 con 0x559e39ba8d40 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe9a003c020 con 0x559e39ba8d40 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 10 client.18081 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 20 client.18081 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39c24420 0x559e39c26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 4 client.18081 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e39b0ad00 con 0x559e39c24420 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9acedb640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39c24420 0x559e39c26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe9acedb640 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39c24420 0x559e39c26840 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935d1121b8adbc88 server_cookie=4ded4f29761c7507 in_seq=0 out_seq=0 2024-09-12T01:04:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.874+0000 7fe987fff640 10 client.18081 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.875+0000 7fe987fff640 1 -- 172.21.15.16:0/2862898113 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e39b0ad00 con 0x559e39c24420 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.875+0000 7fe987fff640 10 client.18081 handle_command_reply: tid=0 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.875+0000 7fe9b1518740 1 client.18081 shutdown 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 msgr2=0x559e39badcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000dc10 tx=0x7fe9a00056f0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 shutdown_connections 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559e39c24420 0x559e39c26840 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e39ba8d40 0x559e39badcb0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe987fff640 0 client.18081 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e39b9e480 0x559e39bb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 --2- 172.21.15.16:0/2862898113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559e39b9db60 0x559e39bb0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 >> 172.21.15.16:0/2862898113 conn(0x559e39b9a0f0 msgr2=0x559e39b98a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 shutdown_connections 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 1 -- 172.21.15.16:0/2862898113 wait complete. 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.876+0000 7fe9b1518740 20 client.18081 trim_cache size 0 max 16384 2024-09-12T01:04:42.879 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7ab40d60-8222-434b-910c-44e90d2d5031", 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 msgr2=0x7fe9980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 0x7fe9980a5230 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0008fd0 tx=0x7fe9a001f000 comp rx=0 tx=0).stop 2024-09-12T01:04:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 msgr2=0x7fe9a80fd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a50 tx=0x7fe994004270 comp rx=0 tx=0).stop 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 shutdown_connections 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9980a2d70 0x7fe9980a5230 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9a8106a60 0x7fe9a80fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9a8101400 0x7fe9a80fd2b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 --2- 172.21.15.16:0/3896236750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a8100a60 0x7fe9a80fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 >> 172.21.15.16:0/3896236750 conn(0x7fe9a80fc780 msgr2=0x7fe9a806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 shutdown_connections 2024-09-12T01:04:42.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:42.877+0000 7fe9af13f640 1 -- 172.21.15.16:0/3896236750 wait complete. 2024-09-12T01:04:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:42 smithi099 ceph-mon[35654]: pgmap v266: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:42 smithi159 ceph-mon[35566]: pgmap v266: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:42 smithi016 ceph-mon[30972]: pgmap v266: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:43.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:43 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:04:43.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:43.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:43 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:04:43.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:43.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:04:44.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:43 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:04:44.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:44.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:43 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:04:44.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:44.063 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:44.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:43 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:04:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:43 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:04:44.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.581+0000 7f36ec416640 1 -- 172.21.15.16:0/3335714687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101de0 msgr2=0x7f36e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.581+0000 7f36ec416640 1 --2- 172.21.15.16:0/3335714687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101de0 0x7f36e41021c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e002a410 comp rx=0 tx=0).stop 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- 172.21.15.16:0/3335714687 shutdown_connections 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- 172.21.15.16:0/3335714687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 0x7f36e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- 172.21.15.16:0/3335714687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4102790 0x7f36e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- 172.21.15.16:0/3335714687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4101de0 0x7f36e41021c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- 172.21.15.16:0/3335714687 >> 172.21.15.16:0/3335714687 conn(0x7f36e40fd8c0 msgr2=0x7f36e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- 172.21.15.16:0/3335714687 shutdown_connections 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- 172.21.15.16:0/3335714687 wait complete. 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 Processor -- start 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- start start 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 0x7f36e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 0x7f36e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4111fc0 con 0x7f36e4102790 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4111e40 con 0x7f36e4101de0 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.582+0000 7f36ec416640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4112140 con 0x7f36e4106f10 2024-09-12T01:04:44.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51498/0 (socket says 172.21.15.16:51498) 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 -- 172.21.15.16:0/1094531349 learned_addr learned my addr 172.21.15.16:0/1094531349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 0x7f36e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 0x7f36e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 -- 172.21.15.16:0/1094531349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 msgr2=0x7f36e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 0x7f36e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 -- 172.21.15.16:0/1094531349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 msgr2=0x7f36e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 0x7f36e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 -- 172.21.15.16:0/1094531349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e41a68e0 con 0x7f36e4102790 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 0x7f36e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36e99b1640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f36d400d950 tx=0x7f36d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36db7fe640 1 -- 172.21.15.16:0/1094531349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4017070 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41a6bd0 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36db7fe640 1 -- 172.21.15.16:0/1094531349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4009c30 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36db7fe640 1 -- 172.21.15.16:0/1094531349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d400f6b0 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.583+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41a7110 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36db7fe640 1 -- 172.21.15.16:0/1094531349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36d40109d0 con 0x7f36e4102790 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 Processor -- start 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 -- start start 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 0x558196776880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x558196781140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5581966dc2e0 con 0x558196781680 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558196683ce0 con 0x5581967764a0 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.586+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5581966d82a0 con 0x558196776dc0 2024-09-12T01:04:44.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51500/0 (socket says 172.21.15.16:51500) 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 -- 172.21.15.16:0/278363747 learned_addr learned my addr 172.21.15.16:0/278363747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 0x558196776880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x558196781140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36db7fe640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 0x7f36d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 -- 172.21.15.16:0/278363747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 msgr2=0x558196781140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x558196781140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 -- 172.21.15.16:0/278363747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 msgr2=0x558196776880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 0x558196776880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36db7fe640 1 -- 172.21.15.16:0/1094531349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f36d40cf110 con 0x7f36e4102790 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 -- 172.21.15.16:0/278363747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558196668440 con 0x558196781680 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 0x558196776880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 0x7f36d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36e99b1640 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f36d409fba0 tx=0x7f36d4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5a12cb7fab02d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.587+0000 7f36d8dea640 1 -- 172.21.15.16:0/278363747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4020070 con 0x558196781680 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36d8dea640 1 -- 172.21.15.16:0/278363747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d409f9e0 con 0x558196781680 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36d8dea640 1 -- 172.21.15.16:0/278363747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4097ca0 con 0x558196781680 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 0x7f36d00a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 -- 172.21.15.16:0/278363747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 msgr2=0x558196783a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f36d409fba0 tx=0x7f36d4010d70 comp rx=0 tx=0).stop 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 -- 172.21.15.16:0/278363747 shutdown_connections 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196781680 0x558196783a70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x558196781140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/278363747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581967764a0 0x558196776880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 -- 172.21.15.16:0/278363747 >> 172.21.15.16:0/278363747 conn(0x558196772a30 msgr2=0x558196772e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 -- 172.21.15.16:0/278363747 shutdown_connections 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.588+0000 7f36ee7ef740 1 -- 172.21.15.16:0/278363747 wait complete. 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 -- start start 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 Processor -- start 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x55819677e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 0x55819677e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5581967841f0 con 0x5581967764a0 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558196697050 con 0x558196781680 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558196784370 con 0x558196776dc0 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ee7ef740 10 client.0 _ll_register_callbacks cb 0x5581966d7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ea1b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ea1b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51510/0 (socket says 172.21.15.16:51510) 2024-09-12T01:04:44.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.589+0000 7f36ea1b2640 1 -- 172.21.15.16:0/2203637528 learned_addr learned my addr 172.21.15.16:0/2203637528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36e99b1640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x55819677e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 0x55819677e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 -- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 msgr2=0x55819677e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x55819677e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 -- 172.21.15.16:0/2203637528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 msgr2=0x55819677e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 0x55819677e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 -- 172.21.15.16:0/2203637528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55819644d3c0 con 0x5581967764a0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 0x55819677e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e23fd1a8120dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0035040 con 0x5581967764a0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f36e0005450 con 0x5581967764a0 2024-09-12T01:04:44.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581966e9850 con 0x5581967764a0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e0005650 con 0x5581967764a0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ee7ef740 10 client.18093 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55819678fb70 con 0x5581967764a0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.590+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5581966e9850 con 0x5581967764a0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f36e0003ae0 con 0x5581967764a0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 10 client.18093 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 20 client.18093 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196791030 0x558196793440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 4 client.18093 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581966e3d60 con 0x558196791030 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36e99b1640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196791030 0x558196793440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36e99b1640 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196791030 0x558196793440 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc7d9d4563d7692 server_cookie=6253a3bfb8fa00f5 in_seq=0 out_seq=0 2024-09-12T01:04:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.591+0000 7f36cdffb640 10 client.18093 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36cdffb640 1 -- 172.21.15.16:0/2203637528 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581966e3d60 con 0x558196791030 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36cdffb640 10 client.18093 handle_command_reply: tid=0 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 client.18093 shutdown 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 msgr2=0x558196780f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009900 tx=0x7f36e00043b0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 shutdown_connections 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196791030 0x558196793440 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196781680 0x55819677e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558196776dc0 0x55819677e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/2203637528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5581967764a0 0x558196780f90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36cdffb640 0 client.18093 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 >> 172.21.15.16:0/2203637528 conn(0x558196772a30 msgr2=0x558196778760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 shutdown_connections 2024-09-12T01:04:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.592+0000 7f36ee7ef740 1 -- 172.21.15.16:0/2203637528 wait complete. 2024-09-12T01:04:44.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.593+0000 7f36ee7ef740 20 client.18093 trim_cache size 0 max 16384 2024-09-12T01:04:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 Processor -- start 2024-09-12T01:04:44.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 -- start start 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 0x55819678d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x5581967797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41afb30 con 0x558196776dc0 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b1670 con 0x558196772a30 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.610+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b0200 con 0x5581967764a0 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51518/0 (socket says 172.21.15.16:51518) 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 -- 172.21.15.16:0/1695338136 learned_addr learned my addr 172.21.15.16:0/1695338136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 0x55819678d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36e99b1640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x5581967797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 -- 172.21.15.16:0/1695338136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 msgr2=0x5581967797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x5581967797a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 -- 172.21.15.16:0/1695338136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 msgr2=0x55819678d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 0x55819678d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 -- 172.21.15.16:0/1695338136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558196772860 con 0x558196776dc0 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 0x55819678d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:44.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009f70 tx=0x7f36dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1416ae3569e6556 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.611+0000 7f36cffff640 1 -- 172.21.15.16:0/1695338136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc01f790 con 0x558196776dc0 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36cffff640 1 -- 172.21.15.16:0/1695338136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc017070 con 0x558196776dc0 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36cffff640 1 -- 172.21.15.16:0/1695338136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0273e0 con 0x558196776dc0 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 -- 172.21.15.16:0/1695338136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 msgr2=0x558196798d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009f70 tx=0x7f36dc015000 comp rx=0 tx=0).stop 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 -- 172.21.15.16:0/1695338136 shutdown_connections 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196776dc0 0x558196798d10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x5581967797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/1695338136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196772a30 0x55819678d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 -- 172.21.15.16:0/1695338136 >> 172.21.15.16:0/1695338136 conn(0x55819677e8e0 msgr2=0x558196779090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.612+0000 7f36ee7ef740 1 -- 172.21.15.16:0/1695338136 shutdown_connections 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 -- 172.21.15.16:0/1695338136 wait complete. 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 -- start start 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 Processor -- start 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x558196784f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196776dc0 0x558196785440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b0fa0 con 0x558196772a30 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b08d0 con 0x558196776dc0 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41adb30 con 0x5581967764a0 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ee7ef740 10 client.0 _ll_register_callbacks cb 0x7f36e0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ea1b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36e99b1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x558196784f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ea1b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51520/0 (socket says 172.21.15.16:51520) 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.613+0000 7f36ea1b2640 1 -- 172.21.15.16:0/3731199381 learned_addr learned my addr 172.21.15.16:0/3731199381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea9b3640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196776dc0 0x558196785440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 -- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 msgr2=0x558196784f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x558196784f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 -- 172.21.15.16:0/3731199381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196776dc0 msgr2=0x558196785440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196776dc0 0x558196785440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 -- 172.21.15.16:0/3731199381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558196785b30 con 0x558196772a30 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ea1b2640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f36e0003eb0 tx=0x7f36e0031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6f74274cab4ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e005a060 con 0x558196772a30 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f36e00035f0 con 0x558196772a30 2024-09-12T01:04:44.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc017070 con 0x558196772a30 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36e0051040 con 0x558196772a30 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ee7ef740 10 client.18099 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.614+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581967fbbb0 con 0x558196772a30 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36dc017070 con 0x558196772a30 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f36e00049c0 con 0x558196772a30 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 10 client.18099 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 20 client.18099 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196781680 0x5581967fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 4 client.18099 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.615+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558196660210 con 0x558196781680 2024-09-12T01:04:44.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36e99b1640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196781680 0x5581967fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36cdffb640 10 client.18099 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36e99b1640 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196781680 0x5581967fe2f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed2856efb2297521 server_cookie=64e5fa9c59b9f4d5 in_seq=0 out_seq=0 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36cdffb640 1 -- 172.21.15.16:0/3731199381 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558196660210 con 0x558196781680 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36cdffb640 10 client.18099 handle_command_reply: tid=0 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 client.18099 shutdown 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 msgr2=0x558196789bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f36e0003eb0 tx=0x7f36e0031770 comp rx=0 tx=0).stop 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 shutdown_connections 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558196781680 0x5581967fe2f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558196776dc0 0x558196785440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581967764a0 0x558196784f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 --2- 172.21.15.16:0/3731199381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558196772a30 0x558196789bd0 secure :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f36e0003eb0 tx=0x7f36e0031770 comp rx=0 tx=0).stop 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.616+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 >> 172.21.15.16:0/3731199381 conn(0x55819677e8e0 msgr2=0x5581967798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36cdffb640 0 client.18099 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 shutdown_connections 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ee7ef740 1 -- 172.21.15.16:0/3731199381 wait complete. 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ee7ef740 20 client.18099 trim_cache size 0 max 16384 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:04:44.620 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 msgr2=0x7f36d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 0x7f36d00a51b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36dc005fd0 tx=0x7f36dc004a40 comp rx=0 tx=0).stop 2024-09-12T01:04:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 msgr2=0x7f36e4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.617+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f36d400d950 tx=0x7f36d400de10 comp rx=0 tx=0).stop 2024-09-12T01:04:44.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 shutdown_connections 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f36d00a2cf0 0x7f36d00a51b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36e4106f10 0x7f36e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36e4102790 0x7f36e4074a30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 --2- 172.21.15.16:0/1094531349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36e4101de0 0x7f36e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 >> 172.21.15.16:0/1094531349 conn(0x7f36e40fd8c0 msgr2=0x7f36e40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 shutdown_connections 2024-09-12T01:04:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:44.618+0000 7f36ec416640 1 -- 172.21.15.16:0/1094531349 wait complete. 2024-09-12T01:04:44.863 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:44 smithi016 ceph-mon[30972]: pgmap v267: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:44 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:04:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:44 smithi099 ceph-mon[35654]: pgmap v267: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:44 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:04:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:44 smithi159 ceph-mon[35566]: pgmap v267: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-09-12T01:04:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:44 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:04:45.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:04:45.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 -- 172.21.15.16:0/3754505994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 msgr2=0x7f8a4410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 --2- 172.21.15.16:0/3754505994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4410e610 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009ca0 tx=0x7f8a3c02a950 comp rx=0 tx=0).stop 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 -- 172.21.15.16:0/3754505994 shutdown_connections 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 --2- 172.21.15.16:0/3754505994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4410e610 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 --2- 172.21.15.16:0/3754505994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44103660 0x7f8a4410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 --2- 172.21.15.16:0/3754505994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44102d40 0x7f8a44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 -- 172.21.15.16:0/3754505994 >> 172.21.15.16:0/3754505994 conn(0x7f8a440fc7a0 msgr2=0x7f8a440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 -- 172.21.15.16:0/3754505994 shutdown_connections 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.005+0000 7f8a499cb640 1 -- 172.21.15.16:0/3754505994 wait complete. 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 Processor -- start 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 -- start start 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 0x7f8a4419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44103660 0x7f8a4419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44111f30 con 0x7f8a4410c170 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44111db0 con 0x7f8a44102d40 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a499cb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441120b0 con 0x7f8a44103660 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51544/0 (socket says 172.21.15.16:51544) 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a437fe640 1 -- 172.21.15.16:0/407739150 learned_addr learned my addr 172.21.15.16:0/407739150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a427fc640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44103660 0x7f8a4419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a42ffd640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 0x7f8a4419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.006+0000 7f8a437fe640 1 -- 172.21.15.16:0/407739150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44103660 msgr2=0x7f8a4419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a437fe640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44103660 0x7f8a4419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a437fe640 1 -- 172.21.15.16:0/407739150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 msgr2=0x7f8a4419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a437fe640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 0x7f8a4419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a437fe640 1 -- 172.21.15.16:0/407739150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4419f5a0 con 0x7f8a4410c170 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a42ffd640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 0x7f8a4419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a437fe640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004af0 tx=0x7f8a3c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c034040 con 0x7f8a4410c170 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a3c030920 con 0x7f8a4410c170 2024-09-12T01:04:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4419f830 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0047b0 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.007+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441a7170 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.008+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a44103120 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.009+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8a3c037020 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.010+0000 7f8a489c9640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 0x7f8a2c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.010+0000 7f8a42ffd640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 0x7f8a2c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.010+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f8a3c0f0110 con 0x7f8a4410c170 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.011+0000 7f8a42ffd640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 0x7f8a2c0a5230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a340058c0 tx=0x7f8a34005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.014+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a3c0bce40 con 0x7f8a4410c170 2024-09-12T01:04:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.231+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a441a73a0 con 0x7f8a4410c170 2024-09-12T01:04:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.232+0000 7f8a489c9640 1 -- 172.21.15.16:0/407739150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f8a3c0bc740 con 0x7f8a4410c170 2024-09-12T01:04:46.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:46.231 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:46.231 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 msgr2=0x7f8a2c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 0x7f8a2c0a5230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a340058c0 tx=0x7f8a34005850 comp rx=0 tx=0).stop 2024-09-12T01:04:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 msgr2=0x7f8a4419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004af0 tx=0x7f8a3c02f5b0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 shutdown_connections 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f8a2c0a2d70 0x7f8a2c0a5230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4410c170 0x7f8a4419eec0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a44103660 0x7f8a4419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 --2- 172.21.15.16:0/407739150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a44102d40 0x7f8a4419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 >> 172.21.15.16:0/407739150 conn(0x7f8a440fc7a0 msgr2=0x7f8a44101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 shutdown_connections 2024-09-12T01:04:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:46.234+0000 7f8a499cb640 1 -- 172.21.15.16:0/407739150 wait complete. 2024-09-12T01:04:46.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:46 smithi016 ceph-mon[30972]: pgmap v268: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-12T01:04:46.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:46 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/407739150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:46.976 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 54f31edd --release --await' 2024-09-12T01:04:46.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 54f31edd --release --await 2024-09-12T01:04:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:46 smithi099 ceph-mon[35654]: pgmap v268: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-12T01:04:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:46 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/407739150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:46 smithi159 ceph-mon[35566]: pgmap v268: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-12T01:04:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:46 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/407739150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 -- 172.21.15.16:0/3863741071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 msgr2=0x7fa800101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 --2- 172.21.15.16:0/3863741071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa800101200 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009900 tx=0x7fa7fc02a480 comp rx=0 tx=0).stop 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 -- 172.21.15.16:0/3863741071 shutdown_connections 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 --2- 172.21.15.16:0/3863741071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 --2- 172.21.15.16:0/3863741071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 0x7fa8001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 --2- 172.21.15.16:0/3863741071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa800101200 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 -- 172.21.15.16:0/3863741071 >> 172.21.15.16:0/3863741071 conn(0x7fa8000fc7d0 msgr2=0x7fa8000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 -- 172.21.15.16:0/3863741071 shutdown_connections 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 -- 172.21.15.16:0/3863741071 wait complete. 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.200+0000 7fa808531640 1 Processor -- start 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 -- start start 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 0x7fa80019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa800110f50 con 0x7fa800100e20 2024-09-12T01:04:47.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa800110dd0 con 0x7fa800102ef0 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa808531640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001110d0 con 0x7fa800102570 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51558/0 (socket says 172.21.15.16:51558) 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 -- 172.21.15.16:0/3214907408 learned_addr learned my addr 172.21.15.16:0/3214907408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa805acc640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 0x7fa80019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa806ace640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 -- 172.21.15.16:0/3214907408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 msgr2=0x7fa80019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 0x7fa80019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 -- 172.21.15.16:0/3214907408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 msgr2=0x7fa80019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa8062cd640 1 -- 172.21.15.16:0/3214907408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80019f640 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.201+0000 7fa806ace640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa8062cd640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006010 tx=0x7fa7fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc03f070 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc031c30 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80019f8d0 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc035030 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.202+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001a7210 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.203+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa800101200 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.204+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa7fc042070 con 0x7fa800100e20 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.205+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 0x7fa7ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:47.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.205+0000 7fa805acc640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 0x7fa7ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:47.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.205+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fa7fc0f0040 con 0x7fa800100e20 2024-09-12T01:04:47.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.205+0000 7fa805acc640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 0x7fa7ec0a5230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa8001055f0 tx=0x7fa7f000a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:47.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.208+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7fc0bcc00 con 0x7fa800100e20 2024-09-12T01:04:47.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.426+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa80019b9f0 con 0x7fa800100e20 2024-09-12T01:04:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.427+0000 7fa7f77fe640 1 -- 172.21.15.16:0/3214907408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa80019b9f0 con 0x7fa800100e20 2024-09-12T01:04:47.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:04:47.426 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:04:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 msgr2=0x7fa7ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 0x7fa7ec0a5230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa8001055f0 tx=0x7fa7f000a2d0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 msgr2=0x7fa80019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc006010 tx=0x7fa7fc003c10 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 shutdown_connections 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7ec0a2d70 0x7fa7ec0a5230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa800102ef0 0x7fa80019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102570 0x7fa80019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 --2- 172.21.15.16:0/3214907408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa800100e20 0x7fa80019a690 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 >> 172.21.15.16:0/3214907408 conn(0x7fa8000fc7d0 msgr2=0x7fa80006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 shutdown_connections 2024-09-12T01:04:47.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:47.430+0000 7fa808531640 1 -- 172.21.15.16:0/3214907408 wait complete. 2024-09-12T01:04:47.863 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3214907408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3214907408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:48.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3214907408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:04:48.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:04:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 -- 172.21.15.16:0/806919704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 msgr2=0x7f4794100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 --2- 172.21.15.16:0/806919704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794100ee0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4788009930 tx=0x7f478802a420 comp rx=0 tx=0).stop 2024-09-12T01:04:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 -- 172.21.15.16:0/806919704 shutdown_connections 2024-09-12T01:04:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 --2- 172.21.15.16:0/806919704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 0x7f479410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 --2- 172.21.15.16:0/806919704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794100ee0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 --2- 172.21.15.16:0/806919704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 0x7f4794106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 -- 172.21.15.16:0/806919704 >> 172.21.15.16:0/806919704 conn(0x7f47940fc7a0 msgr2=0x7f47940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 -- 172.21.15.16:0/806919704 shutdown_connections 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.423+0000 7f479ae70640 1 -- 172.21.15.16:0/806919704 wait complete. 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.424+0000 7f479ae70640 1 Processor -- start 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.424+0000 7f479ae70640 1 -- start start 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.424+0000 7f479ae70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479ae70640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 0x7f47941978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479ae70640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 0x7f479419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479ae70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4794110e80 con 0x7f4794100a80 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479ae70640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4794110d00 con 0x7f4794101420 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479ae70640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4794111000 con 0x7f4794106a80 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58592/0 (socket says 172.21.15.16:58592) 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 -- 172.21.15.16:0/3515867945 learned_addr learned my addr 172.21.15.16:0/3515867945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4793fff640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 0x7f47941978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f479940d640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 0x7f479419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 -- 172.21.15.16:0/3515867945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 msgr2=0x7f479419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 0x7f479419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 -- 172.21.15.16:0/3515867945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 msgr2=0x7f47941978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 0x7f47941978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 -- 172.21.15.16:0/3515867945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479419c310 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4798c0c640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f478c00eb90 tx=0x7f478c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.425+0000 7f4791ffb640 1 -- 172.21.15.16:0/3515867945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c004d20 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.426+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479419c600 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.426+0000 7f4791ffb640 1 -- 172.21.15.16:0/3515867945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f478c00ee90 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.426+0000 7f4791ffb640 1 -- 172.21.15.16:0/3515867945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c010830 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.426+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a3ee0 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.428+0000 7f4791ffb640 1 -- 172.21.15.16:0/3515867945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f478c005020 con 0x7f4794100a80 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 Processor -- start 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 -- start start 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f4791ffb640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 0x7f477c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a16b0 0x56168069ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806a0480 0x5616806a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561680605620 con 0x5616806af300 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5616805ace50 con 0x5616806a16b0 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479d249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5616806015e0 con 0x5616806a0480 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f4791ffb640 1 -- 172.21.15.16:0/3515867945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f478c0cfac0 con 0x7f4794100a80 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479940d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.429+0000 7f479940d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58602/0 (socket says 172.21.15.16:58602) 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 -- 172.21.15.16:0/837373972 learned_addr learned my addr 172.21.15.16:0/837373972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 0x7f477c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f4793fff640 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806a0480 0x5616806a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f4798c0c640 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a16b0 0x56168069ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 -- 172.21.15.16:0/837373972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806a0480 msgr2=0x5616806a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806a0480 0x5616806a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 -- 172.21.15.16:0/837373972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a16b0 msgr2=0x56168069ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a16b0 0x56168069ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 -- 172.21.15.16:0/837373972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168059dd40 con 0x5616806af300 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4784013bf0 tx=0x7f47840103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be14e1c3a1a3c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f479940d640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 0x7f477c0a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4784005fd0 tx=0x7f4784009000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.430+0000 7f477affd640 1 -- 172.21.15.16:0/837373972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784005220 con 0x5616806af300 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f477affd640 1 -- 172.21.15.16:0/837373972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784004c80 con 0x5616806af300 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f477affd640 1 -- 172.21.15.16:0/837373972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840089e0 con 0x5616806af300 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 -- 172.21.15.16:0/837373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 msgr2=0x5616806b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4784013bf0 tx=0x7f47840103d0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 -- 172.21.15.16:0/837373972 shutdown_connections 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b16f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806a0480 0x5616806a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 --2- 172.21.15.16:0/837373972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a16b0 0x56168069ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 -- 172.21.15.16:0/837373972 >> 172.21.15.16:0/837373972 conn(0x56168069c5f0 msgr2=0x56168069c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.431+0000 7f479d249740 1 -- 172.21.15.16:0/837373972 shutdown_connections 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.432+0000 7f479d249740 1 -- 172.21.15.16:0/837373972 wait complete. 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.432+0000 7f479d249740 1 -- start start 2024-09-12T01:04:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.432+0000 7f479d249740 1 Processor -- start 2024-09-12T01:04:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806af300 0x5616806acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5616806b21b0 con 0x5616806a16b0 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56168058df10 con 0x5616806a0480 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5616806b2330 con 0x5616806af300 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58604/0 (socket says 172.21.15.16:58604) 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f4793fff640 1 -- 172.21.15.16:0/2432601027 learned_addr learned my addr 172.21.15.16:0/2432601027 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479d249740 10 client.0 _ll_register_callbacks cb 0x5616805b1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f4798c0c640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.433+0000 7f479940d640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806af300 0x5616806acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 -- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806af300 msgr2=0x5616806acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806af300 0x5616806acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 -- 172.21.15.16:0/2432601027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 msgr2=0x5616806ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 -- 172.21.15.16:0/2432601027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561680314580 con 0x5616806a16b0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4798c0c640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4793fff640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4788000c00 tx=0x7f47880319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8afaeefc9502c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4773fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788035040 con 0x5616806a16b0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f4788004300 con 0x5616806a16b0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4784004c80 con 0x5616806a16b0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4788005020 con 0x5616806a16b0 2024-09-12T01:04:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f479d249740 10 client.18123 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.434+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616803de960 con 0x5616806a16b0 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784004c80 con 0x5616806a16b0 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47880035f0 con 0x5616806a16b0 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 10 client.18123 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 10 client.18123 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 20 client.18123 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806bf290 0x5616806c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 4 client.18123 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561680612b90 con 0x5616806bf290 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.435+0000 7f4798c0c640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806bf290 0x5616806c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.436+0000 7f4798c0c640 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806bf290 0x5616806c16c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3259ad6874758d server_cookie=6edeb35b02292346 in_seq=0 out_seq=0 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.436+0000 7f4773fff640 10 client.18123 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.436+0000 7f4773fff640 1 -- 172.21.15.16:0/2432601027 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561680612b90 con 0x5616806bf290 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.436+0000 7f4773fff640 10 client.18123 handle_command_reply: tid=0 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.436+0000 7f479d249740 1 client.18123 shutdown 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 msgr2=0x5616806ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4788000c00 tx=0x7f47880319b0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 shutdown_connections 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806bf290 0x5616806c16c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806af300 0x5616806acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806a16b0 0x5616806ac850 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 --2- 172.21.15.16:0/2432601027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 >> 172.21.15.16:0/2432601027 conn(0x56168069c5f0 msgr2=0x5616806a2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f4773fff640 0 client.18123 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 shutdown_connections 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 1 -- 172.21.15.16:0/2432601027 wait complete. 2024-09-12T01:04:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.437+0000 7f479d249740 20 client.18123 trim_cache size 0 max 16384 2024-09-12T01:04:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 Processor -- start 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 -- start start 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47941ab590 con 0x5616806af300 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47941ad0d0 con 0x5616806a0480 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479d249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47941abc60 con 0x5616806b16f0 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-09-12T01:04:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f4793fff640 1 -- 172.21.15.16:0/3149596547 learned_addr learned my addr 172.21.15.16:0/3149596547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f479940d640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.456+0000 7f4798c0c640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 -- 172.21.15.16:0/3149596547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 msgr2=0x5616806c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 -- 172.21.15.16:0/3149596547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 msgr2=0x5616806a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 -- 172.21.15.16:0/3149596547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616806a1090 con 0x5616806af300 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f479940d640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f4793fff640 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4788005940 tx=0x7f47880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664c8cb0be34ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.457+0000 7f477a7fc640 1 -- 172.21.15.16:0/3149596547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788035030 con 0x5616806af300 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.458+0000 7f477a7fc640 1 -- 172.21.15.16:0/3149596547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47880035f0 con 0x5616806af300 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.458+0000 7f477a7fc640 1 -- 172.21.15.16:0/3149596547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788053040 con 0x5616806af300 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.458+0000 7f479d249740 1 -- 172.21.15.16:0/3149596547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 msgr2=0x5616806c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.458+0000 7f479d249740 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4788005940 tx=0x7f47880046c0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 -- 172.21.15.16:0/3149596547 shutdown_connections 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806c6f90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 --2- 172.21.15.16:0/3149596547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 -- 172.21.15.16:0/3149596547 >> 172.21.15.16:0/3149596547 conn(0x56168069c5f0 msgr2=0x5616806a1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 -- 172.21.15.16:0/3149596547 shutdown_connections 2024-09-12T01:04:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 -- 172.21.15.16:0/3149596547 wait complete. 2024-09-12T01:04:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 -- start start 2024-09-12T01:04:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.459+0000 7f479d249740 1 Processor -- start 2024-09-12T01:04:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47941aca00 con 0x5616806af300 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47941ac330 con 0x5616806a0480 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47941a9590 con 0x5616806b16f0 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479d249740 10 client.0 _ll_register_callbacks cb 0x7f4788031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f4793fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58634/0 (socket says 172.21.15.16:58634) 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f4793fff640 1 -- 172.21.15.16:0/4252930345 learned_addr learned my addr 172.21.15.16:0/4252930345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f4798c0c640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.460+0000 7f479940d640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 -- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 msgr2=0x5616806b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 -- 172.21.15.16:0/4252930345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 msgr2=0x5616806b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 -- 172.21.15.16:0/4252930345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616806b4d90 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4793fff640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4788009850 tx=0x7f4788004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b15be5c34b31c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4773fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788053070 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f4788004100 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47880035f0 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4788068050 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f479d249740 10 client.18129 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.461+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5616806a50b0 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788003820 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4788004a10 con 0x5616806af300 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 10 client.18129 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 10 client.18129 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 20 client.18129 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806a5810 0x5616806a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 4 client.18129 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "54f31edd", "release": true, "await": true}] 2024-09-12T01:04:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "54f31edd", "release": true, "await": true}) -- 0x7f47880035f0 con 0x5616806a5810 2024-09-12T01:04:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.462+0000 7f4798c0c640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806a5810 0x5616806a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.463+0000 7f4798c0c640 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806a5810 0x5616806a7c30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4243e785e75226d server_cookie=f6e57adc4c3828a1 in_seq=0 out_seq=0 2024-09-12T01:04:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.463+0000 7f4773fff640 10 client.18129 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f4773fff640 1 -- 172.21.15.16:0/4252930345 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f47880035f0 con 0x5616806a5810 2024-09-12T01:04:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f4773fff640 10 client.18129 handle_command_reply: tid=0 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 client.18129 shutdown 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 msgr2=0x5616806b7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4788009850 tx=0x7f4788004e70 comp rx=0 tx=0).stop 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 shutdown_connections 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5616806a5810 0x5616806a7c30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616806b16f0 0x5616806b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616806af300 0x5616806b7ce0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f4773fff640 0 client.18129 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 --2- 172.21.15.16:0/4252930345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5616806a0480 0x5616806b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 >> 172.21.15.16:0/4252930345 conn(0x56168069c5f0 msgr2=0x5616806a9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 shutdown_connections 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 1 -- 172.21.15.16:0/4252930345 wait complete. 2024-09-12T01:04:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.467+0000 7f479d249740 20 client.18129 trim_cache size 0 max 16384 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 36, 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "54f31edd": { 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 36, 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:48.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 139.4, 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:48.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 msgr2=0x7f477c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 0x7f477c0a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4784005fd0 tx=0x7f4784009000 comp rx=0 tx=0).stop 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 msgr2=0x7f4794197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f478c00eb90 tx=0x7f478c00c6a0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 shutdown_connections 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f477c0a2cf0 0x7f477c0a51b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4794106a80 0x7f479419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4794101420 0x7f47941978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 --2- 172.21.15.16:0/3515867945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794100a80 0x7f4794197360 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 >> 172.21.15.16:0/3515867945 conn(0x7f47940fc7a0 msgr2=0x7f47940fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.468+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 shutdown_connections 2024-09-12T01:04:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:48.469+0000 7f479ae70640 1 -- 172.21.15.16:0/3515867945 wait complete. 2024-09-12T01:04:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:48 smithi099 ceph-mon[35654]: pgmap v269: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:04:49.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:48 smithi016 ceph-mon[30972]: pgmap v269: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:04:49.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:48 smithi159 ceph-mon[35566]: pgmap v269: 97 pgs: 97 active+clean; 9.9 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:04:49.140 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '54f31edd', total seconds elapsed: 24.4 2024-09-12T01:04:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 -- 172.21.15.16:0/1692744158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 msgr2=0x7f564010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 --2- 172.21.15.16:0/1692744158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564010e610 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5638009ca0 tx=0x7f563802a950 comp rx=0 tx=0).stop 2024-09-12T01:04:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 -- 172.21.15.16:0/1692744158 shutdown_connections 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 --2- 172.21.15.16:0/1692744158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564010e610 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 --2- 172.21.15.16:0/1692744158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640103660 0x7f564010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 --2- 172.21.15.16:0/1692744158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640102d40 0x7f5640103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 -- 172.21.15.16:0/1692744158 >> 172.21.15.16:0/1692744158 conn(0x7f56400fc7a0 msgr2=0x7f56400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 -- 172.21.15.16:0/1692744158 shutdown_connections 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.602+0000 7f5645720640 1 -- 172.21.15.16:0/1692744158 wait complete. 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 Processor -- start 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 -- start start 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640102d40 0x7f5640198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 0x7f5640198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5640111f70 con 0x7f564010c170 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5640111df0 con 0x7f5640102d40 2024-09-12T01:04:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f5645720640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401120f0 con 0x7f5640103660 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f563f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f563f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58660/0 (socket says 172.21.15.16:58660) 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f563f7fe640 1 -- 172.21.15.16:0/1262058295 learned_addr learned my addr 172.21.15.16:0/1262058295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f563e7fc640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 0x7f5640198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.603+0000 7f563effd640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640102d40 0x7f5640198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 -- 172.21.15.16:0/1262058295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 msgr2=0x7f5640198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 0x7f5640198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 -- 172.21.15.16:0/1262058295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640102d40 msgr2=0x7f5640198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640102d40 0x7f5640198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 -- 172.21.15.16:0/1262058295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564019d3c0 con 0x7f564010c170 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563e7fc640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 0x7f5640198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f563f7fe640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f563802ae70 tx=0x7f563802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f5627fff640 1 -- 172.21.15.16:0/1262058295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638034040 con 0x7f564010c170 2024-09-12T01:04:49.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564019d650 con 0x7f564010c170 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f5627fff640 1 -- 172.21.15.16:0/1262058295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5638030920 con 0x7f564010c170 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f5627fff640 1 -- 172.21.15.16:0/1262058295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638045480 con 0x7f564010c170 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.604+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56401a4f90 con 0x7f564010c170 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.606+0000 7f5627fff640 1 -- 172.21.15.16:0/1262058295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5638037020 con 0x7f564010c170 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.606+0000 7f5647af9740 1 Processor -- start 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 -- start start 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eeb45d0 0x55697eeb4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55697ee1a2f0 con 0x55697eec3780 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55697edc1cf0 con 0x55697eeb5b40 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5647af9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55697ee162b0 con 0x55697eeb45d0 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5627fff640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 0x7f56280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58666/0 (socket says 172.21.15.16:58666) 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 -- 172.21.15.16:0/2902466367 learned_addr learned my addr 172.21.15.16:0/2902466367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563f7fe640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 0x7f56280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563effd640 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eeb45d0 0x55697eeb4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f5627fff640 1 -- 172.21.15.16:0/1262058295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f56380f03a0 con 0x7f564010c170 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 -- 172.21.15.16:0/2902466367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eeb45d0 msgr2=0x55697eeb4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.607+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eeb45d0 0x55697eeb4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f563e7fc640 1 -- 172.21.15.16:0/2902466367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 msgr2=0x55697eeb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f563e7fc640 1 -- 172.21.15.16:0/2902466367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697eda63a0 con 0x55697eec3780 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5634013950 tx=0x7f563400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de51fc9db7fb5435 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f563f7fe640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 0x7f56280a5230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f56380bc830 tx=0x7f5638088900 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f56257fa640 1 -- 172.21.15.16:0/2902466367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634004220 con 0x55697eec3780 2024-09-12T01:04:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f56257fa640 1 -- 172.21.15.16:0/2902466367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56340043c0 con 0x55697eec3780 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.608+0000 7f56257fa640 1 -- 172.21.15.16:0/2902466367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634003af0 con 0x55697eec3780 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- 172.21.15.16:0/2902466367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 msgr2=0x55697eec5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f5634013950 tx=0x7f563400b670 comp rx=0 tx=0).stop 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- 172.21.15.16:0/2902466367 shutdown_connections 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eec3780 0x55697eec5b70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eeb45d0 0x55697eeb4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 --2- 172.21.15.16:0/2902466367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- 172.21.15.16:0/2902466367 >> 172.21.15.16:0/2902466367 conn(0x55697ee7d310 msgr2=0x55697eeb0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- 172.21.15.16:0/2902466367 shutdown_connections 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- 172.21.15.16:0/2902466367 wait complete. 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.609+0000 7f5647af9740 1 -- start start 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 Processor -- start 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eec31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55697eec6480 con 0x55697eeb5b40 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55697edd5060 con 0x55697eeb45d0 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55697eec6600 con 0x55697eec3780 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f5647af9740 10 client.0 _ll_register_callbacks cb 0x55697eb288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58682/0 (socket says 172.21.15.16:58682) 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 -- 172.21.15.16:0/2379509171 learned_addr learned my addr 172.21.15.16:0/2379509171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eec31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 -- 172.21.15.16:0/2379509171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 msgr2=0x55697eec0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 -- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 msgr2=0x55697eec31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eec31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563effd640 1 -- 172.21.15.16:0/2379509171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697eb8b3c0 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.610+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f563effd640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f563000bf00 tx=0x7f5630012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c149501e668ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300108a0 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f5630010a40 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56340043c0 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5630010c40 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 10 client.18141 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55697eed2000 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56340043c0 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5630019070 con 0x55697eeb5b40 2024-09-12T01:04:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 10 client.18141 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 10 client.18141 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 20 client.18141 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697eed34c0 0x55697eed58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 4 client.18141 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.611+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55697ee27860 con 0x55697eed34c0 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.612+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697eed34c0 0x55697eed58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.612+0000 7f5622ff5640 10 client.18141 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.612+0000 7f563e7fc640 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697eed34c0 0x55697eed58f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1b0046d6f28139b server_cookie=e28b8d6065b2b843 in_seq=0 out_seq=0 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5622ff5640 1 -- 172.21.15.16:0/2379509171 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55697ee27860 con 0x55697eed34c0 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5622ff5640 10 client.18141 handle_command_reply: tid=0 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 client.18141 shutdown 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 msgr2=0x55697eec0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f563000bf00 tx=0x7f5630012820 comp rx=0 tx=0).stop 2024-09-12T01:04:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 shutdown_connections 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697eed34c0 0x55697eed58f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eec0490 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 --2- 172.21.15.16:0/2379509171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eec31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 >> 172.21.15.16:0/2379509171 conn(0x55697ee7d310 msgr2=0x55697eeb6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5622ff5640 0 client.18141 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 shutdown_connections 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 1 -- 172.21.15.16:0/2379509171 wait complete. 2024-09-12T01:04:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.613+0000 7f5647af9740 20 client.18141 trim_cache size 0 max 16384 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 Processor -- start 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 -- start start 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eedb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56401ab590 con 0x55697eeb45d0 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56401ad0d0 con 0x55697eeb5b40 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f5647af9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401abc60 con 0x55697eec3780 2024-09-12T01:04:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f563e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f563e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58696/0 (socket says 172.21.15.16:58696) 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.626+0000 7f563e7fc640 1 -- 172.21.15.16:0/3286195794 learned_addr learned my addr 172.21.15.16:0/3286195794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 -- 172.21.15.16:0/3286195794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 msgr2=0x55697eedb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eedb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 -- 172.21.15.16:0/3286195794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 msgr2=0x55697eeb0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563effd640 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 -- 172.21.15.16:0/3286195794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55697eeb1960 con 0x55697eeb45d0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5634013760 tx=0x7f5634003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dd2493a9a7c5ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f5624ff9640 1 -- 172.21.15.16:0/3286195794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634032070 con 0x55697eeb45d0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f5624ff9640 1 -- 172.21.15.16:0/3286195794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56340117b0 con 0x55697eeb45d0 2024-09-12T01:04:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.627+0000 7f5624ff9640 1 -- 172.21.15.16:0/3286195794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634004680 con 0x55697eeb45d0 2024-09-12T01:04:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- 172.21.15.16:0/3286195794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 msgr2=0x55697eeb0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5634013760 tx=0x7f5634003690 comp rx=0 tx=0).stop 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- 172.21.15.16:0/3286195794 shutdown_connections 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eedb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb5b40 0x55697eeb0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 --2- 172.21.15.16:0/3286195794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb45d0 0x55697eeb0700 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- 172.21.15.16:0/3286195794 >> 172.21.15.16:0/3286195794 conn(0x55697ee7d310 msgr2=0x55697eebb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- 172.21.15.16:0/3286195794 shutdown_connections 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- 172.21.15.16:0/3286195794 wait complete. 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 -- start start 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.628+0000 7f5647af9740 1 Processor -- start 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eeb7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56401aca00 con 0x55697eeb5b40 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56401ac330 con 0x55697eeb45d0 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401a9570 con 0x55697eec3780 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5647af9740 10 client.0 _ll_register_callbacks cb 0x7f563000b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58706/0 (socket says 172.21.15.16:58706) 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 -- 172.21.15.16:0/3027252931 learned_addr learned my addr 172.21.15.16:0/3027252931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eeb7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 -- 172.21.15.16:0/3027252931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 msgr2=0x55697eec8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 -- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 msgr2=0x55697eeb7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eeb7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 -- 172.21.15.16:0/3027252931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55697eec9680 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f563effd640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x55697eec1ad0 tx=0x7f5630023480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cf4b70c9876043 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.629+0000 7f5622ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630005020 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f56300051c0 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56340117b0 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56300053e0 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 10 client.18147 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55697eebbc30 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56340117b0 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5630019040 con 0x55697eeb5b40 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 10 client.18147 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 10 client.18147 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:04:49.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 20 client.18147 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697ef42b50 0x55697ef44f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 4 client.18147 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.630+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55697ee21d70 con 0x55697ef42b50 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697ef42b50 0x55697ef44f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f563e7fc640 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697ef42b50 0x55697ef44f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86c14c3b5de48fd2 server_cookie=1a074c85c2450d16 in_seq=0 out_seq=0 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5622ff5640 10 client.18147 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5622ff5640 1 -- 172.21.15.16:0/3027252931 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5634017020 con 0x55697ef42b50 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5622ff5640 10 client.18147 handle_command_reply: tid=0 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5647af9740 1 client.18147 shutdown 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 msgr2=0x55697eeb7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.631+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x55697eec1ad0 tx=0x7f5630023480 comp rx=0 tx=0).stop 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 shutdown_connections 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55697ef42b50 0x55697ef44f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55697eec3780 0x55697eec8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55697eeb5b40 0x55697eeb7960 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5622ff5640 0 client.18147 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 --2- 172.21.15.16:0/3027252931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55697eeb45d0 0x55697eeb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 >> 172.21.15.16:0/3027252931 conn(0x55697ee7d310 msgr2=0x55697eec1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 shutdown_connections 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 1 -- 172.21.15.16:0/3027252931 wait complete. 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5647af9740 20 client.18147 trim_cache size 0 max 16384 2024-09-12T01:04:49.636 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 msgr2=0x7f56280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 0x7f56280a5230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f56380bc830 tx=0x7f5638088900 comp rx=0 tx=0).stop 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 msgr2=0x7f564019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f563802ae70 tx=0x7f563802f5b0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 shutdown_connections 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56280a2d70 0x7f56280a5230 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f564010c170 0x7f564019cce0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5640103660 0x7f5640198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.632+0000 7f5645720640 1 --2- 172.21.15.16:0/1262058295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5640102d40 0x7f5640198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.633+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 >> 172.21.15.16:0/1262058295 conn(0x7f56400fc7a0 msgr2=0x7f56400fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.633+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 shutdown_connections 2024-09-12T01:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:49.633+0000 7f5645720640 1 -- 172.21.15.16:0/1262058295 wait complete. 2024-09-12T01:04:50.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:04:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:50.995 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:50 smithi016 ceph-mon[30972]: pgmap v270: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:04:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:50 smithi099 ceph-mon[35654]: pgmap v270: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:04:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:50 smithi159 ceph-mon[35566]: pgmap v270: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 KiB/s wr, 13 op/s 2024-09-12T01:04:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 -- 172.21.15.16:0/3667479717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100350 msgr2=0x7fb0dc100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 --2- 172.21.15.16:0/3667479717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100350 0x7fb0dc100730 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009cd0 tx=0x7fb0d802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 -- 172.21.15.16:0/3667479717 shutdown_connections 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 --2- 172.21.15.16:0/3667479717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 --2- 172.21.15.16:0/3667479717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100d00 0x7fb0dc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 --2- 172.21.15.16:0/3667479717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100350 0x7fb0dc100730 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 -- 172.21.15.16:0/3667479717 >> 172.21.15.16:0/3667479717 conn(0x7fb0dc077210 msgr2=0x7fb0dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 -- 172.21.15.16:0/3667479717 shutdown_connections 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.216+0000 7fb0e468f640 1 -- 172.21.15.16:0/3667479717 wait complete. 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 Processor -- start 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 -- start start 2024-09-12T01:04:51.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100350 0x7fb0dc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc112020 con 0x7fb0dc100d00 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc111ea0 con 0x7fb0dc106f50 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e468f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1121a0 con 0x7fb0dc100350 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e1c2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e1c2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58726/0 (socket says 172.21.15.16:58726) 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e1c2a640 1 -- 172.21.15.16:0/2568694944 learned_addr learned my addr 172.21.15.16:0/2568694944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e242b640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100350 0x7fb0dc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.217+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 -- 172.21.15.16:0/2568694944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100350 msgr2=0x7fb0dc198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100350 0x7fb0dc198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 -- 172.21.15.16:0/2568694944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 msgr2=0x7fb0dc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 -- 172.21.15.16:0/2568694944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc19d4f0 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d740 tx=0x7fb0d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0cf7fe640 1 -- 172.21.15.16:0/2568694944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0017070 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc19d7e0 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0cf7fe640 1 -- 172.21.15.16:0/2568694944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d000de50 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0cf7fe640 1 -- 172.21.15.16:0/2568694944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0004e00 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.218+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1a50c0 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0cf7fe640 1 -- 172.21.15.16:0/2568694944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb0d0004fa0 con 0x7fb0dc100d00 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0e6a68740 1 Processor -- start 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0e6a68740 1 -- start start 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce55265d0 0x555ce5526a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.221+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce5537b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ce548c2f0 con 0x555ce5527b40 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ce5433cf0 con 0x555ce5535780 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555ce54882b0 con 0x555ce55265d0 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58740/0 (socket says 172.21.15.16:58740) 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3869594691 learned_addr learned my addr 172.21.15.16:0/3869594691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e242b640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce55265d0 0x555ce5526a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce5537b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3869594691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce55265d0 msgr2=0x555ce5526a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce55265d0 0x555ce5526a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3869594691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 msgr2=0x555ce5537b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce5537b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3869594691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce54183a0 con 0x555ce5527b40 2024-09-12T01:04:51.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce5537b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0cf7fe640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 0x7fb0c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e242b640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 0x7fb0c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0cf7fe640 1 -- 172.21.15.16:0/2568694944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fb0d00cee50 con 0x7fb0dc100d00 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.222+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400a990 tx=0x7fb0d400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e575a73788977a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0ccff9640 1 -- 172.21.15.16:0/3869594691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4015070 con 0x555ce5527b40 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0ccff9640 1 -- 172.21.15.16:0/3869594691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d400cbe0 con 0x555ce5527b40 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0ccff9640 1 -- 172.21.15.16:0/3869594691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4010590 con 0x555ce5527b40 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e242b640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 0x7fb0c80a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8006010 tx=0x7fb0d8005fa0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3869594691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 msgr2=0x555ce5526090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400a990 tx=0x7fb0d400ae50 comp rx=0 tx=0).stop 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3869594691 shutdown_connections 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce5537b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce55265d0 0x555ce5526a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3869594691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5527b40 0x555ce5526090 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.223+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3869594691 >> 172.21.15.16:0/3869594691 conn(0x555ce54ef310 msgr2=0x555ce5522c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3869594691 shutdown_connections 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3869594691 wait complete. 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- start start 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 Processor -- start 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce55350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce55323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555ce5538480 con 0x555ce5535780 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ce5447060 con 0x555ce55265d0 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555ce5538600 con 0x555ce5527b40 2024-09-12T01:04:51.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e6a68740 10 client.0 _ll_register_callbacks cb 0x555ce519a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58746/0 (socket says 172.21.15.16:58746) 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3653281626 learned_addr learned my addr 172.21.15.16:0/3653281626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.224+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce55323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e242b640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce55350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 msgr2=0x555ce55323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce55323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3653281626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 msgr2=0x555ce55350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce55350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/3653281626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce51fd3c0 con 0x555ce5535780 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e242b640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce55350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4011d80 tx=0x7fb0d4011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e009ff0be085828 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4020070 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d400cbe0 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fb0d401a450 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0d4015040 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e6a68740 10 client.18159 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ce5543e90 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.225+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d400cbe0 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d402c670 con 0x555ce5535780 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 10 client.18159 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 10 client.18159 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 20 client.18159 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55452f0 0x555ce55476e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 4 client.18159 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:51.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce5499860 con 0x555ce55452f0 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.226+0000 7fb0e242b640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55452f0 0x555ce55476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0c5ffb640 10 client.18159 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e242b640 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55452f0 0x555ce55476e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38f0dc653668638c server_cookie=2cc3d045242e3429 in_seq=0 out_seq=0 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/3653281626 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ce5499860 con 0x555ce55452f0 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0c5ffb640 10 client.18159 handle_command_reply: tid=0 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 client.18159 shutdown 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 msgr2=0x555ce5532900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4011d80 tx=0x7fb0d4011b40 comp rx=0 tx=0).stop 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 shutdown_connections 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55452f0 0x555ce55476e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce5532900 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce55323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0c5ffb640 0 client.18159 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/3653281626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce55350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.227+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 >> 172.21.15.16:0/3653281626 conn(0x555ce54ef310 msgr2=0x555ce5528960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.228+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 shutdown_connections 2024-09-12T01:04:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.228+0000 7fb0e6a68740 1 -- 172.21.15.16:0/3653281626 wait complete. 2024-09-12T01:04:51.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.228+0000 7fb0e6a68740 20 client.18159 trim_cache size 0 max 16384 2024-09-12T01:04:51.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 Processor -- start 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 -- start start 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce554cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce554d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ab6c0 con 0x555ce55265d0 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ad200 con 0x555ce5535780 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1abd90 con 0x555ce5527b40 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58756/0 (socket says 172.21.15.16:58756) 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 -- 172.21.15.16:0/1656281847 learned_addr learned my addr 172.21.15.16:0/1656281847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce554cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce554d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 -- 172.21.15.16:0/1656281847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 msgr2=0x555ce554cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce554cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 -- 172.21.15.16:0/1656281847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 msgr2=0x555ce554d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce554d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e242b640 1 -- 172.21.15.16:0/1656281847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce5521310 con 0x555ce55265d0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce554d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.241+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce554cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009cd0 tx=0x7fb0d8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ba35e3a76951f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0c7fff640 1 -- 172.21.15.16:0/1656281847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80305a0 con 0x555ce55265d0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0c7fff640 1 -- 172.21.15.16:0/1656281847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d8036040 con 0x555ce55265d0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0c7fff640 1 -- 172.21.15.16:0/1656281847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8046a10 con 0x555ce55265d0 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1656281847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 msgr2=0x555ce5541720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009cd0 tx=0x7fb0d8044000 comp rx=0 tx=0).stop 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1656281847 shutdown_connections 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce5535780 0x555ce554d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce554cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1656281847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce55265d0 0x555ce5541720 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1656281847 >> 172.21.15.16:0/1656281847 conn(0x555ce54ef310 msgr2=0x555ce5534420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1656281847 shutdown_connections 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.242+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1656281847 wait complete. 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 -- start start 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 Processor -- start 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce553dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce552f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1acb30 con 0x555ce5535780 2024-09-12T01:04:51.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ac460 con 0x555ce55265d0 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1a96c0 con 0x555ce5527b40 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e6a68740 10 client.0 _ll_register_callbacks cb 0x7fb0d8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e2c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58770/0 (socket says 172.21.15.16:58770) 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/1820576144 learned_addr learned my addr 172.21.15.16:0/1820576144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e1c2a640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce552f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.243+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce553dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 msgr2=0x555ce552f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce552f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/1820576144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 msgr2=0x555ce553dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce553dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 -- 172.21.15.16:0/1820576144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce552fe90 con 0x555ce5535780 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce553dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:51.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e2c2c640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cd60 tx=0x7fb0d402c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199a585161d2abd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4020040 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fb0d40246f0 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d8036040 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0d403a050 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e6a68740 10 client.18165 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ce55afce0 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.244+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d8036040 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d401a730 con 0x555ce5535780 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 10 client.18165 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 10 client.18165 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 20 client.18165 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55b03a0 0x555ce55b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:51.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 4 client.18165 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ce5493d70 con 0x555ce55b03a0 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55b03a0 0x555ce55b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0c5ffb640 10 client.18165 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.245+0000 7fb0e242b640 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55b03a0 0x555ce55b2790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f704fa7b73a169b7 server_cookie=5c7abf158b44e3e in_seq=0 out_seq=0 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0c5ffb640 1 -- 172.21.15.16:0/1820576144 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ce5493d70 con 0x555ce55b03a0 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0c5ffb640 10 client.18165 handle_command_reply: tid=0 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 client.18165 shutdown 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 msgr2=0x555ce552f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cd60 tx=0x7fb0d402c5b0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 shutdown_connections 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x555ce55b03a0 0x555ce55b2790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ce5535780 0x555ce552f660 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ce5527b40 0x555ce552f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 --2- 172.21.15.16:0/1820576144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ce55265d0 0x555ce553dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 >> 172.21.15.16:0/1820576144 conn(0x555ce54ef310 msgr2=0x555ce5528110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0c5ffb640 0 client.18165 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 shutdown_connections 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 1 -- 172.21.15.16:0/1820576144 wait complete. 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.246+0000 7fb0e6a68740 20 client.18165 trim_cache size 0 max 16384 2024-09-12T01:04:51.252 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 msgr2=0x7fb0c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 0x7fb0c80a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8006010 tx=0x7fb0d8005fa0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 msgr2=0x7fb0dc198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d740 tx=0x7fb0d000dc00 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 shutdown_connections 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0c80a2cf0 0x7fb0c80a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc106f50 0x7fb0dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0dc100d00 0x7fb0dc198a80 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 --2- 172.21.15.16:0/2568694944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0dc100350 0x7fb0dc198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 >> 172.21.15.16:0/2568694944 conn(0x7fb0dc077210 msgr2=0x7fb0dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:51.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 shutdown_connections 2024-09-12T01:04:51.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:51.247+0000 7fb0e468f640 1 -- 172.21.15.16:0/2568694944 wait complete. 2024-09-12T01:04:51.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:04:52.258 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- 172.21.15.16:0/3118975316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101dd0 msgr2=0x7f517c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 --2- 172.21.15.16:0/3118975316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101dd0 0x7f517c1021b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f5178009900 tx=0x7f517802a410 comp rx=0 tx=0).stop 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- 172.21.15.16:0/3118975316 shutdown_connections 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 --2- 172.21.15.16:0/3118975316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c106f00 0x7f517c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 --2- 172.21.15.16:0/3118975316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 0x7f517c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 --2- 172.21.15.16:0/3118975316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101dd0 0x7f517c1021b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- 172.21.15.16:0/3118975316 >> 172.21.15.16:0/3118975316 conn(0x7f517c0fd890 msgr2=0x7f517c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- 172.21.15.16:0/3118975316 shutdown_connections 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- 172.21.15.16:0/3118975316 wait complete. 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 Processor -- start 2024-09-12T01:04:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.860+0000 7f51846d5640 1 -- start start 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c101dd0 0x7f517c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 0x7f517c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f517c111f80 con 0x7f517c106f00 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f517c111e00 con 0x7f517c102780 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51846d5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f517c112100 con 0x7f517c101dd0 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58794/0 (socket says 172.21.15.16:58794) 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 -- 172.21.15.16:0/3263515860 learned_addr learned my addr 172.21.15.16:0/3263515860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182471640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c101dd0 0x7f517c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5181c70640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 0x7f517c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 -- 172.21.15.16:0/3263515860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c101dd0 msgr2=0x7f517c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c101dd0 0x7f517c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 -- 172.21.15.16:0/3263515860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 msgr2=0x7f517c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 0x7f517c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 -- 172.21.15.16:0/3263515860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c19d410 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f5182c72640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f517400b970 tx=0x7f517400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.861+0000 7f51737fe640 1 -- 172.21.15.16:0/3263515860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174019070 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.862+0000 7f51737fe640 1 -- 172.21.15.16:0/3263515860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5174014440 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.862+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c19d700 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.862+0000 7f51737fe640 1 -- 172.21.15.16:0/3263515860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740075d0 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.862+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c1a4fe0 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.864+0000 7f51737fe640 1 -- 172.21.15.16:0/3263515860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f517401c070 con 0x7f517c106f00 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.864+0000 7f5186aae740 1 Processor -- start 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.864+0000 7f5186aae740 1 -- start start 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55785232b2f0 con 0x5578523c6b40 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5578522d2cf0 con 0x5578523c55d0 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5186aae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578523272b0 con 0x5578523d0370 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58806/0 (socket says 172.21.15.16:58806) 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5181c70640 1 -- 172.21.15.16:0/3501545101 learned_addr learned my addr 172.21.15.16:0/3501545101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f51737fe640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 0x7f51680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182c72640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 0x7f51680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5181c70640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 -- 172.21.15.16:0/3501545101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 msgr2=0x5578523d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523d2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 -- 172.21.15.16:0/3501545101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 msgr2=0x5578523c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f51737fe640 1 -- 172.21.15.16:0/3263515860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f51740ce860 con 0x7f517c106f00 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5182471640 1 -- 172.21.15.16:0/3501545101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578522b73a0 con 0x5578523c55d0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.865+0000 7f5181c70640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.866+0000 7f5182471640 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c5a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5178005940 tx=0x7f51780316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e1ad7ec506ea71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.866+0000 7f5182c72640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 0x7f51680a5230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5174097910 tx=0x7f51740148c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.866+0000 7f5170ff9640 1 -- 172.21.15.16:0/3501545101 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178040070 con 0x5578523c55d0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.866+0000 7f5170ff9640 1 -- 172.21.15.16:0/3501545101 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5178005020 con 0x5578523c55d0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.866+0000 7f5170ff9640 1 -- 172.21.15.16:0/3501545101 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178035040 con 0x5578523c55d0 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- 172.21.15.16:0/3501545101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 msgr2=0x5578523c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c5a10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5178005940 tx=0x7f51780316e0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- 172.21.15.16:0/3501545101 shutdown_connections 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c5a10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 --2- 172.21.15.16:0/3501545101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- 172.21.15.16:0/3501545101 >> 172.21.15.16:0/3501545101 conn(0x55785238e310 msgr2=0x5578523c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- 172.21.15.16:0/3501545101 shutdown_connections 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- 172.21.15.16:0/3501545101 wait complete. 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 -- start start 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.867+0000 7f5186aae740 1 Processor -- start 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5578523d3010 con 0x5578523c6b40 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5578522e6060 con 0x5578523c55d0 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578523d3190 con 0x5578523d0370 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5186aae740 10 client.0 _ll_register_callbacks cb 0x5578520398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58810/0 (socket says 172.21.15.16:58810) 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 -- 172.21.15.16:0/1380983724 learned_addr learned my addr 172.21.15.16:0/1380983724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5181c70640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5181c70640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 -- 172.21.15.16:0/1380983724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 msgr2=0x5578523cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 -- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 msgr2=0x5578523cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 -- 172.21.15.16:0/1380983724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785209c3c0 con 0x5578523c6b40 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5182471640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5178009a20 tx=0x7f5178009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fe0d6d844f4677 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.868+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178040070 con 0x5578523c6b40 2024-09-12T01:04:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5178005020 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f51780053e0 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5178035050 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 10 client.18177 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578523dea20 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5178005020 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f517803ca00 con 0x5578523c6b40 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 10 client.18177 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 10 client.18177 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 20 client.18177 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5578523dfd70 0x5578523e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 4 client.18177 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557852338860 con 0x5578523dfd70 2024-09-12T01:04:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.869+0000 7f5181c70640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5578523dfd70 0x5578523e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.870+0000 7f5165ffb640 10 client.18177 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.870+0000 7f5181c70640 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5578523dfd70 0x5578523e2160 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=904848e93236ade4 server_cookie=17608d19aa1d7126 in_seq=0 out_seq=0 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5165ffb640 1 -- 172.21.15.16:0/1380983724 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557852338860 con 0x5578523dfd70 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5165ffb640 10 client.18177 handle_command_reply: tid=0 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 client.18177 shutdown 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 msgr2=0x5578523ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5178009a20 tx=0x7f5178009520 comp rx=0 tx=0).stop 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 shutdown_connections 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5578523dfd70 0x5578523e2160 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523d0370 0x5578523cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523c6b40 0x5578523ccf90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5165ffb640 0 client.18177 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 --2- 172.21.15.16:0/1380983724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 >> 172.21.15.16:0/1380983724 conn(0x55785238e310 msgr2=0x5578523c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 shutdown_connections 2024-09-12T01:04:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 1 -- 172.21.15.16:0/1380983724 wait complete. 2024-09-12T01:04:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.871+0000 7f5186aae740 20 client.18177 trim_cache size 0 max 16384 2024-09-12T01:04:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 Processor -- start 2024-09-12T01:04:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 -- start start 2024-09-12T01:04:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c55d0 0x5578523c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 0x5578523cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1ad840 con 0x5578523d0370 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1af380 con 0x5578523c6b40 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.884+0000 7f5186aae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1adf10 con 0x5578523c55d0 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58822/0 (socket says 172.21.15.16:58822) 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 -- 172.21.15.16:0/2449510757 learned_addr learned my addr 172.21.15.16:0/2449510757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c55d0 0x5578523c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 -- 172.21.15.16:0/2449510757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c55d0 msgr2=0x5578523c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c55d0 0x5578523c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5182471640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 0x5578523cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 -- 172.21.15.16:0/2449510757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 msgr2=0x5578523cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 0x5578523cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 -- 172.21.15.16:0/2449510757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578523d2ba0 con 0x5578523d0370 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5182471640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 0x5578523cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5181c70640 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x5578523d32a0 tx=0x7f516c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b4e7b640de1cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5167fff640 1 -- 172.21.15.16:0/2449510757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c01c040 con 0x5578523d0370 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5167fff640 1 -- 172.21.15.16:0/2449510757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c027030 con 0x5578523d0370 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.885+0000 7f5167fff640 1 -- 172.21.15.16:0/2449510757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c005630 con 0x5578523d0370 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- 172.21.15.16:0/2449510757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 msgr2=0x5578523d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x5578523d32a0 tx=0x7f516c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- 172.21.15.16:0/2449510757 shutdown_connections 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523d1760 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c6b40 0x5578523cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 --2- 172.21.15.16:0/2449510757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c55d0 0x5578523c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- 172.21.15.16:0/2449510757 >> 172.21.15.16:0/2449510757 conn(0x55785238e310 msgr2=0x5578523cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- 172.21.15.16:0/2449510757 shutdown_connections 2024-09-12T01:04:52.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- 172.21.15.16:0/2449510757 wait complete. 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 -- start start 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.886+0000 7f5186aae740 1 Processor -- start 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c6b40 0x5578523ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1aecb0 con 0x5578523d0370 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1ae5e0 con 0x5578523c55d0 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1ab840 con 0x5578523c6b40 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5186aae740 10 client.0 _ll_register_callbacks cb 0x7f516c00a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58836/0 (socket says 172.21.15.16:58836) 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 -- 172.21.15.16:0/620430977 learned_addr learned my addr 172.21.15.16:0/620430977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5182471640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c6b40 0x5578523ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 -- 172.21.15.16:0/620430977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c6b40 msgr2=0x5578523ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c6b40 0x5578523ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 -- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 msgr2=0x5578523c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 -- 172.21.15.16:0/620430977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578523c1510 con 0x5578523d0370 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5181c70640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f516c005310 tx=0x7f516c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04503ee04f952eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0055e0 con 0x5578523d0370 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c027030 con 0x5578523d0370 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.887+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f516c011910 con 0x5578523d0370 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f516c011b10 con 0x5578523d0370 2024-09-12T01:04:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 10 client.18183 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55785244a680 con 0x5578523d0370 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f516c027030 con 0x5578523d0370 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f516c022040 con 0x5578523d0370 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 10 client.18183 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 10 client.18183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 20 client.18183 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55785244adb0 0x55785244d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 4 client.18183 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557852332d70 con 0x55785244adb0 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.888+0000 7f5182471640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55785244adb0 0x55785244d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5182471640 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55785244adb0 0x55785244d1d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c42f212fd39e3ff4 server_cookie=f40dee51acfee7e in_seq=0 out_seq=0 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5165ffb640 10 client.18183 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5165ffb640 1 -- 172.21.15.16:0/620430977 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557852332d70 con 0x55785244adb0 2024-09-12T01:04:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5165ffb640 10 client.18183 handle_command_reply: tid=0 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 client.18183 shutdown 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 msgr2=0x5578523ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f516c005310 tx=0x7f516c01c040 comp rx=0 tx=0).stop 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 shutdown_connections 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55785244adb0 0x55785244d1d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578523d0370 0x5578523ca9a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578523c6b40 0x5578523ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5165ffb640 0 client.18183 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 --2- 172.21.15.16:0/620430977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5578523c55d0 0x5578523c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 >> 172.21.15.16:0/620430977 conn(0x55785238e310 msgr2=0x5578523cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 shutdown_connections 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 1 -- 172.21.15.16:0/620430977 wait complete. 2024-09-12T01:04:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.889+0000 7f5186aae740 20 client.18183 trim_cache size 0 max 16384 2024-09-12T01:04:52.894 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 msgr2=0x7f51680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 0x7f51680a5230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5174097910 tx=0x7f51740148c0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 msgr2=0x7f517c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f517400b970 tx=0x7f517400bd70 comp rx=0 tx=0).stop 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 shutdown_connections 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f51680a2d70 0x7f51680a5230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c106f00 0x7f517c19cd30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517c102780 0x7f517c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 --2- 172.21.15.16:0/3263515860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f517c101dd0 0x7f517c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 >> 172.21.15.16:0/3263515860 conn(0x7f517c0fd890 msgr2=0x7f517c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 shutdown_connections 2024-09-12T01:04:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:52.890+0000 7f51846d5640 1 -- 172.21.15.16:0/3263515860 wait complete. 2024-09-12T01:04:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:52 smithi099 ceph-mon[35654]: pgmap v271: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:53.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:52 smithi159 ceph-mon[35566]: pgmap v271: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:53.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:52 smithi016 ceph-mon[30972]: pgmap v271: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:53.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:04:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:54.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 -- 172.21.15.16:0/2045147312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f4102d40 msgr2=0x7f64f4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 --2- 172.21.15.16:0/2045147312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f4102d40 0x7f64f4103120 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402a480 comp rx=0 tx=0).stop 2024-09-12T01:04:54.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 -- 172.21.15.16:0/2045147312 shutdown_connections 2024-09-12T01:04:54.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 --2- 172.21.15.16:0/2045147312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f410c170 0x7f64f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 --2- 172.21.15.16:0/2045147312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4103660 0x7f64f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 --2- 172.21.15.16:0/2045147312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f4102d40 0x7f64f4103120 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 -- 172.21.15.16:0/2045147312 >> 172.21.15.16:0/2045147312 conn(0x7f64f40fc7a0 msgr2=0x7f64f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 -- 172.21.15.16:0/2045147312 shutdown_connections 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 -- 172.21.15.16:0/2045147312 wait complete. 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.308+0000 7f64f99c7640 1 Processor -- start 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 -- start start 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4102d40 0x7f64f4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f4103660 0x7f64f4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4112030 con 0x7f64f410c170 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64f4111eb0 con 0x7f64f4102d40 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f99c7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41121b0 con 0x7f64f4103660 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58850/0 (socket says 172.21.15.16:58850) 2024-09-12T01:04:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 -- 172.21.15.16:0/614150938 learned_addr learned my addr 172.21.15.16:0/614150938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4102d40 0x7f64f4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f27fc640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f4103660 0x7f64f4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 -- 172.21.15.16:0/614150938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f4103660 msgr2=0x7f64f4198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f4103660 0x7f64f4198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 -- 172.21.15.16:0/614150938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4102d40 msgr2=0x7f64f4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4102d40 0x7f64f4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 -- 172.21.15.16:0/614150938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f419d5c0 con 0x7f64f410c170 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.309+0000 7f64f37fe640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009fd0 tx=0x7f64ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.310+0000 7f64f89c5640 1 -- 172.21.15.16:0/614150938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011830 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.310+0000 7f64f89c5640 1 -- 172.21.15.16:0/614150938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec016070 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.310+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f419d8b0 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.310+0000 7f64f89c5640 1 -- 172.21.15.16:0/614150938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec00f2e0 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.310+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f41a5190 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.312+0000 7f64f89c5640 1 -- 172.21.15.16:0/614150938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f64ec0119d0 con 0x7f64f410c170 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.312+0000 7f64fbda0740 1 Processor -- start 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.312+0000 7f64fbda0740 1 -- start start 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e68427a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68432370 0x564e68434760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e6838d2f0 con 0x564e68428b40 2024-09-12T01:04:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e68334cf0 con 0x564e684275d0 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64fbda0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e683892b0 con 0x564e68432370 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f89c5640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58856/0 (socket says 172.21.15.16:58856) 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 -- 172.21.15.16:0/784893246 learned_addr learned my addr 172.21.15.16:0/784893246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e68427a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f37fe640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68432370 0x564e68434760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f89c5640 1 -- 172.21.15.16:0/614150938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f64ec0ce9e0 con 0x7f64f410c170 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 -- 172.21.15.16:0/784893246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68432370 msgr2=0x564e68434760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68432370 0x564e68434760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 -- 172.21.15.16:0/784893246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 msgr2=0x564e68427a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e68427a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 -- 172.21.15.16:0/784893246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e683193a0 con 0x564e68428b40 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f27fc640 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009440 tx=0x7f64e8009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a38a64a545bff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.313+0000 7f64f37fe640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 0x7f64dc0a5230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0973c0 tx=0x7f64ec0972d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64d9ffb640 1 -- 172.21.15.16:0/784893246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e800c2d0 con 0x564e68428b40 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64d9ffb640 1 -- 172.21.15.16:0/784893246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8010030 con 0x564e68428b40 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64d9ffb640 1 -- 172.21.15.16:0/784893246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e800abf0 con 0x564e68428b40 2024-09-12T01:04:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 -- 172.21.15.16:0/784893246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 msgr2=0x564e68427090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009440 tx=0x7f64e8009a10 comp rx=0 tx=0).stop 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 -- 172.21.15.16:0/784893246 shutdown_connections 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68432370 0x564e68434760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e68427a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 --2- 172.21.15.16:0/784893246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e68427090 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 -- 172.21.15.16:0/784893246 >> 172.21.15.16:0/784893246 conn(0x564e683f0310 msgr2=0x564e68423c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 -- 172.21.15.16:0/784893246 shutdown_connections 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.314+0000 7f64fbda0740 1 -- 172.21.15.16:0/784893246 wait complete. 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 -- start start 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 Processor -- start 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e68431d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 0x564e6842efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e68435070 con 0x564e68432370 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e68348060 con 0x564e68428b40 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e684351f0 con 0x564e684275d0 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64fbda0740 10 client.0 _ll_register_callbacks cb 0x564e6809b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58866/0 (socket says 172.21.15.16:58866) 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64f27fc640 1 -- 172.21.15.16:0/1071387211 learned_addr learned my addr 172.21.15.16:0/1071387211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 0x564e6842efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.315+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e68431d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 -- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 msgr2=0x564e68431d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e68431d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 -- 172.21.15.16:0/1071387211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 msgr2=0x564e6842f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 -- 172.21.15.16:0/1071387211 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e680fe3c0 con 0x564e68428b40 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 0x564e6842efe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009230 tx=0x7f64e4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d6521583f60d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4035040 con 0x564e68428b40 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f64e4005020 con 0x564e68428b40 2024-09-12T01:04:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e8010030 con 0x564e68428b40 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64e4005d50 con 0x564e68428b40 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64fbda0740 10 client.27095 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.316+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e68440ad0 con 0x564e68428b40 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8010030 con 0x564e68428b40 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64e4003610 con 0x564e68428b40 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 10 client.27095 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 10 client.27095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 20 client.27095 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e68441f30 0x564e68444320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 4 client.27095 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e6839a860 con 0x564e68441f30 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.317+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e68441f30 0x564e68444320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.318+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e68441f30 0x564e68444320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f306f990d1c92fd server_cookie=8212625181a27106 in_seq=0 out_seq=0 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.318+0000 7f64d2ffd640 10 client.27095 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1071387211 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e6839a860 con 0x564e68441f30 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64d2ffd640 10 client.27095 handle_command_reply: tid=0 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 client.27095 shutdown 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 msgr2=0x564e6842efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 0x564e6842efe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009230 tx=0x7f64e4004270 comp rx=0 tx=0).stop 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 shutdown_connections 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e6842f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e68441f30 0x564e68444320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68428b40 0x564e6842efe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1071387211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e68431d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64d2ffd640 0 client.27095 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 >> 172.21.15.16:0/1071387211 conn(0x564e683f0310 msgr2=0x564e68429960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 shutdown_connections 2024-09-12T01:04:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 1 -- 172.21.15.16:0/1071387211 wait complete. 2024-09-12T01:04:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.319+0000 7f64fbda0740 20 client.27095 trim_cache size 0 max 16384 2024-09-12T01:04:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 Processor -- start 2024-09-12T01:04:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 -- start start 2024-09-12T01:04:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e6843e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 0x564e6842a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41ad9d0 con 0x564e68428b40 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41af510 con 0x564e68432370 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64fbda0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41ae0a0 con 0x564e684275d0 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58878/0 (socket says 172.21.15.16:58878) 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 -- 172.21.15.16:0/2544378439 learned_addr learned my addr 172.21.15.16:0/2544378439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f27fc640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e6843e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f27fc640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 0x564e6842a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 -- 172.21.15.16:0/2544378439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 msgr2=0x564e6843e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e6843e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 -- 172.21.15.16:0/2544378439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 msgr2=0x564e6842a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 0x564e6842a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f2ffd640 1 -- 172.21.15.16:0/2544378439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e68423ed0 con 0x564e68428b40 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.333+0000 7f64f27fc640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 0x564e6842a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.334+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f64e404e4a0 tx=0x7f64e40044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400ffb8f325a2ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.334+0000 7f64d97fa640 1 -- 172.21.15.16:0/2544378439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4035070 con 0x564e68428b40 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.334+0000 7f64d97fa640 1 -- 172.21.15.16:0/2544378439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40037f0 con 0x564e68428b40 2024-09-12T01:04:54.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.334+0000 7f64d97fa640 1 -- 172.21.15.16:0/2544378439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e404ee20 con 0x564e68428b40 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- 172.21.15.16:0/2544378439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 msgr2=0x564e6842a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f64e404e4a0 tx=0x7f64e40044f0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- 172.21.15.16:0/2544378439 shutdown_connections 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e68432370 0x564e6842a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68428b40 0x564e6842a120 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 --2- 172.21.15.16:0/2544378439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e684275d0 0x564e6843e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- 172.21.15.16:0/2544378439 >> 172.21.15.16:0/2544378439 conn(0x564e683f0310 msgr2=0x564e68431060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- 172.21.15.16:0/2544378439 shutdown_connections 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- 172.21.15.16:0/2544378439 wait complete. 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 -- start start 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.335+0000 7f64fbda0740 1 Processor -- start 2024-09-12T01:04:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e6843a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68428b40 0x564e6843ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41aee40 con 0x564e68432370 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41ae770 con 0x564e684275d0 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41ab9d0 con 0x564e68428b40 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64fbda0740 10 client.0 _ll_register_callbacks cb 0x7f64e4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58886/0 (socket says 172.21.15.16:58886) 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 -- 172.21.15.16:0/1927266599 learned_addr learned my addr 172.21.15.16:0/1927266599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e6843a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68428b40 0x564e6843ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 -- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68428b40 msgr2=0x564e6843ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68428b40 0x564e6843ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 -- 172.21.15.16:0/1927266599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 msgr2=0x564e6843a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e6843a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 -- 172.21.15.16:0/1927266599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e684365c0 con 0x564e68432370 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64f27fc640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f64e800a760 tx=0x7f64e801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced9fb6256a976a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.336+0000 7f64d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8030070 con 0x564e68432370 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f64e800c0d0 con 0x564e68432370 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e40037f0 con 0x564e68432370 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64e800c2f0 con 0x564e68432370 2024-09-12T01:04:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 10 client.18201 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e684ac7b0 con 0x564e68432370 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40037f0 con 0x564e68432370 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64e8007cc0 con 0x564e68432370 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 10 client.18201 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 10 client.18201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 20 client.18201 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e684acb50 0x564e684aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 4 client.18201 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e68394d70 con 0x564e684acb50 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.337+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e684acb50 0x564e684aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64f2ffd640 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e684acb50 0x564e684aef40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fa093ee701f8930 server_cookie=e8b1e5b44bfb42da in_seq=0 out_seq=0 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64d2ffd640 10 client.18201 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64d2ffd640 1 -- 172.21.15.16:0/1927266599 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e68394d70 con 0x564e684acb50 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64d2ffd640 10 client.18201 handle_command_reply: tid=0 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 client.18201 shutdown 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 msgr2=0x564e68435dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f64e800a760 tx=0x7f64e801e040 comp rx=0 tx=0).stop 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 shutdown_connections 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e68432370 0x564e68435dc0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e68428b40 0x564e6843ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564e684acb50 0x564e684aef40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 --2- 172.21.15.16:0/1927266599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e684275d0 0x564e6843a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 >> 172.21.15.16:0/1927266599 conn(0x564e683f0310 msgr2=0x564e684224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64d2ffd640 0 client.18201 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 shutdown_connections 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.338+0000 7f64fbda0740 1 -- 172.21.15.16:0/1927266599 wait complete. 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64fbda0740 20 client.18201 trim_cache size 0 max 16384 2024-09-12T01:04:54.351 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 msgr2=0x7f64dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 0x7f64dc0a5230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0973c0 tx=0x7f64ec0972d0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 msgr2=0x7f64f419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009fd0 tx=0x7f64ec00b8e0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 shutdown_connections 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64dc0a2d70 0x7f64dc0a5230 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64f410c170 0x7f64f419cee0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64f4103660 0x7f64f4198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 --2- 172.21.15.16:0/614150938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f4102d40 0x7f64f4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 >> 172.21.15.16:0/614150938 conn(0x7f64f40fc7a0 msgr2=0x7f64f40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 shutdown_connections 2024-09-12T01:04:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:54.339+0000 7f64f99c7640 1 -- 172.21.15.16:0/614150938 wait complete. 2024-09-12T01:04:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:54 smithi099 ceph-mon[35654]: pgmap v272: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:54 smithi159 ceph-mon[35566]: pgmap v272: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:55.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:54 smithi016 ceph-mon[30972]: pgmap v272: 97 pgs: 97 active+clean; 10 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-09-12T01:04:55.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:04:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:55.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 -- 172.21.15.16:0/1610766395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084100e20 msgr2=0x7fd084101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 --2- 172.21.15.16:0/1610766395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084100e20 0x7fd084101200 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd080009900 tx=0x7fd08002a480 comp rx=0 tx=0).stop 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 -- 172.21.15.16:0/1610766395 shutdown_connections 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 --2- 172.21.15.16:0/1610766395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102ef0 0x7fd08410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 --2- 172.21.15.16:0/1610766395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084102570 0x7fd0841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 --2- 172.21.15.16:0/1610766395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084100e20 0x7fd084101200 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 -- 172.21.15.16:0/1610766395 >> 172.21.15.16:0/1610766395 conn(0x7fd0840fc7d0 msgr2=0x7fd0840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 -- 172.21.15.16:0/1610766395 shutdown_connections 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 -- 172.21.15.16:0/1610766395 wait complete. 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.893+0000 7fd08ba83640 1 Processor -- start 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 -- start start 2024-09-12T01:04:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084100e20 0x7fd0840fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102570 0x7fd0840fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd084110ff0 con 0x7fd084102ef0 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd084110e70 con 0x7fd084102570 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08ba83640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd084111170 con 0x7fd084100e20 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58914/0 (socket says 172.21.15.16:58914) 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 -- 172.21.15.16:0/3308901498 learned_addr learned my addr 172.21.15.16:0/3308901498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08901e640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102570 0x7fd0840fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08981f640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084100e20 0x7fd0840fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 -- 172.21.15.16:0/3308901498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084100e20 msgr2=0x7fd0840fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084100e20 0x7fd0840fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 -- 172.21.15.16:0/3308901498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102570 msgr2=0x7fd0840fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102570 0x7fd0840fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 -- 172.21.15.16:0/3308901498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0841a6aa0 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.894+0000 7fd08a020640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00eaa0 tx=0x7fd07c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.895+0000 7fd07affd640 1 -- 172.21.15.16:0/3308901498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004d20 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.895+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0841a6d90 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.895+0000 7fd07affd640 1 -- 172.21.15.16:0/3308901498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c005020 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.895+0000 7fd07affd640 1 -- 172.21.15.16:0/3308901498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0116d0 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.895+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841a7280 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.897+0000 7fd07affd640 1 -- 172.21.15.16:0/3308901498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd07c0051d0 con 0x7fd084102ef0 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.897+0000 7fd08de5c740 1 Processor -- start 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.897+0000 7fd08de5c740 1 -- start start 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d03b5392f0 con 0x55d03b5d4b40 2024-09-12T01:04:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d03b4e0cf0 con 0x55d03b5de370 2024-09-12T01:04:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08de5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d03b5352b0 con 0x55d03b5d35d0 2024-09-12T01:04:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd07affd640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 0x7fd0700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58930/0 (socket says 172.21.15.16:58930) 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 -- 172.21.15.16:0/141462522 learned_addr learned my addr 172.21.15.16:0/141462522 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08981f640 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08a020640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 0x7fd0700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd07affd640 1 -- 172.21.15.16:0/3308901498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd07c0cf680 con 0x7fd084102ef0 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 -- 172.21.15.16:0/141462522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 msgr2=0x55d03b5d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 -- 172.21.15.16:0/141462522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 msgr2=0x55d03b5e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5e0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 -- 172.21.15.16:0/141462522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03b4c53a0 con 0x55d03b5d4b40 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08901e640 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd074006aa0 tx=0x7fd0740096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92223733c05c91a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.898+0000 7fd08a020640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 0x7fd0700a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd07c09bad0 tx=0x7fd07c004f70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd06ffff640 1 -- 172.21.15.16:0/141462522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074009e90 con 0x55d03b5d4b40 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd06ffff640 1 -- 172.21.15.16:0/141462522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07401a070 con 0x55d03b5d4b40 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd06ffff640 1 -- 172.21.15.16:0/141462522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074012610 con 0x55d03b5d4b40 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 -- 172.21.15.16:0/141462522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 msgr2=0x55d03b5d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd074006aa0 tx=0x7fd0740096f0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 -- 172.21.15.16:0/141462522 shutdown_connections 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 --2- 172.21.15.16:0/141462522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d3090 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.899+0000 7fd08de5c740 1 -- 172.21.15.16:0/141462522 >> 172.21.15.16:0/141462522 conn(0x55d03b59c310 msgr2=0x55d03b5cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:56.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.900+0000 7fd08de5c740 1 -- 172.21.15.16:0/141462522 shutdown_connections 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.900+0000 7fd08de5c740 1 -- 172.21.15.16:0/141462522 wait complete. 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.900+0000 7fd08de5c740 1 -- start start 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.900+0000 7fd08de5c740 1 Processor -- start 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d03b5e1010 con 0x55d03b5d4b40 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d03b4f4060 con 0x55d03b5de370 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d03b5e1190 con 0x55d03b5d35d0 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 10 client.0 _ll_register_callbacks cb 0x55d03b2478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58940/0 (socket says 172.21.15.16:58940) 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 -- 172.21.15.16:0/3965746868 learned_addr learned my addr 172.21.15.16:0/3965746868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:56.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08901e640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08901e640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 -- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 msgr2=0x55d03b5ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5ddce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 -- 172.21.15.16:0/3965746868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 msgr2=0x55d03b5db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 -- 172.21.15.16:0/3965746868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03b2aa3c0 con 0x55d03b5d4b40 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08901e640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08981f640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd080009900 tx=0x7fd0800045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876f51469cc6e858 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd06d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080035040 con 0x55d03b5d4b40 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd0800046c0 con 0x55d03b5d4b40 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07401a070 con 0x55d03b5d4b40 2024-09-12T01:04:56.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd080005610 con 0x55d03b5d4b40 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 10 client.18213 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.901+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d03b5eca20 con 0x55d03b5d4b40 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07401a070 con 0x55d03b5d4b40 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd080003c30 con 0x55d03b5d4b40 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 10 client.18213 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 10 client.18213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 20 client.18213 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b5ede80 0x55d03b5f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 4 client.18213 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03b546860 con 0x55d03b5ede80 2024-09-12T01:04:56.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.902+0000 7fd08901e640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b5ede80 0x55d03b5f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.903+0000 7fd06d7fa640 10 client.18213 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.903+0000 7fd08901e640 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b5ede80 0x55d03b5f02a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf9b5bdc832467d server_cookie=89316734f01a4c97 in_seq=0 out_seq=0 2024-09-12T01:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.903+0000 7fd06d7fa640 1 -- 172.21.15.16:0/3965746868 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d03b546860 con 0x55d03b5ede80 2024-09-12T01:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.903+0000 7fd06d7fa640 10 client.18213 handle_command_reply: tid=0 2024-09-12T01:04:56.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.903+0000 7fd08de5c740 1 client.18213 shutdown 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 msgr2=0x55d03b5daf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd080009900 tx=0x7fd0800045a0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 shutdown_connections 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b5ede80 0x55d03b5f02a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5daf90 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 --2- 172.21.15.16:0/3965746868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd06d7fa640 0 client.18213 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 >> 172.21.15.16:0/3965746868 conn(0x55d03b59c310 msgr2=0x55d03b5d5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 shutdown_connections 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 1 -- 172.21.15.16:0/3965746868 wait complete. 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.904+0000 7fd08de5c740 20 client.18213 trim_cache size 0 max 16384 2024-09-12T01:04:56.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 Processor -- start 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 -- start start 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5d35d0 0x55d03b5ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d4b40 0x55d03b5d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841afa60 con 0x55d03b5de370 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841b15a0 con 0x55d03b5d35d0 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08de5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841b0130 con 0x55d03b5d4b40 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08901e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08901e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58946/0 (socket says 172.21.15.16:58946) 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08901e640 1 -- 172.21.15.16:0/4019702937 learned_addr learned my addr 172.21.15.16:0/4019702937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.917+0000 7fd08981f640 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d4b40 0x55d03b5d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 -- 172.21.15.16:0/4019702937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d4b40 msgr2=0x55d03b5d9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d4b40 0x55d03b5d9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 -- 172.21.15.16:0/4019702937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5d35d0 msgr2=0x55d03b5ea2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5d35d0 0x55d03b5ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 -- 172.21.15.16:0/4019702937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03b5d0960 con 0x55d03b5de370 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd08901e640 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x55d03b5d96c0 tx=0x7fd074015d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2535ed475a38f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd06f7fe640 1 -- 172.21.15.16:0/4019702937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07402f070 con 0x55d03b5de370 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd06f7fe640 1 -- 172.21.15.16:0/4019702937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07401a030 con 0x55d03b5de370 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.918+0000 7fd06f7fe640 1 -- 172.21.15.16:0/4019702937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074021950 con 0x55d03b5de370 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- 172.21.15.16:0/4019702937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 msgr2=0x55d03b5f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x55d03b5d96c0 tx=0x7fd074015d80 comp rx=0 tx=0).stop 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- 172.21.15.16:0/4019702937 shutdown_connections 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5de370 0x55d03b5f5b70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d4b40 0x55d03b5d9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 --2- 172.21.15.16:0/4019702937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5d35d0 0x55d03b5ea2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- 172.21.15.16:0/4019702937 >> 172.21.15.16:0/4019702937 conn(0x55d03b59c310 msgr2=0x55d03b5d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- 172.21.15.16:0/4019702937 shutdown_connections 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- 172.21.15.16:0/4019702937 wait complete. 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 -- start start 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 Processor -- start 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.919+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841b0ed0 con 0x55d03b5d4b40 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841b0800 con 0x55d03b5de370 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0841ada60 con 0x55d03b5d35d0 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 10 client.0 _ll_register_callbacks cb 0x7fd080009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08901e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58958/0 (socket says 172.21.15.16:58958) 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 -- 172.21.15.16:0/1659881358 learned_addr learned my addr 172.21.15.16:0/1659881358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08901e640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 -- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 msgr2=0x55d03b5d8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 -- 172.21.15.16:0/1659881358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 msgr2=0x55d03b5d8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5d8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 -- 172.21.15.16:0/1659881358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03b660550 con 0x55d03b5d4b40 2024-09-12T01:04:56.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08901e640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08981f640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd08002a9a0 tx=0x7fd080004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca8b99e9fc577cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd06d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080035040 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fd0800035f0 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.920+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07401a030 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0800037f0 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 10 client.18219 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d03b6617f0 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07401a030 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd080004c50 con 0x55d03b5d4b40 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 10 client.18219 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 20 client.18219 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:56.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b661f10 0x55d03b664330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 4 client.18219 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d03b540d70 con 0x55d03b661f10 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.921+0000 7fd08901e640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b661f10 0x55d03b664330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08901e640 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b661f10 0x55d03b664330 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10dd112dffb2994 server_cookie=88c7d69ccc987695 in_seq=0 out_seq=0 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd06d7fa640 10 client.18219 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd06d7fa640 1 -- 172.21.15.16:0/1659881358 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d03b540d70 con 0x55d03b661f10 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd06d7fa640 10 client.18219 handle_command_reply: tid=0 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 client.18219 shutdown 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 msgr2=0x55d03b5d86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd08002a9a0 tx=0x7fd080004300 comp rx=0 tx=0).stop 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 shutdown_connections 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d03b661f10 0x55d03b664330 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d03b5de370 0x55d03b5d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d03b5d4b40 0x55d03b5d86a0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd06d7fa640 0 client.18219 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 --2- 172.21.15.16:0/1659881358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d03b5d35d0 0x55d03b5d8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 >> 172.21.15.16:0/1659881358 conn(0x55d03b59c310 msgr2=0x55d03b5da080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 shutdown_connections 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.922+0000 7fd08de5c740 1 -- 172.21.15.16:0/1659881358 wait complete. 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08de5c740 20 client.18219 trim_cache size 0 max 16384 2024-09-12T01:04:56.019 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:04:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 msgr2=0x7fd0700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 0x7fd0700a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd07c09bad0 tx=0x7fd07c004f70 comp rx=0 tx=0).stop 2024-09-12T01:04:56.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 msgr2=0x7fd0840fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00eaa0 tx=0x7fd07c00eea0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 shutdown_connections 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0700a2cf0 0x7fd0700a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd084102ef0 0x7fd0840fd700 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd084102570 0x7fd0840fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 --2- 172.21.15.16:0/3308901498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd084100e20 0x7fd0840fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 >> 172.21.15.16:0/3308901498 conn(0x7fd0840fc7d0 msgr2=0x7fd084069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 shutdown_connections 2024-09-12T01:04:56.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:55.923+0000 7fd08ba83640 1 -- 172.21.15.16:0/3308901498 wait complete. 2024-09-12T01:04:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:56 smithi016 ceph-mon[30972]: pgmap v273: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-12T01:04:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:56 smithi099 ceph-mon[35654]: pgmap v273: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-12T01:04:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:56 smithi159 ceph-mon[35566]: pgmap v273: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-09-12T01:04:57.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:04:57.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:04:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 -- 172.21.15.16:0/1003836080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 msgr2=0x7fbfb4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/1003836080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb4106e60 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009930 tx=0x7fbfa402a420 comp rx=0 tx=0).stop 2024-09-12T01:04:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 -- 172.21.15.16:0/1003836080 shutdown_connections 2024-09-12T01:04:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/1003836080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4101420 0x7fbfb410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/1003836080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4100a80 0x7fbfb4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/1003836080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb4106e60 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 -- 172.21.15.16:0/1003836080 >> 172.21.15.16:0/1003836080 conn(0x7fbfb40fc7a0 msgr2=0x7fbfb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 -- 172.21.15.16:0/1003836080 shutdown_connections 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.404+0000 7fbfb9d56640 1 -- 172.21.15.16:0/1003836080 wait complete. 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 Processor -- start 2024-09-12T01:04:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 -- start start 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4100a80 0x7fbfb419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4101420 0x7fbfb419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4111fa0 con 0x7fbfb4106a80 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4111e20 con 0x7fbfb4101420 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb9d56640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb4112120 con 0x7fbfb4100a80 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54596/0 (socket says 172.21.15.16:54596) 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb3fff640 1 -- 172.21.15.16:0/4088366124 learned_addr learned my addr 172.21.15.16:0/4088366124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4100a80 0x7fbfb419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.405+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4101420 0x7fbfb419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 -- 172.21.15.16:0/4088366124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4100a80 msgr2=0x7fbfb419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4100a80 0x7fbfb419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 -- 172.21.15.16:0/4088366124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4101420 msgr2=0x7fbfb419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4101420 0x7fbfb419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 -- 172.21.15.16:0/4088366124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419f690 con 0x7fbfb4106a80 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00eaa0 tx=0x7fbfac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/4088366124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00cdb0 con 0x7fbfb4106a80 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb419f980 con 0x7fbfb4106a80 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/4088366124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac005020 con 0x7fbfb4106a80 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a7260 con 0x7fbfb4106a80 2024-09-12T01:04:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.406+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/4088366124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01f060 con 0x7fbfb4106a80 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.409+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/4088366124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbfac005560 con 0x7fbfb4106a80 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.409+0000 7fbfbc12f740 1 Processor -- start 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.409+0000 7fbfbc12f740 1 -- start start 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.409+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091887b60 0x55c091887f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.409+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091892800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c0917ed280 con 0x55c091892d40 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c091794b10 con 0x55c091888480 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c0917e9240 con 0x55c091887b60 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54608/0 (socket says 172.21.15.16:54608) 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2860549389 learned_addr learned my addr 172.21.15.16:0/2860549389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091887b60 0x55c091887f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091892800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb0ff9640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 0x7fbf9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2860549389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091887b60 msgr2=0x55c091887f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091887b60 0x55c091887f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2860549389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 msgr2=0x55c091892800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091892800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2860549389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0917a11f0 con 0x55c091892d40 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8016530 tx=0x7fbfa800a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb6b6ba8c5302fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 0x7fbf9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/4088366124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbfac0cf640 con 0x7fbfb4106a80 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.410+0000 7fbf99ffb640 1 -- 172.21.15.16:0/2860549389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8018430 con 0x55c091892d40 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbf99ffb640 1 -- 172.21.15.16:0/2860549389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa8018ab0 con 0x55c091892d40 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbf99ffb640 1 -- 172.21.15.16:0/2860549389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80179b0 con 0x55c091892d40 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2860549389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 msgr2=0x55c091895130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 0x7fbf9c0a51b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0ccdf0 tx=0x7fbfac005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8016530 tx=0x7fbfa800a2b0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2860549389 shutdown_connections 2024-09-12T01:04:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c091895130 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091892800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2860549389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091887b60 0x55c091887f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2860549389 >> 172.21.15.16:0/2860549389 conn(0x55c0918840f0 msgr2=0x55c0918844d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2860549389 shutdown_connections 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.411+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2860549389 wait complete. 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 -- start start 2024-09-12T01:04:58.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 Processor -- start 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c091890170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091892d40 0x55c0918906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c0918958b0 con 0x55c091887b60 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c0917a7f10 con 0x55c091892d40 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c091895a30 con 0x55c091888480 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfbc12f740 10 client.0 _ll_register_callbacks cb 0x55c091770a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54620/0 (socket says 172.21.15.16:54620) 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.412+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2473205222 learned_addr learned my addr 172.21.15.16:0/2473205222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c091890170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 msgr2=0x55c091890170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c091890170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2473205222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091892d40 msgr2=0x55c0918906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091892d40 0x55c0918906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/2473205222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09155e6a0 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c091890170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800a220 tx=0x7fbfa8018e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0aa5c6a1eb2e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbf92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8025020 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbfa8018840 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa8018ab0 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfa8030050 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfbc12f740 10 client.18231 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.413+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c0918a14c0 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa8018ab0 con 0x55c091887b60 2024-09-12T01:04:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbfa8012050 con 0x55c091887b60 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 10 client.18231 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 10 client.18231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 20 client.18231 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c0918a28c0 0x55c0918a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 4 client.18231 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0917fa7f0 con 0x55c0918a28c0 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c0918a28c0 0x55c0918a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c0918a28c0 0x55c0918a4cb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fa89063a8bae97 server_cookie=9529501741f2226 in_seq=0 out_seq=0 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.414+0000 7fbf92ffd640 10 client.18231 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbf92ffd640 1 -- 172.21.15.16:0/2473205222 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0917fa7f0 con 0x55c0918a28c0 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbf92ffd640 10 client.18231 handle_command_reply: tid=0 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 client.18231 shutdown 2024-09-12T01:04:58.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 msgr2=0x55c09188fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800a220 tx=0x7fbfa8018e70 comp rx=0 tx=0).stop 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 shutdown_connections 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c0918a28c0 0x55c0918a4cb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091892d40 0x55c0918906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c091890170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbf92ffd640 0 client.18231 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/2473205222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188fc30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.415+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 >> 172.21.15.16:0/2473205222 conn(0x55c0918840f0 msgr2=0x55c091883270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.416+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 shutdown_connections 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.416+0000 7fbfbc12f740 1 -- 172.21.15.16:0/2473205222 wait complete. 2024-09-12T01:04:58.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.416+0000 7fbfbc12f740 20 client.18231 trim_cache size 0 max 16384 2024-09-12T01:04:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 Processor -- start 2024-09-12T01:04:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 -- start start 2024-09-12T01:04:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091883160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091892d40 0x55c0918aa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41afb90 con 0x55c091887b60 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b16d0 con 0x55c091888480 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b0260 con 0x55c091892d40 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54626/0 (socket says 172.21.15.16:54626) 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.433+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/3438824394 learned_addr learned my addr 172.21.15.16:0/3438824394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091883160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/3438824394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091892d40 msgr2=0x55c0918aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091892d40 0x55c0918aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/3438824394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 msgr2=0x55c091883160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091883160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/3438824394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0918959e0 con 0x55c091887b60 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8016f20 tx=0x7fbfa800ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed310318a4cb623 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbf997fa640 1 -- 172.21.15.16:0/3438824394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8025070 con 0x55c091887b60 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbf997fa640 1 -- 172.21.15.16:0/3438824394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa80219f0 con 0x55c091887b60 2024-09-12T01:04:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.434+0000 7fbf997fa640 1 -- 172.21.15.16:0/3438824394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8017940 con 0x55c091887b60 2024-09-12T01:04:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- 172.21.15.16:0/3438824394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 msgr2=0x55c09188f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8016f20 tx=0x7fbfa800ad10 comp rx=0 tx=0).stop 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- 172.21.15.16:0/3438824394 shutdown_connections 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091892d40 0x55c0918aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091888480 0x55c091883160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/3438824394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091887b60 0x55c09188f3e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- 172.21.15.16:0/3438824394 >> 172.21.15.16:0/3438824394 conn(0x55c0918840f0 msgr2=0x55c091882a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- 172.21.15.16:0/3438824394 shutdown_connections 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- 172.21.15.16:0/3438824394 wait complete. 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 -- start start 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.435+0000 7fbfbc12f740 1 Processor -- start 2024-09-12T01:04:58.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091887b60 0x55c09189b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c09188c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b1000 con 0x55c091892d40 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41b0930 con 0x55c091887b60 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41adb90 con 0x55c091888480 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 10 client.0 _ll_register_callbacks cb 0x7fbfa800ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54638/0 (socket says 172.21.15.16:54638) 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/160672693 learned_addr learned my addr 172.21.15.16:0/160672693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091887b60 0x55c09189b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c09188c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 msgr2=0x55c09188c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c09188c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/160672693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091887b60 msgr2=0x55c09189b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091887b60 0x55c09189b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/160672693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c091888a90 con 0x55c091892d40 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfb2ffd640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800aa00 tx=0x7fbfa80189d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f55ec2606de0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbf92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8025070 con 0x55c091892d40 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbfa8017240 con 0x55c091892d40 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.436+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa80219f0 con 0x55c091892d40 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfa8017670 con 0x55c091892d40 2024-09-12T01:04:58.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 10 client.18237 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c09190d050 con 0x55c091892d40 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfa80219f0 con 0x55c091892d40 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbfa8012040 con 0x55c091892d40 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 10 client.18237 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 10 client.18237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 20 client.18237 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c09190d660 0x55c09190fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 4 client.18237 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0917f4d00 con 0x55c09190d660 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.437+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c09190d660 0x55c09190fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.438+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c09190d660 0x55c09190fa50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908188310e24d0ef server_cookie=ee96f71438166ea6 in_seq=0 out_seq=0 2024-09-12T01:04:58.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.438+0000 7fbf92ffd640 10 client.18237 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbf92ffd640 1 -- 172.21.15.16:0/160672693 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0917f4d00 con 0x55c09190d660 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbf92ffd640 10 client.18237 handle_command_reply: tid=0 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 client.18237 shutdown 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 msgr2=0x55c09188ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800aa00 tx=0x7fbfa80189d0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 shutdown_connections 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55c09190d660 0x55c09190fa50 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c091892d40 0x55c09188ce60 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c091888480 0x55c09188c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 --2- 172.21.15.16:0/160672693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c091887b60 0x55c09189b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 >> 172.21.15.16:0/160672693 conn(0x55c0918840f0 msgr2=0x55c091889da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbf92ffd640 0 client.18237 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:58.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 shutdown_connections 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 1 -- 172.21.15.16:0/160672693 wait complete. 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.439+0000 7fbfbc12f740 20 client.18237 trim_cache size 0 max 16384 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4ee1c197-3d23-4361-b2a1-782484547a7d", 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:04:58.444 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 msgr2=0x7fbf9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 0x7fbf9c0a51b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0ccdf0 tx=0x7fbfac005850 comp rx=0 tx=0).stop 2024-09-12T01:04:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 msgr2=0x7fbfb419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00eaa0 tx=0x7fbfac00ef60 comp rx=0 tx=0).stop 2024-09-12T01:04:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 shutdown_connections 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbf9c0a2cf0 0x7fbf9c0a51b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4106a80 0x7fbfb419efb0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfb4101420 0x7fbfb419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 --2- 172.21.15.16:0/4088366124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfb4100a80 0x7fbfb419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 >> 172.21.15.16:0/4088366124 conn(0x7fbfb40fc7a0 msgr2=0x7fbfb410c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 shutdown_connections 2024-09-12T01:04:58.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:58.440+0000 7fbfb9d56640 1 -- 172.21.15.16:0/4088366124 wait complete. 2024-09-12T01:04:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:58 smithi099 ceph-mon[35654]: pgmap v274: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-12T01:04:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:58 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:04:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:58 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:04:59.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:58 smithi016 ceph-mon[30972]: pgmap v274: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-12T01:04:59.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:58 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:04:59.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:58 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:04:59.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:58 smithi159 ceph-mon[35566]: pgmap v274: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-09-12T01:04:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:58 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:04:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:58 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:04:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:04:59.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:04:59.460 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:04:59.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:04:59 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:04:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.951+0000 7fe92e9dd640 1 -- 172.21.15.16:0/3312126382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9281036e0 msgr2=0x7fe928107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.951+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/3312126382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9281036e0 0x7fe928107b90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe91c009930 tx=0x7fe91c02a420 comp rx=0 tx=0).stop 2024-09-12T01:04:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 -- 172.21.15.16:0/3312126382 shutdown_connections 2024-09-12T01:04:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/3312126382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281082a0 0x7fe92810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/3312126382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9281036e0 0x7fe928107b90 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/3312126382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928102d30 0x7fe928103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 -- 172.21.15.16:0/3312126382 >> 172.21.15.16:0/3312126382 conn(0x7fe9280fc7d0 msgr2=0x7fe9280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 -- 172.21.15.16:0/3312126382 shutdown_connections 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 -- 172.21.15.16:0/3312126382 wait complete. 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 Processor -- start 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.952+0000 7fe92e9dd640 1 -- start start 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 0x7fe928100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 0x7fe928101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281120c0 con 0x7fe928102d30 2024-09-12T01:04:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe928111f40 con 0x7fe9281082a0 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe928112240 con 0x7fe9281036e0 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54656/0 (socket says 172.21.15.16:54656) 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 -- 172.21.15.16:0/1476824231 learned_addr learned my addr 172.21.15.16:0/1476824231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 0x7fe928101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe9277fe640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 0x7fe928100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 -- 172.21.15.16:0/1476824231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 msgr2=0x7fe928100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 0x7fe928100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 -- 172.21.15.16:0/1476824231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 msgr2=0x7fe928101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 0x7fe928101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 -- 172.21.15.16:0/1476824231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281a6a80 con 0x7fe928102d30 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe9277fe640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 0x7fe928100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 0x7fe928101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe927fff640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c920 tx=0x7fe91800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe9257fa640 1 -- 172.21.15.16:0/1476824231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918007c70 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe9257fa640 1 -- 172.21.15.16:0/1476824231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918007e10 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281a6d70 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.953+0000 7fe9257fa640 1 -- 172.21.15.16:0/1476824231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918018890 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.954+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a7250 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.956+0000 7fe9257fa640 1 -- 172.21.15.16:0/1476824231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe918018a70 con 0x7fe928102d30 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.956+0000 7fe930db6740 1 Processor -- start 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.956+0000 7fe930db6740 1 -- start start 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec028880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec035a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627ebf8e2e0 con 0x5627ec028dc0 2024-09-12T01:04:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5627ebf35ce0 con 0x5627ec0284a0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe930db6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627ebf8a2a0 con 0x5627ec033680 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54666/0 (socket says 172.21.15.16:54666) 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe92cf7a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec028880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 -- 172.21.15.16:0/368923829 learned_addr learned my addr 172.21.15.16:0/368923829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe9257fa640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 0x7fe9100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe9277fe640 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec035a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 0x7fe9100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 -- 172.21.15.16:0/368923829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 msgr2=0x5627ec035a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec035a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 -- 172.21.15.16:0/368923829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 msgr2=0x5627ec028880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec028880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe9257fa640 1 -- 172.21.15.16:0/1476824231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe9180cf6b0 con 0x7fe928102d30 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 -- 172.21.15.16:0/368923829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ebf1a440 con 0x5627ec028dc0 2024-09-12T01:04:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec028880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.957+0000 7fe927fff640 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe9180ccd10 tx=0x7fe918000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de65c90fdf52c33d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 0x7fe9100a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9200139c0 tx=0x7fe920004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe90e7fc640 1 -- 172.21.15.16:0/368923829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91800de20 con 0x5627ec028dc0 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe90e7fc640 1 -- 172.21.15.16:0/368923829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91809bb70 con 0x5627ec028dc0 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe90e7fc640 1 -- 172.21.15.16:0/368923829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180981e0 con 0x5627ec028dc0 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 -- 172.21.15.16:0/368923829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 msgr2=0x5627ec033140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe9180ccd10 tx=0x7fe918000bc0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 -- 172.21.15.16:0/368923829 shutdown_connections 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec035a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec028dc0 0x5627ec033140 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 --2- 172.21.15.16:0/368923829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec028880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 -- 172.21.15.16:0/368923829 >> 172.21.15.16:0/368923829 conn(0x5627ec024a30 msgr2=0x5627ec024e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.958+0000 7fe930db6740 1 -- 172.21.15.16:0/368923829 shutdown_connections 2024-09-12T01:04:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 -- 172.21.15.16:0/368923829 wait complete. 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 -- start start 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 Processor -- start 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec028dc0 0x5627ec0301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec030730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5627ec036250 con 0x5627ec0284a0 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5627ebf49050 con 0x5627ec028dc0 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627ec0363d0 con 0x5627ec033680 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe930db6740 10 client.0 _ll_register_callbacks cb 0x5627ebf89160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe927fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54676/0 (socket says 172.21.15.16:54676) 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe927fff640 1 -- 172.21.15.16:0/2407151202 learned_addr learned my addr 172.21.15.16:0/2407151202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.959+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec030730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9277fe640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec028dc0 0x5627ec0301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 -- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 msgr2=0x5627ec030730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec030730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 -- 172.21.15.16:0/2407151202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec028dc0 msgr2=0x5627ec0301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec028dc0 0x5627ec0301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 -- 172.21.15.16:0/2407151202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ebcff3c0 con 0x5627ec0284a0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe927fff640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe918097fd0 tx=0x7fe91809a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d2966549d6724f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91800de20 con 0x5627ec0284a0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe91805eae0 con 0x5627ec0284a0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91809bb70 con 0x5627ec0284a0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe918018e60 con 0x5627ec0284a0 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe930db6740 10 client.18249 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627ec041ba0 con 0x5627ec0284a0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91809bb70 con 0x5627ec0284a0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.960+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe91809afc0 con 0x5627ec0284a0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 10 client.18249 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 20 client.18249 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec042ef0 0x5627ec0452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 4 client.18249 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627ebf9b850 con 0x5627ec042ef0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe9277fe640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec042ef0 0x5627ec0452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe9077fe640 10 client.18249 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.961+0000 7fe9277fe640 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec042ef0 0x5627ec0452e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488cdbd40d36584c server_cookie=537eadf3f5a380c6 in_seq=0 out_seq=0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe9077fe640 1 -- 172.21.15.16:0/2407151202 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627ebf9b850 con 0x5627ec042ef0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe9077fe640 10 client.18249 handle_command_reply: tid=0 2024-09-12T01:04:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 client.18249 shutdown 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 msgr2=0x5627ec032f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe918097fd0 tx=0x7fe91809a080 comp rx=0 tx=0).stop 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 shutdown_connections 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec042ef0 0x5627ec0452e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec033680 0x5627ec030730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec028dc0 0x5627ec0301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 --2- 172.21.15.16:0/2407151202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec0284a0 0x5627ec032f40 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe9077fe640 0 client.18249 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 >> 172.21.15.16:0/2407151202 conn(0x5627ec024a30 msgr2=0x5627ec02a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 shutdown_connections 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 1 -- 172.21.15.16:0/2407151202 wait complete. 2024-09-12T01:04:59.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.962+0000 7fe930db6740 20 client.18249 trim_cache size 0 max 16384 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.979+0000 7fe930db6740 1 Processor -- start 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.979+0000 7fe930db6740 1 -- start start 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec048310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec048850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281afba0 con 0x5627ec033680 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b16e0 con 0x5627ec0284a0 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe930db6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b0270 con 0x5627ec028dc0 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54688/0 (socket says 172.21.15.16:54688) 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 -- 172.21.15.16:0/4147679880 learned_addr learned my addr 172.21.15.16:0/4147679880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe9277fe640 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec048850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe927fff640 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec048310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 -- 172.21.15.16:0/4147679880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 msgr2=0x5627ec048850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec048850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 -- 172.21.15.16:0/4147679880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 msgr2=0x5627ec048310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec048310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 -- 172.21.15.16:0/4147679880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ec02f1a0 con 0x5627ec033680 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.980+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe9200109e0 tx=0x7fe9200078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99aeaa3ecdc8afb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe90dffb640 1 -- 172.21.15.16:0/4147679880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920012b60 con 0x5627ec033680 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe90dffb640 1 -- 172.21.15.16:0/4147679880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920017030 con 0x5627ec033680 2024-09-12T01:04:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe90dffb640 1 -- 172.21.15.16:0/4147679880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92001f650 con 0x5627ec033680 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 -- 172.21.15.16:0/4147679880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 msgr2=0x5627ec02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe9200109e0 tx=0x7fe9200078a0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 -- 172.21.15.16:0/4147679880 shutdown_connections 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec02d450 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec048850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 --2- 172.21.15.16:0/4147679880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec048310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 -- 172.21.15.16:0/4147679880 >> 172.21.15.16:0/4147679880 conn(0x5627ec024a30 msgr2=0x5627ec023b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 -- 172.21.15.16:0/4147679880 shutdown_connections 2024-09-12T01:04:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.981+0000 7fe930db6740 1 -- 172.21.15.16:0/4147679880 wait complete. 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 -- start start 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 Processor -- start 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec0a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec0a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b1010 con 0x5627ec033680 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b0940 con 0x5627ec0284a0 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281adba0 con 0x5627ec028dc0 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe930db6740 10 client.0 _ll_register_callbacks cb 0x7fe920007700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe92cf7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe92cf7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54692/0 (socket says 172.21.15.16:54692) 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec0a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.982+0000 7fe92cf7a640 1 -- 172.21.15.16:0/2194713016 learned_addr learned my addr 172.21.15.16:0/2194713016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe927fff640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec0a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 -- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 msgr2=0x5627ec0a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec0a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 -- 172.21.15.16:0/2194713016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 msgr2=0x5627ec0a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec0a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 -- 172.21.15.16:0/2194713016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ec0332b0 con 0x5627ec033680 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe927fff640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec0a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe92cf7a640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe920007080 tx=0x7fe920013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56e370e580c2394 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe9077fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920036060 con 0x5627ec033680 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe920017030 con 0x5627ec033680 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fe920012c30 con 0x5627ec033680 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe920012e30 con 0x5627ec033680 2024-09-12T01:04:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe930db6740 10 client.18255 fetch_fsmap learned FSMap version 28 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.983+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5627ec0adef0 con 0x5627ec033680 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920017030 con 0x5627ec033680 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe92001f910 con 0x5627ec033680 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 10 client.18255 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 20 client.18255 populate_metadata read hostname 'smithi016' 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec0ae510 0x5627ec0b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 4 client.18255 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627ebf95d60 con 0x5627ec0ae510 2024-09-12T01:04:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe927fff640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec0ae510 0x5627ec0b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe927fff640 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec0ae510 0x5627ec0b0930 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dec160d7589b5ef server_cookie=bd338fd59384e520 in_seq=0 out_seq=0 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.984+0000 7fe9077fe640 10 client.18255 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe9077fe640 1 -- 172.21.15.16:0/2194713016 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627ebf95d60 con 0x5627ec0ae510 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe9077fe640 10 client.18255 handle_command_reply: tid=0 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 client.18255 shutdown 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 msgr2=0x5627ec0ac840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe920007080 tx=0x7fe920013690 comp rx=0 tx=0).stop 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 shutdown_connections 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5627ec0ae510 0x5627ec0b0930 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627ec033680 0x5627ec0ac840 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627ec028dc0 0x5627ec0a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe9077fe640 0 client.18255 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:04:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 --2- 172.21.15.16:0/2194713016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5627ec0284a0 0x5627ec0a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 >> 172.21.15.16:0/2194713016 conn(0x5627ec024a30 msgr2=0x5627ec031a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 shutdown_connections 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 1 -- 172.21.15.16:0/2194713016 wait complete. 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.985+0000 7fe930db6740 20 client.18255 trim_cache size 0 max 16384 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:04:59.989 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 msgr2=0x7fe9100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 0x7fe9100a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe9200139c0 tx=0x7fe920004a40 comp rx=0 tx=0).stop 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 msgr2=0x7fe928100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe91800c920 tx=0x7fe91800cde0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 shutdown_connections 2024-09-12T01:04:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe9100a2cf0 0x7fe9100a51b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281082a0 0x7fe928101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9281036e0 0x7fe928100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 --2- 172.21.15.16:0/1476824231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe928102d30 0x7fe928100990 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 >> 172.21.15.16:0/1476824231 conn(0x7fe9280fc7d0 msgr2=0x7fe928106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 shutdown_connections 2024-09-12T01:04:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:04:59.986+0000 7fe92e9dd640 1 -- 172.21.15.16:0/1476824231 wait complete. 2024-09-12T01:05:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:04:59 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:05:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:04:59 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:05:00.699 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:05:00.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:05:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:01.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:00 smithi099 ceph-mon[35654]: pgmap v275: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-12T01:05:01.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:00 smithi016 ceph-mon[30972]: pgmap v275: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-12T01:05:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:00 smithi159 ceph-mon[35566]: pgmap v275: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 -- 172.21.15.16:0/3863821271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 msgr2=0x7f6c40100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 --2- 172.21.15.16:0/3863821271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40100ee0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009930 tx=0x7f6c3002a420 comp rx=0 tx=0).stop 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 -- 172.21.15.16:0/3863821271 shutdown_connections 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 --2- 172.21.15.16:0/3863821271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40101420 0x7f6c4010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 --2- 172.21.15.16:0/3863821271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40100ee0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 --2- 172.21.15.16:0/3863821271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40106a80 0x7f6c40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 -- 172.21.15.16:0/3863821271 >> 172.21.15.16:0/3863821271 conn(0x7f6c400fc7a0 msgr2=0x7f6c400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 -- 172.21.15.16:0/3863821271 shutdown_connections 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.458+0000 7f6c4836e640 1 -- 172.21.15.16:0/3863821271 wait complete. 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 Processor -- start 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 -- start start 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 0x7f6c401989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 0x7f6c4019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40110e80 con 0x7f6c40100a80 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40110d00 con 0x7f6c40106a80 2024-09-12T01:05:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4836e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40111000 con 0x7f6c40101420 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4610a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4610a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54712/0 (socket says 172.21.15.16:54712) 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c4610a640 1 -- 172.21.15.16:0/238718207 learned_addr learned my addr 172.21.15.16:0/238718207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.459+0000 7f6c45909640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 0x7f6c401989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4690b640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 0x7f6c4019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 -- 172.21.15.16:0/238718207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 msgr2=0x7f6c401989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 0x7f6c401989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 -- 172.21.15.16:0/238718207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 msgr2=0x7f6c4019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 0x7f6c4019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 -- 172.21.15.16:0/238718207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4019d430 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c45909640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 0x7f6c401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4690b640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 0x7f6c4019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4610a640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00d770 tx=0x7f6c3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c017070 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4019d720 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c3c005500 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004e00 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.460+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401a5000 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.461+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c40100ee0 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.462+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c3c004fa0 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.463+0000 7f6c377fe640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 0x7f6c2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.463+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f6c3c0cef20 con 0x7f6c40100a80 2024-09-12T01:05:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.463+0000 7f6c45909640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 0x7f6c2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.464+0000 7f6c45909640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 0x7f6c2c0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c30000c00 tx=0x7f6c30005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.467+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c3c09bb40 con 0x7f6c40100a80 2024-09-12T01:05:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.680+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4010a000 con 0x7f6c40100a80 2024-09-12T01:05:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.680+0000 7f6c377fe640 1 -- 172.21.15.16:0/238718207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f6c3c09b440 con 0x7f6c40100a80 2024-09-12T01:05:01.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:05:01.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:05:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:05:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 msgr2=0x7f6c2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 0x7f6c2c0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c30000c00 tx=0x7f6c30005a20 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 msgr2=0x7f6c40198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00d770 tx=0x7f6c3c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 shutdown_connections 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6c2c0a2cf0 0x7f6c2c0a51b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c40106a80 0x7f6c4019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c40101420 0x7f6c401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 --2- 172.21.15.16:0/238718207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c40100a80 0x7f6c40198480 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 >> 172.21.15.16:0/238718207 conn(0x7f6c400fc7a0 msgr2=0x7f6c4006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 shutdown_connections 2024-09-12T01:05:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:01.683+0000 7f6c4836e640 1 -- 172.21.15.16:0/238718207 wait complete. 2024-09-12T01:05:02.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:01 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/238718207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:02.054 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:01 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/238718207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:02.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:01 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/238718207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:02.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:05:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:02 smithi099 ceph-mon[35654]: pgmap v276: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:03.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:02 smithi016 ceph-mon[30972]: pgmap v276: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:02 smithi159 ceph-mon[35566]: pgmap v276: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 -- 172.21.15.16:0/2298495501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102ef0 msgr2=0x7f26f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2298495501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102ef0 0x7f26f410da40 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ca0 tx=0x7f26ec02a950 comp rx=0 tx=0).stop 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 -- 172.21.15.16:0/2298495501 shutdown_connections 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2298495501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4102ef0 0x7f26f410da40 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2298495501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2298495501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4100e20 0x7f26f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 -- 172.21.15.16:0/2298495501 >> 172.21.15.16:0/2298495501 conn(0x7f26f40fc7d0 msgr2=0x7f26f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 -- 172.21.15.16:0/2298495501 shutdown_connections 2024-09-12T01:05:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.144+0000 7f26fbd88640 1 -- 172.21.15.16:0/2298495501 wait complete. 2024-09-12T01:05:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 Processor -- start 2024-09-12T01:05:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 -- start start 2024-09-12T01:05:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4102ef0 0x7f26f419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41120a0 con 0x7f26f4100e20 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4111f20 con 0x7f26f4102ef0 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fbd88640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4112220 con 0x7f26f4102570 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54728/0 (socket says 172.21.15.16:54728) 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 -- 172.21.15.16:0/2856418150 learned_addr learned my addr 172.21.15.16:0/2856418150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26fa325640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4102ef0 0x7f26f419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9323640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 -- 172.21.15.16:0/2856418150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 msgr2=0x7f26f41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 -- 172.21.15.16:0/2856418150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4102ef0 msgr2=0x7f26f419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4102ef0 0x7f26f419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9b24640 1 -- 172.21.15.16:0/2856418150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f419d430 con 0x7f26f4100e20 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.145+0000 7f26f9323640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e9f0 tx=0x7f26f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26eaffd640 1 -- 172.21.15.16:0/2856418150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f000cd80 con 0x7f26f4100e20 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26eaffd640 1 -- 172.21.15.16:0/2856418150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f0010910 con 0x7f26f4100e20 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f419d720 con 0x7f26f4100e20 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26eaffd640 1 -- 172.21.15.16:0/2856418150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0018990 con 0x7f26f4100e20 2024-09-12T01:05:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.146+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f41a5000 con 0x7f26f4100e20 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.148+0000 7f26eaffd640 1 -- 172.21.15.16:0/2856418150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26f0010ac0 con 0x7f26f4100e20 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.148+0000 7f26fe161740 1 Processor -- start 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 -- start start 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 0x555870d6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555870cc62f0 con 0x555870d61b40 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555870c6dcf0 con 0x555870d605d0 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fe161740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555870cc22b0 con 0x555870d6b370 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fa325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fa325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d60090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54736/0 (socket says 172.21.15.16:54736) 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26fa325640 1 -- 172.21.15.16:0/3184194397 learned_addr learned my addr 172.21.15.16:0/3184194397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26eaffd640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26f9b24640 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.149+0000 7f26f9323640 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 0x555870d6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26fa325640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26eaffd640 1 -- 172.21.15.16:0/2856418150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f26f00cf870 con 0x7f26f4100e20 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 -- 172.21.15.16:0/3184194397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 msgr2=0x555870d60a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d60a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 -- 172.21.15.16:0/3184194397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 msgr2=0x555870d60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 -- 172.21.15.16:0/3184194397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555870c523a0 con 0x555870d6b370 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26fa325640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 0x7f26e00a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f26ec005fe0 tx=0x7f26ec02a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26f9323640 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 0x555870d6d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26f40fec10 tx=0x7f26e400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734a4a8fd7a2fc21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26dffff640 1 -- 172.21.15.16:0/3184194397 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004030 con 0x555870d6b370 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26dffff640 1 -- 172.21.15.16:0/3184194397 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e4005500 con 0x555870d6b370 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.150+0000 7f26dffff640 1 -- 172.21.15.16:0/3184194397 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004bb0 con 0x555870d6b370 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 -- 172.21.15.16:0/3184194397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 msgr2=0x555870d6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 0x555870d6d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f26f40fec10 tx=0x7f26e400f730 comp rx=0 tx=0).stop 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 -- 172.21.15.16:0/3184194397 shutdown_connections 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d6b370 0x555870d6d760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 --2- 172.21.15.16:0/3184194397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 -- 172.21.15.16:0/3184194397 >> 172.21.15.16:0/3184194397 conn(0x555870d29310 msgr2=0x555870d5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 -- 172.21.15.16:0/3184194397 shutdown_connections 2024-09-12T01:05:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.151+0000 7f26fe161740 1 -- 172.21.15.16:0/3184194397 wait complete. 2024-09-12T01:05:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 -- start start 2024-09-12T01:05:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 Processor -- start 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d6acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d67f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555870d6e070 con 0x555870d6b370 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555870c81060 con 0x555870d605d0 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555870d6e1f0 con 0x555870d61b40 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fe161740 10 client.0 _ll_register_callbacks cb 0x5558709d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fa325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26f9323640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d67f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fa325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54748/0 (socket says 172.21.15.16:54748) 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26fa325640 1 -- 172.21.15.16:0/2072635265 learned_addr learned my addr 172.21.15.16:0/2072635265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.152+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d6acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 -- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 msgr2=0x555870d6acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d6acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 -- 172.21.15.16:0/2072635265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 msgr2=0x555870d684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 -- 172.21.15.16:0/2072635265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555870a373c0 con 0x555870d61b40 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26fa325640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26f9323640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d67f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7438704af9c89623 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e402b060 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e4005500 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f26e401b660 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.153+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26e40046c0 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 10 client.27385 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555870d79a60 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e4005500 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26e40119d0 con 0x555870d61b40 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 10 client.27385 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 10 client.27385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 20 client.27385 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870d7aec0 0x555870d7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 4 client.27385 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555870cd3860 con 0x555870d7aec0 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.154+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870d7aec0 0x555870d7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.155+0000 7f26dd7fa640 10 client.27385 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.155+0000 7f26f9b24640 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870d7aec0 0x555870d7d2b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b83e5edfac96d6ee server_cookie=2f6a8efbbbf8bb22 in_seq=0 out_seq=0 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.155+0000 7f26dd7fa640 1 -- 172.21.15.16:0/2072635265 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555870cd3860 con 0x555870d7aec0 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.155+0000 7f26dd7fa640 10 client.27385 handle_command_reply: tid=0 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.155+0000 7f26fe161740 1 client.27385 shutdown 2024-09-12T01:05:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 msgr2=0x555870d67f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d67f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e40044a0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 shutdown_connections 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870d7aec0 0x555870d7d2b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d6b370 0x555870d684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26dd7fa640 0 client.27385 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d67f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 --2- 172.21.15.16:0/2072635265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d605d0 0x555870d6acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 >> 172.21.15.16:0/2072635265 conn(0x555870d29310 msgr2=0x555870d62930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 shutdown_connections 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 1 -- 172.21.15.16:0/2072635265 wait complete. 2024-09-12T01:05:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.156+0000 7f26fe161740 20 client.27385 trim_cache size 0 max 16384 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 Processor -- start 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 -- start start 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870d82df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ad860 con 0x555870d605d0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41af3a0 con 0x555870d6b370 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26fe161740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41adf30 con 0x555870d61b40 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54754/0 (socket says 172.21.15.16:54754) 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 -- 172.21.15.16:0/4013718336 learned_addr learned my addr 172.21.15.16:0/4013718336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9323640 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 -- 172.21.15.16:0/4013718336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 msgr2=0x555870d828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 -- 172.21.15.16:0/4013718336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 msgr2=0x555870d82df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870d82df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.169+0000 7f26f9b24640 1 -- 172.21.15.16:0/4013718336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555870d6e1e0 con 0x555870d605d0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c6a0 tx=0x7f26f0097fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56842ce5252f8889 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26df7fe640 1 -- 172.21.15.16:0/4013718336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f00a49a0 con 0x555870d605d0 2024-09-12T01:05:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26df7fe640 1 -- 172.21.15.16:0/4013718336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f009a030 con 0x555870d605d0 2024-09-12T01:05:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26df7fe640 1 -- 172.21.15.16:0/4013718336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0018d90 con 0x555870d605d0 2024-09-12T01:05:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 -- 172.21.15.16:0/4013718336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 msgr2=0x555870d77310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c6a0 tx=0x7f26f0097fd0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 -- 172.21.15.16:0/4013718336 shutdown_connections 2024-09-12T01:05:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870d82df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d61b40 0x555870d828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 --2- 172.21.15.16:0/4013718336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d605d0 0x555870d77310 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.170+0000 7f26fe161740 1 -- 172.21.15.16:0/4013718336 >> 172.21.15.16:0/4013718336 conn(0x555870d29310 msgr2=0x555870d66830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- 172.21.15.16:0/4013718336 shutdown_connections 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- 172.21.15.16:0/4013718336 wait complete. 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- start start 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 Processor -- start 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d605d0 0x555870d73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870dde320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41aecd0 con 0x555870d61b40 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ae600 con 0x555870d6b370 2024-09-12T01:05:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26f41ab860 con 0x555870d605d0 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26fe161740 10 client.0 _ll_register_callbacks cb 0x7f26e4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.171+0000 7f26f9323640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54770/0 (socket says 172.21.15.16:54770) 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 -- 172.21.15.16:0/4069305693 learned_addr learned my addr 172.21.15.16:0/4069305693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d605d0 0x555870d73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26fa325640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870dde320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 -- 172.21.15.16:0/4069305693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d605d0 msgr2=0x555870d73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d605d0 0x555870d73730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 -- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 msgr2=0x555870dde320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870dde320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 -- 172.21.15.16:0/4069305693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555870d5d960 con 0x555870d61b40 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26fa325640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870dde320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26f9323640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x555870d677f0 tx=0x7f26e400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992b87e08d1f910d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e402b040 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f009a030 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f26e40269c0 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f26e403d050 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26fe161740 10 client.18279 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.172+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555870d700b0 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f009a030 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f26e401b660 con 0x555870d61b40 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 10 client.18279 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 10 client.18279 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 20 client.18279 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870de9e90 0x555870dec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 4 client.18279 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555870ccdd70 con 0x555870de9e90 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.173+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870de9e90 0x555870dec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26dd7fa640 10 client.18279 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26f9b24640 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870de9e90 0x555870dec280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49db565a96989804 server_cookie=93b84fa184090c40 in_seq=0 out_seq=0 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26dd7fa640 1 -- 172.21.15.16:0/4069305693 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555870ccdd70 con 0x555870de9e90 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26dd7fa640 10 client.18279 handle_command_reply: tid=0 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 client.18279 shutdown 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 msgr2=0x555870d73c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x555870d677f0 tx=0x7f26e400fcb0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 shutdown_connections 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x555870de9e90 0x555870dec280 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555870d6b370 0x555870dde320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555870d61b40 0x555870d73c70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 --2- 172.21.15.16:0/4069305693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555870d605d0 0x555870d73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26dd7fa640 0 client.18279 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 >> 172.21.15.16:0/4069305693 conn(0x555870d29310 msgr2=0x555870d5bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 shutdown_connections 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 1 -- 172.21.15.16:0/4069305693 wait complete. 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.174+0000 7f26fe161740 20 client.18279 trim_cache size 0 max 16384 2024-09-12T01:05:03.178 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 msgr2=0x7f26e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 0x7f26e00a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f26ec005fe0 tx=0x7f26ec02a950 comp rx=0 tx=0).stop 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 msgr2=0x7f26f4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e9f0 tx=0x7f26f000eeb0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 shutdown_connections 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26e00a2cf0 0x7f26e00a51b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26f4102ef0 0x7f26f419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f4102570 0x7f26f41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 --2- 172.21.15.16:0/2856418150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f4100e20 0x7f26f4198480 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 >> 172.21.15.16:0/2856418150 conn(0x7f26f40fc7d0 msgr2=0x7f26f4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 shutdown_connections 2024-09-12T01:05:03.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:03.175+0000 7f26fbd88640 1 -- 172.21.15.16:0/2856418150 wait complete. 2024-09-12T01:05:03.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:05:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 -- 172.21.15.16:0/1181464100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50101de0 msgr2=0x7f2e501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 --2- 172.21.15.16:0/1181464100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50101de0 0x7f2e501021c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009ca0 tx=0x7f2e4802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 -- 172.21.15.16:0/1181464100 shutdown_connections 2024-09-12T01:05:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 --2- 172.21.15.16:0/1181464100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50106f10 0x7f2e5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 --2- 172.21.15.16:0/1181464100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 --2- 172.21.15.16:0/1181464100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50101de0 0x7f2e501021c0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 -- 172.21.15.16:0/1181464100 >> 172.21.15.16:0/1181464100 conn(0x7f2e500fd8c0 msgr2=0x7f2e500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 -- 172.21.15.16:0/1181464100 shutdown_connections 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.677+0000 7f2e55243640 1 -- 172.21.15.16:0/1181464100 wait complete. 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.678+0000 7f2e55243640 1 Processor -- start 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.678+0000 7f2e55243640 1 -- start start 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.678+0000 7f2e55243640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 0x7f2e500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e55243640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e55243640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e55243640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50111f70 con 0x7f2e50106f10 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e55243640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50111df0 con 0x7f2e50102790 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e55243640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501120f0 con 0x7f2e50101de0 2024-09-12T01:05:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 0x7f2e500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54786/0 (socket says 172.21.15.16:54786) 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 0x7f2e500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:60510/0 (socket says 172.21.15.16:60510) 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 -- 172.21.15.16:0/2977081782 learned_addr learned my addr 172.21.15.16:0/2977081782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 -- 172.21.15.16:0/2977081782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 msgr2=0x7f2e500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 0x7f2e500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 -- 172.21.15.16:0/2977081782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 msgr2=0x7f2e50074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e50074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 -- 172.21.15.16:0/2977081782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e501a68f0 con 0x7f2e50106f10 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e4f59e640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000d750 tx=0x7f2e4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.679+0000 7f2e47fff640 1 -- 172.21.15.16:0/2977081782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40017070 con 0x7f2e50106f10 2024-09-12T01:05:04.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.680+0000 7f2e47fff640 1 -- 172.21.15.16:0/2977081782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e40005500 con 0x7f2e50106f10 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.680+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e501a6be0 con 0x7f2e50106f10 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.680+0000 7f2e47fff640 1 -- 172.21.15.16:0/2977081782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40004d90 con 0x7f2e50106f10 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.680+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e501a7120 con 0x7f2e50106f10 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.682+0000 7f2e47fff640 1 -- 172.21.15.16:0/2977081782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2e40004f30 con 0x7f2e50106f10 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.682+0000 7f2e5761c740 1 Processor -- start 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.682+0000 7f2e5761c740 1 -- start start 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528bbb40 0x55a2528ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528ba5d0 0x55a2528baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a2528202f0 con 0x55a2528c5370 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a2527c7cf0 con 0x55a2528bbb40 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e5761c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a25281c2b0 con 0x55a2528ba5d0 2024-09-12T01:05:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54788/0 (socket says 172.21.15.16:54788) 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 -- 172.21.15.16:0/2237774903 learned_addr learned my addr 172.21.15.16:0/2237774903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e47fff640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 0x7f2e3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528bbb40 0x55a2528ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528ba5d0 0x55a2528baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e47fff640 1 -- 172.21.15.16:0/2977081782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2e400ced00 con 0x7f2e50106f10 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4f59e640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 0x7f2e3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 -- 172.21.15.16:0/2237774903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528ba5d0 msgr2=0x55a2528baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528ba5d0 0x55a2528baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 -- 172.21.15.16:0/2237774903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528bbb40 msgr2=0x55a2528ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528bbb40 0x55a2528ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.683+0000 7f2e4e59c640 1 -- 172.21.15.16:0/2237774903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2527ac3a0 con 0x55a2528c5370 2024-09-12T01:05:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.684+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2e38012b70 tx=0x7f2e3800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb15bac9b2e0297c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.684+0000 7f2e457fa640 1 -- 172.21.15.16:0/2237774903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38012c60 con 0x55a2528c5370 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.684+0000 7f2e4f59e640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 0x7f2e3c0a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009fd0 tx=0x7f2e400bb040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.684+0000 7f2e457fa640 1 -- 172.21.15.16:0/2237774903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38012e00 con 0x55a2528c5370 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.684+0000 7f2e457fa640 1 -- 172.21.15.16:0/2237774903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004bc0 con 0x55a2528c5370 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- 172.21.15.16:0/2237774903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 msgr2=0x55a2528c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2e38012b70 tx=0x7f2e3800bb70 comp rx=0 tx=0).stop 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- 172.21.15.16:0/2237774903 shutdown_connections 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528c5370 0x55a2528c7760 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528ba5d0 0x55a2528baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2237774903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528bbb40 0x55a2528ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- 172.21.15.16:0/2237774903 >> 172.21.15.16:0/2237774903 conn(0x55a252883310 msgr2=0x55a2528b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- 172.21.15.16:0/2237774903 shutdown_connections 2024-09-12T01:05:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- 172.21.15.16:0/2237774903 wait complete. 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- start start 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 Processor -- start 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a2528c8070 con 0x55a2528bbb40 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a2527db060 con 0x55a2528ba5d0 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2528c81f0 con 0x55a2528c5370 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.685+0000 7f2e5761c740 10 client.0 _ll_register_callbacks cb 0x55a25252e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54792/0 (socket says 172.21.15.16:54792) 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/2678329569 learned_addr learned my addr 172.21.15.16:0/2678329569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 msgr2=0x55a2528c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/2678329569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 msgr2=0x55a2528c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/2678329569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2525913c0 con 0x55a2528bbb40 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009ca0 tx=0x7f2e4802f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468a38c51f0286e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e367fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4802f610 con 0x55a2528bbb40 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2e4802f7b0 con 0x55a2528bbb40 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.686+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e38012e00 con 0x55a2528bbb40 2024-09-12T01:05:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e48053050 con 0x55a2528bbb40 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 10 client.18291 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a2528d3ae0 con 0x55a2528bbb40 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38012e00 con 0x55a2528bbb40 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4804e030 con 0x55a2528bbb40 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 10 client.18291 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 10 client.18291 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 20 client.18291 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2528d5040 0x55a2528d7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 4 client.18291 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25282d860 con 0x55a2528d5040 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.687+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2528d5040 0x55a2528d7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.688+0000 7f2e367fc640 10 client.18291 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.688+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2528d5040 0x55a2528d7460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbd8228fe62d8217 server_cookie=bc78df00551251c in_seq=0 out_seq=0 2024-09-12T01:05:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.688+0000 7f2e367fc640 1 -- 172.21.15.16:0/2678329569 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a25282d860 con 0x55a2528d5040 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.688+0000 7f2e367fc640 10 client.18291 handle_command_reply: tid=0 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 client.18291 shutdown 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 msgr2=0x55a2528c1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009ca0 tx=0x7f2e4802f4e0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 shutdown_connections 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2528d5040 0x55a2528d7460 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528c1ff0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 --2- 172.21.15.16:0/2678329569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e367fc640 0 client.18291 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 >> 172.21.15.16:0/2678329569 conn(0x55a252883310 msgr2=0x55a2528bc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 shutdown_connections 2024-09-12T01:05:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 1 -- 172.21.15.16:0/2678329569 wait complete. 2024-09-12T01:05:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.689+0000 7f2e5761c740 20 client.18291 trim_cache size 0 max 16384 2024-09-12T01:05:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 Processor -- start 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 -- start start 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528bbb40 0x55a2528c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528c5370 0x55a2528bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501afaa0 con 0x55a2528ba5d0 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501b15e0 con 0x55a2528c5370 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e5761c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501b0170 con 0x55a2528bbb40 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.702+0000 7f2e4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54802/0 (socket says 172.21.15.16:54802) 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 -- 172.21.15.16:0/1625976173 learned_addr learned my addr 172.21.15.16:0/1625976173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528bbb40 0x55a2528c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 -- 172.21.15.16:0/1625976173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528bbb40 msgr2=0x55a2528c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528bbb40 0x55a2528c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 -- 172.21.15.16:0/1625976173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528c5370 msgr2=0x55a2528bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528c5370 0x55a2528bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 -- 172.21.15.16:0/1625976173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2528d7ae0 con 0x55a2528ba5d0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2e38015040 tx=0x7f2e3800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5ced64ea59aabe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e44ff9640 1 -- 172.21.15.16:0/1625976173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38031070 con 0x55a2528ba5d0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e44ff9640 1 -- 172.21.15.16:0/1625976173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38016030 con 0x55a2528ba5d0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.703+0000 7f2e44ff9640 1 -- 172.21.15.16:0/1625976173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004810 con 0x55a2528ba5d0 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- 172.21.15.16:0/1625976173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 msgr2=0x55a2528b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2e38015040 tx=0x7f2e3800bb70 comp rx=0 tx=0).stop 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- 172.21.15.16:0/1625976173 shutdown_connections 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528c5370 0x55a2528bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528bbb40 0x55a2528c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 --2- 172.21.15.16:0/1625976173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528ba5d0 0x55a2528b6d70 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- 172.21.15.16:0/1625976173 >> 172.21.15.16:0/1625976173 conn(0x55a252883310 msgr2=0x55a2528c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- 172.21.15.16:0/1625976173 shutdown_connections 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- 172.21.15.16:0/1625976173 wait complete. 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 -- start start 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.704+0000 7f2e5761c740 1 Processor -- start 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501b0f10 con 0x55a2528bbb40 2024-09-12T01:05:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501b0840 con 0x55a2528ba5d0 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501adaa0 con 0x55a2528c5370 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e5761c740 10 client.0 _ll_register_callbacks cb 0x7f2e4802f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54810/0 (socket says 172.21.15.16:54810) 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/3226889632 learned_addr learned my addr 172.21.15.16:0/3226889632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 msgr2=0x55a2528bf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528bf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/3226889632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 msgr2=0x55a2528cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 -- 172.21.15.16:0/3226889632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2528bfbc0 con 0x55a2528bbb40 2024-09-12T01:05:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528bf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e4ed9d640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f80 tx=0x7f2e480314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3871d4ee90de55a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.705+0000 7f2e367fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480307c0 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2e48030960 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e38016030 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e48064050 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 10 client.18297 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a25293fda0 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e38016030 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4805f030 con 0x55a2528bbb40 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 10 client.18297 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 10 client.18297 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 20 client.18297 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:04.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2529404e0 0x55a252942900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 4 client.18297 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a252827d70 con 0x55a2529404e0 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.706+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2529404e0 0x55a252942900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e4e59c640 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2529404e0 0x55a252942900 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dffe1cf4c8493412 server_cookie=f51c1648b98999a9 in_seq=0 out_seq=0 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e367fc640 10 client.18297 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e367fc640 1 -- 172.21.15.16:0/3226889632 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a252827d70 con 0x55a2529404e0 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e367fc640 10 client.18297 handle_command_reply: tid=0 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 client.18297 shutdown 2024-09-12T01:05:04.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 msgr2=0x55a2528bee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000f80 tx=0x7f2e480314c0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 shutdown_connections 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2528c5370 0x55a2528bf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a2529404e0 0x55a252942900 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e367fc640 0 client.18297 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a2528bbb40 0x55a2528bee10 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 --2- 172.21.15.16:0/3226889632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a2528ba5d0 0x55a2528cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 >> 172.21.15.16:0/3226889632 conn(0x55a252883310 msgr2=0x55a2528c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 shutdown_connections 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 1 -- 172.21.15.16:0/3226889632 wait complete. 2024-09-12T01:05:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.707+0000 7f2e5761c740 20 client.18297 trim_cache size 0 max 16384 2024-09-12T01:05:04.714 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 msgr2=0x7f2e3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 0x7f2e3c0a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009fd0 tx=0x7f2e400bb040 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 msgr2=0x7f2e50074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000d750 tx=0x7f2e4000dc10 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 shutdown_connections 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2e3c0a2cf0 0x7f2e3c0a51b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e50106f10 0x7f2e50074f70 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e50102790 0x7f2e50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 --2- 172.21.15.16:0/2977081782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e50101de0 0x7f2e500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 >> 172.21.15.16:0/2977081782 conn(0x7f2e500fd8c0 msgr2=0x7f2e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 shutdown_connections 2024-09-12T01:05:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:04.708+0000 7f2e55243640 1 -- 172.21.15.16:0/2977081782 wait complete. 2024-09-12T01:05:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:04 smithi099 ceph-mon[35654]: pgmap v277: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:05.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:04 smithi016 ceph-mon[30972]: pgmap v277: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:04 smithi159 ceph-mon[35566]: pgmap v277: 97 pgs: 97 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-09-12T01:05:05.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:05:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:06.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 -- 172.21.15.16:0/4200163038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb010c170 msgr2=0x7fedb010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 --2- 172.21.15.16:0/4200163038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb010c170 0x7fedb010e610 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7feda8009ca0 tx=0x7feda802a950 comp rx=0 tx=0).stop 2024-09-12T01:05:06.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 -- 172.21.15.16:0/4200163038 shutdown_connections 2024-09-12T01:05:06.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 --2- 172.21.15.16:0/4200163038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb010c170 0x7fedb010e610 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 --2- 172.21.15.16:0/4200163038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0103660 0x7fedb010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 --2- 172.21.15.16:0/4200163038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb0102d40 0x7fedb0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 -- 172.21.15.16:0/4200163038 >> 172.21.15.16:0/4200163038 conn(0x7fedb00fc7a0 msgr2=0x7fedb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 -- 172.21.15.16:0/4200163038 shutdown_connections 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.512+0000 7fedb5504640 1 -- 172.21.15.16:0/4200163038 wait complete. 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.513+0000 7fedb5504640 1 Processor -- start 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.513+0000 7fedb5504640 1 -- start start 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0102d40 0x7fedb0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 0x7fedb019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0111f70 con 0x7fedb0103660 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0111df0 con 0x7fedb010c170 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01120f0 con 0x7fedb0102d40 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54838/0 (socket says 172.21.15.16:54838) 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 -- 172.21.15.16:0/570328182 learned_addr learned my addr 172.21.15.16:0/570328182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 0x7fedb019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedaeffd640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0102d40 0x7fedb0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 -- 172.21.15.16:0/570328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0102d40 msgr2=0x7fedb0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0102d40 0x7fedb0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 -- 172.21.15.16:0/570328182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 msgr2=0x7fedb019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 0x7fedb019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 -- 172.21.15.16:0/570328182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb019d2e0 con 0x7fedb0103660 2024-09-12T01:05:06.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 0x7fedb019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedae7fc640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7feda400c970 tx=0x7feda400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fed97fff640 1 -- 172.21.15.16:0/570328182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4007bb0 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fed97fff640 1 -- 172.21.15.16:0/570328182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda4007d50 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb019d5d0 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fed97fff640 1 -- 172.21.15.16:0/570328182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4018820 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.514+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb01a4eb0 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.517+0000 7fedb78dd740 1 Processor -- start 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 -- start start 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fed97fff640 1 -- 172.21.15.16:0/570328182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feda40189c0 con 0x7fedb0103660 2024-09-12T01:05:06.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557265ffda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557266008370 0x55726600a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557265f632f0 con 0x557265ffeb40 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557265f0acf0 con 0x557265ffd5d0 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedb78dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557265f5f2b0 con 0x557266008370 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54850/0 (socket says 172.21.15.16:54850) 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 -- 172.21.15.16:0/1243025237 learned_addr learned my addr 172.21.15.16:0/1243025237 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaeffd640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557265ffda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedae7fc640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557266008370 0x55726600a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 -- 172.21.15.16:0/1243025237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557266008370 msgr2=0x55726600a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557266008370 0x55726600a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 -- 172.21.15.16:0/1243025237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 msgr2=0x557265ffda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557265ffda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaf7fe640 1 -- 172.21.15.16:0/1243025237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557265eef3a0 con 0x557265ffeb40 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.518+0000 7fedaeffd640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557265ffda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fed97fff640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 0x7fed980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7feda8009a50 tx=0x7feda802f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d69c1d87ec52816 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fedaeffd640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 0x7fed980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fed97fff640 1 -- 172.21.15.16:0/570328182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7feda40cf510 con 0x7fedb0103660 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fed957fa640 1 -- 172.21.15.16:0/1243025237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8034040 con 0x557265ffeb40 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fed957fa640 1 -- 172.21.15.16:0/1243025237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8003970 con 0x557265ffeb40 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.519+0000 7fed957fa640 1 -- 172.21.15.16:0/1243025237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda80046f0 con 0x557265ffeb40 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- 172.21.15.16:0/1243025237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 msgr2=0x557265ffd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedaeffd640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 0x7fed980a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557265ffaac0 tx=0x7feda000b710 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7feda8009a50 tx=0x7feda802f940 comp rx=0 tx=0).stop 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- 172.21.15.16:0/1243025237 shutdown_connections 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557266008370 0x55726600a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557265ffda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- 172.21.15.16:0/1243025237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffeb40 0x557265ffd090 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- 172.21.15.16:0/1243025237 >> 172.21.15.16:0/1243025237 conn(0x557265fc6310 msgr2=0x557265ff9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- 172.21.15.16:0/1243025237 shutdown_connections 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- 172.21.15.16:0/1243025237 wait complete. 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- start start 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 Processor -- start 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x5572660057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55726600b010 con 0x557265ffd5d0 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557265f1e060 con 0x557266008370 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55726600b190 con 0x557265ffeb40 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedb78dd740 10 client.0 _ll_register_callbacks cb 0x557265c718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.520+0000 7fedaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54860/0 (socket says 172.21.15.16:54860) 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 -- 172.21.15.16:0/4175432385 learned_addr learned my addr 172.21.15.16:0/4175432385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedae7fc640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 -- 172.21.15.16:0/4175432385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 msgr2=0x557266005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266005050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 -- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 msgr2=0x5572660057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x5572660057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x5572660057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 -- 172.21.15.16:0/4175432385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557265cd43c0 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedaeffd640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feda0012700 tx=0x7feda00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7835e6fa54c8b776 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fed92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000f9a0 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7feda000fb40 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda8003970 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feda000fd40 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedb78dd740 10 client.18309 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.521+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557266016b50 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8003970 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feda001f680 con 0x557265ffd5d0 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 10 client.18309 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:06.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 10 client.18309 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 20 client.18309 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266017f60 0x55726601a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 4 client.18309 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557265f70860 con 0x557266017f60 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedae7fc640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266017f60 0x55726601a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fedae7fc640 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266017f60 0x55726601a350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=245209b87ab0e163 server_cookie=c655222265169fc1 in_seq=0 out_seq=0 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.522+0000 7fed92ff5640 10 client.18309 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fed92ff5640 1 -- 172.21.15.16:0/4175432385 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557265f70860 con 0x557266017f60 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fed92ff5640 10 client.18309 handle_command_reply: tid=0 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 client.18309 shutdown 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 msgr2=0x557266007da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feda0012700 tx=0x7feda00046c0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 shutdown_connections 2024-09-12T01:05:06.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266017f60 0x55726601a350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x5572660057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266005050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fed92ff5640 0 client.18309 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 --2- 172.21.15.16:0/4175432385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x557266007da0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.523+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 >> 172.21.15.16:0/4175432385 conn(0x557265fc6310 msgr2=0x557265fff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.524+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 shutdown_connections 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.524+0000 7fedb78dd740 1 -- 172.21.15.16:0/4175432385 wait complete. 2024-09-12T01:05:06.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.524+0000 7fedb78dd740 20 client.18309 trim_cache size 0 max 16384 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.536+0000 7fedb78dd740 1 Processor -- start 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.536+0000 7fedb78dd740 1 -- start start 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557266010800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266010d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ad850 con 0x557266008370 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01af390 con 0x557265ffd5d0 2024-09-12T01:05:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedb78dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01adf20 con 0x557265ffeb40 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54864/0 (socket says 172.21.15.16:54864) 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 -- 172.21.15.16:0/558007485 learned_addr learned my addr 172.21.15.16:0/558007485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaeffd640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557266010800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedae7fc640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266010d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 -- 172.21.15.16:0/558007485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 msgr2=0x557266010d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266010d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 -- 172.21.15.16:0/558007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 msgr2=0x557266010800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557266010800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 -- 172.21.15.16:0/558007485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557266014790 con 0x557266008370 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaeffd640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557266010800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feda802ae70 tx=0x7feda802afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac118c36b80937a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.537+0000 7fed94ff9640 1 -- 172.21.15.16:0/558007485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda804b070 con 0x557266008370 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fed94ff9640 1 -- 172.21.15.16:0/558007485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8003970 con 0x557266008370 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fed94ff9640 1 -- 172.21.15.16:0/558007485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8034040 con 0x557266008370 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- 172.21.15.16:0/558007485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 msgr2=0x55726601fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feda802ae70 tx=0x7feda802afd0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- 172.21.15.16:0/558007485 shutdown_connections 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557266008370 0x55726601fbd0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x557266010d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 --2- 172.21.15.16:0/558007485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557265ffd5d0 0x557266010800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- 172.21.15.16:0/558007485 >> 172.21.15.16:0/558007485 conn(0x557265fc6310 msgr2=0x5572660002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- 172.21.15.16:0/558007485 shutdown_connections 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- 172.21.15.16:0/558007485 wait complete. 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.538+0000 7fedb78dd740 1 -- start start 2024-09-12T01:05:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 Processor -- start 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x55726607b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x55726607ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01aecc0 con 0x557265ffd5d0 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ae5f0 con 0x557266008370 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ab850 con 0x557265ffeb40 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedb78dd740 10 client.0 _ll_register_callbacks cb 0x7feda0005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54878/0 (socket says 172.21.15.16:54878) 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 -- 172.21.15.16:0/2531897464 learned_addr learned my addr 172.21.15.16:0/2531897464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedae7fc640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x55726607b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 -- 172.21.15.16:0/2531897464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 msgr2=0x55726607b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x55726607b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaf7fe640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x55726607ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 -- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 msgr2=0x55726607ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x55726607ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.539+0000 7fedaeffd640 1 -- 172.21.15.16:0/2531897464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726607f2b0 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedaeffd640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7feda0005840 tx=0x7feda001f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a866509708d6e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda003f060 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7feda0004230 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda8003970 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feda0036040 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 10 client.18315 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557266087840 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feda8003970 con 0x557265ffd5d0 2024-09-12T01:05:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feda0039030 con 0x557265ffd5d0 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 10 client.18315 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 10 client.18315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 20 client.18315 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266087f90 0x55726608a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 4 client.18315 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.540+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557265f6ad70 con 0x557266087f90 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedae7fc640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266087f90 0x55726608a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fed92ff5640 10 client.18315 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedae7fc640 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266087f90 0x55726608a3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71f05ddeef89521 server_cookie=70d99a3378d0a963 in_seq=0 out_seq=0 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fed92ff5640 1 -- 172.21.15.16:0/2531897464 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557265f6ad70 con 0x557266087f90 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fed92ff5640 10 client.18315 handle_command_reply: tid=0 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 client.18315 shutdown 2024-09-12T01:05:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 msgr2=0x55726607ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7feda0005840 tx=0x7feda001f650 comp rx=0 tx=0).stop 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 shutdown_connections 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x557266087f90 0x55726608a3b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557266008370 0x55726607ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557265ffeb40 0x55726607b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 --2- 172.21.15.16:0/2531897464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557265ffd5d0 0x55726607ad40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.541+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 >> 172.21.15.16:0/2531897464 conn(0x557265fc6310 msgr2=0x557265fff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fed92ff5640 0 client.18315 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 shutdown_connections 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb78dd740 1 -- 172.21.15.16:0/2531897464 wait complete. 2024-09-12T01:05:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb78dd740 20 client.18315 trim_cache size 0 max 16384 2024-09-12T01:05:06.550 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 msgr2=0x7fed980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 0x7fed980a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x557265ffaac0 tx=0x7feda000b710 comp rx=0 tx=0).stop 2024-09-12T01:05:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 msgr2=0x7fedb0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7feda400c970 tx=0x7feda400ce30 comp rx=0 tx=0).stop 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 shutdown_connections 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fed980a2cf0 0x7fed980a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedb010c170 0x7fedb019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedb0103660 0x7fedb0198870 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 --2- 172.21.15.16:0/570328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fedb0102d40 0x7fedb0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 >> 172.21.15.16:0/570328182 conn(0x7fedb00fc7a0 msgr2=0x7fedb00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.542+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 shutdown_connections 2024-09-12T01:05:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:06.543+0000 7fedb5504640 1 -- 172.21.15.16:0/570328182 wait complete. 2024-09-12T01:05:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:06 smithi099 ceph-mon[35654]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:05:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:06 smithi159 ceph-mon[35566]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:05:07.127 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:06 smithi016 ceph-mon[30972]: pgmap v278: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 80 KiB/s wr, 34 op/s 2024-09-12T01:05:07.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 -- 172.21.15.16:0/799850171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c102ef0 msgr2=0x7fc51c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 --2- 172.21.15.16:0/799850171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c102ef0 0x7fc51c10da40 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc514009ca0 tx=0x7fc51402a950 comp rx=0 tx=0).stop 2024-09-12T01:05:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 -- 172.21.15.16:0/799850171 shutdown_connections 2024-09-12T01:05:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 --2- 172.21.15.16:0/799850171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c102ef0 0x7fc51c10da40 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 --2- 172.21.15.16:0/799850171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102570 0x7fc51c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 --2- 172.21.15.16:0/799850171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c100e20 0x7fc51c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 -- 172.21.15.16:0/799850171 >> 172.21.15.16:0/799850171 conn(0x7fc51c0fc7d0 msgr2=0x7fc51c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 -- 172.21.15.16:0/799850171 shutdown_connections 2024-09-12T01:05:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.021+0000 7fc522823640 1 -- 172.21.15.16:0/799850171 wait complete. 2024-09-12T01:05:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 Processor -- start 2024-09-12T01:05:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 -- start start 2024-09-12T01:05:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 0x7fc51c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 0x7fc51c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c110f80 con 0x7fc51c100e20 2024-09-12T01:05:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c110e00 con 0x7fc51c102570 2024-09-12T01:05:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc522823640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c111100 con 0x7fc51c102ef0 2024-09-12T01:05:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54900/0 (socket says 172.21.15.16:54900) 2024-09-12T01:05:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 -- 172.21.15.16:0/3934464048 learned_addr learned my addr 172.21.15.16:0/3934464048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 0x7fc51c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc513fff640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 0x7fc51c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 -- 172.21.15.16:0/3934464048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 msgr2=0x7fc51c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 0x7fc51c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 -- 172.21.15.16:0/3934464048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 msgr2=0x7fc51c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 0x7fc51c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc51bfff640 1 -- 172.21.15.16:0/3934464048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c19c2d0 con 0x7fc51c100e20 2024-09-12T01:05:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc513fff640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 0x7fc51c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:08.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.022+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 0x7fc51c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c920 tx=0x7fc50c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc519ffb640 1 -- 172.21.15.16:0/3934464048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c007c70 con 0x7fc51c100e20 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc519ffb640 1 -- 172.21.15.16:0/3934464048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50c007e10 con 0x7fc51c100e20 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc519ffb640 1 -- 172.21.15.16:0/3934464048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c018890 con 0x7fc51c100e20 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c19c5c0 con 0x7fc51c100e20 2024-09-12T01:05:08.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.023+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1a3ea0 con 0x7fc51c100e20 2024-09-12T01:05:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.025+0000 7fc519ffb640 1 -- 172.21.15.16:0/3934464048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc50c018a70 con 0x7fc51c100e20 2024-09-12T01:05:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 Processor -- start 2024-09-12T01:05:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 -- start start 2024-09-12T01:05:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x55922703ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x559227048760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559226fa12f0 con 0x55922703cb40 2024-09-12T01:05:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559226f48cf0 con 0x55922703b5d0 2024-09-12T01:05:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc524bfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559226f9d2b0 con 0x559227046370 2024-09-12T01:05:08.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54916/0 (socket says 172.21.15.16:54916) 2024-09-12T01:05:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 -- 172.21.15.16:0/3733891098 learned_addr learned my addr 172.21.15.16:0/3733891098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x55922703ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc513fff640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x559227048760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 -- 172.21.15.16:0/3733891098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 msgr2=0x559227048760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x559227048760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc519ffb640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 0x7fc5080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 -- 172.21.15.16:0/3733891098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 msgr2=0x55922703ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x55922703ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc520dc0640 1 -- 172.21.15.16:0/3733891098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559226f2d3a0 con 0x55922703cb40 2024-09-12T01:05:08.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.026+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x55922703ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc519ffb640 1 -- 172.21.15.16:0/3934464048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc50c0cf6b0 con 0x7fc51c100e20 2024-09-12T01:05:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 0x7fc5080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc514009a20 tx=0x7fc51402f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97b37c58b2aeaa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc5127fc640 1 -- 172.21.15.16:0/3733891098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514034040 con 0x55922703cb40 2024-09-12T01:05:08.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc5127fc640 1 -- 172.21.15.16:0/3733891098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51402f6e0 con 0x55922703cb40 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.027+0000 7fc5127fc640 1 -- 172.21.15.16:0/3733891098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140307c0 con 0x55922703cb40 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- 172.21.15.16:0/3733891098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 msgr2=0x55922703b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 0x7fc5080a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc514049000 tx=0x7fc51404a000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc514009a20 tx=0x7fc51402f930 comp rx=0 tx=0).stop 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- 172.21.15.16:0/3733891098 shutdown_connections 2024-09-12T01:05:08.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x559227048760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x55922703ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3733891098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703cb40 0x55922703b090 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- 172.21.15.16:0/3733891098 >> 172.21.15.16:0/3733891098 conn(0x559227004310 msgr2=0x559227037c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- 172.21.15.16:0/3733891098 shutdown_connections 2024-09-12T01:05:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- 172.21.15.16:0/3733891098 wait complete. 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 -- start start 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.028+0000 7fc524bfc740 1 Processor -- start 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 0x559227042f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x5592270434f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55922704b280 con 0x55922703b5d0 2024-09-12T01:05:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559226f5c060 con 0x55922703cb40 2024-09-12T01:05:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55922704b400 con 0x559227046370 2024-09-12T01:05:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 10 client.0 _ll_register_callbacks cb 0x559226caf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54932/0 (socket says 172.21.15.16:54932) 2024-09-12T01:05:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 -- 172.21.15.16:0/3986200241 learned_addr learned my addr 172.21.15.16:0/3986200241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc513fff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 0x559227042f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x5592270434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 -- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 msgr2=0x5592270434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x5592270434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 -- 172.21.15.16:0/3986200241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 msgr2=0x559227042f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 0x559227042f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 -- 172.21.15.16:0/3986200241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559226d123c0 con 0x55922703b5d0 2024-09-12T01:05:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc513fff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 0x559227042f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x5592270434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c8f0 tx=0x7fc50c09b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ac1ef8199b8925 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc4fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c09bb70 con 0x55922703b5d0 2024-09-12T01:05:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc50c0cd020 con 0x55922703b5d0 2024-09-12T01:05:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.029+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51402f6e0 con 0x55922703b5d0 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc50c0b9050 con 0x55922703b5d0 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 10 client.18327 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559227056d20 con 0x55922703b5d0 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51402f6e0 con 0x55922703b5d0 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc50c0d1ce0 con 0x55922703b5d0 2024-09-12T01:05:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 10 client.18327 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 10 client.18327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:05:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 20 client.18327 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270581e0 0x55922705a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 4 client.18327 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559226fae860 con 0x5592270581e0 2024-09-12T01:05:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.030+0000 7fc513fff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270581e0 0x55922705a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.031+0000 7fc513fff640 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270581e0 0x55922705a610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f28c53b46b695a9b server_cookie=bc92311451627c0b in_seq=0 out_seq=0 2024-09-12T01:05:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.031+0000 7fc4fb7fe640 10 client.18327 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3986200241 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559226fae860 con 0x5592270581e0 2024-09-12T01:05:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc4fb7fe640 10 client.18327 handle_command_reply: tid=0 2024-09-12T01:05:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 client.18327 shutdown 2024-09-12T01:05:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 msgr2=0x559227045c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c8f0 tx=0x7fc50c09b590 comp rx=0 tx=0).stop 2024-09-12T01:05:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 shutdown_connections 2024-09-12T01:05:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559227046370 0x5592270434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270581e0 0x55922705a610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703cb40 0x559227042f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc4fb7fe640 0 client.18327 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3986200241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227045c90 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 >> 172.21.15.16:0/3986200241 conn(0x559227004310 msgr2=0x55922703d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 shutdown_connections 2024-09-12T01:05:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 1 -- 172.21.15.16:0/3986200241 wait complete. 2024-09-12T01:05:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.032+0000 7fc524bfc740 20 client.18327 trim_cache size 0 max 16384 2024-09-12T01:05:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 Processor -- start 2024-09-12T01:05:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 -- start start 2024-09-12T01:05:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x559227036bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1ab600 con 0x559227046370 2024-09-12T01:05:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1ad140 con 0x55922703b5d0 2024-09-12T01:05:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc524bfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1abcd0 con 0x55922703cb40 2024-09-12T01:05:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc520dc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc520dc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54946/0 (socket says 172.21.15.16:54946) 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.045+0000 7fc520dc0640 1 -- 172.21.15.16:0/2411764924 learned_addr learned my addr 172.21.15.16:0/2411764924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc513fff640 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc51bfff640 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x559227036bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 -- 172.21.15.16:0/2411764924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 msgr2=0x55922703e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 -- 172.21.15.16:0/2411764924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 msgr2=0x559227036bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x559227036bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 -- 172.21.15.16:0/2411764924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922703eba0 con 0x559227046370 2024-09-12T01:05:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc520dc0640 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc514009a20 tx=0x7fc514050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c884fc8f83b1ce27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc511ffb640 1 -- 172.21.15.16:0/2411764924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514058070 con 0x559227046370 2024-09-12T01:05:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc511ffb640 1 -- 172.21.15.16:0/2411764924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51402f6e0 con 0x559227046370 2024-09-12T01:05:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.046+0000 7fc511ffb640 1 -- 172.21.15.16:0/2411764924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514034040 con 0x559227046370 2024-09-12T01:05:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- 172.21.15.16:0/2411764924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 msgr2=0x55922703e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc514009a20 tx=0x7fc514050000 comp rx=0 tx=0).stop 2024-09-12T01:05:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- 172.21.15.16:0/2411764924 shutdown_connections 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559227046370 0x55922703e660 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 --2- 172.21.15.16:0/2411764924 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922703b5d0 0x559227036bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- 172.21.15.16:0/2411764924 >> 172.21.15.16:0/2411764924 conn(0x559227004310 msgr2=0x5592270364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- 172.21.15.16:0/2411764924 shutdown_connections 2024-09-12T01:05:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- 172.21.15.16:0/2411764924 wait complete. 2024-09-12T01:05:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 -- start start 2024-09-12T01:05:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.047+0000 7fc524bfc740 1 Processor -- start 2024-09-12T01:05:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559227046370 0x559227040320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1aca70 con 0x55922703b5d0 2024-09-12T01:05:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1ac3a0 con 0x559227046370 2024-09-12T01:05:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c1a9600 con 0x55922703cb40 2024-09-12T01:05:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 10 client.0 _ll_register_callbacks cb 0x7fc50c00cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:08.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54958/0 (socket says 172.21.15.16:54958) 2024-09-12T01:05:08.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 -- 172.21.15.16:0/3262839169 learned_addr learned my addr 172.21.15.16:0/3262839169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:08.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc520dc0640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559227046370 0x559227040320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc513fff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 -- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 msgr2=0x55922703fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 -- 172.21.15.16:0/3262839169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559227046370 msgr2=0x559227040320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559227046370 0x559227040320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 -- 172.21.15.16:0/3262839169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559227040b50 con 0x55922703b5d0 2024-09-12T01:05:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc51bfff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0a4750 tx=0x7fc50c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a90dab12de303c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc4fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c064070 con 0x55922703b5d0 2024-09-12T01:05:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc50c0ba440 con 0x55922703b5d0 2024-09-12T01:05:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.048+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51402f6e0 con 0x55922703b5d0 2024-09-12T01:05:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc50c09b940 con 0x55922703b5d0 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 10 client.18333 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592270c2f20 con 0x55922703b5d0 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51402f6e0 con 0x55922703b5d0 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc50c09aa30 con 0x55922703b5d0 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 10 client.18333 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 10 client.18333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 20 client.18333 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270c3690 0x5592270c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 4 client.18333 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559226fa8d70 con 0x5592270c3690 2024-09-12T01:05:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.049+0000 7fc513fff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270c3690 0x5592270c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc4fb7fe640 10 client.18333 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc513fff640 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270c3690 0x5592270c5ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=986a27e0e11a11ec server_cookie=67e7d03b289214e1 in_seq=0 out_seq=0 2024-09-12T01:05:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc4fb7fe640 1 -- 172.21.15.16:0/3262839169 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559226fa8d70 con 0x5592270c3690 2024-09-12T01:05:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc4fb7fe640 10 client.18333 handle_command_reply: tid=0 2024-09-12T01:05:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 client.18333 shutdown 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 msgr2=0x559227050e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0a4750 tx=0x7fc50c00fd60 comp rx=0 tx=0).stop 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 shutdown_connections 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5592270c3690 0x5592270c5ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559227046370 0x559227040320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922703cb40 0x55922703fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 --2- 172.21.15.16:0/3262839169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922703b5d0 0x559227050e50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 >> 172.21.15.16:0/3262839169 conn(0x559227004310 msgr2=0x559227036d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc4fb7fe640 0 client.18333 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 shutdown_connections 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 1 -- 172.21.15.16:0/3262839169 wait complete. 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.050+0000 7fc524bfc740 20 client.18333 trim_cache size 0 max 16384 2024-09-12T01:05:08.085 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 msgr2=0x7fc5080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 0x7fc5080a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc514049000 tx=0x7fc51404a000 comp rx=0 tx=0).stop 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 msgr2=0x7fc51c197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c920 tx=0x7fc50c00cde0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 shutdown_connections 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc5080a2cf0 0x7fc5080a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc51c102ef0 0x7fc51c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51c102570 0x7fc51c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 --2- 172.21.15.16:0/3934464048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc51c100e20 0x7fc51c197320 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 >> 172.21.15.16:0/3934464048 conn(0x7fc51c0fc7d0 msgr2=0x7fc51c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 shutdown_connections 2024-09-12T01:05:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:08.051+0000 7fc522823640 1 -- 172.21.15.16:0/3934464048 wait complete. 2024-09-12T01:05:08.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:05:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:08 smithi099 ceph-mon[35654]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-12T01:05:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:08 smithi159 ceph-mon[35566]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-12T01:05:09.140 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:09.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:08 smithi016 ceph-mon[30972]: pgmap v279: 97 pgs: 97 active+clean; 11 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 -- 172.21.15.16:0/2262445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8106f10 msgr2=0x7f32c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 --2- 172.21.15.16:0/2262445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8106f10 0x7f32c810ea90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009ca0 tx=0x7f32c002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 -- 172.21.15.16:0/2262445113 shutdown_connections 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 --2- 172.21.15.16:0/2262445113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8106f10 0x7f32c810ea90 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 --2- 172.21.15.16:0/2262445113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8102790 0x7f32c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 --2- 172.21.15.16:0/2262445113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8101de0 0x7f32c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 -- 172.21.15.16:0/2262445113 >> 172.21.15.16:0/2262445113 conn(0x7f32c80fd8c0 msgr2=0x7f32c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 -- 172.21.15.16:0/2262445113 shutdown_connections 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.655+0000 7f32ccee1640 1 -- 172.21.15.16:0/2262445113 wait complete. 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 Processor -- start 2024-09-12T01:05:09.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 -- start start 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 0x7f32c8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8106f10 0x7f32c819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111f70 con 0x7f32c8101de0 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c8111df0 con 0x7f32c8102790 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32ccee1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81120f0 con 0x7f32c8106f10 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54846/0 (socket says 172.21.15.16:54846) 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32c659c640 1 -- 172.21.15.16:0/1976334279 learned_addr learned my addr 172.21.15.16:0/1976334279 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.656+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8106f10 0x7f32c819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 0x7f32c8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 -- 172.21.15.16:0/1976334279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8106f10 msgr2=0x7f32c819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8106f10 0x7f32c819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 -- 172.21.15.16:0/1976334279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 msgr2=0x7f32c8198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 0x7f32c8198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 -- 172.21.15.16:0/1976334279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c819d2f0 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 0x7f32c8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32c659c640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d950 tx=0x7f32b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32b77fe640 1 -- 172.21.15.16:0/1976334279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8017070 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32b77fe640 1 -- 172.21.15.16:0/1976334279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b8009c30 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c819d5e0 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32b77fe640 1 -- 172.21.15.16:0/1976334279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800f6b0 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.657+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c81a4ec0 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32b77fe640 1 -- 172.21.15.16:0/1976334279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f32b80109d0 con 0x7f32c8101de0 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 Processor -- start 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 -- start start 2024-09-12T01:05:09.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e90af22f0 con 0x562e90b9b780 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e90a99cf0 con 0x562e90b8c5d0 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e90aee2b0 con 0x562e90b8db40 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54848/0 (socket says 172.21.15.16:54848) 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.660+0000 7f32c5d9b640 1 -- 172.21.15.16:0/761965343 learned_addr learned my addr 172.21.15.16:0/761965343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32b77fe640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 0x7f32b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c659c640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 -- 172.21.15.16:0/761965343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 msgr2=0x562e90b8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 0x7f32b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 -- 172.21.15.16:0/761965343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 msgr2=0x562e90b8ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b8ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 -- 172.21.15.16:0/761965343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e90a7e3a0 con 0x562e90b9b780 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c659c640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b8ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32b77fe640 1 -- 172.21.15.16:0/1976334279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f32b80cf0e0 con 0x7f32c8101de0 2024-09-12T01:05:09.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00bb60 tx=0x7f32bc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf414f1bb47ab58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32b4ff9640 1 -- 172.21.15.16:0/761965343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00be60 con 0x562e90b9b780 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 0x7f32b00a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32c0004af0 tx=0x7f32c002a8c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32b4ff9640 1 -- 172.21.15.16:0/761965343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc017b00 con 0x562e90b9b780 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.661+0000 7f32b4ff9640 1 -- 172.21.15.16:0/761965343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00fc90 con 0x562e90b9b780 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- 172.21.15.16:0/761965343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 msgr2=0x562e90b9db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00bb60 tx=0x7f32bc00d710 comp rx=0 tx=0).stop 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- 172.21.15.16:0/761965343 shutdown_connections 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9db70 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b8ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/761965343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- 172.21.15.16:0/761965343 >> 172.21.15.16:0/761965343 conn(0x562e90b55310 msgr2=0x562e90b88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- 172.21.15.16:0/761965343 shutdown_connections 2024-09-12T01:05:09.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- 172.21.15.16:0/761965343 wait complete. 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.662+0000 7f32cf2ba740 1 -- start start 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 Processor -- start 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e90b9e480 con 0x562e90b9b780 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e90aad060 con 0x562e90b8c5d0 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e90b9e600 con 0x562e90b8db40 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32cf2ba740 10 client.0 _ll_register_callbacks cb 0x562e908008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54866/0 (socket says 172.21.15.16:54866) 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 -- 172.21.15.16:0/2121184206 learned_addr learned my addr 172.21.15.16:0/2121184206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:09.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c659c640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 -- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 msgr2=0x562e90b983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 -- 172.21.15.16:0/2121184206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 msgr2=0x562e90b9b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b9b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.663+0000 7f32c6d9d640 1 -- 172.21.15.16:0/2121184206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e908633c0 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009ca0 tx=0x7f32c00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce78777b576c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c004f070 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f32c0004720 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc017b00 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32c0004b40 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 10 client.18345 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e90ba9ee0 con 0x562e90b9b780 2024-09-12T01:05:09.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc017b00 con 0x562e90b9b780 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32c004a040 con 0x562e90b9b780 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 10 client.18345 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 20 client.18345 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90bad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 4 client.18345 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.664+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e90aff860 con 0x562e90bab2f0 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.665+0000 7f32c659c640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90bad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.665+0000 7f32c659c640 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90bad790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819ac94dabc32963 server_cookie=292792e3259b4c21 in_seq=0 out_seq=0 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.665+0000 7f32adffb640 10 client.18345 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32adffb640 1 -- 172.21.15.16:0/2121184206 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e90aff860 con 0x562e90bab2f0 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32adffb640 10 client.18345 handle_command_reply: tid=0 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 client.18345 shutdown 2024-09-12T01:05:09.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 msgr2=0x562e90b989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009ca0 tx=0x7f32c00095c0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 shutdown_connections 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90bad790 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b989e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8db40 0x562e90b983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32adffb640 0 client.18345 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/2121184206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8c5d0 0x562e90b9b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 >> 172.21.15.16:0/2121184206 conn(0x562e90b55310 msgr2=0x562e90b8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 shutdown_connections 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 1 -- 172.21.15.16:0/2121184206 wait complete. 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.666+0000 7f32cf2ba740 20 client.18345 trim_cache size 0 max 16384 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 Processor -- start 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 -- start start 2024-09-12T01:05:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90ba7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 0x562e90bb35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ab620 con 0x562e90b8db40 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ad160 con 0x562e90b9b780 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81abcf0 con 0x562e90b8c5d0 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32c5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54880/0 (socket says 172.21.15.16:54880) 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.679+0000 7f32c5d9b640 1 -- 172.21.15.16:0/162276538 learned_addr learned my addr 172.21.15.16:0/162276538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 0x562e90bb35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c659c640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90ba7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 -- 172.21.15.16:0/162276538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 msgr2=0x562e90ba7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90ba7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 -- 172.21.15.16:0/162276538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 msgr2=0x562e90bb35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 0x562e90bb35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 -- 172.21.15.16:0/162276538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e90b89960 con 0x562e90b8db40 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 0x562e90bb35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00bb30 tx=0x7f32bc00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c035cb5d16a4cff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32affff640 1 -- 172.21.15.16:0/162276538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc028070 con 0x562e90b8db40 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32affff640 1 -- 172.21.15.16:0/162276538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc00fd40 con 0x562e90b8db40 2024-09-12T01:05:09.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.680+0000 7f32affff640 1 -- 172.21.15.16:0/162276538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc005640 con 0x562e90b8db40 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- 172.21.15.16:0/162276538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 msgr2=0x562e90bb3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00bb30 tx=0x7f32bc00b700 comp rx=0 tx=0).stop 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- 172.21.15.16:0/162276538 shutdown_connections 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b9b780 0x562e90bb35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b8db40 0x562e90bb3060 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/162276538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90ba7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- 172.21.15.16:0/162276538 >> 172.21.15.16:0/162276538 conn(0x562e90b55310 msgr2=0x562e90b96cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- 172.21.15.16:0/162276538 shutdown_connections 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- 172.21.15.16:0/162276538 wait complete. 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 -- start start 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 Processor -- start 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90b9f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8db40 0x562e90b9fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.681+0000 7f32cf2ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81aca90 con 0x562e90b9b780 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81ac3c0 con 0x562e90b8db40 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32cf2ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81a9620 con 0x562e90b8c5d0 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32cf2ba740 10 client.0 _ll_register_callbacks cb 0x7f32c00098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90b9f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90b9f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:36078/0 (socket says 172.21.15.16:36078) 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c659c640 1 -- 172.21.15.16:0/1102450647 learned_addr learned my addr 172.21.15.16:0/1102450647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c5d9b640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8db40 0x562e90b9fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 -- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 msgr2=0x562e90b9f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90b9f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 -- 172.21.15.16:0/1102450647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8db40 msgr2=0x562e90b9fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8db40 0x562e90b9fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 -- 172.21.15.16:0/1102450647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e90c15030 con 0x562e90b9b780 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32c6d9d640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9fff0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009bf0 tx=0x7f32c0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752c23b275086988 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32adffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0064060 con 0x562e90b9b780 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f32c002f610 con 0x562e90b9b780 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.682+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc00fd40 con 0x562e90b9b780 2024-09-12T01:05:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32c004f070 con 0x562e90b9b780 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 10 client.18351 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e90c16290 con 0x562e90b9b780 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc00fd40 con 0x562e90b9b780 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32c004a040 con 0x562e90b9b780 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 10 client.18351 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 20 client.18351 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90c18960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 4 client.18351 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e90af9d70 con 0x562e90bab2f0 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.683+0000 7f32c659c640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90c18960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32c659c640 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90c18960 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c187fa8486995af server_cookie=19516f957da41d56 in_seq=0 out_seq=0 2024-09-12T01:05:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32adffb640 10 client.18351 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32adffb640 1 -- 172.21.15.16:0/1102450647 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e90af9d70 con 0x562e90bab2f0 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32adffb640 10 client.18351 handle_command_reply: tid=0 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 client.18351 shutdown 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 msgr2=0x562e90b9fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9fff0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009bf0 tx=0x7f32c0005fa0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 shutdown_connections 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562e90bab2f0 0x562e90c18960 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e90b9b780 0x562e90b9fff0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e90b8db40 0x562e90b9fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 --2- 172.21.15.16:0/1102450647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e90b8c5d0 0x562e90b9f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 >> 172.21.15.16:0/1102450647 conn(0x562e90b55310 msgr2=0x562e90b874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32adffb640 0 client.18351 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 shutdown_connections 2024-09-12T01:05:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 1 -- 172.21.15.16:0/1102450647 wait complete. 2024-09-12T01:05:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.684+0000 7f32cf2ba740 20 client.18351 trim_cache size 0 max 16384 2024-09-12T01:05:09.698 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:09.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 msgr2=0x7f32b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 0x7f32b00a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32c0004af0 tx=0x7f32c002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 msgr2=0x7f32c8198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d950 tx=0x7f32b800de10 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 shutdown_connections 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32b00a2cf0 0x7f32b00a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32c8106f10 0x7f32c819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c8102790 0x7f32c8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 --2- 172.21.15.16:0/1976334279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c8101de0 0x7f32c8198340 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 >> 172.21.15.16:0/1976334279 conn(0x7f32c80fd8c0 msgr2=0x7f32c80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 shutdown_connections 2024-09-12T01:05:09.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:09.685+0000 7f32ccee1640 1 -- 172.21.15.16:0/1976334279 wait complete. 2024-09-12T01:05:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:10 smithi099 ceph-mon[35654]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 84 KiB/s wr, 34 op/s 2024-09-12T01:05:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:10 smithi159 ceph-mon[35566]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 84 KiB/s wr, 34 op/s 2024-09-12T01:05:11.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:10 smithi016 ceph-mon[30972]: pgmap v280: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 84 KiB/s wr, 34 op/s 2024-09-12T01:05:11.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:05:11.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:05:11.710 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3610332652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 msgr2=0x7ff0dc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3610332652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc106840 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009930 tx=0x7ff0d002a420 comp rx=0 tx=0).stop 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3610332652 shutdown_connections 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3610332652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc106f50 0x7ff0dc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3610332652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc106840 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3610332652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc1002f0 0x7ff0dc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3610332652 >> 172.21.15.16:0/3610332652 conn(0x7ff0dc077210 msgr2=0x7ff0dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3610332652 shutdown_connections 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3610332652 wait complete. 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.223+0000 7ff0e31b0640 1 Processor -- start 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 -- start start 2024-09-12T01:05:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc1002f0 0x7ff0dc101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc106f50 0x7ff0dc101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc111fa0 con 0x7ff0dc100ca0 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc111e20 con 0x7ff0dc106f50 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e31b0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc112120 con 0x7ff0dc1002f0 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54904/0 (socket says 172.21.15.16:54904) 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 -- 172.21.15.16:0/3079599424 learned_addr learned my addr 172.21.15.16:0/3079599424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e174d640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc106f50 0x7ff0dc101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc1002f0 0x7ff0dc101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 -- 172.21.15.16:0/3079599424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc1002f0 msgr2=0x7ff0dc101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc1002f0 0x7ff0dc101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 -- 172.21.15.16:0/3079599424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc106f50 msgr2=0x7ff0dc101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc106f50 0x7ff0dc101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.224+0000 7ff0dbfff640 1 -- 172.21.15.16:0/3079599424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc1a69e0 con 0x7ff0dc100ca0 2024-09-12T01:05:12.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0dbfff640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005940 tx=0x7ff0d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0d9ffb640 1 -- 172.21.15.16:0/3079599424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d003f030 con 0x7ff0dc100ca0 2024-09-12T01:05:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc1a6c70 con 0x7ff0dc100ca0 2024-09-12T01:05:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0d9ffb640 1 -- 172.21.15.16:0/3079599424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d0003ce0 con 0x7ff0dc100ca0 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0d9ffb640 1 -- 172.21.15.16:0/3079599424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0035070 con 0x7ff0dc100ca0 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.225+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a7150 con 0x7ff0dc100ca0 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.227+0000 7ff0d9ffb640 1 -- 172.21.15.16:0/3079599424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0d0042030 con 0x7ff0dc100ca0 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 Processor -- start 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 -- start start 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0d9ffb640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 0x7ff0c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbb5d40 0x55847cbb8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55847cb10280 con 0x55847cbaab60 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55847cab7b10 con 0x55847cbab480 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e5589740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847cb0c240 con 0x55847cbb5d40 2024-09-12T01:05:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54906/0 (socket says 172.21.15.16:54906) 2024-09-12T01:05:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 -- 172.21.15.16:0/2205059032 learned_addr learned my addr 172.21.15.16:0/2205059032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0dbfff640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 0x7ff0c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbb5d40 0x55847cbb8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0d9ffb640 1 -- 172.21.15.16:0/3079599424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7ff0d00ef5e0 con 0x7ff0dc100ca0 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 -- 172.21.15.16:0/2205059032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbb5d40 msgr2=0x55847cbb8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbb5d40 0x55847cbb8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.228+0000 7ff0e174d640 1 -- 172.21.15.16:0/2205059032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 msgr2=0x55847cbb5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0e174d640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0e174d640 1 -- 172.21.15.16:0/2205059032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847cac41f0 con 0x55847cbaab60 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0dbfff640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 0x7ff0c40a51b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ecde0 tx=0x7ff0d00be040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0e174d640 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x55847cba8470 tx=0x7ff0d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07458632adf8b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0c2ffd640 1 -- 172.21.15.16:0/2205059032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d401a030 con 0x55847cbaab60 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0c2ffd640 1 -- 172.21.15.16:0/2205059032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d4014e40 con 0x55847cbaab60 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.229+0000 7ff0c2ffd640 1 -- 172.21.15.16:0/2205059032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4005240 con 0x55847cbaab60 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- 172.21.15.16:0/2205059032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 msgr2=0x55847cbaaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x55847cba8470 tx=0x7ff0d4009c30 comp rx=0 tx=0).stop 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- 172.21.15.16:0/2205059032 shutdown_connections 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbb5d40 0x55847cbb8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 --2- 172.21.15.16:0/2205059032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbaaf40 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- 172.21.15.16:0/2205059032 >> 172.21.15.16:0/2205059032 conn(0x55847cba70f0 msgr2=0x55847cba74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- 172.21.15.16:0/2205059032 shutdown_connections 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- 172.21.15.16:0/2205059032 wait complete. 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 -- start start 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 Processor -- start 2024-09-12T01:05:12.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.230+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbaab60 0x55847cbb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55847cbb8910 con 0x55847cbb5d40 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55847cacaf10 con 0x55847cbab480 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847cbb8a90 con 0x55847cbaab60 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e5589740 10 client.0 _ll_register_callbacks cb 0x55847ca93a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54908/0 (socket says 172.21.15.16:54908) 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 -- 172.21.15.16:0/3768734100 learned_addr learned my addr 172.21.15.16:0/3768734100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbaab60 0x55847cbb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 -- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbaab60 msgr2=0x55847cbb2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbaab60 0x55847cbb2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 -- 172.21.15.16:0/3768734100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 msgr2=0x55847cbb30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 -- 172.21.15.16:0/3768734100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847c8816a0 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0e174d640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009fd0 tx=0x7ff0d4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac162fb918ea27a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.231+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4004e50 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7ff0d4014030 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55847cb1d7f0 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0d4014230 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 10 client.18363 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55847cbc44c0 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55847cb1d7f0 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0d400f040 con 0x55847cbb5d40 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 10 client.18363 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:12.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 10 client.18363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 20 client.18363 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbc5810 0x55847cbc7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 4 client.18363 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.232+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847cb17d00 con 0x55847cbc5810 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.233+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbc5810 0x55847cbc7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.233+0000 7ff0bbfff640 10 client.18363 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:12.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.233+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbc5810 0x55847cbc7c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3ad081d7b525f0 server_cookie=9351fce80a2035b8 in_seq=0 out_seq=0 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0bbfff640 1 -- 172.21.15.16:0/3768734100 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55847cb17d00 con 0x55847cbc5810 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0bbfff640 10 client.18363 handle_command_reply: tid=0 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 client.18363 shutdown 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 msgr2=0x55847cbb35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009fd0 tx=0x7ff0d4010cb0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 shutdown_connections 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbc5810 0x55847cbc7c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbb5d40 0x55847cbb35f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbb30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0bbfff640 0 client.18363 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 --2- 172.21.15.16:0/3768734100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbaab60 0x55847cbb2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 >> 172.21.15.16:0/3768734100 conn(0x55847cba70f0 msgr2=0x55847cbace20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 shutdown_connections 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 1 -- 172.21.15.16:0/3768734100 wait complete. 2024-09-12T01:05:12.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.234+0000 7ff0e5589740 20 client.18363 trim_cache size 0 max 16384 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 Processor -- start 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 -- start start 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 0x55847cbad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbab480 0x55847cbcd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1afb80 con 0x55847cbaab60 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b16c0 con 0x55847cba70f0 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e5589740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b0250 con 0x55847cbab480 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54914/0 (socket says 172.21.15.16:54914) 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/830427487 learned_addr learned my addr 172.21.15.16:0/830427487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e174d640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbab480 0x55847cbcd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e174d640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 0x55847cbad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/830427487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbab480 msgr2=0x55847cbcd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbab480 0x55847cbcd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/830427487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 msgr2=0x55847cbad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 0x55847cbad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/830427487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847cba6f20 con 0x55847cbaab60 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.252+0000 7ff0e174d640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 0x55847cbad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:12.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721e99d4a5cefa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0c27fc640 1 -- 172.21.15.16:0/830427487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc004020 con 0x55847cbaab60 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0c27fc640 1 -- 172.21.15.16:0/830427487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc0041c0 con 0x55847cbaab60 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0c27fc640 1 -- 172.21.15.16:0/830427487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc01c6a0 con 0x55847cbaab60 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 -- 172.21.15.16:0/830427487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 msgr2=0x55847cbb1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc0138f0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 -- 172.21.15.16:0/830427487 shutdown_connections 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cbab480 0x55847cbcd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbb1ab0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 --2- 172.21.15.16:0/830427487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cba70f0 0x55847cbad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.253+0000 7ff0e5589740 1 -- 172.21.15.16:0/830427487 >> 172.21.15.16:0/830427487 conn(0x55847cbae560 msgr2=0x55847cbb4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- 172.21.15.16:0/830427487 shutdown_connections 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- 172.21.15.16:0/830427487 wait complete. 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- start start 2024-09-12T01:05:12.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 Processor -- start 2024-09-12T01:05:12.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cba70f0 0x55847cbb9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b0ff0 con 0x55847cbaab60 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1b0920 con 0x55847cbab480 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1adb80 con 0x55847cba70f0 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.254+0000 7ff0e5589740 10 client.0 _ll_register_callbacks cb 0x7ff0cc00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54922/0 (socket says 172.21.15.16:54922) 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/1256298372 learned_addr learned my addr 172.21.15.16:0/1256298372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e174d640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cba70f0 0x55847cbb9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e174d640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cba70f0 msgr2=0x55847cbb9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cba70f0 0x55847cbb9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/1256298372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 msgr2=0x55847cbba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 -- 172.21.15.16:0/1256298372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847cc2f650 con 0x55847cbaab60 2024-09-12T01:05:12.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e0f4c640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc003fd0 tx=0x7ff0cc004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7089aaf8fa9bde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc033060 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7ff0cc0035f0 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc0041c0 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0cc0037f0 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e5589740 10 client.18369 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.255+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55847cc30900 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc0041c0 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff0cc01cce0 con 0x55847cbaab60 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 10 client.18369 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 10 client.18369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 20 client.18369 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbb5d40 0x55847cc33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 4 client.18369 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55847ca94110 con 0x55847cbb5d40 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.256+0000 7ff0e174d640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbb5d40 0x55847cc33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.257+0000 7ff0e174d640 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbb5d40 0x55847cc33010 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1b083853055d6e6 server_cookie=7081e8197f53df85 in_seq=0 out_seq=0 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.257+0000 7ff0bbfff640 10 client.18369 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.257+0000 7ff0bbfff640 1 -- 172.21.15.16:0/1256298372 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55847ca94110 con 0x55847cbb5d40 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.257+0000 7ff0bbfff640 10 client.18369 handle_command_reply: tid=0 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.257+0000 7ff0e5589740 1 client.18369 shutdown 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 msgr2=0x55847cbba310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc003fd0 tx=0x7ff0cc004020 comp rx=0 tx=0).stop 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 shutdown_connections 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55847cbb5d40 0x55847cc33010 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cbab480 0x55847cbba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0bbfff640 0 client.18369 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847cbaab60 0x55847cbba310 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 --2- 172.21.15.16:0/1256298372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847cba70f0 0x55847cbb9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 >> 172.21.15.16:0/1256298372 conn(0x55847cbae560 msgr2=0x55847cba5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 shutdown_connections 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 1 -- 172.21.15.16:0/1256298372 wait complete. 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.258+0000 7ff0e5589740 20 client.18369 trim_cache size 0 max 16384 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c6c961cb-31de-4455-b641-6f59fa010894", 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:05:12.262 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 msgr2=0x7ff0c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 0x7ff0c40a51b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ecde0 tx=0x7ff0d00be040 comp rx=0 tx=0).stop 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 msgr2=0x7ff0dc1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005940 tx=0x7ff0d00042f0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 shutdown_connections 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff0c40a2cf0 0x7ff0c40a51b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0dc106f50 0x7ff0dc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0dc100ca0 0x7ff0dc1017b0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 --2- 172.21.15.16:0/3079599424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0dc1002f0 0x7ff0dc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 >> 172.21.15.16:0/3079599424 conn(0x7ff0dc077210 msgr2=0x7ff0dc105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 shutdown_connections 2024-09-12T01:05:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:12.259+0000 7ff0e31b0640 1 -- 172.21.15.16:0/3079599424 wait complete. 2024-09-12T01:05:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:12 smithi099 ceph-mon[35654]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:12 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:05:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:12 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:12 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:05:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:12 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:12 smithi159 ceph-mon[35566]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:12 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:05:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:12 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:12 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:05:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:12 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:12 smithi016 ceph-mon[30972]: pgmap v281: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:12 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:05:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:12 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:12 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:05:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:12 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:13.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:05:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 -- 172.21.15.16:0/806739390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8101420 msgr2=0x7f72f810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/806739390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8101420 0x7f72f810d950 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009cd0 tx=0x7f72f002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 -- 172.21.15.16:0/806739390 shutdown_connections 2024-09-12T01:05:13.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/806739390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8101420 0x7f72f810d950 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/806739390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/806739390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8106a80 0x7f72f8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 -- 172.21.15.16:0/806739390 >> 172.21.15.16:0/806739390 conn(0x7f72f80fc7a0 msgr2=0x7f72f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.912+0000 7f72fe5f8640 1 -- 172.21.15.16:0/806739390 shutdown_connections 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 -- 172.21.15.16:0/806739390 wait complete. 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 Processor -- start 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 -- start start 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8101420 0x7f72f81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8110e80 con 0x7f72f8106a80 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8110d00 con 0x7f72f8100a80 2024-09-12T01:05:13.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.913+0000 7f72fe5f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72f8111000 con 0x7f72f8101420 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54936/0 (socket says 172.21.15.16:54936) 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 -- 172.21.15.16:0/278958247 learned_addr learned my addr 172.21.15.16:0/278958247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f7fff640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f77fe640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8101420 0x7f72f81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 -- 172.21.15.16:0/278958247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8101420 msgr2=0x7f72f81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8101420 0x7f72f81997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 -- 172.21.15.16:0/278958247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 msgr2=0x7f72f8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 -- 172.21.15.16:0/278958247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f81a4850 con 0x7f72f8106a80 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f7fff640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:13.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fcb95640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f72f0005cc0 tx=0x7f72f0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f57fa640 1 -- 172.21.15.16:0/278958247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0004040 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f81a4a20 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f57fa640 1 -- 172.21.15.16:0/278958247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f00046c0 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72f57fa640 1 -- 172.21.15.16:0/278958247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0030e00 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.914+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f81a4ed0 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f72f57fa640 1 -- 172.21.15.16:0/278958247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f72f00305a0 con 0x7f72f8106a80 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 Processor -- start 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 -- start start 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b4dc0 0x56523e3bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3bf680 0x56523e3c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56523e31a2e0 con 0x56523e3b44a0 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56523e2c1ce0 con 0x56523e3b4dc0 2024-09-12T01:05:13.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f73009d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56523e3162a0 con 0x56523e3bf680 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f72f57fa640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 0x7f72e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.917+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54938/0 (socket says 172.21.15.16:54938) 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 -- 172.21.15.16:0/1589918065 learned_addr learned my addr 172.21.15.16:0/1589918065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72fcb95640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 0x7f72e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f7fff640 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b4dc0 0x56523e3bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 -- 172.21.15.16:0/1589918065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3bf680 msgr2=0x56523e3c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3bf680 0x56523e3c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 -- 172.21.15.16:0/1589918065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b4dc0 msgr2=0x56523e3bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b4dc0 0x56523e3bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f57fa640 1 -- 172.21.15.16:0/278958247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f72f002e030 con 0x7f72f8106a80 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 -- 172.21.15.16:0/1589918065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523e2a6440 con 0x56523e3b44a0 2024-09-12T01:05:13.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005e00 tx=0x7f72ec0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387e5298f8c8e879 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72fcb95640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 0x7f72e00a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72f00ee4f0 tx=0x7f72f00308d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72de7fc640 1 -- 172.21.15.16:0/1589918065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec016070 con 0x56523e3b44a0 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72de7fc640 1 -- 172.21.15.16:0/1589918065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec012900 con 0x56523e3b44a0 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.918+0000 7f72de7fc640 1 -- 172.21.15.16:0/1589918065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec011750 con 0x56523e3b44a0 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- 172.21.15.16:0/1589918065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 msgr2=0x56523e3b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005e00 tx=0x7f72ec0106e0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- 172.21.15.16:0/1589918065 shutdown_connections 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3bf680 0x56523e3c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b4dc0 0x56523e3bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 --2- 172.21.15.16:0/1589918065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b44a0 0x56523e3b4880 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- 172.21.15.16:0/1589918065 >> 172.21.15.16:0/1589918065 conn(0x56523e3b0a30 msgr2=0x56523e3b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- 172.21.15.16:0/1589918065 shutdown_connections 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- 172.21.15.16:0/1589918065 wait complete. 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.919+0000 7f73009d1740 1 -- start start 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 Processor -- start 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3bef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56523e3c2250 con 0x56523e3bf680 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56523e2d5050 con 0x56523e3b44a0 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56523e3c23d0 con 0x56523e3b4dc0 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f73009d1740 10 client.0 _ll_register_callbacks cb 0x56523e315160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54940/0 (socket says 172.21.15.16:54940) 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 -- 172.21.15.16:0/503363404 learned_addr learned my addr 172.21.15.16:0/503363404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3bef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f7fff640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 -- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 msgr2=0x56523e3bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 -- 172.21.15.16:0/503363404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 msgr2=0x56523e3bef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3bef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.920+0000 7f72f77fe640 1 -- 172.21.15.16:0/503363404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523e08b3c0 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72f77fe640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f72ec011800 tx=0x7f72ec009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490254dacc19a825 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec01f070 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f72ec012cc0 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec012900 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72ec031050 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 10 client.18381 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56523e3cdbf0 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec012900 con 0x56523e3bf680 2024-09-12T01:05:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72ec011860 con 0x56523e3bf680 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 10 client.18381 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 10 client.18381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 20 client.18381 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3cf000 0x56523e3d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 4 client.18381 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.921+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56523e327850 con 0x56523e3cf000 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.922+0000 7f72f7fff640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3cf000 0x56523e3d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.922+0000 7f72d77fe640 10 client.18381 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.922+0000 7f72f7fff640 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3cf000 0x56523e3d13f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22ef8814ac18205 server_cookie=e7176f71cad10a7c in_seq=0 out_seq=0 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f72d77fe640 1 -- 172.21.15.16:0/503363404 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56523e327850 con 0x56523e3cf000 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f72d77fe640 10 client.18381 handle_command_reply: tid=0 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 client.18381 shutdown 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 msgr2=0x56523e3bc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f72ec011800 tx=0x7f72ec009380 comp rx=0 tx=0).stop 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 shutdown_connections 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3cf000 0x56523e3d13f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3bf680 0x56523e3bc720 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 --2- 172.21.15.16:0/503363404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3bef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f72d77fe640 0 client.18381 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 >> 172.21.15.16:0/503363404 conn(0x56523e3b0a30 msgr2=0x56523e3b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 shutdown_connections 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 1 -- 172.21.15.16:0/503363404 wait complete. 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.923+0000 7f73009d1740 20 client.18381 trim_cache size 0 max 16384 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 Processor -- start 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 -- start start 2024-09-12T01:05:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b0a30 0x56523e3be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b44a0 0x56523e3d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81ad7b0 con 0x56523e3b4dc0 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81af2f0 con 0x56523e3b0a30 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f73009d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81ade80 con 0x56523e3b44a0 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54952/0 (socket says 172.21.15.16:54952) 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 -- 172.21.15.16:0/1131673946 learned_addr learned my addr 172.21.15.16:0/1131673946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 -- 172.21.15.16:0/1131673946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b44a0 msgr2=0x56523e3d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b44a0 0x56523e3d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 -- 172.21.15.16:0/1131673946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b0a30 msgr2=0x56523e3be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b0a30 0x56523e3be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 -- 172.21.15.16:0/1131673946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523e3b0860 con 0x56523e3b4dc0 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.941+0000 7f72f77fe640 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0117d0 tx=0x7f72ec011460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a006b0a3ae556b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f72ddffb640 1 -- 172.21.15.16:0/1131673946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec01f040 con 0x56523e3b4dc0 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f72ddffb640 1 -- 172.21.15.16:0/1131673946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec02e030 con 0x56523e3b4dc0 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f72ddffb640 1 -- 172.21.15.16:0/1131673946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec016030 con 0x56523e3b4dc0 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- 172.21.15.16:0/1131673946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 msgr2=0x56523e3d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0117d0 tx=0x7f72ec011460 comp rx=0 tx=0).stop 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- 172.21.15.16:0/1131673946 shutdown_connections 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b4dc0 0x56523e3d7140 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b44a0 0x56523e3d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 --2- 172.21.15.16:0/1131673946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b0a30 0x56523e3be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- 172.21.15.16:0/1131673946 >> 172.21.15.16:0/1131673946 conn(0x56523e3bc880 msgr2=0x56523e3bda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- 172.21.15.16:0/1131673946 shutdown_connections 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- 172.21.15.16:0/1131673946 wait complete. 2024-09-12T01:05:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.942+0000 7f73009d1740 1 -- start start 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 Processor -- start 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81aec20 con 0x56523e3b0a30 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81ae550 con 0x56523e3b44a0 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72f81ab7b0 con 0x56523e3b4dc0 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f73009d1740 10 client.0 _ll_register_callbacks cb 0x7f72ec009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54966/0 (socket says 172.21.15.16:54966) 2024-09-12T01:05:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 -- 172.21.15.16:0/3575871553 learned_addr learned my addr 172.21.15.16:0/3575871553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f7fff640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 -- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 msgr2=0x56523e3c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 -- 172.21.15.16:0/3575871553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 msgr2=0x56523e3c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.943+0000 7f72f77fe640 1 -- 172.21.15.16:0/3575871553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523e3c5df0 con 0x56523e3b0a30 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72f77fe640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f72ec011d50 tx=0x7f72ec009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09fae50045eb99c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec03c020 con 0x56523e3b0a30 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f72ec012420 con 0x56523e3b0a30 2024-09-12T01:05:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ec02e030 con 0x56523e3b0a30 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72ec047050 con 0x56523e3b0a30 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 10 client.18387 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56523e439bc0 con 0x56523e3b0a30 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec02e030 con 0x56523e3b0a30 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72ec004920 con 0x56523e3b0a30 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 10 client.18387 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 10 client.18387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 20 client.18387 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3bf680 0x56523e43c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 4 client.18387 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:05:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.944+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56523e321d60 con 0x56523e3bf680 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f72f7fff640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3bf680 0x56523e43c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f72f7fff640 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3bf680 0x56523e43c1d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8139e4b3fb5979 server_cookie=5e6f9f076aa63622 in_seq=0 out_seq=0 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f72d77fe640 10 client.18387 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f72d77fe640 1 -- 172.21.15.16:0/3575871553 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56523e321d60 con 0x56523e3bf680 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f72d77fe640 10 client.18387 handle_command_reply: tid=0 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.945+0000 7f73009d1740 1 client.18387 shutdown 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 msgr2=0x56523e3c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f72ec011d50 tx=0x7f72ec009cf0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 shutdown_connections 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56523e3bf680 0x56523e43c1d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523e3b4dc0 0x56523e3c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56523e3b44a0 0x56523e3c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 --2- 172.21.15.16:0/3575871553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56523e3b0a30 0x56523e3c7d60 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 >> 172.21.15.16:0/3575871553 conn(0x56523e3bc880 msgr2=0x56523e3be260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f72d77fe640 0 client.18387 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 shutdown_connections 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 1 -- 172.21.15.16:0/3575871553 wait complete. 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.946+0000 7f73009d1740 20 client.18387 trim_cache size 0 max 16384 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:05:13.963 INFO:teuthology.orchestra.run.smithi016.stdout: "c6c961cb-31de-4455-b641-6f59fa010894": { 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c6c961cb-31de-4455-b641-6f59fa010894", 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 msgr2=0x7f72e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 0x7f72e00a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72f00ee4f0 tx=0x7f72f00308d0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 msgr2=0x7f72f81a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f72f0005cc0 tx=0x7f72f0032f40 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 shutdown_connections 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f72e00a2cf0 0x7f72e00a51b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72f8106a80 0x7f72f81a4310 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72f8101420 0x7f72f81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 --2- 172.21.15.16:0/278958247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72f8100a80 0x7f72f8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 >> 172.21.15.16:0/278958247 conn(0x7f72f80fc7a0 msgr2=0x7f72f80fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 shutdown_connections 2024-09-12T01:05:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:13.947+0000 7f72fe5f8640 1 -- 172.21.15.16:0/278958247 wait complete. 2024-09-12T01:05:14.727 INFO:tasks.cephfs.filesystem:scrub status for tag:c6c961cb-31de-4455-b641-6f59fa010894 - {'path': '/', 'tag': 'c6c961cb-31de-4455-b641-6f59fa010894', 'options': 'recursive,force'} 2024-09-12T01:05:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:14 smithi099 ceph-mon[35654]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:14 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:05:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:14 smithi159 ceph-mon[35566]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:14 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:05:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:14 smithi016 ceph-mon[30972]: pgmap v282: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 66 KiB/s wr, 27 op/s 2024-09-12T01:05:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:14 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:05:17.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:16 smithi099 ceph-mon[35654]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-12T01:05:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:16 smithi159 ceph-mon[35566]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-12T01:05:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:16 smithi016 ceph-mon[30972]: pgmap v283: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 88 KiB/s wr, 34 op/s 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:251/600: open fd = 3 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:252/600: open fd = 3 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:253/600: open fd = 3 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.685 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:254/600: open fd = 3 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:255/600: open fd = 3 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:256/600: open fd = 3 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:257/600: open fd = 3 2024-09-12T01:05:17.686 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:258/600: open fd = 3 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:259/600: open fd = 3 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:260/600: open fd = 3 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.687 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:261/600: open fd = 3 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:262/600: open fd = 3 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:263/600: open fd = 3 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.688 INFO:tasks.workunit.client.0.smithi016.stdout:264/600: open fd = 3 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:265/600: open fd = 3 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:266/600: open fd = 3 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:267/600: open fd = 3 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.689 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:268/600: open fd = 3 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:269/600: open fd = 3 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:270/600: open fd = 3 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:271/600: open fd = 3 2024-09-12T01:05:17.690 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:272/600: open fd = 3 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:273/600: open fd = 3 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.691 INFO:tasks.workunit.client.0.smithi016.stdout:274/600: open fd = 3 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:275/600: open fd = 3 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:276/600: open fd = 3 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.692 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:277/600: open fd = 3 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:278/600: open fd = 3 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:279/600: open fd = 3 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.693 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:280/600: open fd = 3 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:281/600: open fd = 3 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:282/600: open fd = 3 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.694 INFO:tasks.workunit.client.0.smithi016.stdout:283/600: open fd = 3 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:284/600: open fd = 3 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:285/600: open fd = 3 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.695 INFO:tasks.workunit.client.0.smithi016.stdout:286/600: open fd = 3 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:287/600: open fd = 3 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:288/600: open fd = 3 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:289/600: open fd = 3 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.696 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:290/600: open fd = 3 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:291/600: open fd = 3 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:292/600: open fd = 3 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.697 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:293/600: open fd = 3 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:294/600: open fd = 3 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:295/600: open fd = 3 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:296/600: open fd = 3 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.698 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:297/600: open fd = 3 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:298/600: open fd = 3 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:299/600: open fd = 3 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:300/600: open fd = 3 2024-09-12T01:05:17.699 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:301/600: open fd = 3 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:302/600: open fd = 3 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.700 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:303/600: open fd = 3 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:304/600: open fd = 3 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:305/600: open fd = 3 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.701 INFO:tasks.workunit.client.0.smithi016.stdout:306/600: open fd = 3 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:307/600: open fd = 3 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:308/600: open fd = 3 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.702 INFO:tasks.workunit.client.0.smithi016.stdout:309/600: open fd = 3 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:310/600: open fd = 3 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:311/600: open fd = 3 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:312/600: open fd = 3 2024-09-12T01:05:17.703 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:17.704 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:05:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:18 smithi159 ceph-mon[35566]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:19.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:18 smithi016 ceph-mon[30972]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:18 smithi099 ceph-mon[35654]: pgmap v284: 97 pgs: 97 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:19 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:05:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:19 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:05:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:19 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:05:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:20 smithi159 ceph-mon[35566]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:05:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:20 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:05:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:20 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:05:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:20 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:05:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:20 smithi016 ceph-mon[30972]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:05:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:20 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:05:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:20 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:05:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:20 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:05:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:20 smithi099 ceph-mon[35654]: pgmap v285: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:05:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:20 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:05:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:20 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:05:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:20 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:05:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:22 smithi159 ceph-mon[35566]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:22 smithi016 ceph-mon[30972]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:22 smithi099 ceph-mon[35654]: pgmap v286: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:24 smithi159 ceph-mon[35566]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:24 smithi016 ceph-mon[30972]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:24 smithi099 ceph-mon[35654]: pgmap v287: 97 pgs: 97 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:26 smithi159 ceph-mon[35566]: pgmap v288: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:27.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:26 smithi016 ceph-mon[30972]: pgmap v288: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:26 smithi099 ceph-mon[35654]: pgmap v288: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:28.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:28 smithi016 ceph-mon[30972]: pgmap v289: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:28.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:28 smithi099 ceph-mon[35654]: pgmap v289: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:28.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:28 smithi159 ceph-mon[35566]: pgmap v289: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:30.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:30 smithi159 ceph-mon[35566]: pgmap v290: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-12T01:05:30.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:30 smithi016 ceph-mon[30972]: pgmap v290: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-12T01:05:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:30 smithi099 ceph-mon[35654]: pgmap v290: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-12T01:05:32.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:32 smithi159 ceph-mon[35566]: pgmap v291: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:32 smithi016 ceph-mon[30972]: pgmap v291: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:32 smithi099 ceph-mon[35654]: pgmap v291: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:34.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:34 smithi159 ceph-mon[35566]: pgmap v292: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:34 smithi016 ceph-mon[30972]: pgmap v292: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:34 smithi099 ceph-mon[35654]: pgmap v292: 97 pgs: 97 active+clean; 12 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:36.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:36 smithi159 ceph-mon[35566]: pgmap v293: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:36.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:36 smithi016 ceph-mon[30972]: pgmap v293: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:36 smithi099 ceph-mon[35654]: pgmap v293: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:38.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:38 smithi159 ceph-mon[35566]: pgmap v294: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:38.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:38 smithi016 ceph-mon[30972]: pgmap v294: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:38 smithi099 ceph-mon[35654]: pgmap v294: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:40.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:40 smithi159 ceph-mon[35566]: pgmap v295: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:40.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:40 smithi016 ceph-mon[30972]: pgmap v295: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:40 smithi099 ceph-mon[35654]: pgmap v295: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:42.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:42 smithi159 ceph-mon[35566]: pgmap v296: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:42.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:42 smithi016 ceph-mon[30972]: pgmap v296: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:42 smithi099 ceph-mon[35654]: pgmap v296: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:44.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:05:44.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:44 smithi159 ceph-mon[35566]: pgmap v297: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:44 smithi016 ceph-mon[30972]: pgmap v297: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:45.001 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:44 smithi099 ceph-mon[35654]: pgmap v297: 97 pgs: 97 active+clean; 12 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 -- 172.21.15.16:0/1266946747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1410103660 msgr2=0x7f141010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 --2- 172.21.15.16:0/1266946747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1410103660 0x7f141010bb50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1400009900 tx=0x7f140002a480 comp rx=0 tx=0).stop 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 -- 172.21.15.16:0/1266946747 shutdown_connections 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 --2- 172.21.15.16:0/1266946747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f141010c170 0x7f141010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 --2- 172.21.15.16:0/1266946747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1410103660 0x7f141010bb50 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 --2- 172.21.15.16:0/1266946747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 0x7f1410103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.572+0000 7f1417f67640 1 -- 172.21.15.16:0/1266946747 >> 172.21.15.16:0/1266946747 conn(0x7f14100fc7a0 msgr2=0x7f14100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- 172.21.15.16:0/1266946747 shutdown_connections 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- 172.21.15.16:0/1266946747 wait complete. 2024-09-12T01:05:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 Processor -- start 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- start start 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 0x7f1410198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1410103660 0x7f14101989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111f70 con 0x7f141010c170 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111df0 con 0x7f1410102d40 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.573+0000 7f1417f67640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14101120f0 con 0x7f1410103660 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58130/0 (socket says 172.21.15.16:58130) 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 -- 172.21.15.16:0/1608943675 learned_addr learned my addr 172.21.15.16:0/1608943675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1415502640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1410103660 0x7f14101989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1415d03640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 0x7f1410198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 -- 172.21.15.16:0/1608943675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1410103660 msgr2=0x7f14101989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1410103660 0x7f14101989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 -- 172.21.15.16:0/1608943675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 msgr2=0x7f1410198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 0x7f1410198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 -- 172.21.15.16:0/1608943675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019d420 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1416504640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1408009fd0 tx=0x7f140800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1406ffd640 1 -- 172.21.15.16:0/1608943675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408017070 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019d710 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1406ffd640 1 -- 172.21.15.16:0/1608943675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408003e50 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1406ffd640 1 -- 172.21.15.16:0/1608943675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080106c0 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.574+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101a4ff0 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f1406ffd640 1 -- 172.21.15.16:0/1608943675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1408003970 con 0x7f141010c170 2024-09-12T01:05:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 Processor -- start 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 -- start start 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744d47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55637443a2e0 con 0x5563744d4ce0 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5563743e1ce0 con 0x5563744d43c0 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.577+0000 7f141a340740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563744362a0 con 0x5563744e3a90 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58134/0 (socket says 172.21.15.16:58134) 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415d03640 1 -- 172.21.15.16:0/831493018 learned_addr learned my addr 172.21.15.16:0/831493018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1406ffd640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 0x7f13fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1416504640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 0x7f13fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744d47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1406ffd640 1 -- 172.21.15.16:0/1608943675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f14080cf130 con 0x7f141010c170 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 -- 172.21.15.16:0/831493018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 msgr2=0x5563744d47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744d47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 -- 172.21.15.16:0/831493018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 msgr2=0x5563744e3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 -- 172.21.15.16:0/831493018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563743c6440 con 0x5563744e3a90 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415d03640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1415502640 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744e5e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1400005940 tx=0x7f14000314e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda11f4d839b28ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.578+0000 7f1416504640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 0x7f13fc0a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f140809b630 tx=0x7f1408004020 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.579+0000 7f13fbfff640 1 -- 172.21.15.16:0/831493018 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400035070 con 0x5563744e3a90 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.579+0000 7f13fbfff640 1 -- 172.21.15.16:0/831493018 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400004030 con 0x5563744e3a90 2024-09-12T01:05:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.579+0000 7f13fbfff640 1 -- 172.21.15.16:0/831493018 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14000054f0 con 0x5563744e3a90 2024-09-12T01:05:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- 172.21.15.16:0/831493018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 msgr2=0x5563744e5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744e5e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1400005940 tx=0x7f14000314e0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- 172.21.15.16:0/831493018 shutdown_connections 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744e5e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 --2- 172.21.15.16:0/831493018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744d47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- 172.21.15.16:0/831493018 >> 172.21.15.16:0/831493018 conn(0x5563744d0a30 msgr2=0x5563744d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- 172.21.15.16:0/831493018 shutdown_connections 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- 172.21.15.16:0/831493018 wait complete. 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 -- start start 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.580+0000 7f141a340740 1 Processor -- start 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 0x5563744e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5563744e65a0 con 0x5563744e3a90 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5563743f5050 con 0x5563744d43c0 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563744e6720 con 0x5563744d4ce0 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f141a340740 10 client.0 _ll_register_callbacks cb 0x556374435160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415502640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 0x5563744e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:38430/0 (socket says 172.21.15.16:38430) 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 -- 172.21.15.16:0/2604376564 learned_addr learned my addr 172.21.15.16:0/2604376564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415502640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 -- 172.21.15.16:0/2604376564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 msgr2=0x5563744e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 -- 172.21.15.16:0/2604376564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 msgr2=0x5563744e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 0x5563744e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415d03640 1 -- 172.21.15.16:0/2604376564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563741ab3c0 con 0x5563744d4ce0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.581+0000 7f1415502640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 0x5563744e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f1415d03640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f140c012840 tx=0x7f140c012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2261d4a38017c854 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f13f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c004050 con 0x5563744d4ce0 2024-09-12T01:05:45.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f140c0041f0 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400004030 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f140c0043f0 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f141a340740 10 client.27493 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563744f2050 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.582+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400004030 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f140c003b90 con 0x5563744d4ce0 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 10 client.27493 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 10 client.27493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 20 client.27493 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5563744f3460 0x5563744f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 4 client.27493 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556374447850 con 0x5563744f3460 2024-09-12T01:05:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f1415502640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5563744f3460 0x5563744f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f1415502640 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5563744f3460 0x5563744f58a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec82ad5d3ca5246 server_cookie=129deb703120b31f in_seq=0 out_seq=0 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.583+0000 7f13f97fa640 10 client.27493 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f13f97fa640 1 -- 172.21.15.16:0/2604376564 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556374447850 con 0x5563744f3460 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f13f97fa640 10 client.27493 handle_command_reply: tid=0 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 client.27493 shutdown 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 msgr2=0x5563744e0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f140c012840 tx=0x7f140c012d00 comp rx=0 tx=0).stop 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 shutdown_connections 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5563744f3460 0x5563744f58a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744e3a90 0x5563744e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d4ce0 0x5563744e0700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f13f97fa640 0 client.27493 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 --2- 172.21.15.16:0/2604376564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 >> 172.21.15.16:0/2604376564 conn(0x5563744d0a30 msgr2=0x5563744d66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 shutdown_connections 2024-09-12T01:05:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 1 -- 172.21.15.16:0/2604376564 wait complete. 2024-09-12T01:05:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.584+0000 7f141a340740 20 client.27493 trim_cache size 0 max 16384 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 Processor -- start 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 -- start start 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744fb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ab6f0 con 0x5563744d4ce0 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ad230 con 0x5563744d43c0 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f141a340740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14101abdc0 con 0x5563744e3a90 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58142/0 (socket says 172.21.15.16:58142) 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.602+0000 7f1415d03640 1 -- 172.21.15.16:0/1007548483 learned_addr learned my addr 172.21.15.16:0/1007548483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415502640 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744fb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415502640 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 -- 172.21.15.16:0/1007548483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 msgr2=0x5563744fb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744fb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 -- 172.21.15.16:0/1007548483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 msgr2=0x5563744ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 -- 172.21.15.16:0/1007548483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563744d1ba0 con 0x5563744d4ce0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f1415d03640 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f140c003f30 tx=0x7f140c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace4735ce114ae6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f13fb7fe640 1 -- 172.21.15.16:0/1007548483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c0256f0 con 0x5563744d4ce0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f13fb7fe640 1 -- 172.21.15.16:0/1007548483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140c003fb0 con 0x5563744d4ce0 2024-09-12T01:05:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.603+0000 7f13fb7fe640 1 -- 172.21.15.16:0/1007548483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c00f790 con 0x5563744d4ce0 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- 172.21.15.16:0/1007548483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 msgr2=0x5563744df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f140c003f30 tx=0x7f140c005db0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- 172.21.15.16:0/1007548483 shutdown_connections 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744e3a90 0x5563744fb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744df6c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 --2- 172.21.15.16:0/1007548483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744d43c0 0x5563744ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- 172.21.15.16:0/1007548483 >> 172.21.15.16:0/1007548483 conn(0x5563744d0a30 msgr2=0x5563744dbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- 172.21.15.16:0/1007548483 shutdown_connections 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- 172.21.15.16:0/1007548483 wait complete. 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.604+0000 7f141a340740 1 -- start start 2024-09-12T01:05:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 Processor -- start 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d43c0 0x5563744ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 0x5563744e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14101acb60 con 0x5563744d4ce0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14101ac490 con 0x5563744e3a90 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14101a96f0 con 0x5563744d43c0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f141a340740 10 client.0 _ll_register_callbacks cb 0x7f140c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415d03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58150/0 (socket says 172.21.15.16:58150) 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415d03640 1 -- 172.21.15.16:0/1745202880 learned_addr learned my addr 172.21.15.16:0/1745202880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415502640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d43c0 0x5563744ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415502640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 0x5563744e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.605+0000 7f1415d03640 1 -- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d43c0 msgr2=0x5563744ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415d03640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d43c0 0x5563744ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415d03640 1 -- 172.21.15.16:0/1745202880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 msgr2=0x5563744e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415d03640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 0x5563744e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415d03640 1 -- 172.21.15.16:0/1745202880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563744e80a0 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415502640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 0x5563744e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f1415d03640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f140c0254f0 tx=0x7f140c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dbf93b8420d94a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c004220 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f140c0055e0 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140c003fb0 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f140c025980 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f141a340740 10 client.18405 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55637455e050 con 0x5563744d4ce0 2024-09-12T01:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140c003fb0 con 0x5563744d4ce0 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f140c0035f0 con 0x5563744d4ce0 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f141a340740 10 client.18405 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.606+0000 7f141a340740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f141a340740 20 client.18405 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55637455e650 0x556374560a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f141a340740 4 client.18405 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556374441d60 con 0x55637455e650 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f1415502640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55637455e650 0x556374560a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f1415502640 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55637455e650 0x556374560a40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b791ccd3abf87ac5 server_cookie=ed88cfd42be95511 in_seq=0 out_seq=0 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f13f97fa640 10 client.18405 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f13f97fa640 1 -- 172.21.15.16:0/1745202880 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556374441d60 con 0x55637455e650 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f13f97fa640 10 client.18405 handle_command_reply: tid=0 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.607+0000 7f141a340740 1 client.18405 shutdown 2024-09-12T01:05:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 msgr2=0x5563744e7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f140c0254f0 tx=0x7f140c009e60 comp rx=0 tx=0).stop 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 shutdown_connections 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55637455e650 0x556374560a40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563744e3a90 0x5563744e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5563744d4ce0 0x5563744e7330 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 --2- 172.21.15.16:0/1745202880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563744d43c0 0x5563744ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f13f97fa640 0 client.18405 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 >> 172.21.15.16:0/1745202880 conn(0x5563744d0a30 msgr2=0x5563744cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 shutdown_connections 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 1 -- 172.21.15.16:0/1745202880 wait complete. 2024-09-12T01:05:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f141a340740 20 client.18405 trim_cache size 0 max 16384 2024-09-12T01:05:45.612 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:05:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:05:45.612 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:05:45.612 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:05:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 msgr2=0x7f13fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 0x7f13fc0a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f140809b630 tx=0x7f1408004020 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 msgr2=0x7f141019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.608+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f1408009fd0 tx=0x7f140800b260 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 shutdown_connections 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f13fc0a2cf0 0x7f13fc0a51b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f141010c170 0x7f141019cd40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1410103660 0x7f14101989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 --2- 172.21.15.16:0/1608943675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1410102d40 0x7f1410198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 >> 172.21.15.16:0/1608943675 conn(0x7f14100fc7a0 msgr2=0x7f14100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 shutdown_connections 2024-09-12T01:05:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:45.609+0000 7f1417f67640 1 -- 172.21.15.16:0/1608943675 wait complete. 2024-09-12T01:05:46.448 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:05:46.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:05:46.707 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:46.828 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:46 smithi016 ceph-mon[30972]: pgmap v298: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-09-12T01:05:46.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:46 smithi159 ceph-mon[35566]: pgmap v298: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-09-12T01:05:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:46 smithi099 ceph-mon[35654]: pgmap v298: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-09-12T01:05:47.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- 172.21.15.16:0/57024252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c101de0 msgr2=0x7f0b8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 --2- 172.21.15.16:0/57024252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c101de0 0x7f0b8c1021c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009900 tx=0x7f0b8402a410 comp rx=0 tx=0).stop 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- 172.21.15.16:0/57024252 shutdown_connections 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 --2- 172.21.15.16:0/57024252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 0x7f0b8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 --2- 172.21.15.16:0/57024252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c102790 0x7f0b8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 --2- 172.21.15.16:0/57024252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c101de0 0x7f0b8c1021c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- 172.21.15.16:0/57024252 >> 172.21.15.16:0/57024252 conn(0x7f0b8c0fd8c0 msgr2=0x7f0b8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- 172.21.15.16:0/57024252 shutdown_connections 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- 172.21.15.16:0/57024252 wait complete. 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 Processor -- start 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.162+0000 7f0b944a5640 1 -- start start 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 0x7f0b8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 0x7f0b8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c111f70 con 0x7f0b8c102790 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c111df0 con 0x7f0b8c106f10 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b944a5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1120f0 con 0x7f0b8c101de0 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58168/0 (socket says 172.21.15.16:58168) 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 -- 172.21.15.16:0/1609190141 learned_addr learned my addr 172.21.15.16:0/1609190141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b92241640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 0x7f0b8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b92a42640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 0x7f0b8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 -- 172.21.15.16:0/1609190141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 msgr2=0x7f0b8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 0x7f0b8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 -- 172.21.15.16:0/1609190141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 msgr2=0x7f0b8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 0x7f0b8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 -- 172.21.15.16:0/1609190141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1a6950 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b92241640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 0x7f0b8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.163+0000 7f0b91a40640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d770 tx=0x7f0b8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.164+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80017070 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.164+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b80005500 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.164+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a6c40 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.164+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80004e00 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.164+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a7130 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.165+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c1021c0 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.166+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0b80004fa0 con 0x7f0b8c102790 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.166+0000 7f0b8b7fe640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 0x7f0b740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.167+0000 7f0b92241640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 0x7f0b740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.167+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f0b800cefe0 con 0x7f0b8c102790 2024-09-12T01:05:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.167+0000 7f0b92241640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 0x7f0b740a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005ab0 tx=0x7f0b84005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.171+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b8009bba0 con 0x7f0b8c102790 2024-09-12T01:05:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.380+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8c10a0f0 con 0x7f0b8c102790 2024-09-12T01:05:47.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.381+0000 7f0b8b7fe640 1 -- 172.21.15.16:0/1609190141 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f0b8009b4a0 con 0x7f0b8c102790 2024-09-12T01:05:47.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:05:47.380 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:05:47.380 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 msgr2=0x7f0b740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 0x7f0b740a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b84005ab0 tx=0x7f0b84005a20 comp rx=0 tx=0).stop 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 msgr2=0x7f0b8c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d770 tx=0x7f0b8000dc30 comp rx=0 tx=0).stop 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 shutdown_connections 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0b740a2cf0 0x7f0b740a51b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b8c106f10 0x7f0b8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b8c102790 0x7f0b8c074a30 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 --2- 172.21.15.16:0/1609190141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b8c101de0 0x7f0b8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.383+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 >> 172.21.15.16:0/1609190141 conn(0x7f0b8c0fd8c0 msgr2=0x7f0b8c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.384+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 shutdown_connections 2024-09-12T01:05:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:47.384+0000 7f0b944a5640 1 -- 172.21.15.16:0/1609190141 wait complete. 2024-09-12T01:05:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1609190141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:47.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1609190141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1609190141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:05:48.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:05:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:48.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:48 smithi016 ceph-mon[30972]: pgmap v299: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:48.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 -- 172.21.15.16:0/4100558468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106f10 msgr2=0x7f2fcc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/4100558468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106f10 0x7f2fcc10ea90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009ca0 tx=0x7f2fc402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 -- 172.21.15.16:0/4100558468 shutdown_connections 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/4100558468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc106f10 0x7f2fcc10ea90 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/4100558468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102790 0x7f2fcc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/4100558468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc101de0 0x7f2fcc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 -- 172.21.15.16:0/4100558468 >> 172.21.15.16:0/4100558468 conn(0x7f2fcc0fd8c0 msgr2=0x7f2fcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 -- 172.21.15.16:0/4100558468 shutdown_connections 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.831+0000 7f2fd46ea640 1 -- 172.21.15.16:0/4100558468 wait complete. 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 Processor -- start 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 -- start start 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc102790 0x7f2fcc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 0x7f2fcc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc112010 con 0x7f2fcc101de0 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc111e90 con 0x7f2fcc102790 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd46ea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc112190 con 0x7f2fcc106f10 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55558/0 (socket says 172.21.15.16:55558) 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.832+0000 7f2fd2486640 1 -- 172.21.15.16:0/3976985847 learned_addr learned my addr 172.21.15.16:0/3976985847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc102790 0x7f2fcc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 0x7f2fcc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 -- 172.21.15.16:0/3976985847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 msgr2=0x7f2fcc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 0x7f2fcc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 -- 172.21.15.16:0/3976985847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc102790 msgr2=0x7f2fcc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc102790 0x7f2fcc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 -- 172.21.15.16:0/3976985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc1a6ab0 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 0x7f2fcc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd2486640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00b4a0 tx=0x7f2fbc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/3976985847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc014070 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/3976985847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc010af0 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc1a6da0 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/3976985847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc017d10 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.833+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a7250 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.836+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/3976985847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2fbc017070 con 0x7f2fcc101de0 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.836+0000 7f2fd6ac3740 1 Processor -- start 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 -- start start 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb8b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb8ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a7ecaf12f0 con 0x55a7ecb96370 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7eca98cf0 con 0x55a7ecb8cb40 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7ecaed2b0 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55564/0 (socket says 172.21.15.16:55564) 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1794560973 learned_addr learned my addr 172.21.15.16:0/1794560973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb8b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd2486640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb8ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1794560973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 msgr2=0x55a7ecb8ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb8ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fcb7fe640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1794560973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 msgr2=0x55a7ecb8b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb8b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1794560973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7eca7d3a0 con 0x55a7ecb96370 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd2486640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb8ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.837+0000 7f2fcb7fe640 1 -- 172.21.15.16:0/3976985847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2fbc0ce750 con 0x7f2fcc101de0 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b7f0 tx=0x7f2fc000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ed5f437c60ccde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fc8ff9640 1 -- 172.21.15.16:0/1794560973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000d950 con 0x55a7ecb96370 2024-09-12T01:05:48.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fc8ff9640 1 -- 172.21.15.16:0/1794560973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc0018070 con 0x55a7ecb96370 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fc8ff9640 1 -- 172.21.15.16:0/1794560973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00123e0 con 0x55a7ecb96370 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1794560973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 msgr2=0x55a7ecb98760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009430 tx=0x7f2fc40095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b7f0 tx=0x7f2fc000bcb0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.838+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1794560973 shutdown_connections 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb96370 0x55a7ecb98760 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb8ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1794560973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb8b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1794560973 >> 172.21.15.16:0/1794560973 conn(0x55a7ecb54310 msgr2=0x55a7ecb87c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1794560973 shutdown_connections 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1794560973 wait complete. 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- start start 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 Processor -- start 2024-09-12T01:05:48.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 0x55a7ecb934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a7ecb99070 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7ecaac060 con 0x55a7ecb96370 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7ecb991f0 con 0x55a7ecb8cb40 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd6ac3740 10 client.0 _ll_register_callbacks cb 0x55a7ec7ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55572/0 (socket says 172.21.15.16:55572) 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.839+0000 7f2fd2486640 1 -- 172.21.15.16:0/2908987120 learned_addr learned my addr 172.21.15.16:0/2908987120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 0x55a7ecb934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 -- 172.21.15.16:0/2908987120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8cb40 msgr2=0x55a7ecb92f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 -- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 msgr2=0x55a7ecb934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 0x55a7ecb934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 -- 172.21.15.16:0/2908987120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ec8623c0 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 0x55a7ecb934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd2486640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0cc4a0 tx=0x7f2fbc09ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91709f852c318c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01e050 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2fbc0cc120 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc0018070 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2fbc09ad80 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd6ac3740 10 client.18423 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7ecba4a80 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.840+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc0018070 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2fbc0b1060 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 10 client.18423 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 10 client.18423 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 20 client.18423 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecba82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 4 client.18423 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ecafe860 con 0x55a7ecba5e90 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecba82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecba82e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c5e9b6efce5169d server_cookie=b4f9394fd863553 in_seq=0 out_seq=0 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.841+0000 7f2fb1ffb640 10 client.18423 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/2908987120 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7ecafe860 con 0x55a7ecba5e90 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fb1ffb640 10 client.18423 handle_command_reply: tid=0 2024-09-12T01:05:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 client.18423 shutdown 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 msgr2=0x55a7ecb95cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0cc4a0 tx=0x7f2fbc09ac10 comp rx=0 tx=0).stop 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 shutdown_connections 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecba82e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb96370 0x55a7ecb934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fb1ffb640 0 client.18423 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/2908987120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb95cc0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 >> 172.21.15.16:0/2908987120 conn(0x55a7ecb54310 msgr2=0x55a7ecb8d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 shutdown_connections 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.842+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/2908987120 wait complete. 2024-09-12T01:05:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.843+0000 7f2fd6ac3740 20 client.18423 trim_cache size 0 max 16384 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 Processor -- start 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 -- start start 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecba2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1afb70 con 0x55a7ecb8cb40 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b16b0 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b0240 con 0x55a7ecb96370 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55574/0 (socket says 172.21.15.16:55574) 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1722251434 learned_addr learned my addr 172.21.15.16:0/1722251434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd2486640 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecba2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1722251434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 msgr2=0x55a7ecb97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1722251434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8b5d0 msgr2=0x55a7ecba2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecba2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 -- 172.21.15.16:0/1722251434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ecb88960 con 0x55a7ecb8cb40 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.856+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b740 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b428ccc2f6c340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fb3fff640 1 -- 172.21.15.16:0/1722251434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc002a020 con 0x55a7ecb8cb40 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fb3fff640 1 -- 172.21.15.16:0/1722251434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc0018040 con 0x55a7ecb8cb40 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fb3fff640 1 -- 172.21.15.16:0/1722251434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00265f0 con 0x55a7ecb8cb40 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1722251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 msgr2=0x55a7ecb92830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000b740 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1722251434 shutdown_connections 2024-09-12T01:05:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb92830 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/1722251434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecba2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1722251434 >> 172.21.15.16:0/1722251434 conn(0x55a7ecb54310 msgr2=0x55a7ecb8e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1722251434 shutdown_connections 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.857+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/1722251434 wait complete. 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 -- start start 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 Processor -- start 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b0fe0 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1b0910 con 0x55a7ecb8cb40 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1adb70 con 0x55a7ecb96370 2024-09-12T01:05:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd6ac3740 10 client.0 _ll_register_callbacks cb 0x7f2fbc00b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd2486640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55586/0 (socket says 172.21.15.16:55586) 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd2486640 1 -- 172.21.15.16:0/380571389 learned_addr learned my addr 172.21.15.16:0/380571389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.858+0000 7f2fd2c87640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 -- 172.21.15.16:0/380571389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 msgr2=0x55a7ecb9ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb9ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 -- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 msgr2=0x55a7ecb9e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb9e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 -- 172.21.15.16:0/380571389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ecb8e120 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd2486640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0cebd0 tx=0x7f2fbc09ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7998ed6467e2916 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01e070 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f2fbc010610 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc0018040 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2fbc09a500 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 10 client.18429 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7ecc10c90 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fc0018040 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2fbc0b1070 con 0x55a7ecb8b5d0 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 10 client.18429 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 10 client.18429 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 20 client.18429 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.859+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecc13280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 4 client.18429 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7ecaf8d70 con 0x55a7ecba5e90 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecc13280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fb1ffb640 10 client.18429 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd1c85640 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecc13280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3da884638e763374 server_cookie=a606f56f8d9db839 in_seq=0 out_seq=0 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fb1ffb640 1 -- 172.21.15.16:0/380571389 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7ecaf8d70 con 0x55a7ecba5e90 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fb1ffb640 10 client.18429 handle_command_reply: tid=0 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 client.18429 shutdown 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 msgr2=0x55a7ecb90bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0cebd0 tx=0x7f2fbc09ac80 comp rx=0 tx=0).stop 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 shutdown_connections 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.860+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55a7ecba5e90 0x55a7ecc13280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7ecb96370 0x55a7ecb9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7ecb8cb40 0x55a7ecb9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 --2- 172.21.15.16:0/380571389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7ecb8b5d0 0x55a7ecb90bc0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 >> 172.21.15.16:0/380571389 conn(0x55a7ecb54310 msgr2=0x55a7ecb917a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fb1ffb640 0 client.18429 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:05:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 shutdown_connections 2024-09-12T01:05:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 1 -- 172.21.15.16:0/380571389 wait complete. 2024-09-12T01:05:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd6ac3740 20 client.18429 trim_cache size 0 max 16384 2024-09-12T01:05:48.874 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 msgr2=0x7f2fb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009430 tx=0x7f2fc40095c0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 msgr2=0x7f2fcc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00b4a0 tx=0x7f2fbc00b960 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 shutdown_connections 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2fb40a2cf0 0x7f2fb40a51b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc106f10 0x7f2fcc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fcc102790 0x7f2fcc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 --2- 172.21.15.16:0/3976985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fcc101de0 0x7f2fcc0744f0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 >> 172.21.15.16:0/3976985847 conn(0x7f2fcc0fd8c0 msgr2=0x7f2fcc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 shutdown_connections 2024-09-12T01:05:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:48.861+0000 7f2fd46ea640 1 -- 172.21.15.16:0/3976985847 wait complete. 2024-09-12T01:05:48.876 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:48 smithi159 ceph-mon[35566]: pgmap v299: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:48 smithi099 ceph-mon[35654]: pgmap v299: 97 pgs: 97 active+clean; 13 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 12 op/s 2024-09-12T01:05:49.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:05:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 -- 172.21.15.16:0/2526945820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc72406a370 msgr2=0x7fc7241009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2526945820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc72406a370 0x7fc7241009c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402a410 comp rx=0 tx=0).stop 2024-09-12T01:05:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 -- 172.21.15.16:0/2526945820 shutdown_connections 2024-09-12T01:05:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2526945820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc724104e80 0x7fc724105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2526945820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc72406a370 0x7fc7241009c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2526945820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.405+0000 7fc72bf61640 1 -- 172.21.15.16:0/2526945820 >> 172.21.15.16:0/2526945820 conn(0x7fc724077210 msgr2=0x7fc7240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.406+0000 7fc72bf61640 1 -- 172.21.15.16:0/2526945820 shutdown_connections 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.406+0000 7fc72bf61640 1 -- 172.21.15.16:0/2526945820 wait complete. 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.406+0000 7fc72bf61640 1 Processor -- start 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.406+0000 7fc72bf61640 1 -- start start 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc72406a370 0x7fc724197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc724110eb0 con 0x7fc724104e80 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc724110d30 con 0x7fc724069f00 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72bf61640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc724111030 con 0x7fc72406a370 2024-09-12T01:05:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55608/0 (socket says 172.21.15.16:55608) 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 -- 172.21.15.16:0/2006692947 learned_addr learned my addr 172.21.15.16:0/2006692947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc7294fc640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc72406a370 0x7fc724197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc729cfd640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 -- 172.21.15.16:0/2006692947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc72406a370 msgr2=0x7fc724197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc72406a370 0x7fc724197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 -- 172.21.15.16:0/2006692947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 msgr2=0x7fc724197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 -- 172.21.15.16:0/2006692947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72419c2e0 con 0x7fc724104e80 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc729cfd640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00eaa0 tx=0x7fc71c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.407+0000 7fc71affd640 1 -- 172.21.15.16:0/2006692947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00cdb0 con 0x7fc724104e80 2024-09-12T01:05:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.408+0000 7fc71affd640 1 -- 172.21.15.16:0/2006692947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c005020 con 0x7fc724104e80 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.408+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72419c5d0 con 0x7fc724104e80 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.408+0000 7fc71affd640 1 -- 172.21.15.16:0/2006692947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0116b0 con 0x7fc724104e80 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.408+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a3eb0 con 0x7fc724104e80 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.410+0000 7fc71affd640 1 -- 172.21.15.16:0/2006692947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc71c005560 con 0x7fc724104e80 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.410+0000 7fc72e33a740 1 Processor -- start 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 -- start start 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc71affd640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 0x7fc7100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca065c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca072970 0x5607ca074d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607c9fc92f0 con 0x5607ca066c30 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5607c9f70cf0 con 0x5607ca0657e0 2024-09-12T01:05:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc72e33a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607c9fc52b0 con 0x5607ca072970 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc71affd640 1 -- 172.21.15.16:0/2006692947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc71c0cf690 con 0x7fc724104e80 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc7294fc640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 0x7fc7100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55620/0 (socket says 172.21.15.16:55620) 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 -- 172.21.15.16:0/407234647 learned_addr learned my addr 172.21.15.16:0/407234647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc7294fc640 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca065c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 -- 172.21.15.16:0/407234647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca072970 msgr2=0x5607ca074d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca072970 0x5607ca074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 -- 172.21.15.16:0/407234647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 msgr2=0x5607ca065c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca065c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.411+0000 7fc729cfd640 1 -- 172.21.15.16:0/407234647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607c9f553a0 con 0x5607ca066c30 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc7294fc640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 0x7fc7100a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc714009230 tx=0x7fc714031800 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc729cfd640 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc72000bd40 tx=0x7fc72000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a754c820c4381f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc70ffff640 1 -- 172.21.15.16:0/407234647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720015070 con 0x5607ca066c30 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc70ffff640 1 -- 172.21.15.16:0/407234647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72000be30 con 0x5607ca066c30 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc70ffff640 1 -- 172.21.15.16:0/407234647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200119a0 con 0x5607ca066c30 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc72e33a740 1 -- 172.21.15.16:0/407234647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 msgr2=0x5607ca0652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.412+0000 7fc72e33a740 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc72000bd40 tx=0x7fc72000f730 comp rx=0 tx=0).stop 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 -- 172.21.15.16:0/407234647 shutdown_connections 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca072970 0x5607ca074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca065c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 --2- 172.21.15.16:0/407234647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca0652a0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 -- 172.21.15.16:0/407234647 >> 172.21.15.16:0/407234647 conn(0x5607ca02c310 msgr2=0x5607ca061e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 -- 172.21.15.16:0/407234647 shutdown_connections 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 -- 172.21.15.16:0/407234647 wait complete. 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 -- start start 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.413+0000 7fc72e33a740 1 Processor -- start 2024-09-12T01:05:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 0x5607ca06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607ca075540 con 0x5607ca072970 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5607c9f84060 con 0x5607ca0657e0 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607ca0756c0 con 0x5607ca066c30 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 10 client.0 _ll_register_callbacks cb 0x5607c9cd78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55630/0 (socket says 172.21.15.16:55630) 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 -- 172.21.15.16:0/1884178561 learned_addr learned my addr 172.21.15.16:0/1884178561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc7294fc640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 0x5607ca06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 -- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 msgr2=0x5607ca06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 0x5607ca06f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 -- 172.21.15.16:0/1884178561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 msgr2=0x5607ca072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 -- 172.21.15.16:0/1884178561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607c9d3a3c0 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc7294fc640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 0x5607ca06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ccde0 tx=0x7fc71c09c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47633d546b1347c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc70d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c01f070 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc71c0a4330 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.414+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72000be30 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc71c0a47c0 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 10 client.18441 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607ca081010 con 0x5607ca072970 2024-09-12T01:05:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72000be30 con 0x5607ca072970 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc71c00f040 con 0x5607ca072970 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 10 client.18441 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 10 client.18441 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 20 client.18441 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca084830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 4 client.18441 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607c9fd6860 con 0x5607ca082420 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.415+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca084830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.416+0000 7fc70d7fa640 10 client.18441 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.416+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca084830 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95ae2b7ee49cf4ca server_cookie=3f1df06a10aec254 in_seq=0 out_seq=0 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.416+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1884178561 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607c9fd6860 con 0x5607ca082420 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.416+0000 7fc70d7fa640 10 client.18441 handle_command_reply: tid=0 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.416+0000 7fc72e33a740 1 client.18441 shutdown 2024-09-12T01:05:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 msgr2=0x5607ca06fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ccde0 tx=0x7fc71c09c250 comp rx=0 tx=0).stop 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 shutdown_connections 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca084830 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca072970 0x5607ca06fbe0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca066c30 0x5607ca06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc70d7fa640 0 client.18441 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1884178561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca0657e0 0x5607ca072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 >> 172.21.15.16:0/1884178561 conn(0x5607ca02c310 msgr2=0x5607ca067bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 shutdown_connections 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 1 -- 172.21.15.16:0/1884178561 wait complete. 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.417+0000 7fc72e33a740 20 client.18441 trim_cache size 0 max 16384 2024-09-12T01:05:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 Processor -- start 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 -- start start 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca07e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241ab660 con 0x5607ca066c30 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241ad1a0 con 0x5607ca072970 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72e33a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241abd30 con 0x5607ca0657e0 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc7294fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc7294fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55632/0 (socket says 172.21.15.16:55632) 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc7294fc640 1 -- 172.21.15.16:0/3987729319 learned_addr learned my addr 172.21.15.16:0/3987729319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc729cfd640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.430+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca07e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 -- 172.21.15.16:0/3987729319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 msgr2=0x5607ca06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 -- 172.21.15.16:0/3987729319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 msgr2=0x5607ca07e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca07e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 -- 172.21.15.16:0/3987729319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607ca061ff0 con 0x5607ca066c30 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca07e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc7294fc640 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc7140058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464dc1849c820c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc70f7fe640 1 -- 172.21.15.16:0/3987729319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714035040 con 0x5607ca066c30 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc70f7fe640 1 -- 172.21.15.16:0/3987729319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc714003cd0 con 0x5607ca066c30 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.431+0000 7fc70f7fe640 1 -- 172.21.15.16:0/3987729319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71403dc20 con 0x5607ca066c30 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- 172.21.15.16:0/3987729319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 msgr2=0x5607ca06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc7140058d0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- 172.21.15.16:0/3987729319 shutdown_connections 2024-09-12T01:05:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca07e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca06eeb0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- 172.21.15.16:0/3987729319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- 172.21.15.16:0/3987729319 >> 172.21.15.16:0/3987729319 conn(0x5607ca02c310 msgr2=0x5607ca06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- 172.21.15.16:0/3987729319 shutdown_connections 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- 172.21.15.16:0/3987729319 wait complete. 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 -- start start 2024-09-12T01:05:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 Processor -- start 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca07b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.432+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca089ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241acad0 con 0x5607ca066c30 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241ac400 con 0x5607ca072970 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241a9660 con 0x5607ca0657e0 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 10 client.0 _ll_register_callbacks cb 0x7fc71c009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55634/0 (socket says 172.21.15.16:55634) 2024-09-12T01:05:50.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 -- 172.21.15.16:0/1380315456 learned_addr learned my addr 172.21.15.16:0/1380315456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca07b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 -- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 msgr2=0x5607ca07b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca07b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 -- 172.21.15.16:0/1380315456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 msgr2=0x5607ca089ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72a4fe640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca089ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca089ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 -- 172.21.15.16:0/1380315456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607ca08a820 con 0x5607ca066c30 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc7294fc640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ec0 tx=0x7fc71403d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f217803181682d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc70d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140037f0 con 0x5607ca066c30 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fc714003990 con 0x5607ca066c30 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.433+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc714003cd0 con 0x5607ca066c30 2024-09-12T01:05:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc71403d3f0 con 0x5607ca066c30 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 10 client.18447 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607ca0ed0f0 con 0x5607ca066c30 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc714003cd0 con 0x5607ca066c30 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc714065730 con 0x5607ca066c30 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 10 client.18447 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 10 client.18447 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 20 client.18447 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca0ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 4 client.18447 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607c9fd0d70 con 0x5607ca082420 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.434+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca0ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc70d7fa640 10 client.18447 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc729cfd640 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca0ef7d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a5d6a80ac82920d server_cookie=fa728b72027b77fc in_seq=0 out_seq=0 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc70d7fa640 1 -- 172.21.15.16:0/1380315456 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607c9fd0d70 con 0x5607ca082420 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc70d7fa640 10 client.18447 handle_command_reply: tid=0 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 client.18447 shutdown 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 msgr2=0x5607ca089ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ec0 tx=0x7fc71403d760 comp rx=0 tx=0).stop 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 shutdown_connections 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5607ca072970 0x5607ca089ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5607ca082420 0x5607ca0ef7d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607ca066c30 0x5607ca089ab0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 --2- 172.21.15.16:0/1380315456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607ca0657e0 0x5607ca07b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc70d7fa640 0 client.18447 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 >> 172.21.15.16:0/1380315456 conn(0x5607ca02c310 msgr2=0x5607ca0605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 shutdown_connections 2024-09-12T01:05:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 1 -- 172.21.15.16:0/1380315456 wait complete. 2024-09-12T01:05:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.435+0000 7fc72e33a740 20 client.18447 trim_cache size 0 max 16384 2024-09-12T01:05:50.449 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 msgr2=0x7fc7100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 0x7fc7100a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc714009230 tx=0x7fc714031800 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 msgr2=0x7fc72419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00eaa0 tx=0x7fc71c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 shutdown_connections 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7100a2cf0 0x7fc7100a51b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc724104e80 0x7fc72419bc00 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc72406a370 0x7fc724197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 --2- 172.21.15.16:0/2006692947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc724069f00 0x7fc724197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 >> 172.21.15.16:0/2006692947 conn(0x7fc724077210 msgr2=0x7fc724102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 shutdown_connections 2024-09-12T01:05:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:50.436+0000 7fc72bf61640 1 -- 172.21.15.16:0/2006692947 wait complete. 2024-09-12T01:05:50.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:50 smithi159 ceph-mon[35566]: pgmap v300: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:50.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:50 smithi016 ceph-mon[30972]: pgmap v300: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:50 smithi099 ceph-mon[35654]: pgmap v300: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:51.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:05:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.033+0000 7f7fd887f640 1 -- 172.21.15.16:0/104838729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd4103660 msgr2=0x7f7fd410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.033+0000 7f7fd887f640 1 --2- 172.21.15.16:0/104838729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd4103660 0x7f7fd410bb50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009900 tx=0x7f7fbc02a480 comp rx=0 tx=0).stop 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 -- 172.21.15.16:0/104838729 shutdown_connections 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 --2- 172.21.15.16:0/104838729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd410c170 0x7f7fd410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 --2- 172.21.15.16:0/104838729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd4103660 0x7f7fd410bb50 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 --2- 172.21.15.16:0/104838729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4102d40 0x7f7fd4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 -- 172.21.15.16:0/104838729 >> 172.21.15.16:0/104838729 conn(0x7f7fd40fc7a0 msgr2=0x7f7fd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 -- 172.21.15.16:0/104838729 shutdown_connections 2024-09-12T01:05:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 -- 172.21.15.16:0/104838729 wait complete. 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 Processor -- start 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.034+0000 7f7fd887f640 1 -- start start 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 0x7f7fd41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4103660 0x7f7fd4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4111f70 con 0x7f7fd410c170 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4111df0 con 0x7f7fd4102d40 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd887f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41120f0 con 0x7f7fd4103660 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55650/0 (socket says 172.21.15.16:55650) 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/3826157842 learned_addr learned my addr 172.21.15.16:0/3826157842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd259c640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 0x7f7fd41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4103660 0x7f7fd4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/3826157842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4103660 msgr2=0x7f7fd4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4103660 0x7f7fd4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/3826157842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 msgr2=0x7f7fd41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 0x7f7fd41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/3826157842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd419d480 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd259c640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 0x7f7fd41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.035+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00eaa0 tx=0x7f7fcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.036+0000 7f7fcb7fe640 1 -- 172.21.15.16:0/3826157842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00cdb0 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.036+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd419d770 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.036+0000 7f7fcb7fe640 1 -- 172.21.15.16:0/3826157842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc005020 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.036+0000 7f7fcb7fe640 1 -- 172.21.15.16:0/3826157842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0116b0 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.036+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd41a5050 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.038+0000 7f7fcb7fe640 1 -- 172.21.15.16:0/3826157842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7fcc005560 con 0x7f7fd410c170 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.038+0000 7f7fdac58740 1 Processor -- start 2024-09-12T01:05:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.038+0000 7f7fdac58740 1 -- start start 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99ae2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fcb7fe640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 0x7f7fc00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99ae2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c99a482f0 con 0x564c99af1780 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c999efcf0 con 0x564c99ae3b40 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fdac58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c99a442b0 con 0x564c99ae25d0 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd259c640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 0x7f7fc00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55664/0 (socket says 172.21.15.16:55664) 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/2468356181 learned_addr learned my addr 172.21.15.16:0/2468356181 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99ae2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fcb7fe640 1 -- 172.21.15.16:0/3826157842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f7fcc0cf750 con 0x7f7fd410c170 2024-09-12T01:05:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99ae2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/2468356181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 msgr2=0x564c99ae2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99ae2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/2468356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 msgr2=0x564c99ae2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99ae2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/2468356181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c999d43a0 con 0x564c99af1780 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99ae2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd259c640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 0x7f7fc00a51b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40058c0 tx=0x7f7fc4005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.039+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00efd0 tx=0x7f7fcc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b5834d68a9574b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fc8ff9640 1 -- 172.21.15.16:0/2468356181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc01f070 con 0x564c99af1780 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fc8ff9640 1 -- 172.21.15.16:0/2468356181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc09bc10 con 0x564c99af1780 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fc8ff9640 1 -- 172.21.15.16:0/2468356181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0b51d0 con 0x564c99af1780 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 -- 172.21.15.16:0/2468356181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 msgr2=0x564c99af3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00efd0 tx=0x7f7fcc0058d0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 -- 172.21.15.16:0/2468356181 shutdown_connections 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99af3b70 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99ae2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2468356181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99ae2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.040+0000 7f7fdac58740 1 -- 172.21.15.16:0/2468356181 >> 172.21.15.16:0/2468356181 conn(0x564c99aab310 msgr2=0x564c99adec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- 172.21.15.16:0/2468356181 shutdown_connections 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- 172.21.15.16:0/2468356181 wait complete. 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- start start 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 Processor -- start 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aee450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99af1780 0x564c99aeea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c99af43c0 con 0x564c99ae25d0 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c99a03060 con 0x564c99ae3b40 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c99af4540 con 0x564c99af1780 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fdac58740 10 client.0 _ll_register_callbacks cb 0x564c997568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fd259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fd259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55680/0 (socket says 172.21.15.16:55680) 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fd259c640 1 -- 172.21.15.16:0/4086607315 learned_addr learned my addr 172.21.15.16:0/4086607315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.041+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99af1780 0x564c99aeea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 -- 172.21.15.16:0/4086607315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99af1780 msgr2=0x564c99aeea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99af1780 0x564c99aeea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 -- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 msgr2=0x564c99aee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 -- 172.21.15.16:0/4086607315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c997b93c0 con 0x564c99ae25d0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fd259c640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40116c0 tx=0x7f7fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8477e91ea6f651bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4022070 con 0x564c99ae25d0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f7fc401e860 con 0x564c99ae25d0 2024-09-12T01:05:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc09bc10 con 0x564c99ae25d0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fc402e050 con 0x564c99ae25d0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fdac58740 10 client.18459 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c99affef0 con 0x564c99ae25d0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.042+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc09bc10 con 0x564c99ae25d0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fc40263b0 con 0x564c99ae25d0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 10 client.18459 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 10 client.18459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 20 client.18459 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b01300 0x564c99b03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 4 client.18459 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c99a55860 con 0x564c99b01300 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b01300 0x564c99b03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fb9ffb640 10 client.18459 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.043+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b01300 0x564c99b03750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6fa0682359e26b server_cookie=adef22e825ff8eaa in_seq=0 out_seq=0 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.044+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/4086607315 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c99a55860 con 0x564c99b01300 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.044+0000 7f7fb9ffb640 10 client.18459 handle_command_reply: tid=0 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.044+0000 7f7fdac58740 1 client.18459 shutdown 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 msgr2=0x564c99af11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40116c0 tx=0x7f7fc40046c0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 shutdown_connections 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b01300 0x564c99b03750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99af1780 0x564c99aeea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 --2- 172.21.15.16:0/4086607315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af11a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fb9ffb640 0 client.18459 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 >> 172.21.15.16:0/4086607315 conn(0x564c99aab310 msgr2=0x564c99ae4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 shutdown_connections 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 1 -- 172.21.15.16:0/4086607315 wait complete. 2024-09-12T01:05:52.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.045+0000 7f7fdac58740 20 client.18459 trim_cache size 0 max 16384 2024-09-12T01:05:52.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 Processor -- start 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 -- start start 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99afd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41ab6f0 con 0x564c99af1780 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41ad230 con 0x564c99ae3b40 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fdac58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41abdc0 con 0x564c99ae25d0 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55686/0 (socket says 172.21.15.16:55686) 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/1387097032 learned_addr learned my addr 172.21.15.16:0/1387097032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd259c640 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99afd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/1387097032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 msgr2=0x564c99afd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.058+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99afd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/1387097032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 msgr2=0x564c99aed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fd2d9d640 1 -- 172.21.15.16:0/1387097032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c99aeedd0 con 0x564c99af1780 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc099ec0 tx=0x7f7fcc097fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c45abf4d57f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fbbfff640 1 -- 172.21.15.16:0/1387097032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc01f070 con 0x564c99af1780 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fbbfff640 1 -- 172.21.15.16:0/1387097032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc0a4880 con 0x564c99af1780 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fbbfff640 1 -- 172.21.15.16:0/1387097032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc076290 con 0x564c99af1780 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fdac58740 1 -- 172.21.15.16:0/1387097032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 msgr2=0x564c99b09070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.059+0000 7f7fdac58740 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc099ec0 tx=0x7f7fcc097fe0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- 172.21.15.16:0/1387097032 shutdown_connections 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99af1780 0x564c99b09070 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99ae3b40 0x564c99aed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- 172.21.15.16:0/1387097032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae25d0 0x564c99afd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- 172.21.15.16:0/1387097032 >> 172.21.15.16:0/1387097032 conn(0x564c99aab310 msgr2=0x564c99aecd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- 172.21.15.16:0/1387097032 shutdown_connections 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- 172.21.15.16:0/1387097032 wait complete. 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- start start 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 Processor -- start 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae3b40 0x564c99af4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 0x564c99af54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41acb60 con 0x564c99ae25d0 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41ac490 con 0x564c99af1780 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41a96f0 con 0x564c99ae3b40 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fdac58740 10 client.0 _ll_register_callbacks cb 0x7f7fc4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.060+0000 7f7fd259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55694/0 (socket says 172.21.15.16:55694) 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 -- 172.21.15.16:0/2152009912 learned_addr learned my addr 172.21.15.16:0/2152009912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 0x564c99af54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae3b40 0x564c99af4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 -- 172.21.15.16:0/2152009912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae3b40 msgr2=0x564c99af4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae3b40 0x564c99af4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 -- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 msgr2=0x564c99af54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 0x564c99af54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 -- 172.21.15.16:0/2152009912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c99af5a20 con 0x564c99ae25d0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd2d9d640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 0x564c99af54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fd259c640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4026940 tx=0x7f7fc400fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7831b3622d69147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4022070 con 0x564c99ae25d0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f7fc40269c0 con 0x564c99ae25d0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc0a4880 con 0x564c99ae25d0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.061+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fc4041050 con 0x564c99ae25d0 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 10 client.18465 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c99b6bdf0 con 0x564c99ae25d0 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc0a4880 con 0x564c99ae25d0 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fc4004a00 con 0x564c99ae25d0 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 10 client.18465 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 10 client.18465 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 20 client.18465 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b6c5c0 0x564c99b6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 4 client.18465 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c99a4fd70 con 0x564c99b6c5c0 2024-09-12T01:05:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.062+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b6c5c0 0x564c99b6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fb9ffb640 10 client.18465 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fd1d9b640 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b6c5c0 0x564c99b6e9e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=700dd13f7616c9e server_cookie=65ef223255a12941 in_seq=0 out_seq=0 2024-09-12T01:05:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fb9ffb640 1 -- 172.21.15.16:0/2152009912 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c99a4fd70 con 0x564c99b6c5c0 2024-09-12T01:05:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fb9ffb640 10 client.18465 handle_command_reply: tid=0 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 client.18465 shutdown 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 msgr2=0x564c99af9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4026940 tx=0x7f7fc400fcc0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 shutdown_connections 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x564c99b6c5c0 0x564c99b6e9e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c99af1780 0x564c99af54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fb9ffb640 0 client.18465 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c99ae3b40 0x564c99af4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 --2- 172.21.15.16:0/2152009912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c99ae25d0 0x564c99af9c70 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 >> 172.21.15.16:0/2152009912 conn(0x564c99aab310 msgr2=0x564c99aed540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 shutdown_connections 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 1 -- 172.21.15.16:0/2152009912 wait complete. 2024-09-12T01:05:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.063+0000 7f7fdac58740 20 client.18465 trim_cache size 0 max 16384 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 msgr2=0x7f7fc00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 0x7f7fc00a51b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40058c0 tx=0x7f7fc4005850 comp rx=0 tx=0).stop 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 msgr2=0x7f7fd419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00eaa0 tx=0x7f7fcc00ef60 comp rx=0 tx=0).stop 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 shutdown_connections 2024-09-12T01:05:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7fc00a2cf0 0x7f7fc00a51b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd410c170 0x7f7fd419cda0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd4103660 0x7f7fd4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 --2- 172.21.15.16:0/3826157842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fd4102d40 0x7f7fd41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 >> 172.21.15.16:0/3826157842 conn(0x7f7fd40fc7a0 msgr2=0x7f7fd40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 shutdown_connections 2024-09-12T01:05:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:52.064+0000 7f7fd887f640 1 -- 172.21.15.16:0/3826157842 wait complete. 2024-09-12T01:05:52.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:52 smithi159 ceph-mon[35566]: pgmap v301: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:52.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:52 smithi016 ceph-mon[30972]: pgmap v301: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:52.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:05:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:52 smithi099 ceph-mon[35654]: pgmap v301: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:53.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 -- 172.21.15.16:0/2329179186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc102790 msgr2=0x7fa4dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 --2- 172.21.15.16:0/2329179186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc102790 0x7fa4dc1067e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc02a410 comp rx=0 tx=0).stop 2024-09-12T01:05:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 -- 172.21.15.16:0/2329179186 shutdown_connections 2024-09-12T01:05:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 --2- 172.21.15.16:0/2329179186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc106f10 0x7fa4dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 --2- 172.21.15.16:0/2329179186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc102790 0x7fa4dc1067e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 --2- 172.21.15.16:0/2329179186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc101de0 0x7fa4dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 -- 172.21.15.16:0/2329179186 >> 172.21.15.16:0/2329179186 conn(0x7fa4dc0fd8c0 msgr2=0x7fa4dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 -- 172.21.15.16:0/2329179186 shutdown_connections 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.684+0000 7fa4e4533640 1 -- 172.21.15.16:0/2329179186 wait complete. 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 Processor -- start 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 -- start start 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc101de0 0x7fa4dc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 0x7fa4dc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc111f70 con 0x7fa4dc106f10 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc111df0 con 0x7fa4dc102790 2024-09-12T01:05:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e4533640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1120f0 con 0x7fa4dc101de0 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e2ad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e2ad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55704/0 (socket says 172.21.15.16:55704) 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e2ad0640 1 -- 172.21.15.16:0/1522610395 learned_addr learned my addr 172.21.15.16:0/1522610395 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.685+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc101de0 0x7fa4dc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 0x7fa4dc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 -- 172.21.15.16:0/1522610395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc101de0 msgr2=0x7fa4dc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc101de0 0x7fa4dc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 -- 172.21.15.16:0/1522610395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 msgr2=0x7fa4dc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 0x7fa4dc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 -- 172.21.15.16:0/1522610395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19d440 con 0x7fa4dc106f10 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 0x7fa4dc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e2ad0640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eaa0 tx=0x7fa4d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4d37fe640 1 -- 172.21.15.16:0/1522610395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400cdb0 con 0x7fa4dc106f10 2024-09-12T01:05:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4d37fe640 1 -- 172.21.15.16:0/1522610395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4005020 con 0x7fa4dc106f10 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19d730 con 0x7fa4dc106f10 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4d37fe640 1 -- 172.21.15.16:0/1522610395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40116b0 con 0x7fa4dc106f10 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.686+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a5010 con 0x7fa4dc106f10 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.688+0000 7fa4d37fe640 1 -- 172.21.15.16:0/1522610395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa4d4005560 con 0x7fa4dc106f10 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 Processor -- start 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 -- start start 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b871b40 0x563b9b870090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b870a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b9b7d62f0 con 0x563b9b87f780 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563b9b77dcf0 con 0x563b9b8705d0 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e690c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b9b7d22b0 con 0x563b9b871b40 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4d37fe640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 0x7fa4c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55710/0 (socket says 172.21.15.16:55710) 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2836258039 learned_addr learned my addr 172.21.15.16:0/2836258039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.689+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b871b40 0x563b9b870090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e2ad0640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 0x7fa4c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4d37fe640 1 -- 172.21.15.16:0/1522610395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fa4d40cf590 con 0x7fa4dc106f10 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b870a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2836258039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b871b40 msgr2=0x563b9b870090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b871b40 0x563b9b870090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2836258039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 msgr2=0x563b9b870a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b870a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2836258039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9b7623a0 con 0x563b9b87f780 2024-09-12T01:05:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4dab546b2284ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4e2ad0640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 0x7fa4c80a5230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40ccd90 tx=0x7fa4d40054d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4d0ff9640 1 -- 172.21.15.16:0/2836258039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc035040 con 0x563b9b87f780 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4d0ff9640 1 -- 172.21.15.16:0/2836258039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc005020 con 0x563b9b87f780 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.690+0000 7fa4d0ff9640 1 -- 172.21.15.16:0/2836258039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0048c0 con 0x563b9b87f780 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 -- 172.21.15.16:0/2836258039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 msgr2=0x563b9b881b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009900 tx=0x7fa4cc005b70 comp rx=0 tx=0).stop 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 -- 172.21.15.16:0/2836258039 shutdown_connections 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b881b70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b870a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2836258039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b871b40 0x563b9b870090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.691+0000 7fa4e690c740 1 -- 172.21.15.16:0/2836258039 >> 172.21.15.16:0/2836258039 conn(0x563b9b839310 msgr2=0x563b9b86cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- 172.21.15.16:0/2836258039 shutdown_connections 2024-09-12T01:05:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- 172.21.15.16:0/2836258039 wait complete. 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- start start 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 Processor -- start 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b87c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b87cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b9b882420 con 0x563b9b8705d0 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563b9b791060 con 0x563b9b871b40 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b9b8825a0 con 0x563b9b87f780 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e690c740 10 client.0 _ll_register_callbacks cb 0x563b9b4e48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:53.746 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.692+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55720/0 (socket says 172.21.15.16:55720) 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2930084303 learned_addr learned my addr 172.21.15.16:0/2930084303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b87c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b87cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 msgr2=0x563b9b87cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b87cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2930084303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 msgr2=0x563b9b87c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b87c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 -- 172.21.15.16:0/2930084303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9b5473c0 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc033000 tx=0x7fa4cc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a58371f21b61ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc035040 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fa4cc0046c0 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4cc005020 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4cc0048c0 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e690c740 10 client.18477 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.693+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b9b88df30 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc005020 con 0x563b9b8705d0 2024-09-12T01:05:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4cc03a040 con 0x563b9b8705d0 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 10 client.18477 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 10 client.18477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 20 client.18477 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b88f280 0x563b9b891670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 4 client.18477 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b9b7e3860 con 0x563b9b88f280 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b88f280 0x563b9b891670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4c5ffb640 10 client.18477 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.694+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b88f280 0x563b9b891670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64e979aea4c327b2 server_cookie=4de24c08d053ec17 in_seq=0 out_seq=0 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2930084303 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b9b7e3860 con 0x563b9b88f280 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4c5ffb640 10 client.18477 handle_command_reply: tid=0 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 client.18477 shutdown 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 msgr2=0x563b9b87f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc033000 tx=0x7fa4cc005a20 comp rx=0 tx=0).stop 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 shutdown_connections 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b87cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b88f280 0x563b9b891670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b87c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2930084303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b8705d0 0x563b9b87f1d0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4c5ffb640 0 client.18477 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.695+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 >> 172.21.15.16:0/2930084303 conn(0x563b9b839310 msgr2=0x563b9b872960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.696+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 shutdown_connections 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.696+0000 7fa4e690c740 1 -- 172.21.15.16:0/2930084303 wait complete. 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.696+0000 7fa4e690c740 20 client.18477 trim_cache size 0 max 16384 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 Processor -- start 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 -- start start 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b8705d0 0x563b9b87b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b86c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1ab680 con 0x563b9b87f780 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1ad1c0 con 0x563b9b871b40 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e690c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1abd50 con 0x563b9b8705d0 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e1ace640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55734/0 (socket says 172.21.15.16:55734) 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e1ace640 1 -- 172.21.15.16:0/3350307285 learned_addr learned my addr 172.21.15.16:0/3350307285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.709+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b8705d0 0x563b9b87b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b86c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 -- 172.21.15.16:0/3350307285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b8705d0 msgr2=0x563b9b87b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b8705d0 0x563b9b87b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 -- 172.21.15.16:0/3350307285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 msgr2=0x563b9b86c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b86c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 -- 172.21.15.16:0/3350307285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9b86d1a0 con 0x563b9b87f780 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b86c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc057ed0 tx=0x7fa4cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef75387633ca7bde server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4c7fff640 1 -- 172.21.15.16:0/3350307285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc031880 con 0x563b9b87f780 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4c7fff640 1 -- 172.21.15.16:0/3350307285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc0053e0 con 0x563b9b87f780 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.710+0000 7fa4c7fff640 1 -- 172.21.15.16:0/3350307285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc004a60 con 0x563b9b87f780 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- 172.21.15.16:0/3350307285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 msgr2=0x563b9b86ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc057ed0 tx=0x7fa4cc000bc0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- 172.21.15.16:0/3350307285 shutdown_connections 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b87f780 0x563b9b86ca30 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b871b40 0x563b9b86c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- 172.21.15.16:0/3350307285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b8705d0 0x563b9b87b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- 172.21.15.16:0/3350307285 >> 172.21.15.16:0/3350307285 conn(0x563b9b839310 msgr2=0x563b9b87ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- 172.21.15.16:0/3350307285 shutdown_connections 2024-09-12T01:05:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- 172.21.15.16:0/3350307285 wait complete. 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 -- start start 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 Processor -- start 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b891370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.711+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b8f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1acaf0 con 0x563b9b871b40 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1ac420 con 0x563b9b8705d0 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a9680 con 0x563b9b87f780 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 10 client.0 _ll_register_callbacks cb 0x7fa4cc032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55744/0 (socket says 172.21.15.16:55744) 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 -- 172.21.15.16:0/2578509781 learned_addr learned my addr 172.21.15.16:0/2578509781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b8f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b891370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 -- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 msgr2=0x563b9b8f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b8f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 -- 172.21.15.16:0/2578509781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 msgr2=0x563b9b891370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b891370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 -- 172.21.15.16:0/2578509781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9b892190 con 0x563b9b871b40 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b8f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e22cf640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000c00 tx=0x7fa4d800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac465ee5cf24564 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8022030 con 0x563b9b871b40 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fa4d8003cb0 con 0x563b9b871b40 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.712+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4cc0053e0 con 0x563b9b871b40 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4d802c050 con 0x563b9b871b40 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 10 client.18483 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b9b8f91d0 con 0x563b9b871b40 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4cc0053e0 con 0x563b9b871b40 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4d80115e0 con 0x563b9b871b40 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 10 client.18483 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 10 client.18483 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 20 client.18483 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b8f9980 0x563b9b8fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 4 client.18483 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b9b7ddd70 con 0x563b9b8f9980 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.713+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b8f9980 0x563b9b8fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e1ace640 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b8f9980 0x563b9b8fbda0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2187e1feb9ed34c2 server_cookie=86a0550bf5e00f1b in_seq=0 out_seq=0 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4c5ffb640 10 client.18483 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4c5ffb640 1 -- 172.21.15.16:0/2578509781 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b9b7ddd70 con 0x563b9b8f9980 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4c5ffb640 10 client.18483 handle_command_reply: tid=0 2024-09-12T01:05:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 client.18483 shutdown 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 msgr2=0x563b9b8918b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000c00 tx=0x7fa4d800bd60 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 shutdown_connections 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b9b87f780 0x563b9b8f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b9b871b40 0x563b9b8918b0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x563b9b8f9980 0x563b9b8fbda0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 --2- 172.21.15.16:0/2578509781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563b9b8705d0 0x563b9b891370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4c5ffb640 0 client.18483 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 >> 172.21.15.16:0/2578509781 conn(0x563b9b839310 msgr2=0x563b9b87b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 shutdown_connections 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 1 -- 172.21.15.16:0/2578509781 wait complete. 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.714+0000 7fa4e690c740 20 client.18483 trim_cache size 0 max 16384 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 msgr2=0x7fa4c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 0x7fa4c80a5230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40ccd90 tx=0x7fa4d40054d0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 msgr2=0x7fa4dc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eaa0 tx=0x7fa4d400ef60 comp rx=0 tx=0).stop 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 shutdown_connections 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa4c80a2d70 0x7fa4c80a5230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4dc106f10 0x7fa4dc19cd60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4dc102790 0x7fa4dc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 --2- 172.21.15.16:0/1522610395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc101de0 0x7fa4dc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 >> 172.21.15.16:0/1522610395 conn(0x7fa4dc0fd8c0 msgr2=0x7fa4dc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 shutdown_connections 2024-09-12T01:05:53.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:53.715+0000 7fa4e4533640 1 -- 172.21.15.16:0/1522610395 wait complete. 2024-09-12T01:05:54.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:05:54.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:54 smithi016 ceph-mon[30972]: pgmap v302: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:54.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:54 smithi159 ceph-mon[35566]: pgmap v302: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:54 smithi099 ceph-mon[35654]: pgmap v302: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.152+0000 7f86b4d9f640 1 -- 172.21.15.16:0/2244977440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 msgr2=0x7f86b0107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.152+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/2244977440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b0107ed0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009930 tx=0x7f86a002a420 comp rx=0 tx=0).stop 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 -- 172.21.15.16:0/2244977440 shutdown_connections 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/2244977440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/2244977440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 0x7f86b0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/2244977440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b0107ed0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 -- 172.21.15.16:0/2244977440 >> 172.21.15.16:0/2244977440 conn(0x7f86b00fd700 msgr2=0x7f86b00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 -- 172.21.15.16:0/2244977440 shutdown_connections 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 -- 172.21.15.16:0/2244977440 wait complete. 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 Processor -- start 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.153+0000 7f86b4d9f640 1 -- start start 2024-09-12T01:05:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 0x7f86b019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0111ef0 con 0x7f86b0107af0 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0111d70 con 0x7f86b01019e0 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86b4d9f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86b0112070 con 0x7f86b0102380 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55764/0 (socket says 172.21.15.16:55764) 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86affff640 1 -- 172.21.15.16:0/1202853551 learned_addr learned my addr 172.21.15.16:0/1202853551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86af7fe640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 0x7f86b019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1202853551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 msgr2=0x7f86b019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 0x7f86b019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 msgr2=0x7f86b019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.154+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1202853551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b019f560 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b019aaf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8698005fd0 tx=0x7f869800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86ad7fa640 1 -- 172.21.15.16:0/1202853551 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698009c30 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b019f850 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86ad7fa640 1 -- 172.21.15.16:0/1202853551 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8698009dd0 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86ad7fa640 1 -- 172.21.15.16:0/1202853551 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698010460 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.155+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b01a7130 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.157+0000 7f86ad7fa640 1 -- 172.21.15.16:0/1202853551 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8698003ef0 con 0x7f86b0102380 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 Processor -- start 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 -- start start 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86ad7fa640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 0x7f869c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd2713fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2714a550 0x55bd2714c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd270a52f0 con 0x55bd2714a550 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd2704ccf0 con 0x55bd2713f7b0 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86b83da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd270a12b0 con 0x55bd27140d20 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86ad7fa640 1 -- 172.21.15.16:0/1202853551 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f86980ceff0 con 0x7f86b0102380 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86affff640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 0x7f869c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:40970/0 (socket says 172.21.15.16:40970) 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 -- 172.21.15.16:0/702517546 learned_addr learned my addr 172.21.15.16:0/702517546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd2713fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86affff640 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2714a550 0x55bd2714c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 -- 172.21.15.16:0/702517546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 msgr2=0x55bd2713fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd2713fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 -- 172.21.15.16:0/702517546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2714a550 msgr2=0x55bd2714c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2714a550 0x55bd2714c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.158+0000 7f86af7fe640 1 -- 172.21.15.16:0/702517546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd270313a0 con 0x55bd27140d20 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.159+0000 7f86affff640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 0x7f869c0a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86a80049d0 tx=0x7f86a8013ba0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.159+0000 7f86af7fe640 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86a002a940 tx=0x7f86a0005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8465a64c49ba100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.159+0000 7f86a5ffb640 1 -- 172.21.15.16:0/702517546 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0040070 con 0x55bd27140d20 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.159+0000 7f86a5ffb640 1 -- 172.21.15.16:0/702517546 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a00318f0 con 0x55bd27140d20 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.159+0000 7f86a5ffb640 1 -- 172.21.15.16:0/702517546 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0035040 con 0x55bd27140d20 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- 172.21.15.16:0/702517546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 msgr2=0x55bd2713f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86a002a940 tx=0x7f86a0005c00 comp rx=0 tx=0).stop 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- 172.21.15.16:0/702517546 shutdown_connections 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd2714a550 0x55bd2714c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd2713fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 --2- 172.21.15.16:0/702517546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd27140d20 0x55bd2713f270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- 172.21.15.16:0/702517546 >> 172.21.15.16:0/702517546 conn(0x55bd27108310 msgr2=0x55bd2713be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- 172.21.15.16:0/702517546 shutdown_connections 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- 172.21.15.16:0/702517546 wait complete. 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 -- start start 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.160+0000 7f86b83da740 1 Processor -- start 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27149e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27147640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd2714d1f0 con 0x55bd27140d20 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bd27060060 con 0x55bd2714a550 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd2714d370 con 0x55bd2713f7b0 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86b83da740 10 client.0 _ll_register_callbacks cb 0x55bd26db38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55792/0 (socket says 172.21.15.16:55792) 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 -- 172.21.15.16:0/867061179 learned_addr learned my addr 172.21.15.16:0/867061179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86affff640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 -- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 msgr2=0x55bd27149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 -- 172.21.15.16:0/867061179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 msgr2=0x55bd27147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 -- 172.21.15.16:0/867061179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd26e163c0 con 0x55bd27140d20 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86af7fe640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f869809b4f0 tx=0x7f869809b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffcbfbc952f9349 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f868effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.161+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698020070 con 0x55bd27140d20 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f8698097bd0 con 0x55bd27140d20 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a00318f0 con 0x55bd27140d20 2024-09-12T01:05:55.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8698013680 con 0x55bd27140d20 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 10 client.18495 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd27158ba0 con 0x55bd27140d20 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a00318f0 con 0x55bd27140d20 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86980b42c0 con 0x55bd27140d20 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 10 client.18495 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 20 client.18495 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd2715a020 0x55bd2715c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 4 client.18495 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd270b2860 con 0x55bd2715a020 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.162+0000 7f86af7fe640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd2715a020 0x55bd2715c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.163+0000 7f868effd640 10 client.18495 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:55.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.163+0000 7f86af7fe640 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd2715a020 0x55bd2715c4c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcb7a2850557211 server_cookie=5c94479aa382695e in_seq=0 out_seq=0 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.163+0000 7f868effd640 1 -- 172.21.15.16:0/867061179 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd270b2860 con 0x55bd2715a020 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.163+0000 7f868effd640 10 client.18495 handle_command_reply: tid=0 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 client.18495 shutdown 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 msgr2=0x55bd27147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f869809b4f0 tx=0x7f869809b1a0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 shutdown_connections 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd2715a020 0x55bd2715c4c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27147100 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f868effd640 0 client.18495 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 --2- 172.21.15.16:0/867061179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 >> 172.21.15.16:0/867061179 conn(0x55bd27108310 msgr2=0x55bd27141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 shutdown_connections 2024-09-12T01:05:55.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 1 -- 172.21.15.16:0/867061179 wait complete. 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.164+0000 7f86b83da740 20 client.18495 trim_cache size 0 max 16384 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.177+0000 7f86b83da740 1 Processor -- start 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.177+0000 7f86b83da740 1 -- start start 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27156490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27142840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01af8b0 con 0x55bd27140d20 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b13f0 con 0x55bd2714a550 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86b83da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01aff80 con 0x55bd2713f7b0 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55808/0 (socket says 172.21.15.16:55808) 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86af7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1547589417 learned_addr learned my addr 172.21.15.16:0/1547589417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86affff640 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1547589417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 msgr2=0x55bd27156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1547589417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 msgr2=0x55bd27142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 -- 172.21.15.16:0/1547589417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd2713cb40 con 0x55bd27140d20 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86af7fe640 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8698097f30 tx=0x7f8698003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684a96a0d0a55f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a57fa640 1 -- 172.21.15.16:0/1547589417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698020070 con 0x55bd27140d20 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a57fa640 1 -- 172.21.15.16:0/1547589417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8698013e50 con 0x55bd27140d20 2024-09-12T01:05:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.178+0000 7f86a57fa640 1 -- 172.21.15.16:0/1547589417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698097bd0 con 0x55bd27140d20 2024-09-12T01:05:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 -- 172.21.15.16:0/1547589417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 msgr2=0x55bd27142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8698097f30 tx=0x7f8698003950 comp rx=0 tx=0).stop 2024-09-12T01:05:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 -- 172.21.15.16:0/1547589417 shutdown_connections 2024-09-12T01:05:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2714a550 0x55bd27142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27142300 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 --2- 172.21.15.16:0/1547589417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2713f7b0 0x55bd27156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 -- 172.21.15.16:0/1547589417 >> 172.21.15.16:0/1547589417 conn(0x55bd27108310 msgr2=0x55bd271461f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 -- 172.21.15.16:0/1547589417 shutdown_connections 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.179+0000 7f86b83da740 1 -- 172.21.15.16:0/1547589417 wait complete. 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 -- start start 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 Processor -- start 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd271527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2714a550 0x55bd27143910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b0d20 con 0x55bd27140d20 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01b0650 con 0x55bd2713f7b0 2024-09-12T01:05:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86b01ad8b0 con 0x55bd2714a550 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86b83da740 10 client.0 _ll_register_callbacks cb 0x7f869809b330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86affff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2714a550 0x55bd27143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55810/0 (socket says 172.21.15.16:55810) 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86a6ffd640 1 -- 172.21.15.16:0/2748525726 learned_addr learned my addr 172.21.15.16:0/2748525726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.180+0000 7f86af7fe640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd271527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 -- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2714a550 msgr2=0x55bd27143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2714a550 0x55bd27143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 -- 172.21.15.16:0/2748525726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 msgr2=0x55bd271527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd271527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 -- 172.21.15.16:0/2748525726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd27144110 con 0x55bd27140d20 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86a6ffd640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8698009610 tx=0x7f869809b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb0d623c6b306a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f868effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698020030 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f8698097bd0 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8698013e50 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f869809f860 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86b83da740 10 client.18501 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd27144110 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.181+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8698013e50 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f869801c970 con 0x55bd27140d20 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 10 client.18501 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 20 client.18501 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd271c4af0 0x55bd271c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 4 client.18501 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd270acd70 con 0x55bd271c4af0 2024-09-12T01:05:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86af7fe640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd271c4af0 0x55bd271c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86af7fe640 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd271c4af0 0x55bd271c6f10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83d9327441c8a18 server_cookie=1cdd54421a18c1fc in_seq=0 out_seq=0 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f868effd640 10 client.18501 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f868effd640 1 -- 172.21.15.16:0/2748525726 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd270acd70 con 0x55bd271c4af0 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f868effd640 10 client.18501 handle_command_reply: tid=0 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.182+0000 7f86b83da740 1 client.18501 shutdown 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 msgr2=0x55bd27152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f8698009610 tx=0x7f869809b130 comp rx=0 tx=0).stop 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 shutdown_connections 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bd271c4af0 0x55bd271c6f10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd2714a550 0x55bd27143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd27140d20 0x55bd27152d20 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 --2- 172.21.15.16:0/2748525726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bd2713f7b0 0x55bd271527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f868effd640 0 client.18501 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 >> 172.21.15.16:0/2748525726 conn(0x55bd27108310 msgr2=0x55bd27148960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 shutdown_connections 2024-09-12T01:05:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 1 -- 172.21.15.16:0/2748525726 wait complete. 2024-09-12T01:05:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b83da740 20 client.18501 trim_cache size 0 max 16384 2024-09-12T01:05:55.195 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:05:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 msgr2=0x7f869c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 0x7f869c0a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86a80049d0 tx=0x7f86a8013ba0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 msgr2=0x7f86b019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b019aaf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8698005fd0 tx=0x7f869800dde0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 shutdown_connections 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f869c0a2cf0 0x7f869c0a51b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.183+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0107af0 0x7f86b019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.184+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86b0102380 0x7f86b019aaf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.184+0000 7f86b4d9f640 1 --2- 172.21.15.16:0/1202853551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86b01019e0 0x7f86b019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.184+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 >> 172.21.15.16:0/1202853551 conn(0x7f86b00fd700 msgr2=0x7f86b010d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.184+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 shutdown_connections 2024-09-12T01:05:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:55.184+0000 7f86b4d9f640 1 -- 172.21.15.16:0/1202853551 wait complete. 2024-09-12T01:05:56.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:56 smithi159 ceph-mon[35566]: pgmap v303: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:56 smithi016 ceph-mon[30972]: pgmap v303: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:56.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:05:56.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:05:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:56 smithi099 ceph-mon[35654]: pgmap v303: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 67 KiB/s wr, 19 op/s 2024-09-12T01:05:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 -- 172.21.15.16:0/665819744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd81055b0 msgr2=0x7fbdd810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 --2- 172.21.15.16:0/665819744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd81055b0 0x7fbdd810d130 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000adb0 tx=0x7fbdd002b7a0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 -- 172.21.15.16:0/665819744 shutdown_connections 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 --2- 172.21.15.16:0/665819744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd81055b0 0x7fbdd810d130 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 --2- 172.21.15.16:0/665819744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 0x7fbdd8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 --2- 172.21.15.16:0/665819744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd8100480 0x7fbdd8100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 -- 172.21.15.16:0/665819744 >> 172.21.15.16:0/665819744 conn(0x7fbdd8077210 msgr2=0x7fbdd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 -- 172.21.15.16:0/665819744 shutdown_connections 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.697+0000 7fbddd41a640 1 -- 172.21.15.16:0/665819744 wait complete. 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 Processor -- start 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 -- start start 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 0x7fbdd81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 0x7fbdd819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd810fdb0 con 0x7fbdd8100480 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd810fc30 con 0x7fbdd8100e30 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.698+0000 7fbddd41a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd810ff30 con 0x7fbdd81055b0 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55828/0 (socket says 172.21.15.16:55828) 2024-09-12T01:05:57.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/2981752372 learned_addr learned my addr 172.21.15.16:0/2981752372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 0x7fbdd81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 0x7fbdd819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/2981752372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 msgr2=0x7fbdd819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 0x7fbdd819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/2981752372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 msgr2=0x7fbdd81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 0x7fbdd81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 -- 172.21.15.16:0/2981752372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd819d430 con 0x7fbdd8100480 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 0x7fbdd819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b4a0 tx=0x7fbdc800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdbffff640 1 -- 172.21.15.16:0/2981752372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8014070 con 0x7fbdd8100480 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd819d720 con 0x7fbdd8100480 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdbffff640 1 -- 172.21.15.16:0/2981752372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8010af0 con 0x7fbdd8100480 2024-09-12T01:05:57.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbdbffff640 1 -- 172.21.15.16:0/2981752372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8017d10 con 0x7fbdd8100480 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.699+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd81a5000 con 0x7fbdd8100480 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdbffff640 1 -- 172.21.15.16:0/2981752372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbdc8017070 con 0x7fbdd8100480 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 Processor -- start 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 -- start start 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d6df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6e480 0x5594c5d78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdbffff640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 0x7fbdc00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594c5cd3280 con 0x5594c5d78d40 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5594c5c7ab10 con 0x5594c5d6db60 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594c5ccf240 con 0x5594c5d6e480 2024-09-12T01:05:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55830/0 (socket says 172.21.15.16:55830) 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdd67fc640 1 -- 172.21.15.16:0/597872200 learned_addr learned my addr 172.21.15.16:0/597872200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdbffff640 1 -- 172.21.15.16:0/2981752372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbdc80ce820 con 0x7fbdd8100480 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.702+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 0x7fbdc00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6e480 0x5594c5d78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d6df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 -- 172.21.15.16:0/597872200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6e480 msgr2=0x5594c5d78800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6e480 0x5594c5d78800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 -- 172.21.15.16:0/597872200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 msgr2=0x5594c5d6df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d6df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 -- 172.21.15.16:0/597872200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c5c871f0 con 0x5594c5d78d40 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:57.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00cc40 tx=0x7fbdcc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccb259bc6fa1f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 0x7fbdc00a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005e30 tx=0x7fbdd0005d80 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdbd7fa640 1 -- 172.21.15.16:0/597872200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc014070 con 0x5594c5d78d40 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdbd7fa640 1 -- 172.21.15.16:0/597872200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc0047a0 con 0x5594c5d78d40 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.703+0000 7fbdbd7fa640 1 -- 172.21.15.16:0/597872200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc010760 con 0x5594c5d78d40 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- 172.21.15.16:0/597872200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 msgr2=0x5594c5d7b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00cc40 tx=0x7fbdcc00e720 comp rx=0 tx=0).stop 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- 172.21.15.16:0/597872200 shutdown_connections 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d78d40 0x5594c5d7b130 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6e480 0x5594c5d78800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/597872200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- 172.21.15.16:0/597872200 >> 172.21.15.16:0/597872200 conn(0x5594c5d6a0f0 msgr2=0x5594c5d6a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- 172.21.15.16:0/597872200 shutdown_connections 2024-09-12T01:05:57.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- 172.21.15.16:0/597872200 wait complete. 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 -- start start 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.704+0000 7fbddf7f3740 1 Processor -- start 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594c5d7b910 con 0x5594c5d6e480 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5594c5c8df10 con 0x5594c5d6db60 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594c5d7ba90 con 0x5594c5d78d40 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbddf7f3740 10 client.0 _ll_register_callbacks cb 0x5594c5c56a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55838/0 (socket says 172.21.15.16:55838) 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 -- 172.21.15.16:0/1706830 learned_addr learned my addr 172.21.15.16:0/1706830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 -- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 msgr2=0x5594c5d766b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d766b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 -- 172.21.15.16:0/1706830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 msgr2=0x5594c5d75c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d75c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 -- 172.21.15.16:0/1706830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c5a446a0 con 0x5594c5d6e480 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005210 tx=0x7fbdcc005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ec9580d0e4a3de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.705+0000 7fbdbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc014070 con 0x5594c5d6e480 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdcc0047a0 con 0x5594c5d6e480 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbdcc005570 con 0x5594c5d6e480 2024-09-12T01:05:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbdcc0104a0 con 0x5594c5d6e480 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 10 client.18513 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594c5d87520 con 0x5594c5d6e480 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc0047a0 con 0x5594c5d6e480 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbdcc023650 con 0x5594c5d6e480 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 10 client.18513 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 20 client.18513 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5d889e0 0x5594c5d8add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 4 client.18513 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c5ce07f0 con 0x5594c5d889e0 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.706+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5d889e0 0x5594c5d8add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.707+0000 7fbdbaff5640 10 client.18513 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.707+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5d889e0 0x5594c5d8add0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34dc1bffea4037 server_cookie=dbd479dec5cbc971 in_seq=0 out_seq=0 2024-09-12T01:05:57.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbdbaff5640 1 -- 172.21.15.16:0/1706830 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594c5ce07f0 con 0x5594c5d889e0 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbdbaff5640 10 client.18513 handle_command_reply: tid=0 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 client.18513 shutdown 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 msgr2=0x5594c5d76170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc005210 tx=0x7fbdcc005240 comp rx=0 tx=0).stop 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 shutdown_connections 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5d889e0 0x5594c5d8add0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d766b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d76170 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbdbaff5640 0 client.18513 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/1706830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 >> 172.21.15.16:0/1706830 conn(0x5594c5d6a0f0 msgr2=0x5594c5d69270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 shutdown_connections 2024-09-12T01:05:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 1 -- 172.21.15.16:0/1706830 wait complete. 2024-09-12T01:05:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.708+0000 7fbddf7f3740 20 client.18513 trim_cache size 0 max 16384 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.725+0000 7fbddf7f3740 1 Processor -- start 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.725+0000 7fbddf7f3740 1 -- start start 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d77a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d90a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ad8d0 con 0x5594c5d6e480 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81af410 con 0x5594c5d6db60 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81adfa0 con 0x5594c5d78d40 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55848/0 (socket says 172.21.15.16:55848) 2024-09-12T01:05:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 -- 172.21.15.16:0/663415854 learned_addr learned my addr 172.21.15.16:0/663415854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d90a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 -- 172.21.15.16:0/663415854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 msgr2=0x5594c5d90a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d90a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 -- 172.21.15.16:0/663415854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 msgr2=0x5594c5d77a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d77a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d77a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 -- 172.21.15.16:0/663415854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c5d74be0 con 0x5594c5d6e480 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d77a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc000c00 tx=0x7fbdcc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841febd286ea3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdbcff9640 1 -- 172.21.15.16:0/663415854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc01c070 con 0x5594c5d6e480 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdbcff9640 1 -- 172.21.15.16:0/663415854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc004780 con 0x5594c5d6e480 2024-09-12T01:05:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.726+0000 7fbdbcff9640 1 -- 172.21.15.16:0/663415854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc014040 con 0x5594c5d6e480 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- 172.21.15.16:0/663415854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 msgr2=0x5594c5d90540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc000c00 tx=0x7fbdcc001e50 comp rx=0 tx=0).stop 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- 172.21.15.16:0/663415854 shutdown_connections 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d78d40 0x5594c5d90a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d90540 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/663415854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d6db60 0x5594c5d77a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- 172.21.15.16:0/663415854 >> 172.21.15.16:0/663415854 conn(0x5594c5d6a0f0 msgr2=0x5594c5d77380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- 172.21.15.16:0/663415854 shutdown_connections 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- 172.21.15.16:0/663415854 wait complete. 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.727+0000 7fbddf7f3740 1 -- start start 2024-09-12T01:05:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 Processor -- start 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 0x5594c5d72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d78d40 0x5594c5d736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81aed40 con 0x5594c5d6e480 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ae670 con 0x5594c5d78d40 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd81ab8d0 con 0x5594c5d6db60 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbddf7f3740 10 client.0 _ll_register_callbacks cb 0x7fbdc8017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55852/0 (socket says 172.21.15.16:55852) 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 -- 172.21.15.16:0/3339143599 learned_addr learned my addr 172.21.15.16:0/3339143599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 0x5594c5d72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd77fe640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d78d40 0x5594c5d736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 -- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 msgr2=0x5594c5d72c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 0x5594c5d72c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 -- 172.21.15.16:0/3339143599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d78d40 msgr2=0x5594c5d736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d78d40 0x5594c5d736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 -- 172.21.15.16:0/3339143599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c5d6e2b0 con 0x5594c5d6e480 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 0x5594c5d72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdd67fc640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00f010 tx=0x7fbdcc004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137284a838fce7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.728+0000 7fbdbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc03a070 con 0x5594c5d6e480 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7fbdcc023630 con 0x5594c5d6e480 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdcc004780 con 0x5594c5d6e480 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbdcc046050 con 0x5594c5d6e480 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 10 client.18519 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594c5df79e0 con 0x5594c5d6e480 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc004780 con 0x5594c5d6e480 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbdcc03f4d0 con 0x5594c5d6e480 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 10 client.18519 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 20 client.18519 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5df8010 0x5594c5dfa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 4 client.18519 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.729+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5594c5cdad00 con 0x5594c5df8010 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.730+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5df8010 0x5594c5dfa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.730+0000 7fbdd6ffd640 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5df8010 0x5594c5dfa430 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac78493168699eb server_cookie=6117504d7ed68abd in_seq=0 out_seq=0 2024-09-12T01:05:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.730+0000 7fbdbaff5640 10 client.18519 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbdbaff5640 1 -- 172.21.15.16:0/3339143599 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5594c5cdad00 con 0x5594c5df8010 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbdbaff5640 10 client.18519 handle_command_reply: tid=0 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 client.18519 shutdown 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 msgr2=0x5594c5d73190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00f010 tx=0x7fbdcc004f70 comp rx=0 tx=0).stop 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 shutdown_connections 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5594c5df8010 0x5594c5dfa430 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5594c5d78d40 0x5594c5d736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594c5d6e480 0x5594c5d73190 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 --2- 172.21.15.16:0/3339143599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594c5d6db60 0x5594c5d72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbdbaff5640 0 client.18519 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 >> 172.21.15.16:0/3339143599 conn(0x5594c5d6a0f0 msgr2=0x5594c5d69270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 shutdown_connections 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.731+0000 7fbddf7f3740 1 -- 172.21.15.16:0/3339143599 wait complete. 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddf7f3740 20 client.18519 trim_cache size 0 max 16384 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c446601f-2cfd-4596-9970-ec9655439204", 2024-09-12T01:05:57.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:05:57.736 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 msgr2=0x7fbdc00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 0x7fbdc00a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005e30 tx=0x7fbdd0005d80 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 msgr2=0x7fbdd8198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800b4a0 tx=0x7fbdc800b960 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 shutdown_connections 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbdc00a2cf0 0x7fbdc00a51b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdd81055b0 0x7fbdd819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdd8100e30 0x7fbdd81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 --2- 172.21.15.16:0/2981752372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdd8100480 0x7fbdd8198480 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 >> 172.21.15.16:0/2981752372 conn(0x7fbdd8077210 msgr2=0x7fbdd8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 shutdown_connections 2024-09-12T01:05:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:57.732+0000 7fbddd41a640 1 -- 172.21.15.16:0/2981752372 wait complete. 2024-09-12T01:05:58.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:05:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:05:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:58 smithi016 ceph-mon[30972]: pgmap v304: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:58 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:05:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:58 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:05:58.864 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:58 smithi159 ceph-mon[35566]: pgmap v304: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:58 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:05:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:58 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:05:58.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:58 smithi099 ceph-mon[35654]: pgmap v304: 97 pgs: 97 active+clean; 13 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:05:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:58 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:05:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:58 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:05:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:05:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 -- 172.21.15.16:0/3215839087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8102d40 msgr2=0x7faab8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 --2- 172.21.15.16:0/3215839087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8102d40 0x7faab8103120 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009900 tx=0x7faaa802a480 comp rx=0 tx=0).stop 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 -- 172.21.15.16:0/3215839087 shutdown_connections 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 --2- 172.21.15.16:0/3215839087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab810c170 0x7faab810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 --2- 172.21.15.16:0/3215839087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8103660 0x7faab810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 --2- 172.21.15.16:0/3215839087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8102d40 0x7faab8103120 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 -- 172.21.15.16:0/3215839087 >> 172.21.15.16:0/3215839087 conn(0x7faab80fc7a0 msgr2=0x7faab80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 -- 172.21.15.16:0/3215839087 shutdown_connections 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.228+0000 7faabdad0640 1 -- 172.21.15.16:0/3215839087 wait complete. 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 Processor -- start 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 -- start start 2024-09-12T01:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab8102d40 0x7faab8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 0x7faab819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faab8111f70 con 0x7faab8103660 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faab8111df0 con 0x7faab8102d40 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.229+0000 7faabdad0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faab81120f0 con 0x7faab810c170 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49460/0 (socket says 172.21.15.16:49460) 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 -- 172.21.15.16:0/318596275 learned_addr learned my addr 172.21.15.16:0/318596275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab7fff640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 0x7faab819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab77fe640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab8102d40 0x7faab8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 -- 172.21.15.16:0/318596275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 msgr2=0x7faab819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 0x7faab819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 -- 172.21.15.16:0/318596275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab8102d40 msgr2=0x7faab8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab8102d40 0x7faab8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 -- 172.21.15.16:0/318596275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab819d420 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab7fff640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 0x7faab819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab6ffd640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c970 tx=0x7faaac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab4ff9640 1 -- 172.21.15.16:0/318596275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac007bb0 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab819d710 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab4ff9640 1 -- 172.21.15.16:0/318596275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac007d50 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faab4ff9640 1 -- 172.21.15.16:0/318596275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac00f450 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.230+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a4ff0 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab4ff9640 1 -- 172.21.15.16:0/318596275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faaac004f60 con 0x7faab8103660 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 Processor -- start 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 -- start start 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 0x55db58c31550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db58b882e0 con 0x55db58c31a90 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db58b2fce0 con 0x55db58c268b0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faabfea9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db58b842a0 con 0x55db58c271d0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49470/0 (socket says 172.21.15.16:49470) 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab4ff9640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 0x7faaa00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab6ffd640 1 -- 172.21.15.16:0/3391960767 learned_addr learned my addr 172.21.15.16:0/3391960767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.233+0000 7faab77fe640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 0x55db58c31550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab7fff640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab7fff640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 0x7faaa00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab4ff9640 1 -- 172.21.15.16:0/318596275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7faaac0cf210 con 0x7faab8103660 2024-09-12T01:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 -- 172.21.15.16:0/3391960767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 msgr2=0x55db58c31550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 0x55db58c31550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 -- 172.21.15.16:0/3391960767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 msgr2=0x55db58c26c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c26c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 -- 172.21.15.16:0/3391960767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db58b14440 con 0x55db58c31a90 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab77fe640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 0x55db58c31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faaac09b430 tx=0x7faaac004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc1e20ec4bb6457 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faab7fff640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 0x7faaa00a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faab0013b30 tx=0x7faab0005c60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faa9dffb640 1 -- 172.21.15.16:0/3391960767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac01d050 con 0x55db58c31a90 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faa9dffb640 1 -- 172.21.15.16:0/3391960767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac0a43a0 con 0x55db58c31a90 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.234+0000 7faa9dffb640 1 -- 172.21.15.16:0/3391960767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0ccd70 con 0x55db58c31a90 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 -- 172.21.15.16:0/3391960767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 msgr2=0x55db58c33e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7faaac09b430 tx=0x7faaac004940 comp rx=0 tx=0).stop 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 -- 172.21.15.16:0/3391960767 shutdown_connections 2024-09-12T01:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c31a90 0x55db58c33e80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c271d0 0x55db58c31550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 --2- 172.21.15.16:0/3391960767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c26c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 -- 172.21.15.16:0/3391960767 >> 172.21.15.16:0/3391960767 conn(0x55db58c22e40 msgr2=0x55db58c23220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 -- 172.21.15.16:0/3391960767 shutdown_connections 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.235+0000 7faabfea9740 1 -- 172.21.15.16:0/3391960767 wait complete. 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 -- start start 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 Processor -- start 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db58c34660 con 0x55db58c271d0 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db58b43050 con 0x55db58c31a90 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db58c347e0 con 0x55db58c268b0 2024-09-12T01:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faabfea9740 10 client.0 _ll_register_callbacks cb 0x55db58b83160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49476/0 (socket says 172.21.15.16:49476) 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faab6ffd640 1 -- 172.21.15.16:0/544481689 learned_addr learned my addr 172.21.15.16:0/544481689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faab77fe640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.236+0000 7faab7fff640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 -- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 msgr2=0x55db58c31410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c31410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 -- 172.21.15.16:0/544481689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 msgr2=0x55db58c2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c2ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 -- 172.21.15.16:0/544481689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db588f93c0 con 0x55db58c271d0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab77fe640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faab6ffd640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faaac097b90 tx=0x7faaac097390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f891bef56edca492 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0d4060 con 0x55db58c271d0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7faaac09ad80 con 0x55db58c271d0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac0a43a0 con 0x55db58c271d0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaac09fae0 con 0x55db58c271d0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faabfea9740 10 client.18531 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db58c34660 con 0x55db58c271d0 2024-09-12T01:05:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac0a43a0 con 0x55db58c271d0 2024-09-12T01:05:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.237+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faaac0a0c70 con 0x55db58c271d0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 10 client.18531 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 20 client.18531 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58c435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 4 client.18531 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db58b95850 con 0x55db58c411d0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faab77fe640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58c435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faab77fe640 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58c435c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c31e5e00f45001 server_cookie=a945a27810471982 in_seq=0 out_seq=0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.238+0000 7faa96ffd640 10 client.18531 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faa96ffd640 1 -- 172.21.15.16:0/544481689 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db58b95850 con 0x55db58c411d0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faa96ffd640 10 client.18531 handle_command_reply: tid=0 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 client.18531 shutdown 2024-09-12T01:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 msgr2=0x55db58c2e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7faaac097b90 tx=0x7faaac097390 comp rx=0 tx=0).stop 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 shutdown_connections 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58c435c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faa96ffd640 0 client.18531 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c2e6c0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 --2- 172.21.15.16:0/544481689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c31410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.239+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 >> 172.21.15.16:0/544481689 conn(0x55db58c22e40 msgr2=0x55db58c1db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.240+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 shutdown_connections 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.240+0000 7faabfea9740 1 -- 172.21.15.16:0/544481689 wait complete. 2024-09-12T01:05:59.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.240+0000 7faabfea9740 20 client.18531 trim_cache size 0 max 16384 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 Processor -- start 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 -- start start 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c31a90 0x55db58c49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ab650 con 0x55db58c271d0 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ad190 con 0x55db58c268b0 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faabfea9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faab81abd20 con 0x55db58c31a90 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49484/0 (socket says 172.21.15.16:49484) 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 -- 172.21.15.16:0/3296237590 learned_addr learned my addr 172.21.15.16:0/3296237590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab77fe640 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 -- 172.21.15.16:0/3296237590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c31a90 msgr2=0x55db58c49190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c31a90 0x55db58c49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 -- 172.21.15.16:0/3296237590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 msgr2=0x55db58c3d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c3d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 -- 172.21.15.16:0/3296237590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db58c2cc10 con 0x55db58c271d0 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.257+0000 7faab6ffd640 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c940 tx=0x7faaac099c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f4602183d6b27f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faa9d7fa640 1 -- 172.21.15.16:0/3296237590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac01d040 con 0x55db58c271d0 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faa9d7fa640 1 -- 172.21.15.16:0/3296237590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac0a43a0 con 0x55db58c271d0 2024-09-12T01:05:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faa9d7fa640 1 -- 172.21.15.16:0/3296237590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac09ab80 con 0x55db58c271d0 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 -- 172.21.15.16:0/3296237590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 msgr2=0x55db58c48c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c940 tx=0x7faaac099c40 comp rx=0 tx=0).stop 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 -- 172.21.15.16:0/3296237590 shutdown_connections 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c31a90 0x55db58c49190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c48c50 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 --2- 172.21.15.16:0/3296237590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c268b0 0x55db58c3d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 -- 172.21.15.16:0/3296237590 >> 172.21.15.16:0/3296237590 conn(0x55db58c22e40 msgr2=0x55db58c2d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 -- 172.21.15.16:0/3296237590 shutdown_connections 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.258+0000 7faabfea9740 1 -- 172.21.15.16:0/3296237590 wait complete. 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 -- start start 2024-09-12T01:05:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 Processor -- start 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c3a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c35a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faab81acac0 con 0x55db58c271d0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ac3f0 con 0x55db58c31a90 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faab81a9670 con 0x55db58c268b0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faabfea9740 10 client.0 _ll_register_callbacks cb 0x7faab000ad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49488/0 (socket says 172.21.15.16:49488) 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.259+0000 7faab6ffd640 1 -- 172.21.15.16:0/2482632777 learned_addr learned my addr 172.21.15.16:0/2482632777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab7fff640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c35a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab77fe640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c3a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 -- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 msgr2=0x55db58c3a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c3a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 -- 172.21.15.16:0/2482632777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 msgr2=0x55db58c35a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c35a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 -- 172.21.15.16:0/2482632777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db58c36230 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faab6ffd640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faaac097b90 tx=0x7faaac09cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641db356a35cc971 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faa96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0d4070 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7faaac097480 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac0a43a0 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaac01d040 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faabfea9740 10 client.18537 fetch_fsmap learned FSMap version 28 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db58c36230 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.260+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac0a43a0 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faaac0bb3e0 con 0x55db58c271d0 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 10 client.18537 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:05:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 10 client.18537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 20 client.18537 populate_metadata read hostname 'smithi016' 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58cadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 4 client.18537 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db58b8fd60 con 0x55db58c411d0 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faab77fe640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58cadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faab77fe640 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58cadec0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3d0cdf7b3c77c6 server_cookie=2bbd2b1b71be389d in_seq=0 out_seq=0 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.261+0000 7faa96ffd640 10 client.18537 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faa96ffd640 1 -- 172.21.15.16:0/2482632777 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55db58b8fd60 con 0x55db58c411d0 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faa96ffd640 10 client.18537 handle_command_reply: tid=0 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 client.18537 shutdown 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 msgr2=0x55db58c354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7faaac097b90 tx=0x7faaac09cd10 comp rx=0 tx=0).stop 2024-09-12T01:05:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 shutdown_connections 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db58c411d0 0x55db58cadec0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db58c31a90 0x55db58c35a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db58c271d0 0x55db58c354f0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 --2- 172.21.15.16:0/2482632777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db58c268b0 0x55db58c3a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 >> 172.21.15.16:0/2482632777 conn(0x55db58c22e40 msgr2=0x55db58c294a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faa96ffd640 0 client.18537 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 shutdown_connections 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 1 -- 172.21.15.16:0/2482632777 wait complete. 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.262+0000 7faabfea9740 20 client.18537 trim_cache size 0 max 16384 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:05:59.265 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: "c446601f-2cfd-4596-9970-ec9655439204": { 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c446601f-2cfd-4596-9970-ec9655439204", 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:05:59.266 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 msgr2=0x7faaa00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 0x7faaa00a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faab0013b30 tx=0x7faab0005c60 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 msgr2=0x7faab81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c970 tx=0x7faaac00ce30 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 shutdown_connections 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7faaa00a2cf0 0x7faaa00a51b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab810c170 0x7faab819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8103660 0x7faab81989b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 --2- 172.21.15.16:0/318596275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faab8102d40 0x7faab8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 >> 172.21.15.16:0/318596275 conn(0x7faab80fc7a0 msgr2=0x7faab80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 shutdown_connections 2024-09-12T01:05:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:05:59.263+0000 7faabdad0640 1 -- 172.21.15.16:0/318596275 wait complete. 2024-09-12T01:05:59.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:05:59 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:05:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:05:59 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:05:59.952 INFO:tasks.cephfs.filesystem:scrub status for tag:c446601f-2cfd-4596-9970-ec9655439204 - {'path': '/', 'tag': 'c446601f-2cfd-4596-9970-ec9655439204', 'options': 'recursive,force'} 2024-09-12T01:06:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:05:59 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:06:00.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:00 smithi159 ceph-mon[35566]: pgmap v305: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:06:00.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:00 smithi016 ceph-mon[30972]: pgmap v305: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:06:01.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:00 smithi099 ceph-mon[35654]: pgmap v305: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 20 op/s 2024-09-12T01:06:01.198 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:06:01.199 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T01:06:01.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- 172.21.15.16:0/1745708167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102570 msgr2=0x7f3a001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 --2- 172.21.15.16:0/1745708167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102570 0x7f3a001029b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ca0 tx=0x7f39f402a950 comp rx=0 tx=0).stop 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- 172.21.15.16:0/1745708167 shutdown_connections 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 --2- 172.21.15.16:0/1745708167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00102ef0 0x7f3a0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 --2- 172.21.15.16:0/1745708167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102570 0x7f3a001029b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 --2- 172.21.15.16:0/1745708167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00100e20 0x7f3a00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- 172.21.15.16:0/1745708167 >> 172.21.15.16:0/1745708167 conn(0x7f3a000fc7d0 msgr2=0x7f3a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- 172.21.15.16:0/1745708167 shutdown_connections 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- 172.21.15.16:0/1745708167 wait complete. 2024-09-12T01:06:01.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 Processor -- start 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.951+0000 7f3a07bab640 1 -- start start 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00100e20 0x7f3a00198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 0x7f3a001989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00110f80 con 0x7f3a00102ef0 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00110e00 con 0x7f3a00100e20 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a07bab640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a00111100 con 0x7f3a00102570 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49502/0 (socket says 172.21.15.16:49502) 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 -- 172.21.15.16:0/395325919 learned_addr learned my addr 172.21.15.16:0/395325919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a05947640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00100e20 0x7f3a00198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a05146640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 0x7f3a001989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 -- 172.21.15.16:0/395325919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 msgr2=0x7f3a001989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:01.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 0x7f3a001989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 -- 172.21.15.16:0/395325919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00100e20 msgr2=0x7f3a00198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00100e20 0x7f3a00198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 -- 172.21.15.16:0/395325919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0019d430 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a05146640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 0x7f3a001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.952+0000 7f3a06148640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f39f800c970 tx=0x7f39f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.953+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8007bb0 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.953+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f8007d50 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.953+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8018800 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.953+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0019d720 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.953+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a001a5000 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.954+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a00101200 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.955+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f39f800fa50 con 0x7f3a00102ef0 2024-09-12T01:06:01.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.955+0000 7f39feffd640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:01.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.956+0000 7f3a05947640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:01.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.956+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f39f80cf4a0 con 0x7f3a00102ef0 2024-09-12T01:06:01.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.956+0000 7f3a05947640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 0x7f39e80a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005fd0 tx=0x7f39f000b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:01.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:01.960+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39f8014030 con 0x7f3a00102ef0 2024-09-12T01:06:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.175+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a001a51d0 con 0x7f3a00102ef0 2024-09-12T01:06:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.175+0000 7f39feffd640 1 -- 172.21.15.16:0/395325919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f39f809c0c0 con 0x7f3a00102ef0 2024-09-12T01:06:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:02.174 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 msgr2=0x7f39e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 0x7f39e80a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005fd0 tx=0x7f39f000b390 comp rx=0 tx=0).stop 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 msgr2=0x7f3a0019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f39f800c970 tx=0x7f39f800ce30 comp rx=0 tx=0).stop 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 shutdown_connections 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f39e80a2cf0 0x7f39e80a51b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a00102ef0 0x7f3a0019cd50 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a00102570 0x7f3a001989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 --2- 172.21.15.16:0/395325919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a00100e20 0x7f3a00198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 >> 172.21.15.16:0/395325919 conn(0x7f3a000fc7d0 msgr2=0x7f3a00069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 shutdown_connections 2024-09-12T01:06:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:02.178+0000 7f3a07bab640 1 -- 172.21.15.16:0/395325919 wait complete. 2024-09-12T01:06:02.812 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T01:06:02.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T01:06:02.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:02 smithi016 ceph-mon[30972]: pgmap v306: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:02.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:02 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/395325919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:02 smithi099 ceph-mon[35654]: pgmap v306: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:02 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/395325919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:02 smithi159 ceph-mon[35566]: pgmap v306: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:02 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/395325919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 -- 172.21.15.16:0/3577327515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c106f10 msgr2=0x7f634c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 --2- 172.21.15.16:0/3577327515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c106f10 0x7f634c10ea90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6344009ca0 tx=0x7f634402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 -- 172.21.15.16:0/3577327515 shutdown_connections 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 --2- 172.21.15.16:0/3577327515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c106f10 0x7f634c10ea90 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 --2- 172.21.15.16:0/3577327515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 0x7f634c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 --2- 172.21.15.16:0/3577327515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c101de0 0x7f634c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 -- 172.21.15.16:0/3577327515 >> 172.21.15.16:0/3577327515 conn(0x7f634c0fd8c0 msgr2=0x7f634c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 -- 172.21.15.16:0/3577327515 shutdown_connections 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.546+0000 7f6351fe5640 1 -- 172.21.15.16:0/3577327515 wait complete. 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 Processor -- start 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 -- start start 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 0x7f634c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c106f10 0x7f634c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c112010 con 0x7f634c101de0 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f634c111e90 con 0x7f634c106f10 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f6351fe5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f634c112190 con 0x7f634c102790 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49514/0 (socket says 172.21.15.16:49514) 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f634b7fe640 1 -- 172.21.15.16:0/1028746474 learned_addr learned my addr 172.21.15.16:0/1028746474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.547+0000 7f634affd640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 0x7f634c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634bfff640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c106f10 0x7f634c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 -- 172.21.15.16:0/1028746474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 msgr2=0x7f634c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 0x7f634c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 -- 172.21.15.16:0/1028746474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c106f10 msgr2=0x7f634c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c106f10 0x7f634c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 -- 172.21.15.16:0/1028746474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c19d560 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f633c00b4a0 tx=0x7f633c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f6348ff9640 1 -- 172.21.15.16:0/1028746474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c015020 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c19d850 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f6348ff9640 1 -- 172.21.15.16:0/1028746474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c011d10 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f6348ff9640 1 -- 172.21.15.16:0/1028746474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c00f2e0 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.548+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a5130 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f6348ff9640 1 -- 172.21.15.16:0/1028746474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f633c018020 con 0x7f634c101de0 2024-09-12T01:06:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 Processor -- start 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 -- start start 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3daf3160 con 0x560f3db99a30 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f3da9a720 con 0x560f3db8f170 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f63543be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f3daef120 con 0x560f3db8e850 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f634affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f634affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49530/0 (socket says 172.21.15.16:49530) 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f634affd640 1 -- 172.21.15.16:0/4173164019 learned_addr learned my addr 172.21.15.16:0/4173164019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f634bfff640 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.551+0000 7f6348ff9640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 0x7f63340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634b7fe640 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 -- 172.21.15.16:0/4173164019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 msgr2=0x560f3db8ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db8ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 -- 172.21.15.16:0/4173164019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 msgr2=0x560f3db994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 -- 172.21.15.16:0/4173164019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3d864ba0 con 0x560f3db99a30 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634bfff640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 0x7f63340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f6348ff9640 1 -- 172.21.15.16:0/1028746474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f633c0cec10 con 0x7f634c101de0 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f634affd640 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6340009a40 tx=0x7f6340009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d595a7b152167e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f6331ffb640 1 -- 172.21.15.16:0/4173164019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340017070 con 0x560f3db99a30 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f6331ffb640 1 -- 172.21.15.16:0/4173164019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340013470 con 0x560f3db99a30 2024-09-12T01:06:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.552+0000 7f6331ffb640 1 -- 172.21.15.16:0/4173164019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340012610 con 0x560f3db99a30 2024-09-12T01:06:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 -- 172.21.15.16:0/4173164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 msgr2=0x560f3db9be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f634bfff640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 0x7f63340a51b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6344005fe0 tx=0x7f634402a8c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6340009a40 tx=0x7f6340009f00 comp rx=0 tx=0).stop 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 -- 172.21.15.16:0/4173164019 shutdown_connections 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9be20 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 --2- 172.21.15.16:0/4173164019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 -- 172.21.15.16:0/4173164019 >> 172.21.15.16:0/4173164019 conn(0x560f3db8ade0 msgr2=0x560f3db8b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 -- 172.21.15.16:0/4173164019 shutdown_connections 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.553+0000 7f63543be740 1 -- 172.21.15.16:0/4173164019 wait complete. 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 -- start start 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 Processor -- start 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f3db9c600 con 0x560f3db99a30 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f3da7b900 con 0x560f3db8f170 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560f3db9c780 con 0x560f3db8e850 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f63543be740 10 client.0 _ll_register_callbacks cb 0x560f3da79780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f634bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f634bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49534/0 (socket says 172.21.15.16:49534) 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f634bfff640 1 -- 172.21.15.16:0/3897972586 learned_addr learned my addr 172.21.15.16:0/3897972586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f634b7fe640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.554+0000 7f634affd640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 -- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 msgr2=0x560f3db96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db96920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 -- 172.21.15.16:0/3897972586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 msgr2=0x560f3db96e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db96e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 -- 172.21.15.16:0/3897972586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3dabbc50 con 0x560f3db99a30 2024-09-12T01:06:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f634bfff640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6344003e90 tx=0x7f63440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154b71af42fc02ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f632affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634404e070 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f6344004740 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6340013470 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f634402f610 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f63543be740 10 client.18555 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3dba82b0 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.555+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340013470 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6344030950 con 0x560f3db99a30 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 10 client.18555 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 10 client.18555 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 20 client.18555 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dba96c0 0x560f3dbabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 4 client.18555 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3db00890 con 0x560f3dba96c0 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f634b7fe640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dba96c0 0x560f3dbabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f634b7fe640 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dba96c0 0x560f3dbabb60 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f624386db31fda server_cookie=8ebcd4fd6395a7bc in_seq=0 out_seq=0 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.556+0000 7f632affd640 10 client.18555 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.557+0000 7f632affd640 1 -- 172.21.15.16:0/3897972586 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f3db00890 con 0x560f3dba96c0 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.557+0000 7f632affd640 10 client.18555 handle_command_reply: tid=0 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.557+0000 7f63543be740 1 client.18555 shutdown 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.557+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 msgr2=0x560f3db973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.557+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6344003e90 tx=0x7f63440095c0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 shutdown_connections 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dba96c0 0x560f3dbabb60 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db973a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3db96e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f632affd640 0 client.18555 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:03.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 --2- 172.21.15.16:0/3897972586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3db96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 >> 172.21.15.16:0/3897972586 conn(0x560f3db8ade0 msgr2=0x560f3db90ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 shutdown_connections 2024-09-12T01:06:03.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 1 -- 172.21.15.16:0/3897972586 wait complete. 2024-09-12T01:06:03.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.558+0000 7f63543be740 20 client.18555 trim_cache size 0 max 16384 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 Processor -- start 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 -- start start 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 0x560f3dbb19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 0x560f3db93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ad9d0 con 0x560f3db8e850 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1af510 con 0x560f3db99a30 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f63543be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ae0a0 con 0x560f3db8f170 2024-09-12T01:06:03.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49536/0 (socket says 172.21.15.16:49536) 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 -- 172.21.15.16:0/1566622773 learned_addr learned my addr 172.21.15.16:0/1566622773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634bfff640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 0x560f3db93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 -- 172.21.15.16:0/1566622773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 msgr2=0x560f3dbb19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634affd640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 0x560f3dbb19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 0x560f3dbb19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 -- 172.21.15.16:0/1566622773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 msgr2=0x560f3db93ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 0x560f3db93ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 -- 172.21.15.16:0/1566622773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3db8ac10 con 0x560f3db8e850 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634bfff640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 0x560f3db93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634affd640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 0x560f3dbb19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.575+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cc520 tx=0x7f633c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f744f12bc3e8b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63317fa640 1 -- 172.21.15.16:0/1566622773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0a3eb0 con 0x560f3db8e850 2024-09-12T01:06:03.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63317fa640 1 -- 172.21.15.16:0/1566622773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c0992e0 con 0x560f3db8e850 2024-09-12T01:06:03.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63317fa640 1 -- 172.21.15.16:0/1566622773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0978b0 con 0x560f3db8e850 2024-09-12T01:06:03.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 -- 172.21.15.16:0/1566622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 msgr2=0x560f3dbb1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f633c0cc520 tx=0x7f633c0bb040 comp rx=0 tx=0).stop 2024-09-12T01:06:03.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 -- 172.21.15.16:0/1566622773 shutdown_connections 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db99a30 0x560f3db93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8f170 0x560f3dbb19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 --2- 172.21.15.16:0/1566622773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db8e850 0x560f3dbb1480 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 -- 172.21.15.16:0/1566622773 >> 172.21.15.16:0/1566622773 conn(0x560f3db8ade0 msgr2=0x560f3db989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 -- 172.21.15.16:0/1566622773 shutdown_connections 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.576+0000 7f63543be740 1 -- 172.21.15.16:0/1566622773 wait complete. 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 -- start start 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 Processor -- start 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3dba1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3dba20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1aee40 con 0x560f3db99a30 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ae770 con 0x560f3db8f170 2024-09-12T01:06:03.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ab9d0 con 0x560f3db8e850 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f63543be740 10 client.0 _ll_register_callbacks cb 0x7f6344009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f634bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f634bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49538/0 (socket says 172.21.15.16:49538) 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f634bfff640 1 -- 172.21.15.16:0/1898215418 learned_addr learned my addr 172.21.15.16:0/1898215418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.577+0000 7f634affd640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3dba20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3dba1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 -- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 msgr2=0x560f3dba1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3dba1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 -- 172.21.15.16:0/1898215418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 msgr2=0x560f3dba20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3dba20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 -- 172.21.15.16:0/1898215418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3db9f1d0 con 0x560f3db99a30 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3dba1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:03.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f634bfff640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f6344031d20 tx=0x7f63440307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108a564d1226abe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f632affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344063060 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633c0992e0 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8521+0+0 (secure 0 0 0) 0x7f634402f9b0 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f634404e040 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f63543be740 10 client.18561 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3dc14e50 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.578+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c0992e0 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f634402fbb0 con 0x560f3db99a30 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 10 client.18561 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 10 client.18561 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 20 client.18561 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dc15610 0x560f3dc17a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:03.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 4 client.18561 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T01:06:03.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x560f3dafabe0 con 0x560f3dc15610 2024-09-12T01:06:03.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dc15610 0x560f3dc17a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:03.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f632affd640 10 client.18561 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:03.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.579+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dc15610 0x560f3dc17a30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72324beebfa2bd9 server_cookie=acbeac4d814c3549 in_seq=0 out_seq=0 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f632affd640 1 -- 172.21.15.16:0/1898215418 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560f3dafabe0 con 0x560f3dc15610 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f632affd640 10 client.18561 handle_command_reply: tid=0 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 client.18561 shutdown 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 msgr2=0x560f3db9e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f6344031d20 tx=0x7f63440307f0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 shutdown_connections 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x560f3dc15610 0x560f3dc17a30 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f3db99a30 0x560f3db9e9a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f3db8f170 0x560f3dba20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 --2- 172.21.15.16:0/1898215418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f3db8e850 0x560f3dba1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.591+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 >> 172.21.15.16:0/1898215418 conn(0x560f3db8ade0 msgr2=0x560f3db989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f632affd640 0 client.18561 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 shutdown_connections 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f63543be740 1 -- 172.21.15.16:0/1898215418 wait complete. 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f63543be740 20 client.18561 trim_cache size 0 max 16384 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:03.591 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 38, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "98e550ce": { 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 38, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T01:06:03.592 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:03.593 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 msgr2=0x7f63340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 0x7f63340a51b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6344005fe0 tx=0x7f634402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 msgr2=0x7f634c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f633c00b4a0 tx=0x7f633c00b8a0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 shutdown_connections 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63340a2cf0 0x7f63340a51b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f634c106f10 0x7f634c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f634c102790 0x7f634c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 --2- 172.21.15.16:0/1028746474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c101de0 0x7f634c1985b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:03.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.592+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 >> 172.21.15.16:0/1028746474 conn(0x7f634c0fd8c0 msgr2=0x7f634c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.593+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 shutdown_connections 2024-09-12T01:06:03.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:03.593+0000 7f6351fe5640 1 -- 172.21.15.16:0/1028746474 wait complete. 2024-09-12T01:06:04.312 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '98e550ce', quiesce took 3.1 seconds. Will release after: 19.45 2024-09-12T01:06:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:04 smithi016 ceph-mon[30972]: pgmap v307: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:04 smithi099 ceph-mon[35654]: pgmap v307: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:04 smithi159 ceph-mon[35566]: pgmap v307: 97 pgs: 97 active+clean; 13 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-12T01:06:05.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:05 smithi159 ceph-mon[35566]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:06:05.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:05 smithi016 ceph-mon[30972]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:06:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:05 smithi099 ceph-mon[35654]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T01:06:06.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:06 smithi016 ceph-mon[30972]: pgmap v309: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:06 smithi099 ceph-mon[35654]: pgmap v309: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:06 smithi159 ceph-mon[35566]: pgmap v309: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:08.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:08 smithi016 ceph-mon[30972]: pgmap v310: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:08 smithi099 ceph-mon[35654]: pgmap v310: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:08 smithi159 ceph-mon[35566]: pgmap v310: 97 pgs: 97 active+clean; 14 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 105 KiB/s wr, 37 op/s 2024-09-12T01:06:10.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:10 smithi016 ceph-mon[30972]: pgmap v311: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:10 smithi099 ceph-mon[35654]: pgmap v311: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:10 smithi159 ceph-mon[35566]: pgmap v311: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:12 smithi016 ceph-mon[30972]: pgmap v312: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:12 smithi016 ceph-mon[30972]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:06:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:12 smithi099 ceph-mon[35654]: pgmap v312: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:12 smithi099 ceph-mon[35654]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:06:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:12 smithi159 ceph-mon[35566]: pgmap v312: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-09-12T01:06:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:12 smithi159 ceph-mon[35566]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T01:06:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:14 smithi099 ceph-mon[35654]: pgmap v314: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T01:06:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:14 smithi159 ceph-mon[35566]: pgmap v314: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T01:06:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:14 smithi016 ceph-mon[30972]: pgmap v314: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T01:06:17.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:16 smithi099 ceph-mon[35654]: pgmap v315: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:16 smithi159 ceph-mon[35566]: pgmap v315: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:16 smithi016 ceph-mon[30972]: pgmap v315: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:18 smithi099 ceph-mon[35654]: pgmap v316: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:18 smithi159 ceph-mon[35566]: pgmap v316: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:19.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:18 smithi016 ceph-mon[30972]: pgmap v316: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T01:06:20.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:20 smithi159 ceph-mon[35566]: pgmap v317: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:20.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:20 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:20 smithi016 ceph-mon[30972]: pgmap v317: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:20 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:06:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:20 smithi099 ceph-mon[35654]: pgmap v317: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:20 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: pgmap v318: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:06:22.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: pgmap v318: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.867 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:06:22.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: pgmap v318: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi016 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: Unable to set osd_memory_target on smithi159 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:06:22.895 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:06:23.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 -- 172.21.15.16:0/463857410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec101400 msgr2=0x7f43ec10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 --2- 172.21.15.16:0/463857410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec101400 0x7f43ec10d930 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009cd0 tx=0x7f43e402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 -- 172.21.15.16:0/463857410 shutdown_connections 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 --2- 172.21.15.16:0/463857410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec101400 0x7f43ec10d930 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 --2- 172.21.15.16:0/463857410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec100a60 0x7f43ec100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 --2- 172.21.15.16:0/463857410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 0x7f43ec106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 -- 172.21.15.16:0/463857410 >> 172.21.15.16:0/463857410 conn(0x7f43ec0fc780 msgr2=0x7f43ec0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 -- 172.21.15.16:0/463857410 shutdown_connections 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.529+0000 7f43f396f640 1 -- 172.21.15.16:0/463857410 wait complete. 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 Processor -- start 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 -- start start 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec101400 0x7f43ec102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 0x7f43ec1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec110e60 con 0x7f43ec100a60 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec110ce0 con 0x7f43ec101400 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f396f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec110fe0 con 0x7f43ec106a60 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f170b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f170b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43070/0 (socket says 172.21.15.16:43070) 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f170b640 1 -- 172.21.15.16:0/481880276 learned_addr learned my addr 172.21.15.16:0/481880276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.530+0000 7f43f1f0c640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 0x7f43ec1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f0f0a640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec101400 0x7f43ec102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 -- 172.21.15.16:0/481880276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 msgr2=0x7f43ec1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 0x7f43ec1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 -- 172.21.15.16:0/481880276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec101400 msgr2=0x7f43ec102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec101400 0x7f43ec102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 -- 172.21.15.16:0/481880276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec1a67d0 con 0x7f43ec100a60 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f170b640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c920 tx=0x7f43e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8007c70 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e8007e10 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec1a6ac0 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018890 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.531+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1a7000 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.532+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec100ec0 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.533+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f43e8018a50 con 0x7f43ec100a60 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.534+0000 7f43e27fc640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 0x7f43d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.534+0000 7f43f0f0a640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 0x7f43d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.534+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f43e80cf750 con 0x7f43ec100a60 2024-09-12T01:06:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.535+0000 7f43f0f0a640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 0x7f43d80a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43ec1040c0 tx=0x7f43dc00b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.537+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43e8014030 con 0x7f43ec100a60 2024-09-12T01:06:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.750+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43ec103b60 con 0x7f43ec100a60 2024-09-12T01:06:24.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:24 smithi016 ceph-mon[30972]: pgmap v319: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T01:06:24.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.751+0000 7f43e27fc640 1 -- 172.21.15.16:0/481880276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f43e809c3a0 con 0x7f43ec100a60 2024-09-12T01:06:24.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:24.751 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:24.751 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 msgr2=0x7f43d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 0x7f43d80a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43ec1040c0 tx=0x7f43dc00b390 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 msgr2=0x7f43ec102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43e800c920 tx=0x7f43e800cde0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 shutdown_connections 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43d80a2cf0 0x7f43d80a51b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43ec106a60 0x7f43ec1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43ec101400 0x7f43ec102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 --2- 172.21.15.16:0/481880276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ec100a60 0x7f43ec102630 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 >> 172.21.15.16:0/481880276 conn(0x7f43ec0fc780 msgr2=0x7f43ec06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 shutdown_connections 2024-09-12T01:06:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:24.753+0000 7f43f396f640 1 -- 172.21.15.16:0/481880276 wait complete. 2024-09-12T01:06:24.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:24 smithi159 ceph-mon[35566]: pgmap v319: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T01:06:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:24 smithi099 ceph-mon[35654]: pgmap v319: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-12T01:06:25.606 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 98e550ce --release --await' 2024-09-12T01:06:25.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 98e550ce --release --await 2024-09-12T01:06:25.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:25 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/481880276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:25.874 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:25.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:25 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/481880276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:25 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/481880276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 -- 172.21.15.16:0/2488356017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0101de0 msgr2=0x7ff4b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 --2- 172.21.15.16:0/2488356017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0101de0 0x7ff4b01021c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009900 tx=0x7ff4ac02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 -- 172.21.15.16:0/2488356017 shutdown_connections 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 --2- 172.21.15.16:0/2488356017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0106f10 0x7ff4b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 --2- 172.21.15.16:0/2488356017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0102790 0x7ff4b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.336+0000 7ff4b8339640 1 --2- 172.21.15.16:0/2488356017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0101de0 0x7ff4b01021c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- 172.21.15.16:0/2488356017 >> 172.21.15.16:0/2488356017 conn(0x7ff4b00fd8c0 msgr2=0x7ff4b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- 172.21.15.16:0/2488356017 shutdown_connections 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- 172.21.15.16:0/2488356017 wait complete. 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 Processor -- start 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- start start 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0101de0 0x7ff4b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 0x7ff4b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0111f70 con 0x7ff4b0102790 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0111df0 con 0x7ff4b0106f10 2024-09-12T01:06:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.337+0000 7ff4b8339640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01120f0 con 0x7ff4b0101de0 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43094/0 (socket says 172.21.15.16:43094) 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 -- 172.21.15.16:0/3017603999 learned_addr learned my addr 172.21.15.16:0/3017603999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0101de0 0x7ff4b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 0x7ff4b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 -- 172.21.15.16:0/3017603999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0101de0 msgr2=0x7ff4b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0101de0 0x7ff4b00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 -- 172.21.15.16:0/3017603999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 msgr2=0x7ff4b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 0x7ff4b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 -- 172.21.15.16:0/3017603999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b01a6850 con 0x7ff4b0102790 2024-09-12T01:06:26.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 0x7ff4b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c920 tx=0x7ff4a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4a77fe640 1 -- 172.21.15.16:0/3017603999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007c70 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4a77fe640 1 -- 172.21.15.16:0/3017603999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0007e10 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b01a6b40 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4a77fe640 1 -- 172.21.15.16:0/3017603999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0018890 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.338+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b01a7080 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4a77fe640 1 -- 172.21.15.16:0/3017603999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff4a0018a70 con 0x7ff4b0102790 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 Processor -- start 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 -- start start 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e59ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5952620 con 0x55d8e59f8240 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d8e58f9e50 con 0x55d8e59ed980 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.341+0000 7ff4ba712740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d8e594e5e0 con 0x55d8e59ed060 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43108/0 (socket says 172.21.15.16:43108) 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b60d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b68d6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e59ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 -- 172.21.15.16:0/2034388061 learned_addr learned my addr 172.21.15.16:0/2034388061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4a77fe640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 0x7ff49c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 -- 172.21.15.16:0/2034388061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 msgr2=0x55d8e59ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e59ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 -- 172.21.15.16:0/2034388061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 msgr2=0x55d8e59f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 -- 172.21.15.16:0/2034388061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e58ead40 con 0x55d8e59f8240 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 0x7ff49c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4a77fe640 1 -- 172.21.15.16:0/3017603999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff4a00cf6b0 con 0x7ff4b0102790 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.342+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00ccce0 tx=0x7ff4a0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677214d547c5cba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4a4dea640 1 -- 172.21.15.16:0/2034388061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a009b4d0 con 0x55d8e59f8240 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4a4dea640 1 -- 172.21.15.16:0/2034388061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a004af60 con 0x55d8e59f8240 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4a4dea640 1 -- 172.21.15.16:0/2034388061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a009ff20 con 0x55d8e59f8240 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 -- 172.21.15.16:0/2034388061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 msgr2=0x55d8e59fa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 0x7ff49c0a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005fd0 tx=0x7ff4a8004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00ccce0 tx=0x7ff4a0020000 comp rx=0 tx=0).stop 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 -- 172.21.15.16:0/2034388061 shutdown_connections 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fa630 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 --2- 172.21.15.16:0/2034388061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e59ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 -- 172.21.15.16:0/2034388061 >> 172.21.15.16:0/2034388061 conn(0x55d8e59e95f0 msgr2=0x55d8e59e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.343+0000 7ff4ba712740 1 -- 172.21.15.16:0/2034388061 shutdown_connections 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 -- 172.21.15.16:0/2034388061 wait complete. 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 -- start start 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 Processor -- start 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 0x55d8e59f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59f8240 0x55d8e59f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d8e59fae10 con 0x55d8e59ed060 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d8e58daf10 con 0x55d8e59f8240 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d8e59faf90 con 0x55d8e59ed980 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4ba712740 10 client.0 _ll_register_callbacks cb 0x55d8e58fedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4b60d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4b60d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43120/0 (socket says 172.21.15.16:43120) 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.344+0000 7ff4b60d5640 1 -- 172.21.15.16:0/4273439569 learned_addr learned my addr 172.21.15.16:0/4273439569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 0x55d8e59f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59f8240 0x55d8e59f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 -- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 msgr2=0x55d8e59f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 0x55d8e59f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 -- 172.21.15.16:0/4273439569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59f8240 msgr2=0x55d8e59f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59f8240 0x55d8e59f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 -- 172.21.15.16:0/4273439569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5661580 con 0x55d8e59ed060 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 0x55d8e59f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac033000 tx=0x7ff4ac031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abac8158e629cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac035040 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff4ac004460 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8e595fb90 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4ac005d50 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4ba712740 10 client.18579 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.345+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8e572b960 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8e595fb90 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4ac0035f0 con 0x55d8e59ed060 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 10 client.18579 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 10 client.18579 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 20 client.18579 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a0a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 4 client.18579 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e595a0a0 con 0x55d8e5a07d40 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.346+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a0a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a0a1b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b83201e2d57529a server_cookie=4a3036309dabce37 in_seq=0 out_seq=0 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff499ffb640 10 client.18579 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff499ffb640 1 -- 172.21.15.16:0/4273439569 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8e595a0a0 con 0x55d8e5a07d40 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff499ffb640 10 client.18579 handle_command_reply: tid=0 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 client.18579 shutdown 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 msgr2=0x55d8e59f5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac033000 tx=0x7ff4ac031a20 comp rx=0 tx=0).stop 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 shutdown_connections 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a0a1b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff499ffb640 0 client.18579 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59f8240 0x55d8e59f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed980 0x55d8e59f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 --2- 172.21.15.16:0/4273439569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f5120 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.347+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 >> 172.21.15.16:0/4273439569 conn(0x55d8e59e95f0 msgr2=0x55d8e59ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.348+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 shutdown_connections 2024-09-12T01:06:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.348+0000 7ff4ba712740 1 -- 172.21.15.16:0/4273439569 wait complete. 2024-09-12T01:06:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.348+0000 7ff4ba712740 20 client.18579 trim_cache size 0 max 16384 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 Processor -- start 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 -- start start 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59f8240 0x55d8e5a0fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01afa40 con 0x55d8e59ed060 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b1580 con 0x55d8e59ed980 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.365+0000 7ff4ba712740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b0110 con 0x55d8e59f8240 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43134/0 (socket says 172.21.15.16:43134) 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 -- 172.21.15.16:0/222458657 learned_addr learned my addr 172.21.15.16:0/222458657 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59f8240 0x55d8e5a0fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 -- 172.21.15.16:0/222458657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59f8240 msgr2=0x55d8e5a0fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59f8240 0x55d8e5a0fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 -- 172.21.15.16:0/222458657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 msgr2=0x55d8e59ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 -- 172.21.15.16:0/222458657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e59f7060 con 0x55d8e59ed060 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac004020 tx=0x7ff4ac02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc0310bdea7af96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff49bfff640 1 -- 172.21.15.16:0/222458657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac000e60 con 0x55d8e59ed060 2024-09-12T01:06:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff49bfff640 1 -- 172.21.15.16:0/222458657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac052030 con 0x55d8e59ed060 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.366+0000 7ff49bfff640 1 -- 172.21.15.16:0/222458657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac035070 con 0x55d8e59ed060 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- 172.21.15.16:0/222458657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 msgr2=0x55d8e59f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac004020 tx=0x7ff4ac02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- 172.21.15.16:0/222458657 shutdown_connections 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59f8240 0x55d8e5a0fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e59ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 --2- 172.21.15.16:0/222458657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59ed060 0x55d8e59f03e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- 172.21.15.16:0/222458657 >> 172.21.15.16:0/222458657 conn(0x55d8e59e95f0 msgr2=0x55d8e59eeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- 172.21.15.16:0/222458657 shutdown_connections 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- 172.21.15.16:0/222458657 wait complete. 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 -- start start 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.367+0000 7ff4ba712740 1 Processor -- start 2024-09-12T01:06:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e5a001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e5a00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b0eb0 con 0x55d8e59f8240 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01b07e0 con 0x55d8e59ed980 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b01ada40 con 0x55d8e59ed060 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4ba712740 10 client.0 _ll_register_callbacks cb 0x7ff4ac0316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43148/0 (socket says 172.21.15.16:43148) 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 -- 172.21.15.16:0/949089127 learned_addr learned my addr 172.21.15.16:0/949089127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e5a001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b58d4640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e5a00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 -- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 msgr2=0x55d8e5a001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e5a001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 -- 172.21.15.16:0/949089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 msgr2=0x55d8e5a00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e5a00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.368+0000 7ff4b68d6640 1 -- 172.21.15.16:0/949089127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e59ed7b0 con 0x55d8e59f8240 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4b68d6640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8012b50 tx=0x7ff4a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b8c288def4bce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a802a440 con 0x55d8e59f8240 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4ac052030 con 0x55d8e59f8240 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff4a80293e0 con 0x55d8e59f8240 2024-09-12T01:06:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4a802a5e0 con 0x55d8e59f8240 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 10 client.18585 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8e5a73190 con 0x55d8e59f8240 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ac052030 con 0x55d8e59f8240 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4a8028070 con 0x55d8e59f8240 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 10 client.18585 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 10 client.18585 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 20 client.18585 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 4 client.18585 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "98e550ce", "release": true, "await": true}] 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.369+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "98e550ce", "release": true, "await": true}) -- 0x55d8e58d65a0 con 0x55d8e5a07d40 2024-09-12T01:06:26.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.370+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.370+0000 7ff499ffb640 10 client.18585 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.370+0000 7ff4b60d5640 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a759d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306f63dc804d6838 server_cookie=2d6ffc8f709d21e3 in_seq=0 out_seq=0 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff499ffb640 1 -- 172.21.15.16:0/949089127 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d8e58d65a0 con 0x55d8e5a07d40 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff499ffb640 10 client.18585 handle_command_reply: tid=0 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 client.18585 shutdown 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 msgr2=0x55d8e59fd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8012b50 tx=0x7ff4a8004ab0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 shutdown_connections 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d8e5a07d40 0x55d8e5a759d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8e59f8240 0x55d8e59fd1b0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d8e59ed980 0x55d8e5a00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 --2- 172.21.15.16:0/949089127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8e59ed060 0x55d8e5a001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 >> 172.21.15.16:0/949089127 conn(0x55d8e59e95f0 msgr2=0x55d8e59e8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.373+0000 7ff499ffb640 0 client.18585 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:26.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 shutdown_connections 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4ba712740 1 -- 172.21.15.16:0/949089127 wait complete. 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4ba712740 20 client.18585 trim_cache size 0 max 16384 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 40, 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "98e550ce": { 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 40, 2024-09-12T01:06:26.377 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:26.378 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:26.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:26.379 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 msgr2=0x7ff49c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 0x7ff49c0a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005fd0 tx=0x7ff4a8004a40 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 msgr2=0x7ff4b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c920 tx=0x7ff4a000cde0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 shutdown_connections 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff49c0a2cf0 0x7ff49c0a51b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b0106f10 0x7ff4b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b0102790 0x7ff4b0074a30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 --2- 172.21.15.16:0/3017603999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b0101de0 0x7ff4b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 >> 172.21.15.16:0/3017603999 conn(0x7ff4b00fd8c0 msgr2=0x7ff4b00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:26.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 shutdown_connections 2024-09-12T01:06:26.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:26.374+0000 7ff4b8339640 1 -- 172.21.15.16:0/3017603999 wait complete. 2024-09-12T01:06:26.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:26 smithi159 ceph-mon[35566]: pgmap v320: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T01:06:26.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:26 smithi016 ceph-mon[30972]: pgmap v320: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T01:06:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:26 smithi099 ceph-mon[35654]: pgmap v320: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T01:06:27.079 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '98e550ce', total seconds elapsed: 25.9 2024-09-12T01:06:27.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T01:06:27.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 -- 172.21.15.16:0/292871512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 msgr2=0x7efc9c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 --2- 172.21.15.16:0/292871512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c1089f0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009900 tx=0x7efc8c02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 -- 172.21.15.16:0/292871512 shutdown_connections 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 --2- 172.21.15.16:0/292871512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 0x7efc9c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 --2- 172.21.15.16:0/292871512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c1089f0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 --2- 172.21.15.16:0/292871512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 -- 172.21.15.16:0/292871512 >> 172.21.15.16:0/292871512 conn(0x7efc9c0ffa90 msgr2=0x7efc9c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 -- 172.21.15.16:0/292871512 shutdown_connections 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.839+0000 7efca37f8640 1 -- 172.21.15.16:0/292871512 wait complete. 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 Processor -- start 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 -- start start 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 0x7efc9c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c114120 con 0x7efc9c1049a0 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c113fa0 con 0x7efc9c103ff0 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca37f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1142a0 con 0x7efc9c109120 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca0d93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca0d93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43174/0 (socket says 172.21.15.16:43174) 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca0d93640 1 -- 172.21.15.16:0/1305738841 learned_addr learned my addr 172.21.15.16:0/1305738841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca1d95640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 0x7efc9c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.840+0000 7efca1594640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 -- 172.21.15.16:0/1305738841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 msgr2=0x7efc9c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 0x7efc9c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 -- 172.21.15.16:0/1305738841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 msgr2=0x7efc9c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 -- 172.21.15.16:0/1305738841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c19f5b0 con 0x7efc9c1049a0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca1594640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca0d93640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c03f030 con 0x7efc9c1049a0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c19f840 con 0x7efc9c1049a0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc8c003ce0 con 0x7efc9c1049a0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c035070 con 0x7efc9c1049a0 2024-09-12T01:06:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.841+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1a7180 con 0x7efc9c1049a0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.842+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c1054d0 con 0x7efc9c1049a0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.843+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc8c042030 con 0x7efc9c1049a0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.844+0000 7efc927fc640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 0x7efc880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.844+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7efc8c0efdb0 con 0x7efc9c1049a0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.844+0000 7efca1594640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 0x7efc880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.845+0000 7efca1594640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 0x7efc880a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc9800baa0 tx=0x7efc98005d90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:27.848+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc8c0bca90 con 0x7efc9c1049a0 2024-09-12T01:06:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.059+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc9c10c360 con 0x7efc9c1049a0 2024-09-12T01:06:28.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.060+0000 7efc927fc640 1 -- 172.21.15.16:0/1305738841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7efc8c0bc390 con 0x7efc9c1049a0 2024-09-12T01:06:28.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:28.070 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 msgr2=0x7efc880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 0x7efc880a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc9800baa0 tx=0x7efc98005d90 comp rx=0 tx=0).stop 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 msgr2=0x7efc9c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 shutdown_connections 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc880a2cf0 0x7efc880a51b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc9c109120 0x7efc9c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:28.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c1049a0 0x7efc9c19ab40 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 --2- 172.21.15.16:0/1305738841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc9c103ff0 0x7efc9c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 >> 172.21.15.16:0/1305738841 conn(0x7efc9c0ffa90 msgr2=0x7efc9c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 shutdown_connections 2024-09-12T01:06:28.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:28.062+0000 7efca37f8640 1 -- 172.21.15.16:0/1305738841 wait complete. 2024-09-12T01:06:28.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:28 smithi016 ceph-mon[30972]: pgmap v321: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-12T01:06:28.834 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:28 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1305738841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:28.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:28 smithi159 ceph-mon[35566]: pgmap v321: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-12T01:06:28.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:28 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1305738841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:28.869 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T01:06:28.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T01:06:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:28 smithi099 ceph-mon[35654]: pgmap v321: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-09-12T01:06:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:28 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1305738841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 -- 172.21.15.16:0/2842138909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754106f10 msgr2=0x7fd75410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 --2- 172.21.15.16:0/2842138909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754106f10 0x7fd75410ea90 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009ca0 tx=0x7fd74c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 -- 172.21.15.16:0/2842138909 shutdown_connections 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 --2- 172.21.15.16:0/2842138909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754106f10 0x7fd75410ea90 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 --2- 172.21.15.16:0/2842138909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754102790 0x7fd7541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 --2- 172.21.15.16:0/2842138909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754101de0 0x7fd7541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 -- 172.21.15.16:0/2842138909 >> 172.21.15.16:0/2842138909 conn(0x7fd7540fd8c0 msgr2=0x7fd7540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 -- 172.21.15.16:0/2842138909 shutdown_connections 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.663+0000 7fd75b950640 1 -- 172.21.15.16:0/2842138909 wait complete. 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 Processor -- start 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 -- start start 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754102790 0x7fd754198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754106f10 0x7fd75419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd754111f70 con 0x7fd754101de0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd754111df0 con 0x7fd754106f10 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd75b950640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541120f0 con 0x7fd754102790 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40352/0 (socket says 172.21.15.16:40352) 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 -- 172.21.15.16:0/3273523761 learned_addr learned my addr 172.21.15.16:0/3273523761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd759eed640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754106f10 0x7fd75419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd758eeb640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754102790 0x7fd754198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 -- 172.21.15.16:0/3273523761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754102790 msgr2=0x7fd754198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754102790 0x7fd754198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 -- 172.21.15.16:0/3273523761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754106f10 msgr2=0x7fd75419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754106f10 0x7fd75419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.664+0000 7fd7596ec640 1 -- 172.21.15.16:0/3273523761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75419d380 con 0x7fd754101de0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd7596ec640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd75000d770 tx=0x7fd75000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd74a7fc640 1 -- 172.21.15.16:0/3273523761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750017070 con 0x7fd754101de0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd74a7fc640 1 -- 172.21.15.16:0/3273523761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd750005500 con 0x7fd754101de0 2024-09-12T01:06:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75419d670 con 0x7fd754101de0 2024-09-12T01:06:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd74a7fc640 1 -- 172.21.15.16:0/3273523761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750004e00 con 0x7fd754101de0 2024-09-12T01:06:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.665+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7541a4f50 con 0x7fd754101de0 2024-09-12T01:06:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.667+0000 7fd74a7fc640 1 -- 172.21.15.16:0/3273523761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd750004fa0 con 0x7fd754101de0 2024-09-12T01:06:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 Processor -- start 2024-09-12T01:06:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 -- start start 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6b9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bad900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bed6b03160 con 0x55bed6bade40 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bed6aaa720 con 0x55bed6b9f070 2024-09-12T01:06:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd75dd29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bed6aff120 con 0x55bed6b9e750 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.668+0000 7fd74a7fc640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 0x7fd7400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd759eed640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 0x7fd7400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40368/0 (socket says 172.21.15.16:40368) 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 -- 172.21.15.16:0/83334186 learned_addr learned my addr 172.21.15.16:0/83334186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd759eed640 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6b9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd74a7fc640 1 -- 172.21.15.16:0/3273523761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd7500ced60 con 0x7fd754101de0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 -- 172.21.15.16:0/83334186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 msgr2=0x55bed6b9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6b9eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 -- 172.21.15.16:0/83334186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 msgr2=0x55bed6bad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 -- 172.21.15.16:0/83334186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed6874ba0 con 0x55bed6bade40 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd758eeb640 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd744009c10 tx=0x7fd744011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83da4cc9d267a189 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd759eed640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 0x7fd7400a5230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004af0 tx=0x7fd74c02f310 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd73f7fe640 1 -- 172.21.15.16:0/83334186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744004230 con 0x55bed6bade40 2024-09-12T01:06:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd73f7fe640 1 -- 172.21.15.16:0/83334186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7440043d0 con 0x55bed6bade40 2024-09-12T01:06:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.669+0000 7fd73f7fe640 1 -- 172.21.15.16:0/83334186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744004e00 con 0x55bed6bade40 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- 172.21.15.16:0/83334186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 msgr2=0x55bed6bb0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd744009c10 tx=0x7fd744011810 comp rx=0 tx=0).stop 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- 172.21.15.16:0/83334186 shutdown_connections 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb0230 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 --2- 172.21.15.16:0/83334186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6b9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- 172.21.15.16:0/83334186 >> 172.21.15.16:0/83334186 conn(0x55bed6b9ade0 msgr2=0x55bed6b9b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- 172.21.15.16:0/83334186 shutdown_connections 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- 172.21.15.16:0/83334186 wait complete. 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.670+0000 7fd75dd29740 1 -- start start 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 Processor -- start 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6bade40 0x55bed6bab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bed6bb0a70 con 0x55bed6b9e750 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bed6a8b900 con 0x55bed6bade40 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bed6bb0bf0 con 0x55bed6b9f070 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd75dd29740 10 client.0 _ll_register_callbacks cb 0x55bed6a89780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40376/0 (socket says 172.21.15.16:40376) 2024-09-12T01:06:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd7596ec640 1 -- 172.21.15.16:0/2710783067 learned_addr learned my addr 172.21.15.16:0/2710783067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd759eed640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6bade40 0x55bed6bab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd7596ec640 1 -- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 msgr2=0x55bed6bab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.671+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd7596ec640 1 -- 172.21.15.16:0/2710783067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6bade40 msgr2=0x55bed6bab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6bade40 0x55bed6bab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd7596ec640 1 -- 172.21.15.16:0/2710783067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed6acbc50 con 0x55bed6b9e750 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd75009b030 tx=0x7fd75006b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03e007d8c3014ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750020050 con 0x55bed6b9e750 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd7500cc730 con 0x55bed6b9e750 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7440043d0 con 0x55bed6b9e750 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd75009f630 con 0x55bed6b9e750 2024-09-12T01:06:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 10 client.18603 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bed6bbc7e0 con 0x55bed6b9e750 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7440043d0 con 0x55bed6b9e750 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7500979a0 con 0x55bed6b9e750 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 10 client.18603 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 10 client.18603 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.672+0000 7fd75dd29740 20 client.18603 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6bbdca0 0x55bed6bc00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd75dd29740 4 client.18603 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bed6b10890 con 0x55bed6bbdca0 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6bbdca0 0x55bed6bc00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6bbdca0 0x55bed6bc00b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9f6286df816650 server_cookie=5f3edd5cb373fc2e in_seq=0 out_seq=0 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.673+0000 7fd73cff9640 10 client.18603 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd73cff9640 1 -- 172.21.15.16:0/2710783067 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bed6b10890 con 0x55bed6bbdca0 2024-09-12T01:06:29.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd73cff9640 10 client.18603 handle_command_reply: tid=0 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 client.18603 shutdown 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 msgr2=0x55bed6baacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd75009b030 tx=0x7fd75006b190 comp rx=0 tx=0).stop 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 shutdown_connections 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6bbdca0 0x55bed6bc00b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6bade40 0x55bed6bab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2710783067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6b9e750 0x55bed6baacf0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 >> 172.21.15.16:0/2710783067 conn(0x55bed6b9ade0 msgr2=0x55bed6ba0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd73cff9640 0 client.18603 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.674+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 shutdown_connections 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.675+0000 7fd75dd29740 1 -- 172.21.15.16:0/2710783067 wait complete. 2024-09-12T01:06:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.675+0000 7fd75dd29740 20 client.18603 trim_cache size 0 max 16384 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.691+0000 7fd75dd29740 1 Processor -- start 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.691+0000 7fd75dd29740 1 -- start start 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6ba95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bc5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ab610 con 0x55bed6bade40 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ad150 con 0x55bed6b9f070 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd75dd29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541abce0 con 0x55bed6b9e750 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40384/0 (socket says 172.21.15.16:40384) 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 -- 172.21.15.16:0/1019108504 learned_addr learned my addr 172.21.15.16:0/1019108504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:29.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd758eeb640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bc5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd7596ec640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6ba95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 -- 172.21.15.16:0/1019108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 msgr2=0x55bed6ba95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6ba95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 -- 172.21.15.16:0/1019108504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 msgr2=0x55bed6bc5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bc5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 -- 172.21.15.16:0/1019108504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed6ba9ca0 con 0x55bed6bade40 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd758eeb640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bc5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd759eed640 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009ca0 tx=0x7fd74c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec55ada02a98bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.692+0000 7fd73effd640 1 -- 172.21.15.16:0/1019108504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c003e00 con 0x55bed6bade40 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd73effd640 1 -- 172.21.15.16:0/1019108504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c004520 con 0x55bed6bade40 2024-09-12T01:06:29.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd73effd640 1 -- 172.21.15.16:0/1019108504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c02fc30 con 0x55bed6bade40 2024-09-12T01:06:29.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- 172.21.15.16:0/1019108504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 msgr2=0x55bed6bc5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009ca0 tx=0x7fd74c004a80 comp rx=0 tx=0).stop 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- 172.21.15.16:0/1019108504 shutdown_connections 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bc5ec0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9f070 0x55bed6bc5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 --2- 172.21.15.16:0/1019108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9e750 0x55bed6ba95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- 172.21.15.16:0/1019108504 >> 172.21.15.16:0/1019108504 conn(0x55bed6b9ade0 msgr2=0x55bed6ba0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- 172.21.15.16:0/1019108504 shutdown_connections 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- 172.21.15.16:0/1019108504 wait complete. 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.693+0000 7fd75dd29740 1 -- start start 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 Processor -- start 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541aca80 con 0x55bed6bade40 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541ac3b0 con 0x55bed6b9e750 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7541a9610 con 0x55bed6b9f070 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd75dd29740 10 client.0 _ll_register_callbacks cb 0x7fd75000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd759eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd7596ec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:33400/0 (socket says 172.21.15.16:33400) 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd7596ec640 1 -- 172.21.15.16:0/2341797335 learned_addr learned my addr 172.21.15.16:0/2341797335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.694+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 -- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 msgr2=0x55bed6bb1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bb1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 -- 172.21.15.16:0/2341797335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 msgr2=0x55bed6bb2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 -- 172.21.15.16:0/2341797335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed6bb2840 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd759eed640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd7596ec640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd750097790 tx=0x7fd750099840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc8dc6318d711ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750020070 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c004520 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd7500bb0b0 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd75009ac90 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd75dd29740 10 client.27398 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bed6c28a40 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c004520 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.695+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd75006e040 con 0x55bed6b9e750 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 10 client.27398 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 10 client.27398 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 20 client.27398 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6c29040 0x55bed6c2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 4 client.27398 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55bed6b0abe0 con 0x55bed6c29040 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6c29040 0x55bed6c2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd73cff9640 10 client.27398 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.696+0000 7fd758eeb640 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6c29040 0x55bed6c2b430 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7506b71043e661 server_cookie=dfe64c06117cc389 in_seq=0 out_seq=0 2024-09-12T01:06:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.708+0000 7fd73cff9640 1 -- 172.21.15.16:0/2341797335 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bed6b0abe0 con 0x55bed6c29040 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.708+0000 7fd73cff9640 10 client.27398 handle_command_reply: tid=0 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.708+0000 7fd75dd29740 1 client.27398 shutdown 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 msgr2=0x55bed6bb66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd750097790 tx=0x7fd750099840 comp rx=0 tx=0).stop 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 shutdown_connections 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bed6c29040 0x55bed6c2b430 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bed6bade40 0x55bed6bb2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bed6b9f070 0x55bed6bb1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd73cff9640 0 client.27398 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 --2- 172.21.15.16:0/2341797335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bed6b9e750 0x55bed6bb66f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 >> 172.21.15.16:0/2341797335 conn(0x55bed6b9ade0 msgr2=0x55bed6ba0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 shutdown_connections 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 1 -- 172.21.15.16:0/2341797335 wait complete. 2024-09-12T01:06:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75dd29740 20 client.27398 trim_cache size 0 max 16384 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 42, 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "233533b6": { 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 42, 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:29.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:29.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 msgr2=0x7fd7400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 0x7fd7400a5230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd74c004af0 tx=0x7fd74c02f310 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 msgr2=0x7fd7541983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.709+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd75000d770 tx=0x7fd75000dc30 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 shutdown_connections 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7400a2d70 0x7fd7400a5230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd754106f10 0x7fd75419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd754102790 0x7fd754198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 --2- 172.21.15.16:0/3273523761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd754101de0 0x7fd7541983d0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 >> 172.21.15.16:0/3273523761 conn(0x7fd7540fd8c0 msgr2=0x7fd7540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 shutdown_connections 2024-09-12T01:06:29.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:29.710+0000 7fd75b950640 1 -- 172.21.15.16:0/3273523761 wait complete. 2024-09-12T01:06:29.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:06:30.491 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '233533b6', quiesce took 3.4 seconds. Will release after: 19.150000000000002 2024-09-12T01:06:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:30.821 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:30 smithi016 ceph-mon[30972]: pgmap v322: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T01:06:30.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:30 smithi159 ceph-mon[35566]: pgmap v322: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T01:06:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:30 smithi099 ceph-mon[35654]: pgmap v322: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 -- 172.21.15.16:0/383669745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 msgr2=0x7fcacc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 --2- 172.21.15.16:0/383669745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc100ee0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009930 tx=0x7fcac002a420 comp rx=0 tx=0).stop 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 -- 172.21.15.16:0/383669745 shutdown_connections 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 --2- 172.21.15.16:0/383669745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc101420 0x7fcacc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 --2- 172.21.15.16:0/383669745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc100ee0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 --2- 172.21.15.16:0/383669745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc106a80 0x7fcacc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 -- 172.21.15.16:0/383669745 >> 172.21.15.16:0/383669745 conn(0x7fcacc0fc7a0 msgr2=0x7fcacc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.057+0000 7fcad1df5640 1 -- 172.21.15.16:0/383669745 shutdown_connections 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.058+0000 7fcad1df5640 1 -- 172.21.15.16:0/383669745 wait complete. 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.058+0000 7fcad1df5640 1 Processor -- start 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.058+0000 7fcad1df5640 1 -- start start 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.058+0000 7fcad1df5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.058+0000 7fcad1df5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 0x7fcacc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc106a80 0x7fcacc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc110e80 con 0x7fcacc100a80 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc110d00 con 0x7fcacc106a80 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc111000 con 0x7fcacc101420 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40410/0 (socket says 172.21.15.16:40410) 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 -- 172.21.15.16:0/3192799968 learned_addr learned my addr 172.21.15.16:0/3192799968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacaffd640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 0x7fcacc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacbfff640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc106a80 0x7fcacc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 -- 172.21.15.16:0/3192799968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 msgr2=0x7fcacc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 0x7fcacc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 -- 172.21.15.16:0/3192799968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc106a80 msgr2=0x7fcacc1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc106a80 0x7fcacc1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 -- 172.21.15.16:0/3192799968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc1a48f0 con 0x7fcacc100a80 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacaffd640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 0x7fcacc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00eb90 tx=0x7fcabc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcac8ff9640 1 -- 172.21.15.16:0/3192799968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc004d20 con 0x7fcacc100a80 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc1a4ac0 con 0x7fcacc100a80 2024-09-12T01:06:31.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcac8ff9640 1 -- 172.21.15.16:0/3192799968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc00ee90 con 0x7fcacc100a80 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcac8ff9640 1 -- 172.21.15.16:0/3192799968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc010830 con 0x7fcacc100a80 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.059+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a4f80 con 0x7fcacc100a80 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcac8ff9640 1 -- 172.21.15.16:0/3192799968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcabc005020 con 0x7fcacc100a80 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 Processor -- start 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 -- start start 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cc73a0 0x5651d2cc7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cd6a90 0x5651d2cd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651d2c2d2e0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5651d2bd4ce0 con 0x5651d2cd6a90 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcad41ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651d2c292a0 con 0x5651d2cc73a0 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40422/0 (socket says 172.21.15.16:40422) 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcacb7fe640 1 -- 172.21.15.16:0/4022579724 learned_addr learned my addr 172.21.15.16:0/4022579724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcacbfff640 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cc73a0 0x5651d2cc7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.062+0000 7fcacaffd640 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cd6a90 0x5651d2cd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 -- 172.21.15.16:0/4022579724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cc73a0 msgr2=0x5651d2cc7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cc73a0 0x5651d2cc7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcac8ff9640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 0x7fcab40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 -- 172.21.15.16:0/4022579724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cd6a90 msgr2=0x5651d2cd8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cd6a90 0x5651d2cd8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 -- 172.21.15.16:0/4022579724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d2bb9440 con 0x5651d2cc7cc0 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacbfff640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 0x7fcab40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cd200 tx=0x7fcabc098160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3f46f1d5acf9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcac8ff9640 1 -- 172.21.15.16:0/3192799968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fcabc0cfbd0 con 0x7fcacc100a80 2024-09-12T01:06:31.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcab1ffb640 1 -- 172.21.15.16:0/4022579724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09be60 con 0x5651d2cc7cc0 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcab1ffb640 1 -- 172.21.15.16:0/4022579724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcabc0aa550 con 0x5651d2cc7cc0 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.063+0000 7fcab1ffb640 1 -- 172.21.15.16:0/4022579724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0aad10 con 0x5651d2cc7cc0 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- 172.21.15.16:0/4022579724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 msgr2=0x5651d2cd6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcacbfff640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 0x7fcab40a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac400d480 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0cd200 tx=0x7fcabc098160 comp rx=0 tx=0).stop 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- 172.21.15.16:0/4022579724 shutdown_connections 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cd6a90 0x5651d2cd8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd6550 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 --2- 172.21.15.16:0/4022579724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cc73a0 0x5651d2cc7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- 172.21.15.16:0/4022579724 >> 172.21.15.16:0/4022579724 conn(0x5651d2cc3a30 msgr2=0x5651d2cc3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- 172.21.15.16:0/4022579724 shutdown_connections 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- 172.21.15.16:0/4022579724 wait complete. 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.064+0000 7fcad41ce740 1 -- start start 2024-09-12T01:06:31.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 Processor -- start 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5651d2cd96c0 con 0x5651d2cc73a0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5651d2be8050 con 0x5651d2cc7cc0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651d2cd9840 con 0x5651d2cd6a90 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcad41ce740 10 client.0 _ll_register_callbacks cb 0x5651d2c28160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40438/0 (socket says 172.21.15.16:40438) 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 -- 172.21.15.16:0/2879698192 learned_addr learned my addr 172.21.15.16:0/2879698192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacaffd640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacbfff640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 -- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 msgr2=0x5651d2cd3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 -- 172.21.15.16:0/2879698192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 msgr2=0x5651d2cd36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 -- 172.21.15.16:0/2879698192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d299e3c0 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacaffd640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00cba0 tx=0x7fcabc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8654ef284bdb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.065+0000 7fcaaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09b9f0 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fcabc09a390 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651d2c3a850 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcabc09bbd0 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 10 client.18621 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651d2ce50f0 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5651d2c3a850 con 0x5651d2cc73a0 2024-09-12T01:06:31.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcabc097de0 con 0x5651d2cc73a0 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 10 client.18621 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 20 client.18621 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2ce6440 0x5651d2ce8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 4 client.18621 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651d2c34d60 con 0x5651d2ce6440 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.066+0000 7fcacaffd640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2ce6440 0x5651d2ce8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.067+0000 7fcaaaffd640 10 client.18621 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.067+0000 7fcacaffd640 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2ce6440 0x5651d2ce8830 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879b10488c2a2519 server_cookie=9ab4b25e2f84550d in_seq=0 out_seq=0 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcaaaffd640 1 -- 172.21.15.16:0/2879698192 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651d2c34d60 con 0x5651d2ce6440 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcaaaffd640 10 client.18621 handle_command_reply: tid=0 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 client.18621 shutdown 2024-09-12T01:06:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 msgr2=0x5651d2cd6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00cba0 tx=0x7fcabc00cd50 comp rx=0 tx=0).stop 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 shutdown_connections 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2ce6440 0x5651d2ce8830 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc7cc0 0x5651d2cd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 --2- 172.21.15.16:0/2879698192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc73a0 0x5651d2cd6420 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 >> 172.21.15.16:0/2879698192 conn(0x5651d2cc3a30 msgr2=0x5651d2cc96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcaaaffd640 0 client.18621 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 shutdown_connections 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 1 -- 172.21.15.16:0/2879698192 wait complete. 2024-09-12T01:06:31.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.068+0000 7fcad41ce740 20 client.18621 trim_cache size 0 max 16384 2024-09-12T01:06:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.085+0000 7fcad41ce740 1 Processor -- start 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.085+0000 7fcad41ce740 1 -- start start 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ceb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1ab730 con 0x5651d2cc7cc0 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1ad270 con 0x5651d2cc73a0 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcad41ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1abe00 con 0x5651d2cd6a90 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40440/0 (socket says 172.21.15.16:40440) 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 -- 172.21.15.16:0/3606492658 learned_addr learned my addr 172.21.15.16:0/3606492658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ceb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacbfff640 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 -- 172.21.15.16:0/3606492658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 msgr2=0x5651d2cd0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 -- 172.21.15.16:0/3606492658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 msgr2=0x5651d2ceb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ceb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 -- 172.21.15.16:0/3606492658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651d2cc8380 con 0x5651d2cc7cc0 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcacaffd640 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcac002a940 tx=0x7fcac0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7777ac5945cb6129 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcab17fa640 1 -- 172.21.15.16:0/3606492658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0049070 con 0x5651d2cc7cc0 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcab17fa640 1 -- 172.21.15.16:0/3606492658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac00044d0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.086+0000 7fcab17fa640 1 -- 172.21.15.16:0/3606492658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0035040 con 0x5651d2cc7cc0 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 -- 172.21.15.16:0/3606492658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 msgr2=0x5651d2ceb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcac002a940 tx=0x7fcac0005b50 comp rx=0 tx=0).stop 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 -- 172.21.15.16:0/3606492658 shutdown_connections 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2cd0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ceb9b0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 --2- 172.21.15.16:0/3606492658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ceb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 -- 172.21.15.16:0/3606492658 >> 172.21.15.16:0/3606492658 conn(0x5651d2cc3a30 msgr2=0x5651d2cced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 -- 172.21.15.16:0/3606492658 shutdown_connections 2024-09-12T01:06:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.087+0000 7fcad41ce740 1 -- 172.21.15.16:0/3606492658 wait complete. 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 -- start start 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 Processor -- start 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ce83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2ce8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1acba0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1ac4d0 con 0x5651d2cc73a0 2024-09-12T01:06:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc1a9730 con 0x5651d2cd6a90 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcad41ce740 10 client.0 _ll_register_callbacks cb 0x7fcabc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcacaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40454/0 (socket says 172.21.15.16:40454) 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcacaffd640 1 -- 172.21.15.16:0/903146917 learned_addr learned my addr 172.21.15.16:0/903146917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcacbfff640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2ce8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.088+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ce83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 -- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 msgr2=0x5651d2ce8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2ce8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 -- 172.21.15.16:0/903146917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 msgr2=0x5651d2ce83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ce83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 -- 172.21.15.16:0/903146917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651d2d4ef70 con 0x5651d2cc7cc0 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ce83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcacaffd640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4a9f3f7d113e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcaaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0052060 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fcac00048c0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac00044d0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcac0049040 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcad41ce740 10 client.18627 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651d2d50220 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.089+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac00044d0 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcac0003a30 con 0x5651d2cc7cc0 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 10 client.18627 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 20 client.18627 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2d50920 0x5651d2d52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 4 client.18627 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651d2bb1210 con 0x5651d2d50920 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2d50920 0x5651d2d52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcacb7fe640 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2d50920 0x5651d2d52d40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35baadcb8ee619bc server_cookie=c3f89216589ed5dd in_seq=0 out_seq=0 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.090+0000 7fcaaaffd640 10 client.18627 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcaaaffd640 1 -- 172.21.15.16:0/903146917 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651d2bb1210 con 0x5651d2d50920 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcaaaffd640 10 client.18627 handle_command_reply: tid=0 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 client.18627 shutdown 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 msgr2=0x5651d2ce88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac00042d0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 shutdown_connections 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5651d2d50920 0x5651d2d52d40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcaaaffd640 0 client.18627 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651d2cd6a90 0x5651d2ce8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651d2cc7cc0 0x5651d2ce88f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 --2- 172.21.15.16:0/903146917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5651d2cc73a0 0x5651d2ce83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 >> 172.21.15.16:0/903146917 conn(0x5651d2cc3a30 msgr2=0x5651d2cd1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 shutdown_connections 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 1 -- 172.21.15.16:0/903146917 wait complete. 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.091+0000 7fcad41ce740 20 client.18627 trim_cache size 0 max 16384 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:06:31.096 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 msgr2=0x7fcab40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 0x7fcab40a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005fd0 tx=0x7fcac400d480 comp rx=0 tx=0).stop 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 msgr2=0x7fcacc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00eb90 tx=0x7fcabc00c6a0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 shutdown_connections 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcab40a2cf0 0x7fcab40a51b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcacc106a80 0x7fcacc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcacc101420 0x7fcacc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 --2- 172.21.15.16:0/3192799968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcacc100a80 0x7fcacc199330 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 >> 172.21.15.16:0/3192799968 conn(0x7fcacc0fc7a0 msgr2=0x7fcacc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 shutdown_connections 2024-09-12T01:06:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:31.092+0000 7fcad1df5640 1 -- 172.21.15.16:0/3192799968 wait complete. 2024-09-12T01:06:31.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:06:31.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 -- 172.21.15.16:0/3566894189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880102570 msgr2=0x7f38801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 --2- 172.21.15.16:0/3566894189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880102570 0x7f38801029b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f386c009900 tx=0x7f386c02a480 comp rx=0 tx=0).stop 2024-09-12T01:06:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 -- 172.21.15.16:0/3566894189 shutdown_connections 2024-09-12T01:06:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 --2- 172.21.15.16:0/3566894189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 0x7f388010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 --2- 172.21.15.16:0/3566894189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880102570 0x7f38801029b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 --2- 172.21.15.16:0/3566894189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880100e20 0x7f3880101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 -- 172.21.15.16:0/3566894189 >> 172.21.15.16:0/3566894189 conn(0x7f38800fc7d0 msgr2=0x7f38800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 -- 172.21.15.16:0/3566894189 shutdown_connections 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.681+0000 7f3886da2640 1 -- 172.21.15.16:0/3566894189 wait complete. 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.682+0000 7f3886da2640 1 Processor -- start 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.682+0000 7f3886da2640 1 -- start start 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.682+0000 7f3886da2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.682+0000 7f3886da2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880102570 0x7f3880198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.682+0000 7f3886da2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 0x7f388019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3886da2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3880110fc0 con 0x7f3880100e20 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3886da2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3880110e40 con 0x7f3880102570 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3886da2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3880111140 con 0x7f3880102ef0 2024-09-12T01:06:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40472/0 (socket says 172.21.15.16:40472) 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 -- 172.21.15.16:0/2909003218 learned_addr learned my addr 172.21.15.16:0/2909003218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f388533f640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 0x7f388019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f387ffff640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880102570 0x7f3880198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 -- 172.21.15.16:0/2909003218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 msgr2=0x7f388019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 0x7f388019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 -- 172.21.15.16:0/2909003218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880102570 msgr2=0x7f3880198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880102570 0x7f3880198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 -- 172.21.15.16:0/2909003218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388019d4f0 con 0x7f3880100e20 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3884b3e640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f387400b4a0 tx=0x7f387400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874014070 con 0x7f3880100e20 2024-09-12T01:06:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388019d7e0 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874010af0 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.683+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874017d10 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.684+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38801a50c0 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.684+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3880101200 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.686+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3874017070 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.686+0000 7f387dffb640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 0x7f38700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.686+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f38740ce660 con 0x7f3880100e20 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.686+0000 7f387ffff640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 0x7f38700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.687+0000 7f387ffff640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 0x7f38700a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f386c009230 tx=0x7f386c0058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.691+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f387409ad40 con 0x7f3880100e20 2024-09-12T01:06:32.868 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:32 smithi159 ceph-mon[35566]: pgmap v323: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:32.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:32 smithi016 ceph-mon[30972]: pgmap v323: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.905+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388010a0a0 con 0x7f3880100e20 2024-09-12T01:06:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.906+0000 7f387dffb640 1 -- 172.21.15.16:0/2909003218 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f38740b2ed0 con 0x7f3880100e20 2024-09-12T01:06:32.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:32.906 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 msgr2=0x7f38700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 0x7f38700a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f386c009230 tx=0x7f386c0058d0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 msgr2=0x7f3880198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f387400b4a0 tx=0x7f387400b960 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 shutdown_connections 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38700a2cf0 0x7f38700a51b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3880102ef0 0x7f388019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3880102570 0x7f3880198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 --2- 172.21.15.16:0/2909003218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3880100e20 0x7f3880198540 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 >> 172.21.15.16:0/2909003218 conn(0x7f38800fc7d0 msgr2=0x7f388006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 shutdown_connections 2024-09-12T01:06:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:32.908+0000 7f3886da2640 1 -- 172.21.15.16:0/2909003218 wait complete. 2024-09-12T01:06:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:32 smithi099 ceph-mon[35654]: pgmap v323: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:33.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:06:33.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:33 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2909003218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:34.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:33 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2909003218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:34.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:33 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2909003218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 -- 172.21.15.16:0/1450847544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381036c0 msgr2=0x7f493810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 --2- 172.21.15.16:0/1450847544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381036c0 0x7f493810bbb0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4928009900 tx=0x7f492802a480 comp rx=0 tx=0).stop 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 -- 172.21.15.16:0/1450847544 shutdown_connections 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 --2- 172.21.15.16:0/1450847544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f493810c180 0x7f493810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 --2- 172.21.15.16:0/1450847544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49381036c0 0x7f493810bbb0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 --2- 172.21.15.16:0/1450847544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4938102da0 0x7f4938103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 -- 172.21.15.16:0/1450847544 >> 172.21.15.16:0/1450847544 conn(0x7f49380fc7a0 msgr2=0x7f49380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 -- 172.21.15.16:0/1450847544 shutdown_connections 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 -- 172.21.15.16:0/1450847544 wait complete. 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.400+0000 7f4940751640 1 Processor -- start 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 -- start start 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49381036c0 0x7f4938100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f493810c180 0x7f4938101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4938111f60 con 0x7f4938102da0 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4938111de0 con 0x7f493810c180 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f4940751640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49381120e0 con 0x7f49381036c0 2024-09-12T01:06:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40482/0 (socket says 172.21.15.16:40482) 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 -- 172.21.15.16:0/347819427 learned_addr learned my addr 172.21.15.16:0/347819427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493dcec640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49381036c0 0x7f4938100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493ecee640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f493810c180 0x7f4938101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 -- 172.21.15.16:0/347819427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49381036c0 msgr2=0x7f4938100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49381036c0 0x7f4938100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 -- 172.21.15.16:0/347819427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f493810c180 msgr2=0x7f4938101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f493810c180 0x7f4938101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.401+0000 7f493e4ed640 1 -- 172.21.15.16:0/347819427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381a68a0 con 0x7f4938102da0 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f493e4ed640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f493400d950 tx=0x7f493400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f492f7fe640 1 -- 172.21.15.16:0/347819427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934017070 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f492f7fe640 1 -- 172.21.15.16:0/347819427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934009c30 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f492f7fe640 1 -- 172.21.15.16:0/347819427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400f6b0 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49381a6b90 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.402+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49381a70f0 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.404+0000 7f492f7fe640 1 -- 172.21.15.16:0/347819427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f49340109d0 con 0x7f4938102da0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 Processor -- start 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 -- start start 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff08090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55812fe6e2f0 con 0x55812ff085d0 2024-09-12T01:06:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55812fe15cf0 con 0x55812ff09b40 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f4942b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55812fe6a2b0 con 0x55812ff13370 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40496/0 (socket says 172.21.15.16:40496) 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f493e4ed640 1 -- 172.21.15.16:0/1031079452 learned_addr learned my addr 172.21.15.16:0/1031079452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f492f7fe640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 0x7f49240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f493dcec640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.405+0000 7f493ecee640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff08090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 -- 172.21.15.16:0/1031079452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 msgr2=0x55812ff15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 -- 172.21.15.16:0/1031079452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 msgr2=0x55812ff08090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff08090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 -- 172.21.15.16:0/1031079452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812fdfa3a0 con 0x55812ff085d0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493ecee640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493ecee640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 0x7f49240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f492f7fe640 1 -- 172.21.15.16:0/347819427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f49340cef90 con 0x7f4938102da0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f49340cc5c0 tx=0x7f4934066130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97212efdaad0460 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f492cff9640 1 -- 172.21.15.16:0/1031079452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934020070 con 0x55812ff085d0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f493ecee640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 0x7f49240a5230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4930007e90 tx=0x7f4930004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f492cff9640 1 -- 172.21.15.16:0/1031079452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934009de0 con 0x55812ff085d0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.406+0000 7f492cff9640 1 -- 172.21.15.16:0/1031079452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493406ea70 con 0x55812ff085d0 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- 172.21.15.16:0/1031079452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 msgr2=0x55812ff08a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f49340cc5c0 tx=0x7f4934066130 comp rx=0 tx=0).stop 2024-09-12T01:06:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- 172.21.15.16:0/1031079452 shutdown_connections 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff08a10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1031079452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff08090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- 172.21.15.16:0/1031079452 >> 172.21.15.16:0/1031079452 conn(0x55812fed1310 msgr2=0x55812ff04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- 172.21.15.16:0/1031079452 shutdown_connections 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- 172.21.15.16:0/1031079452 wait complete. 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.407+0000 7f4942b2a740 1 -- start start 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 Processor -- start 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff12d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff10570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55812ff16010 con 0x55812ff09b40 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55812fe29060 con 0x55812ff085d0 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55812ff16190 con 0x55812ff13370 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f4942b2a740 10 client.0 _ll_register_callbacks cb 0x55812fb7c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff12d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40512/0 (socket says 172.21.15.16:40512) 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 -- 172.21.15.16:0/4206198455 learned_addr learned my addr 172.21.15.16:0/4206198455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493ecee640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff10570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 -- 172.21.15.16:0/4206198455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 msgr2=0x55812ff10570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff10570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 -- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 msgr2=0x55812ff12d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff12d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.408+0000 7f493dcec640 1 -- 172.21.15.16:0/4206198455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812fbdf3c0 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f493dcec640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f492802a9a0 tx=0x7f49280319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4770f59f7458ee7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928035040 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f4928004300 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4934009de0 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4928005020 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 10 client.18645 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55812ff21a20 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934009de0 con 0x55812ff09b40 2024-09-12T01:06:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f49280035f0 con 0x55812ff09b40 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 10 client.18645 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 10 client.18645 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 20 client.18645 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff22e30 0x55812ff25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 4 client.18645 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.409+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55812fe7b860 con 0x55812ff22e30 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.410+0000 7f493e4ed640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff22e30 0x55812ff25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.410+0000 7f4921ffb640 10 client.18645 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.410+0000 7f493e4ed640 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff22e30 0x55812ff25290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c149db90aad9db5 server_cookie=abd39f1f759bdb0c in_seq=0 out_seq=0 2024-09-12T01:06:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4921ffb640 1 -- 172.21.15.16:0/4206198455 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55812fe7b860 con 0x55812ff22e30 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4921ffb640 10 client.18645 handle_command_reply: tid=0 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 client.18645 shutdown 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 msgr2=0x55812ff0ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f492802a9a0 tx=0x7f49280319b0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 shutdown_connections 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff22e30 0x55812ff25290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff10570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff09b40 0x55812ff0ffd0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 --2- 172.21.15.16:0/4206198455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff12d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 >> 172.21.15.16:0/4206198455 conn(0x55812fed1310 msgr2=0x55812ff0a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4921ffb640 0 client.18645 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 shutdown_connections 2024-09-12T01:06:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 1 -- 172.21.15.16:0/4206198455 wait complete. 2024-09-12T01:06:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.411+0000 7f4942b2a740 20 client.18645 trim_cache size 0 max 16384 2024-09-12T01:06:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 Processor -- start 2024-09-12T01:06:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 -- start start 2024-09-12T01:06:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff2ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49381afa30 con 0x55812ff085d0 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b1570 con 0x55812ff09b40 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f4942b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b0100 con 0x55812ff13370 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40528/0 (socket says 172.21.15.16:40528) 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 -- 172.21.15.16:0/3831335641 learned_addr learned my addr 172.21.15.16:0/3831335641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493ecee640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493dcec640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff2ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 -- 172.21.15.16:0/3831335641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 msgr2=0x55812ff2b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff2b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 -- 172.21.15.16:0/3831335641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 msgr2=0x55812ff2ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff2ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493e4ed640 1 -- 172.21.15.16:0/3831335641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812ff04ed0 con 0x55812ff085d0 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.425+0000 7f493ecee640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f493e4ed640 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4934097ed0 tx=0x7f4934000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed84c323983f5888 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f4923fff640 1 -- 172.21.15.16:0/3831335641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934020040 con 0x55812ff085d0 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f4923fff640 1 -- 172.21.15.16:0/3831335641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493409b500 con 0x55812ff085d0 2024-09-12T01:06:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f4923fff640 1 -- 172.21.15.16:0/3831335641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49340995a0 con 0x55812ff085d0 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f4942b2a740 1 -- 172.21.15.16:0/3831335641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 msgr2=0x55812ff1f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.426+0000 7f4942b2a740 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f4934097ed0 tx=0x7f4934000bc0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- 172.21.15.16:0/3831335641 shutdown_connections 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff13370 0x55812ff2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff09b40 0x55812ff2ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- 172.21.15.16:0/3831335641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff085d0 0x55812ff1f2b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- 172.21.15.16:0/3831335641 >> 172.21.15.16:0/3831335641 conn(0x55812fed1310 msgr2=0x55812ff0b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- 172.21.15.16:0/3831335641 shutdown_connections 2024-09-12T01:06:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- 172.21.15.16:0/3831335641 wait complete. 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- start start 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 Processor -- start 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff09b40 0x55812ff17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b0ea0 con 0x55812ff13370 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b07d0 con 0x55812ff085d0 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49381ada30 con 0x55812ff09b40 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f4942b2a740 10 client.0 _ll_register_callbacks cb 0x7f4928009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f493ecee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f493ecee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40532/0 (socket says 172.21.15.16:40532) 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f493ecee640 1 -- 172.21.15.16:0/1138331151 learned_addr learned my addr 172.21.15.16:0/1138331151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.427+0000 7f493dcec640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff09b40 0x55812ff17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 -- 172.21.15.16:0/1138331151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff09b40 msgr2=0x55812ff17f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff09b40 0x55812ff17f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 -- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 msgr2=0x55812ff17a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff17a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 -- 172.21.15.16:0/1138331151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812ff87140 con 0x55812ff13370 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f493ecee640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4930012ca0 tx=0x7f4930007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee50f78f5219c492 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4921ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493001e9a0 con 0x55812ff13370 2024-09-12T01:06:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f493001eb40 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493409b500 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f493001ed40 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4942b2a740 10 client.18651 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.428+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55812ff8e480 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493409b500 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4930018040 con 0x55812ff13370 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 10 client.18651 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 10 client.18651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 20 client.18651 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff8ec00 0x55812ff91020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 4 client.18651 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55812fe75d70 con 0x55812ff8ec00 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.429+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff8ec00 0x55812ff91020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4921ffb640 10 client.18651 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f493e4ed640 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff8ec00 0x55812ff91020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e7060ac4df4983d server_cookie=298387074e1e4ae7 in_seq=0 out_seq=0 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4921ffb640 1 -- 172.21.15.16:0/1138331151 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55812fe75d70 con 0x55812ff8ec00 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4921ffb640 10 client.18651 handle_command_reply: tid=0 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 client.18651 shutdown 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 msgr2=0x55812ff86b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4930012ca0 tx=0x7f4930007520 comp rx=0 tx=0).stop 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 shutdown_connections 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55812ff8ec00 0x55812ff91020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812ff13370 0x55812ff86b00 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55812ff09b40 0x55812ff17f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4921ffb640 0 client.18651 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 --2- 172.21.15.16:0/1138331151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55812ff085d0 0x55812ff17a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 >> 172.21.15.16:0/1138331151 conn(0x55812fed1310 msgr2=0x55812ff11830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 shutdown_connections 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 1 -- 172.21.15.16:0/1138331151 wait complete. 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.430+0000 7f4942b2a740 20 client.18651 trim_cache size 0 max 16384 2024-09-12T01:06:34.436 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 msgr2=0x7f49240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 0x7f49240a5230 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4930007e90 tx=0x7f4930004a40 comp rx=0 tx=0).stop 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 msgr2=0x7f4938100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f493400d950 tx=0x7f493400de10 comp rx=0 tx=0).stop 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 shutdown_connections 2024-09-12T01:06:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f49240a2d70 0x7f49240a5230 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f493810c180 0x7f4938101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49381036c0 0x7f4938100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 --2- 172.21.15.16:0/347819427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938102da0 0x7f4938100a00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 >> 172.21.15.16:0/347819427 conn(0x7f49380fc7a0 msgr2=0x7f49380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 shutdown_connections 2024-09-12T01:06:34.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:34.431+0000 7f4940751640 1 -- 172.21.15.16:0/347819427 wait complete. 2024-09-12T01:06:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:34 smithi016 ceph-mon[30972]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:34 smithi099 ceph-mon[35654]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:34 smithi159 ceph-mon[35566]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 9 op/s 2024-09-12T01:06:35.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:06:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 -- 172.21.15.16:0/2912542286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102de0 msgr2=0x7ff7f81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 --2- 172.21.15.16:0/2912542286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102de0 0x7ff7f81031c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e802a480 comp rx=0 tx=0).stop 2024-09-12T01:06:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 -- 172.21.15.16:0/2912542286 shutdown_connections 2024-09-12T01:06:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 --2- 172.21.15.16:0/2912542286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f810c180 0x7ff7f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 --2- 172.21.15.16:0/2912542286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8103700 0x7ff7f810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 --2- 172.21.15.16:0/2912542286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102de0 0x7ff7f81031c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:35.999+0000 7ff7fe467640 1 -- 172.21.15.16:0/2912542286 >> 172.21.15.16:0/2912542286 conn(0x7ff7f80fc7a0 msgr2=0x7ff7f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.000+0000 7ff7fe467640 1 -- 172.21.15.16:0/2912542286 shutdown_connections 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.000+0000 7ff7fe467640 1 -- 172.21.15.16:0/2912542286 wait complete. 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.000+0000 7ff7fe467640 1 Processor -- start 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.000+0000 7ff7fe467640 1 -- start start 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102de0 0x7ff7f8100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 0x7ff7f81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111f70 con 0x7ff7f8103700 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111df0 con 0x7ff7f8102de0 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fe467640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81120f0 con 0x7ff7f810c180 2024-09-12T01:06:36.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40544/0 (socket says 172.21.15.16:40544) 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 -- 172.21.15.16:0/280142167 learned_addr learned my addr 172.21.15.16:0/280142167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102de0 0x7ff7f8100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fca04640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 0x7ff7f81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 -- 172.21.15.16:0/280142167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 msgr2=0x7ff7f81014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 0x7ff7f81014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 -- 172.21.15.16:0/280142167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102de0 msgr2=0x7ff7f8100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102de0 0x7ff7f8100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 -- 172.21.15.16:0/280142167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a6930 con 0x7ff7f8103700 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7fca04640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 0x7ff7f81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000d950 tx=0x7ff7e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.001+0000 7ff7f57fa640 1 -- 172.21.15.16:0/280142167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0017070 con 0x7ff7f8103700 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.002+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a6c20 con 0x7ff7f8103700 2024-09-12T01:06:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.002+0000 7ff7f57fa640 1 -- 172.21.15.16:0/280142167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e0009c30 con 0x7ff7f8103700 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.002+0000 7ff7f57fa640 1 -- 172.21.15.16:0/280142167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e000f6b0 con 0x7ff7f8103700 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.002+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a71b0 con 0x7ff7f8103700 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff7f57fa640 1 -- 172.21.15.16:0/280142167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff7e00109d0 con 0x7ff7f8103700 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 Processor -- start 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 -- start start 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7bd5d0 0x56457d7bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 0x56457d7ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56457d7232f0 con 0x56457d7beb40 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56457d6cacf0 con 0x56457d7c8370 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff800840740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56457d71f2b0 con 0x56457d7bd5d0 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff7fca04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff7fca04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40552/0 (socket says 172.21.15.16:40552) 2024-09-12T01:06:36.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.004+0000 7ff7fca04640 1 -- 172.21.15.16:0/1279530728 learned_addr learned my addr 172.21.15.16:0/1279530728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7bd5d0 0x56457d7bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7f57fa640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 0x56457d7ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 -- 172.21.15.16:0/1279530728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7bd5d0 msgr2=0x56457d7bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7bd5d0 0x56457d7bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 -- 172.21.15.16:0/1279530728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 msgr2=0x56457d7ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 0x56457d7ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 -- 172.21.15.16:0/1279530728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457d6af3a0 con 0x56457d7beb40 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 0x56457d7ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7f57fa640 1 -- 172.21.15.16:0/280142167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff7e00cf060 con 0x7ff7f8103700 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7fca04640 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005f40 tx=0x7ff7f0010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee00691381e72067 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7ee7fc640 1 -- 172.21.15.16:0/1279530728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0019070 con 0x56457d7beb40 2024-09-12T01:06:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7ee7fc640 1 -- 172.21.15.16:0/1279530728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0022070 con 0x56457d7beb40 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.005+0000 7ff7ee7fc640 1 -- 172.21.15.16:0/1279530728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0011c10 con 0x56457d7beb40 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- 172.21.15.16:0/1279530728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 msgr2=0x56457d7bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff7fca04640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0021000 tx=0x7ff7f001f000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005f40 tx=0x7ff7f0010a00 comp rx=0 tx=0).stop 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- 172.21.15.16:0/1279530728 shutdown_connections 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7c8370 0x56457d7ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7bd5d0 0x56457d7bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 --2- 172.21.15.16:0/1279530728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7bd090 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- 172.21.15.16:0/1279530728 >> 172.21.15.16:0/1279530728 conn(0x56457d786310 msgr2=0x56457d7b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- 172.21.15.16:0/1279530728 shutdown_connections 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- 172.21.15.16:0/1279530728 wait complete. 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.006+0000 7ff800840740 1 -- start start 2024-09-12T01:06:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 Processor -- start 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7beb40 0x56457d7c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56457d7cb070 con 0x56457d7c8370 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56457d6de060 con 0x56457d7bd5d0 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56457d7cb1f0 con 0x56457d7beb40 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff800840740 10 client.0 _ll_register_callbacks cb 0x56457d4318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40564/0 (socket says 172.21.15.16:40564) 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 -- 172.21.15.16:0/3649966661 learned_addr learned my addr 172.21.15.16:0/3649966661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7beb40 0x56457d7c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 -- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7beb40 msgr2=0x56457d7c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7beb40 0x56457d7c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 -- 172.21.15.16:0/3649966661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 msgr2=0x56457d7c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.007+0000 7ff7fca04640 1 -- 172.21.15.16:0/3649966661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457d4943c0 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7fca04640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0013e70 tx=0x7ff7f0013b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0843b503096ddf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0019050 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff7f0011440 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f0022070 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7f000e5d0 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 10 client.18663 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56457d7d6ae0 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f0022070 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff7f001d990 con 0x56457d7c8370 2024-09-12T01:06:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 10 client.18663 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 10 client.18663 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 20 client.18663 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d7d7e90 0x56457d7da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 4 client.18663 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.008+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56457d730860 con 0x56457d7d7e90 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.009+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d7d7e90 0x56457d7da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.009+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d7d7e90 0x56457d7da280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf68c2248fa4314a server_cookie=c0764cd62b870dc4 in_seq=0 out_seq=0 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.009+0000 7ff7d77fe640 10 client.18663 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3649966661 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56457d730860 con 0x56457d7d7e90 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff7d77fe640 10 client.18663 handle_command_reply: tid=0 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 client.18663 shutdown 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 msgr2=0x56457d7c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0013e70 tx=0x7ff7f0013b60 comp rx=0 tx=0).stop 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 shutdown_connections 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d7d7e90 0x56457d7da280 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7c8370 0x56457d7c5640 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7beb40 0x56457d7c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 --2- 172.21.15.16:0/3649966661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff7d77fe640 0 client.18663 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 >> 172.21.15.16:0/3649966661 conn(0x56457d786310 msgr2=0x56457d7bf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 shutdown_connections 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 1 -- 172.21.15.16:0/3649966661 wait complete. 2024-09-12T01:06:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.010+0000 7ff800840740 20 client.18663 trim_cache size 0 max 16384 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.023+0000 7ff800840740 1 Processor -- start 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 -- start start 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 0x56457d7c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81afa50 con 0x56457d7bd5d0 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b1590 con 0x56457d7beb40 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff800840740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b0120 con 0x56457d7c8370 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40572/0 (socket says 172.21.15.16:40572) 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 -- 172.21.15.16:0/1185313887 learned_addr learned my addr 172.21.15.16:0/1185313887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 0x56457d7c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7fca04640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 -- 172.21.15.16:0/1185313887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 msgr2=0x56457d7c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 -- 172.21.15.16:0/1185313887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 msgr2=0x56457d7c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 0x56457d7c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 -- 172.21.15.16:0/1185313887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56457d7c80f0 con 0x56457d7bd5d0 2024-09-12T01:06:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7fca04640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 0x56457d7c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.024+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff7e802a9a0 tx=0x7ff7e80319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747e362cc2366fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff7edffb640 1 -- 172.21.15.16:0/1185313887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e804d030 con 0x56457d7bd5d0 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff7edffb640 1 -- 172.21.15.16:0/1185313887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e8005020 con 0x56457d7bd5d0 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff7edffb640 1 -- 172.21.15.16:0/1185313887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8035040 con 0x56457d7bd5d0 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 -- 172.21.15.16:0/1185313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 msgr2=0x56457d7d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff7e802a9a0 tx=0x7ff7e80319a0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 -- 172.21.15.16:0/1185313887 shutdown_connections 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7beb40 0x56457d7c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 --2- 172.21.15.16:0/1185313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7bd5d0 0x56457d7d4340 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 -- 172.21.15.16:0/1185313887 >> 172.21.15.16:0/1185313887 conn(0x56457d786310 msgr2=0x56457d7c4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 -- 172.21.15.16:0/1185313887 shutdown_connections 2024-09-12T01:06:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.025+0000 7ff800840740 1 -- 172.21.15.16:0/1185313887 wait complete. 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 -- start start 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 Processor -- start 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b0ec0 con 0x56457d7beb40 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81b07f0 con 0x56457d7bd5d0 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81ada50 con 0x56457d7c8370 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff800840740 10 client.0 _ll_register_callbacks cb 0x7ff7e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40580/0 (socket says 172.21.15.16:40580) 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.026+0000 7ff7f77fe640 1 -- 172.21.15.16:0/3684218106 learned_addr learned my addr 172.21.15.16:0/3684218106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:36.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7fca04640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 -- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 msgr2=0x56457d7c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 -- 172.21.15.16:0/3684218106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 msgr2=0x56457d7d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 -- 172.21.15.16:0/3684218106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457d7c2480 con 0x56457d7beb40 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7f77fe640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00979e0 tx=0x7ff7e009b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0606cbcf3a956 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0020040 con 0x56457d7beb40 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7ff7e00d10b0 con 0x56457d7beb40 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e8005020 con 0x56457d7beb40 2024-09-12T01:06:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7e009bc80 con 0x56457d7beb40 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff800840740 10 client.18669 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56457d842640 con 0x56457d7beb40 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.027+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e8005020 con 0x56457d7beb40 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e00b42b0 con 0x56457d7beb40 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 10 client.18669 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 10 client.18669 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 20 client.18669 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d842dc0 0x56457d8451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 4 client.18669 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56457d72ad70 con 0x56457d842dc0 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d842dc0 0x56457d8451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff7f7fff640 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d842dc0 0x56457d8451e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f5e2aa4e10c7d2d server_cookie=1e4304f50aea76d in_seq=0 out_seq=0 2024-09-12T01:06:36.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.028+0000 7ff7d77fe640 10 client.18669 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff7d77fe640 1 -- 172.21.15.16:0/3684218106 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56457d72ad70 con 0x56457d842dc0 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff7d77fe640 10 client.18669 handle_command_reply: tid=0 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 client.18669 shutdown 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 msgr2=0x56457d7d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00979e0 tx=0x7ff7e009b0c0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 shutdown_connections 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56457d7c8370 0x56457d7c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56457d842dc0 0x56457d8451e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56457d7beb40 0x56457d7d0da0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 --2- 172.21.15.16:0/3684218106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56457d7bd5d0 0x56457d7d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff7d77fe640 0 client.18669 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 >> 172.21.15.16:0/3684218106 conn(0x56457d786310 msgr2=0x56457d7c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 shutdown_connections 2024-09-12T01:06:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 1 -- 172.21.15.16:0/3684218106 wait complete. 2024-09-12T01:06:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff800840740 20 client.18669 trim_cache size 0 max 16384 2024-09-12T01:06:36.037 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.029+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 msgr2=0x7ff7e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0021000 tx=0x7ff7f001f000 comp rx=0 tx=0).stop 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 msgr2=0x7ff7f8100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000d950 tx=0x7ff7e000de10 comp rx=0 tx=0).stop 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 shutdown_connections 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f810c180 0x7ff7f81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f8103700 0x7ff7f8100f80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 --2- 172.21.15.16:0/280142167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7f8102de0 0x7ff7f8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 >> 172.21.15.16:0/280142167 conn(0x7ff7f80fc7a0 msgr2=0x7ff7f80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 shutdown_connections 2024-09-12T01:06:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:36.030+0000 7ff7fe467640 1 -- 172.21.15.16:0/280142167 wait complete. 2024-09-12T01:06:36.771 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:36 smithi016 ceph-mon[30972]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:06:36.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:06:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:36 smithi099 ceph-mon[35654]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:06:37.065 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:37.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:36 smithi159 ceph-mon[35566]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 -- 172.21.15.16:0/3746851166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 msgr2=0x7fa7f8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/3746851166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f8103110 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009930 tx=0x7fa7e802a420 comp rx=0 tx=0).stop 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 -- 172.21.15.16:0/3746851166 shutdown_connections 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/3746851166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 0x7fa7f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/3746851166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/3746851166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f8103110 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 -- 172.21.15.16:0/3746851166 >> 172.21.15.16:0/3746851166 conn(0x7fa7f80fc7d0 msgr2=0x7fa7f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 -- 172.21.15.16:0/3746851166 shutdown_connections 2024-09-12T01:06:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.569+0000 7fa7fddcb640 1 -- 172.21.15.16:0/3746851166 wait complete. 2024-09-12T01:06:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 Processor -- start 2024-09-12T01:06:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 -- start start 2024-09-12T01:06:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 0x7fa7f819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81120a0 con 0x7fa7f8102d30 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8111f20 con 0x7fa7f81036e0 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7fddcb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8112220 con 0x7fa7f81082a0 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40590/0 (socket says 172.21.15.16:40590) 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7f77fe640 1 -- 172.21.15.16:0/2653108589 learned_addr learned my addr 172.21.15.16:0/2653108589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.570+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 0x7fa7f819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 -- 172.21.15.16:0/2653108589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 msgr2=0x7fa7f819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 0x7fa7f819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 -- 172.21.15.16:0/2653108589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 msgr2=0x7fa7f8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 -- 172.21.15.16:0/2653108589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f819d570 con 0x7fa7f8102d30 2024-09-12T01:06:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009230 tx=0x7fa7e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f4ff9640 1 -- 172.21.15.16:0/2653108589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e803f030 con 0x7fa7f8102d30 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f819d800 con 0x7fa7f8102d30 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f4ff9640 1 -- 172.21.15.16:0/2653108589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e8003ad0 con 0x7fa7f8102d30 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f81a5140 con 0x7fa7f8102d30 2024-09-12T01:06:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.571+0000 7fa7f4ff9640 1 -- 172.21.15.16:0/2653108589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8035070 con 0x7fa7f8102d30 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa7f4ff9640 1 -- 172.21.15.16:0/2653108589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa7e8042030 con 0x7fa7f8102d30 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 Processor -- start 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 -- start start 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 0x55bea76d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bea76292f0 con 0x55bea76c35d0 2024-09-12T01:06:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bea75d0cf0 con 0x55bea76c4b40 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa8001a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bea76252b0 con 0x55bea76d2780 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa7f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40592/0 (socket says 172.21.15.16:40592) 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa7f6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 0x55bea76d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.574+0000 7fa7f77fe640 1 -- 172.21.15.16:0/10816477 learned_addr learned my addr 172.21.15.16:0/10816477 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f4ff9640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 0x7fa7e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 0x7fa7e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 -- 172.21.15.16:0/10816477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 msgr2=0x55bea76d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 0x55bea76d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 -- 172.21.15.16:0/10816477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 msgr2=0x55bea76c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f4ff9640 1 -- 172.21.15.16:0/2653108589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7e80ef950 con 0x7fa7f8102d30 2024-09-12T01:06:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 -- 172.21.15.16:0/10816477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea75b53a0 con 0x55bea76c35d0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 0x55bea76d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ed040 tx=0x7fa7e8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81607befc58cad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7ddffb640 1 -- 172.21.15.16:0/10816477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8049070 con 0x55bea76c35d0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 0x7fa7e00a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005fd0 tx=0x7fa7f0004ab0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7ddffb640 1 -- 172.21.15.16:0/10816477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80bc570 con 0x55bea76c35d0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.575+0000 7fa7ddffb640 1 -- 172.21.15.16:0/10816477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e80ed320 con 0x55bea76c35d0 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 -- 172.21.15.16:0/10816477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 msgr2=0x55bea76c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80ed040 tx=0x7fa7e8009260 comp rx=0 tx=0).stop 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 -- 172.21.15.16:0/10816477 shutdown_connections 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76d2780 0x55bea76d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76c35d0 0x55bea76c3a10 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 --2- 172.21.15.16:0/10816477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 -- 172.21.15.16:0/10816477 >> 172.21.15.16:0/10816477 conn(0x55bea768c310 msgr2=0x55bea76bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 -- 172.21.15.16:0/10816477 shutdown_connections 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.576+0000 7fa8001a4740 1 -- 172.21.15.16:0/10816477 wait complete. 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 -- start start 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 Processor -- start 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bea76292f0 con 0x55bea76d2780 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bea75e4060 con 0x55bea76c4b40 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bea76d5240 con 0x55bea76c35d0 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa8001a4740 10 client.0 _ll_register_callbacks cb 0x55bea73378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.577+0000 7fa7f7fff640 1 -- 172.21.15.16:0/801481369 learned_addr learned my addr 172.21.15.16:0/801481369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 -- 172.21.15.16:0/801481369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 msgr2=0x55bea76d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 -- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 msgr2=0x55bea76cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 -- 172.21.15.16:0/801481369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea739a3c0 con 0x55bea76d2780 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010a60 tx=0x7fa7f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22888d2b5bb67626 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0018070 con 0x55bea76d2780 2024-09-12T01:06:37.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7f001dd40 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e80bc570 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7f0004b20 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa8001a4740 10 client.18681 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.578+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bea76e0ba0 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80bc570 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa7f000f040 con 0x55bea76d2780 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 10 client.18681 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 10 client.18681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 20 client.18681 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea76e2000 0x55bea76e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 4 client.18681 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea7636860 con 0x55bea76e2000 2024-09-12T01:06:37.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.579+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea76e2000 0x55bea76e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa7d6ffd640 10 client.18681 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:37.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea76e2000 0x55bea76e4450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6c0814155cdb0ad server_cookie=433c63af5678a926 in_seq=0 out_seq=0 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/801481369 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bea7636860 con 0x55bea76e2000 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa7d6ffd640 10 client.18681 handle_command_reply: tid=0 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa8001a4740 1 client.18681 shutdown 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 msgr2=0x55bea76cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.580+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010a60 tx=0x7fa7f0004a40 comp rx=0 tx=0).stop 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 shutdown_connections 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea76e2000 0x55bea76e4450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cf9e0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa7d6ffd640 0 client.18681 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:37.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 --2- 172.21.15.16:0/801481369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 >> 172.21.15.16:0/801481369 conn(0x55bea768c310 msgr2=0x55bea76c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 shutdown_connections 2024-09-12T01:06:37.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 1 -- 172.21.15.16:0/801481369 wait complete. 2024-09-12T01:06:37.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.581+0000 7fa8001a4740 20 client.18681 trim_cache size 0 max 16384 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 Processor -- start 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 -- start start 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81ab6e0 con 0x55bea76d2780 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81ad220 con 0x55bea76c4b40 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa8001a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81abdb0 con 0x55bea76c35d0 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40608/0 (socket says 172.21.15.16:40608) 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1294610484 learned_addr learned my addr 172.21.15.16:0/1294610484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1294610484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 msgr2=0x55bea76de360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.594+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1294610484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 msgr2=0x55bea76ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1294610484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea76bf4f0 con 0x55bea76d2780 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010690 tx=0x7fa7f0010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f840277916c224c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7dd7fa640 1 -- 172.21.15.16:0/1294610484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0018040 con 0x55bea76d2780 2024-09-12T01:06:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7dd7fa640 1 -- 172.21.15.16:0/1294610484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f00316f0 con 0x55bea76d2780 2024-09-12T01:06:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa7dd7fa640 1 -- 172.21.15.16:0/1294610484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0028690 con 0x55bea76d2780 2024-09-12T01:06:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa8001a4740 1 -- 172.21.15.16:0/1294610484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 msgr2=0x55bea76e9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.595+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010690 tx=0x7fa7f0010490 comp rx=0 tx=0).stop 2024-09-12T01:06:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 -- 172.21.15.16:0/1294610484 shutdown_connections 2024-09-12T01:06:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76e9d20 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c4b40 0x55bea76ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1294610484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c35d0 0x55bea76de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 -- 172.21.15.16:0/1294610484 >> 172.21.15.16:0/1294610484 conn(0x55bea768c310 msgr2=0x55bea76cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 -- 172.21.15.16:0/1294610484 shutdown_connections 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 -- 172.21.15.16:0/1294610484 wait complete. 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 -- start start 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.596+0000 7fa8001a4740 1 Processor -- start 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c35d0 0x55bea76cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c4b40 0x55bea76ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81acb50 con 0x55bea76d2780 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81ac480 con 0x55bea76c35d0 2024-09-12T01:06:37.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81a96c0 con 0x55bea76c4b40 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 10 client.0 _ll_register_callbacks cb 0x7fa7f00106e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40616/0 (socket says 172.21.15.16:40616) 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1220263282 learned_addr learned my addr 172.21.15.16:0/1220263282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c35d0 0x55bea76cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f6ffd640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c4b40 0x55bea76ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1220263282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c4b40 msgr2=0x55bea76ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c4b40 0x55bea76ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c35d0 msgr2=0x55bea76cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c35d0 0x55bea76cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 -- 172.21.15.16:0/1220263282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea774bef0 con 0x55bea76d2780 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7f7fff640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000a010 tx=0x7fa7f00122c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12e7c67413ef21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:37.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f003f070 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa7f00318a0 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.597+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f00316f0 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7f0048050 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 10 client.18687 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bea774d0e0 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f00316f0 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa7f0042020 con 0x55bea76d2780 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 10 client.18687 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 10 client.18687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 20 client.18687 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea774d790 0x55bea774fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 4 client.18687 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bea7630d70 con 0x55bea774d790 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.598+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea774d790 0x55bea774fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa7f77fe640 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea774d790 0x55bea774fb80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=362cc886138f439a server_cookie=e78032e8851f704e in_seq=0 out_seq=0 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa7d6ffd640 10 client.18687 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/1220263282 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bea7630d70 con 0x55bea774d790 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa7d6ffd640 10 client.18687 handle_command_reply: tid=0 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 client.18687 shutdown 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 msgr2=0x55bea76cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000a010 tx=0x7fa7f00122c0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 shutdown_connections 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bea774d790 0x55bea774fb80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bea76d2780 0x55bea76cd040 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bea76c4b40 0x55bea76ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 --2- 172.21.15.16:0/1220263282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bea76c35d0 0x55bea76cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 >> 172.21.15.16:0/1220263282 conn(0x55bea768c310 msgr2=0x55bea76ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa7d6ffd640 0 client.18687 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 shutdown_connections 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 1 -- 172.21.15.16:0/1220263282 wait complete. 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.599+0000 7fa8001a4740 20 client.18687 trim_cache size 0 max 16384 2024-09-12T01:06:37.604 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 msgr2=0x7fa7e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 0x7fa7e00a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005fd0 tx=0x7fa7f0004ab0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 msgr2=0x7fa7f81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009230 tx=0x7fa7e8004270 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 shutdown_connections 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa7e00a2cf0 0x7fa7e00a51b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7f81082a0 0x7fa7f819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7f81036e0 0x7fa7f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 --2- 172.21.15.16:0/2653108589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7f8102d30 0x7fa7f81985c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 >> 172.21.15.16:0/2653108589 conn(0x7fa7f80fc7d0 msgr2=0x7fa7f8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 shutdown_connections 2024-09-12T01:06:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:37.600+0000 7fa7fddcb640 1 -- 172.21.15.16:0/2653108589 wait complete. 2024-09-12T01:06:38.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:06:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:38.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:38 smithi016 ceph-mon[30972]: pgmap v326: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-12T01:06:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:38 smithi099 ceph-mon[35654]: pgmap v326: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-12T01:06:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:38 smithi159 ceph-mon[35566]: pgmap v326: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-12T01:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 -- 172.21.15.16:0/2376262221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c104e80 msgr2=0x7f2c3c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 --2- 172.21.15.16:0/2376262221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c104e80 0x7f2c3c105330 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009ca0 tx=0x7f2c3402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 -- 172.21.15.16:0/2376262221 shutdown_connections 2024-09-12T01:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 --2- 172.21.15.16:0/2376262221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c104e80 0x7f2c3c105330 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 --2- 172.21.15.16:0/2376262221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 --2- 172.21.15.16:0/2376262221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c069f00 0x7f2c3c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.228+0000 7f2c4367f640 1 -- 172.21.15.16:0/2376262221 >> 172.21.15.16:0/2376262221 conn(0x7f2c3c077210 msgr2=0x7f2c3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- 172.21.15.16:0/2376262221 shutdown_connections 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- 172.21.15.16:0/2376262221 wait complete. 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 Processor -- start 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- start start 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c104e80 0x7f2c3c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c110eb0 con 0x7f2c3c069f00 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c110d30 con 0x7f2c3c104e80 2024-09-12T01:06:39.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4367f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c111030 con 0x7f2c3c06a370 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.229+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33604/0 (socket says 172.21.15.16:33604) 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 -- 172.21.15.16:0/257064248 learned_addr learned my addr 172.21.15.16:0/257064248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c104e80 0x7f2c3c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 -- 172.21.15.16:0/257064248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 msgr2=0x7f2c3c199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 -- 172.21.15.16:0/257064248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c104e80 msgr2=0x7f2c3c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c104e80 0x7f2c3c1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 -- 172.21.15.16:0/257064248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1a47c0 con 0x7f2c3c069f00 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4141b640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800d950 tx=0x7f2c3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c327fc640 1 -- 172.21.15.16:0/257064248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38017070 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c1a4990 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c327fc640 1 -- 172.21.15.16:0/257064248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38009c30 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c327fc640 1 -- 172.21.15.16:0/257064248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800f6b0 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.230+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c196ff0 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c327fc640 1 -- 172.21.15.16:0/257064248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2c380109d0 con 0x7f2c3c069f00 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 Processor -- start 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 -- start start 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a348a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a355760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df5a2ae2f0 con 0x55df5a349b40 2024-09-12T01:06:39.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df5a255cf0 con 0x55df5a353370 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c45a58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55df5a2aa2b0 con 0x55df5a3485d0 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.233+0000 7f2c327fc640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 0x7f2c280a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a348a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33616/0 (socket says 172.21.15.16:33616) 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 -- 172.21.15.16:0/3823175484 learned_addr learned my addr 172.21.15.16:0/3823175484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 0x7f2c280a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a355760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c327fc640 1 -- 172.21.15.16:0/257064248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f2c380cf140 con 0x7f2c3c069f00 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 -- 172.21.15.16:0/3823175484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 msgr2=0x55df5a355760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a355760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 -- 172.21.15.16:0/3823175484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 msgr2=0x55df5a348090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 -- 172.21.15.16:0/3823175484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5a23a3a0 con 0x55df5a3485d0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c4141b640 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a348a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c380cc830 tx=0x7f2c3806e670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9404c26ab8df574e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 0x7f2c280a95c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fe0 tx=0x7f2c34004a80 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.234+0000 7f2c277fe640 1 -- 172.21.15.16:0/3823175484 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38020070 con 0x55df5a3485d0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c277fe640 1 -- 172.21.15.16:0/3823175484 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3809bd00 con 0x55df5a3485d0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c277fe640 1 -- 172.21.15.16:0/3823175484 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380a4270 con 0x55df5a3485d0 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 -- 172.21.15.16:0/3823175484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 msgr2=0x55df5a348a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a348a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c380cc830 tx=0x7f2c3806e670 comp rx=0 tx=0).stop 2024-09-12T01:06:39.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 -- 172.21.15.16:0/3823175484 shutdown_connections 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a355760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a348a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 --2- 172.21.15.16:0/3823175484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a348090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 -- 172.21.15.16:0/3823175484 >> 172.21.15.16:0/3823175484 conn(0x55df5a311310 msgr2=0x55df5a344c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 -- 172.21.15.16:0/3823175484 shutdown_connections 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.235+0000 7f2c45a58740 1 -- 172.21.15.16:0/3823175484 wait complete. 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 -- start start 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 Processor -- start 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a350060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a3507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df5a356010 con 0x55df5a3485d0 2024-09-12T01:06:39.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55df5a269060 con 0x55df5a349b40 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55df5a356190 con 0x55df5a353370 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c45a58740 10 client.0 _ll_register_callbacks cb 0x55df59fbc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33628/0 (socket says 172.21.15.16:33628) 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.236+0000 7f2c4141b640 1 -- 172.21.15.16:0/771818767 learned_addr learned my addr 172.21.15.16:0/771818767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a350060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a3507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 -- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 msgr2=0x55df5a3507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a3507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 -- 172.21.15.16:0/771818767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 msgr2=0x55df5a350060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a350060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 -- 172.21.15.16:0/771818767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5a01f3c0 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c4141b640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2c3809b300 tx=0x7f2c38097ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f649f70d8ad1212b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38097050 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f2c380a3de0 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c3809bd00 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c38020040 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c45a58740 10 client.18699 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.237+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df5a361b00 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3809bd00 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2c38084b40 con 0x55df5a3485d0 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 10 client.18699 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 10 client.18699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:06:39.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 20 client.18699 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a365240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 4 client.18699 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df5a2bb860 con 0x55df5a362e50 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.238+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a365240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.239+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a365240 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de7a42ba5f802e1b server_cookie=13193a906e433483 in_seq=0 out_seq=0 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.239+0000 7f2c24ff9640 10 client.18699 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.239+0000 7f2c24ff9640 1 -- 172.21.15.16:0/771818767 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df5a2bb860 con 0x55df5a362e50 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.239+0000 7f2c24ff9640 10 client.18699 handle_command_reply: tid=0 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.239+0000 7f2c45a58740 1 client.18699 shutdown 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 msgr2=0x55df5a352db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2c3809b300 tx=0x7f2c38097ea0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 shutdown_connections 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a3507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a365240 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c24ff9640 0 client.18699 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a350060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 --2- 172.21.15.16:0/771818767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a352db0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 >> 172.21.15.16:0/771818767 conn(0x55df5a311310 msgr2=0x55df5a34a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 shutdown_connections 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 1 -- 172.21.15.16:0/771818767 wait complete. 2024-09-12T01:06:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.240+0000 7f2c45a58740 20 client.18699 trim_cache size 0 max 16384 2024-09-12T01:06:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 Processor -- start 2024-09-12T01:06:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 -- start start 2024-09-12T01:06:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a344e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a36ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1ab630 con 0x55df5a349b40 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1ad170 con 0x55df5a353370 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.253+0000 7f2c45a58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1abd00 con 0x55df5a3485d0 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33634/0 (socket says 172.21.15.16:33634) 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 -- 172.21.15.16:0/2871575779 learned_addr learned my addr 172.21.15.16:0/2871575779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a36ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c4141b640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a344e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 -- 172.21.15.16:0/2871575779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 msgr2=0x55df5a344e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a344e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 -- 172.21.15.16:0/2871575779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 msgr2=0x55df5a36ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a36ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 -- 172.21.15.16:0/2871575779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5a349750 con 0x55df5a349b40 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a36ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005d80 tx=0x7f2c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafcb26f3c5d1daa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c26ffd640 1 -- 172.21.15.16:0/2871575779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0040f0 con 0x55df5a349b40 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c26ffd640 1 -- 172.21.15.16:0/2871575779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c004290 con 0x55df5a349b40 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.254+0000 7f2c26ffd640 1 -- 172.21.15.16:0/2871575779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0038c0 con 0x55df5a349b40 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- 172.21.15.16:0/2871575779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 msgr2=0x55df5a36a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c005d80 tx=0x7f2c2c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- 172.21.15.16:0/2871575779 shutdown_connections 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a353370 0x55df5a36ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a349b40 0x55df5a36a810 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 --2- 172.21.15.16:0/2871575779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a3485d0 0x55df5a344e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- 172.21.15.16:0/2871575779 >> 172.21.15.16:0/2871575779 conn(0x55df5a311310 msgr2=0x55df5a344700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- 172.21.15.16:0/2871575779 shutdown_connections 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- 172.21.15.16:0/2871575779 wait complete. 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 -- start start 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.255+0000 7f2c45a58740 1 Processor -- start 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a3579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a357f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1acaa0 con 0x55df5a3485d0 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1ac3d0 con 0x55df5a349b40 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c1a9630 con 0x55df5a353370 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c45a58740 10 client.0 _ll_register_callbacks cb 0x7f2c34009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33642/0 (socket says 172.21.15.16:33642) 2024-09-12T01:06:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 -- 172.21.15.16:0/258286501 learned_addr learned my addr 172.21.15.16:0/258286501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a3579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 -- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 msgr2=0x55df5a357f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a357f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a357f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 -- 172.21.15.16:0/258286501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 msgr2=0x55df5a3579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a3579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 -- 172.21.15.16:0/258286501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5a3cca80 con 0x55df5a3485d0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a3579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c41c1c640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a357f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c4141b640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2c38084f60 tx=0x7f2c3809b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4443563a30cb312 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.256+0000 7f2c24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38097040 con 0x55df5a3485d0 2024-09-12T01:06:39.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f2c380b42b0 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c004290 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2c380cccc0 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 10 client.18705 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55df5a3cdd80 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2c004290 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2c3807ead0 con 0x55df5a3485d0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 10 client.18705 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 10 client.18705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 20 client.18705 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a3d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 4 client.18705 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.257+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df5a2b5d70 con 0x55df5a362e50 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a3d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c40c1a640 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a3d0310 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9806977b51a64b1 server_cookie=bf9b36c34fe6b0be in_seq=0 out_seq=0 2024-09-12T01:06:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c24ff9640 10 client.18705 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c24ff9640 1 -- 172.21.15.16:0/258286501 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df5a2b5d70 con 0x55df5a362e50 2024-09-12T01:06:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c24ff9640 10 client.18705 handle_command_reply: tid=0 2024-09-12T01:06:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 client.18705 shutdown 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 msgr2=0x55df5a357480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2c38084f60 tx=0x7f2c3809b6d0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 shutdown_connections 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df5a353370 0x55df5a357f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55df5a362e50 0x55df5a3d0310 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55df5a349b40 0x55df5a3579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 --2- 172.21.15.16:0/258286501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5a3485d0 0x55df5a357480 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c24ff9640 0 client.18705 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.258+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 >> 172.21.15.16:0/258286501 conn(0x55df5a311310 msgr2=0x55df5a34a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 shutdown_connections 2024-09-12T01:06:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c45a58740 1 -- 172.21.15.16:0/258286501 wait complete. 2024-09-12T01:06:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c45a58740 20 client.18705 trim_cache size 0 max 16384 2024-09-12T01:06:39.265 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 msgr2=0x7f2c280a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 0x7f2c280a95c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005fe0 tx=0x7f2c34004a80 comp rx=0 tx=0).stop 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 msgr2=0x7f2c3c199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800d950 tx=0x7f2c3800de10 comp rx=0 tx=0).stop 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 shutdown_connections 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2c280a7100 0x7f2c280a95c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c3c104e80 0x7f2c3c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c3c06a370 0x7f2c3c199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 --2- 172.21.15.16:0/257064248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c069f00 0x7f2c3c199200 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 >> 172.21.15.16:0/257064248 conn(0x7f2c3c077210 msgr2=0x7f2c3c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 shutdown_connections 2024-09-12T01:06:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:39.259+0000 7f2c4367f640 1 -- 172.21.15.16:0/257064248 wait complete. 2024-09-12T01:06:40.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:06:40.418 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:40.826 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:40 smithi016 ceph-mon[30972]: pgmap v327: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-12T01:06:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 -- 172.21.15.16:0/1711355581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 msgr2=0x7f145c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 --2- 172.21.15.16:0/1711355581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c100ee0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f144c009930 tx=0x7f144c02a420 comp rx=0 tx=0).stop 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 -- 172.21.15.16:0/1711355581 shutdown_connections 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 --2- 172.21.15.16:0/1711355581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 0x7f145c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 --2- 172.21.15.16:0/1711355581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c100ee0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 --2- 172.21.15.16:0/1711355581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 -- 172.21.15.16:0/1711355581 >> 172.21.15.16:0/1711355581 conn(0x7f145c0fc7a0 msgr2=0x7f145c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 -- 172.21.15.16:0/1711355581 shutdown_connections 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.913+0000 7f1464663640 1 -- 172.21.15.16:0/1711355581 wait complete. 2024-09-12T01:06:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 Processor -- start 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 -- start start 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 0x7f145c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f145c110ed0 con 0x7f145c100a80 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f145c110d50 con 0x7f145c101420 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1464663640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f145c111050 con 0x7f145c106a80 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33660/0 (socket says 172.21.15.16:33660) 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 -- 172.21.15.16:0/1847082340 learned_addr learned my addr 172.21.15.16:0/1847082340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1462c00640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f1461bfe640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 0x7f145c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 -- 172.21.15.16:0/1847082340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 msgr2=0x7f145c19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 -- 172.21.15.16:0/1847082340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 msgr2=0x7f145c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.914+0000 7f14623ff640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 0x7f145c197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f14623ff640 1 -- 172.21.15.16:0/1847082340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c19c2c0 con 0x7f145c100a80 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f1462c00640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f14623ff640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f145800eb90 tx=0x7f145800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f14537fe640 1 -- 172.21.15.16:0/1847082340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458004d20 con 0x7f145c100a80 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f14537fe640 1 -- 172.21.15.16:0/1847082340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f145800ee90 con 0x7f145c100a80 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f14537fe640 1 -- 172.21.15.16:0/1847082340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458010830 con 0x7f145c100a80 2024-09-12T01:06:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c19c5b0 con 0x7f145c100a80 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.915+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c1a3e90 con 0x7f145c100a80 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.917+0000 7f14537fe640 1 -- 172.21.15.16:0/1847082340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1458005020 con 0x7f145c100a80 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 Processor -- start 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 -- start start 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5016d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4f6b2f0 con 0x55d9c50077e0 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4f12cf0 con 0x55d9c5014970 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f1466a3c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4f672b0 con 0x55d9c5008c30 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f14537fe640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 0x7f14480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.918+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33674/0 (socket says 172.21.15.16:33674) 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 -- 172.21.15.16:0/4233957730 learned_addr learned my addr 172.21.15.16:0/4233957730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1462c00640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 0x7f14480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1462c00640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14537fe640 1 -- 172.21.15.16:0/1847082340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f14580cfb60 con 0x7f145c100a80 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1461bfe640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5016d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 -- 172.21.15.16:0/4233957730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 msgr2=0x55d9c50072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 -- 172.21.15.16:0/4233957730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 msgr2=0x55d9c5016d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5016d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 -- 172.21.15.16:0/4233957730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c4ef73a0 con 0x55d9c50077e0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1461bfe640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5016d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1462c00640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 0x7f14480a5230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f145400afd0 tx=0x7f145400a9b0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f14623ff640 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f145800cba0 tx=0x7f1458005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b14a4ada43d6433 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1450ff9640 1 -- 172.21.15.16:0/4233957730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145809b980 con 0x55d9c50077e0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1450ff9640 1 -- 172.21.15.16:0/4233957730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1458085560 con 0x55d9c50077e0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.919+0000 7f1450ff9640 1 -- 172.21.15.16:0/4233957730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458016db0 con 0x55d9c50077e0 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 -- 172.21.15.16:0/4233957730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 msgr2=0x55d9c5007c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f145800cba0 tx=0x7f1458005960 comp rx=0 tx=0).stop 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 -- 172.21.15.16:0/4233957730 shutdown_connections 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5016d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5007c20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 --2- 172.21.15.16:0/4233957730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 -- 172.21.15.16:0/4233957730 >> 172.21.15.16:0/4233957730 conn(0x55d9c4fce310 msgr2=0x55d9c5003e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 -- 172.21.15.16:0/4233957730 shutdown_connections 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.920+0000 7f1466a3c740 1 -- 172.21.15.16:0/4233957730 wait complete. 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 -- start start 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 Processor -- start 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5011580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 0x55d9c5011ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d9c5017540 con 0x55d9c50077e0 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d9c4f26060 con 0x55d9c5008c30 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9c50176c0 con 0x55d9c5014970 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1466a3c740 10 client.0 _ll_register_callbacks cb 0x55d9c4c798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33676/0 (socket says 172.21.15.16:33676) 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 -- 172.21.15.16:0/3681603479 learned_addr learned my addr 172.21.15.16:0/3681603479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1462c00640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 0x55d9c5011ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 -- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 msgr2=0x55d9c5011ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1461bfe640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5011580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 0x55d9c5011ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 -- 172.21.15.16:0/3681603479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 msgr2=0x55d9c5011580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5011580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f14623ff640 1 -- 172.21.15.16:0/3681603479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c4cdc3c0 con 0x55d9c50077e0 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.921+0000 7f1462c00640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 0x55d9c5011ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1461bfe640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5011580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f14623ff640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f14580989f0 tx=0x7f14580a4c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89675cad1c4fa97e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14580982c0 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f1458016db0 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1458085560 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14580a03d0 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 10 client.18717 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9c5022f50 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1458085560 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f145809c280 con 0x55d9c50077e0 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 10 client.18717 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 20 client.18717 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5026690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 4 client.18717 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.922+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c4f78860 con 0x55d9c50242a0 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.923+0000 7f1461bfe640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5026690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.923+0000 7f1461bfe640 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5026690 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adcfd1bc166893d2 server_cookie=403eaa6f23555995 in_seq=0 out_seq=0 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.923+0000 7f1445ffb640 10 client.18717 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1445ffb640 1 -- 172.21.15.16:0/3681603479 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9c4f78860 con 0x55d9c50242a0 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1445ffb640 10 client.18717 handle_command_reply: tid=0 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 client.18717 shutdown 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 msgr2=0x55d9c50142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f14580989f0 tx=0x7f14580a4c90 comp rx=0 tx=0).stop 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 shutdown_connections 2024-09-12T01:06:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5026690 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5014970 0x55d9c5011ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1445ffb640 0 client.18717 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5011580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 --2- 172.21.15.16:0/3681603479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c50142d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 >> 172.21.15.16:0/3681603479 conn(0x55d9c4fce310 msgr2=0x55d9c5009bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 shutdown_connections 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.924+0000 7f1466a3c740 1 -- 172.21.15.16:0/3681603479 wait complete. 2024-09-12T01:06:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.925+0000 7f1466a3c740 20 client.18717 trim_cache size 0 max 16384 2024-09-12T01:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 Processor -- start 2024-09-12T01:06:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 -- start start 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 0x55d9c5004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5001320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1ab6b0 con 0x55d9c5014970 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1ad1f0 con 0x55d9c5008c30 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1466a3c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1abd80 con 0x55d9c50077e0 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33686/0 (socket says 172.21.15.16:33686) 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 -- 172.21.15.16:0/1358265600 learned_addr learned my addr 172.21.15.16:0/1358265600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f14623ff640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 0x55d9c5004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1461bfe640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5001320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 -- 172.21.15.16:0/1358265600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 msgr2=0x55d9c5004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 0x55d9c5004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 -- 172.21.15.16:0/1358265600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 msgr2=0x55d9c5001320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5001320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f1462c00640 1 -- 172.21.15.16:0/1358265600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c5011e50 con 0x55d9c5014970 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.938+0000 7f14623ff640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 0x55d9c5004020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1462c00640 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1454013010 tx=0x7f145401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfaa015605f0f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1447fff640 1 -- 172.21.15.16:0/1358265600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145401f800 con 0x55d9c5014970 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1447fff640 1 -- 172.21.15.16:0/1358265600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f145402a480 con 0x55d9c5014970 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1447fff640 1 -- 172.21.15.16:0/1358265600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14540105f0 con 0x55d9c5014970 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 -- 172.21.15.16:0/1358265600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 msgr2=0x55d9c5001860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1454013010 tx=0x7f145401c000 comp rx=0 tx=0).stop 2024-09-12T01:06:40.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 -- 172.21.15.16:0/1358265600 shutdown_connections 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c5014970 0x55d9c5001860 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5008c30 0x55d9c5001320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 --2- 172.21.15.16:0/1358265600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c50077e0 0x55d9c5004020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 -- 172.21.15.16:0/1358265600 >> 172.21.15.16:0/1358265600 conn(0x55d9c4fce310 msgr2=0x55d9c5003910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 -- 172.21.15.16:0/1358265600 shutdown_connections 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.939+0000 7f1466a3c740 1 -- 172.21.15.16:0/1358265600 wait complete. 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 -- start start 2024-09-12T01:06:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 Processor -- start 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5026c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1acb20 con 0x55d9c50077e0 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1ac450 con 0x55d9c5014970 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f145c1a96b0 con 0x55d9c5008c30 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1466a3c740 10 client.0 _ll_register_callbacks cb 0x7f145809a320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33690/0 (socket says 172.21.15.16:33690) 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 -- 172.21.15.16:0/452505097 learned_addr learned my addr 172.21.15.16:0/452505097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:40.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f1461bfe640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 -- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 msgr2=0x55d9c50864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.940+0000 7f14623ff640 1 -- 172.21.15.16:0/452505097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 msgr2=0x55d9c5026c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f14623ff640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5026c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f14623ff640 1 -- 172.21.15.16:0/452505097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c50274a0 con 0x55d9c50077e0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f14623ff640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1458098480 tx=0x7f145800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e7aedaa3741bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14580982c0 con 0x55d9c50077e0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f1458085120 con 0x55d9c50077e0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145402a480 con 0x55d9c50077e0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f145809b980 con 0x55d9c50077e0 2024-09-12T01:06:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 10 client.18723 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9c508e680 con 0x55d9c50077e0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f145402a480 con 0x55d9c50077e0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f145809b540 con 0x55d9c50077e0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 10 client.18723 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.941+0000 7f1466a3c740 20 client.18723 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5090b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1466a3c740 4 client.18723 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9c4f72d70 con 0x55d9c50242a0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1461bfe640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5090b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1461bfe640 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5090b60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ab12df961e8e55 server_cookie=36f495f58b70b522 in_seq=0 out_seq=0 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.942+0000 7f1445ffb640 10 client.18723 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:40.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1445ffb640 1 -- 172.21.15.16:0/452505097 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9c4f72d70 con 0x55d9c50242a0 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1445ffb640 10 client.18723 handle_command_reply: tid=0 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 client.18723 shutdown 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 msgr2=0x55d9c5085f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1458098480 tx=0x7f145800cf90 comp rx=0 tx=0).stop 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 shutdown_connections 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d9c50242a0 0x55d9c5090b60 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9c5014970 0x55d9c5026c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9c5008c30 0x55d9c50864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 --2- 172.21.15.16:0/452505097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d9c50077e0 0x55d9c5085f70 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 >> 172.21.15.16:0/452505097 conn(0x55d9c4fce310 msgr2=0x55d9c5012de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1445ffb640 0 client.18723 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:40.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 shutdown_connections 2024-09-12T01:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 1 -- 172.21.15.16:0/452505097 wait complete. 2024-09-12T01:06:40.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.943+0000 7f1466a3c740 20 client.18723 trim_cache size 0 max 16384 2024-09-12T01:06:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 msgr2=0x7f14480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 0x7f14480a5230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f145400afd0 tx=0x7f145400a9b0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 msgr2=0x7f145c197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f145800eb90 tx=0x7f145800c6a0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 shutdown_connections 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f14480a2d70 0x7f14480a5230 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f145c106a80 0x7f145c19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145c101420 0x7f145c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 --2- 172.21.15.16:0/1847082340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f145c100a80 0x7f145c197310 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 >> 172.21.15.16:0/1847082340 conn(0x7f145c0fc7a0 msgr2=0x7f145c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:40.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 shutdown_connections 2024-09-12T01:06:40.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:40.944+0000 7f1464663640 1 -- 172.21.15.16:0/1847082340 wait complete. 2024-09-12T01:06:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:40 smithi099 ceph-mon[35654]: pgmap v327: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-12T01:06:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:40 smithi159 ceph-mon[35566]: pgmap v327: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-12T01:06:42.789 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:06:42.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:06:42.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:42 smithi016 ceph-mon[30972]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:42 smithi099 ceph-mon[35654]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:43.059 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:42 smithi159 ceph-mon[35566]: pgmap v328: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/2026347435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664101de0 msgr2=0x7fa6641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- 172.21.15.16:0/2026347435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664101de0 0x7fa6641021c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa654009900 tx=0x7fa65402a410 comp rx=0 tx=0).stop 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/2026347435 shutdown_connections 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- 172.21.15.16:0/2026347435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664106f10 0x7fa66410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- 172.21.15.16:0/2026347435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 0x7fa6641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- 172.21.15.16:0/2026347435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664101de0 0x7fa6641021c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/2026347435 >> 172.21.15.16:0/2026347435 conn(0x7fa6640fd8c0 msgr2=0x7fa6640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/2026347435 shutdown_connections 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/2026347435 wait complete. 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 Processor -- start 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- start start 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664101de0 0x7fa6641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 0x7fa664198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa664111f70 con 0x7fa664106f10 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa664111df0 con 0x7fa664102790 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641120f0 con 0x7fa664101de0 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33712/0 (socket says 172.21.15.16:33712) 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 -- 172.21.15.16:0/4136777658 learned_addr learned my addr 172.21.15.16:0/4136777658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa668c7c640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664101de0 0x7fa6641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa663fff640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 0x7fa664198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 -- 172.21.15.16:0/4136777658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664101de0 msgr2=0x7fa6641983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664101de0 0x7fa6641983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 -- 172.21.15.16:0/4136777658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 msgr2=0x7fa664198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 0x7fa664198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 -- 172.21.15.16:0/4136777658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66419d380 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66947d640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00cce0 tx=0x7fa65c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa661ffb640 1 -- 172.21.15.16:0/4136777658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00f450 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66419d670 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa661ffb640 1 -- 172.21.15.16:0/4136777658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c00fad0 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.515+0000 7fa661ffb640 1 -- 172.21.15.16:0/4136777658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c018c10 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.516+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641a4f50 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa661ffb640 1 -- 172.21.15.16:0/4136777658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa65c00f5f0 con 0x7fa664106f10 2024-09-12T01:06:43.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 Processor -- start 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 -- start start 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec3f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec402130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dec35a280 con 0x558dec3f5480 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558dec301b10 con 0x558dec3ffd40 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558dec356240 con 0x558dec3f4b60 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33714/0 (socket says 172.21.15.16:33714) 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 -- 172.21.15.16:0/3792045243 learned_addr learned my addr 172.21.15.16:0/3792045243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa661ffb640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 0x7fa64c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa663fff640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec402130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa663fff640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec3f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66947d640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 0x7fa64c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa661ffb640 1 -- 172.21.15.16:0/4136777658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa65c0cfa30 con 0x7fa664106f10 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 -- 172.21.15.16:0/3792045243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 msgr2=0x558dec3f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec3f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 -- 172.21.15.16:0/3792045243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 msgr2=0x558dec402130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec402130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 -- 172.21.15.16:0/3792045243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dec30e1f0 con 0x558dec3f5480 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa663fff640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec402130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa654009900 tx=0x7fa6540316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d6eaa71833feab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa66947d640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 0x7fa64c0a51b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa65c09bbf0 tx=0x7fa65c0186b0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa64affd640 1 -- 172.21.15.16:0/3792045243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654035040 con 0x558dec3f5480 2024-09-12T01:06:43.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa64affd640 1 -- 172.21.15.16:0/3792045243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa654005020 con 0x558dec3f5480 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.519+0000 7fa64affd640 1 -- 172.21.15.16:0/3792045243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65403fbb0 con 0x558dec3f5480 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3792045243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 msgr2=0x558dec3ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fa654009900 tx=0x7fa6540316e0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3792045243 shutdown_connections 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec402130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec3ff800 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3792045243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec3f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3792045243 >> 172.21.15.16:0/3792045243 conn(0x558dec3f10f0 msgr2=0x558dec3f14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3792045243 shutdown_connections 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.520+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3792045243 wait complete. 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 -- start start 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 Processor -- start 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3f4b60 0x558dec3fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558dec402910 con 0x558dec3ffd40 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558dec314f10 con 0x558dec3f4b60 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558dec402a90 con 0x558dec3f5480 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa66d2b9740 10 client.0 _ll_register_callbacks cb 0x558dec2dda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33716/0 (socket says 172.21.15.16:33716) 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa663fff640 1 -- 172.21.15.16:0/3420376194 learned_addr learned my addr 172.21.15.16:0/3420376194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.521+0000 7fa663fff640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3f4b60 0x558dec3fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 -- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 msgr2=0x558dec3fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 -- 172.21.15.16:0/3420376194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3f4b60 msgr2=0x558dec3fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3f4b60 0x558dec3fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 -- 172.21.15.16:0/3420376194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dec0cb6a0 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa663fff640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa658011e70 tx=0x7fa658013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3de43b49ee132 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa643fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658004020 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa654005020 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa6580041c0 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6580043c0 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa66d2b9740 10 client.18735 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dec40e4f0 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.522+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa654005020 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa658004be0 con 0x558dec3ffd40 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 10 client.18735 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:43.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 10 client.18735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 20 client.18735 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec40f9b0 0x558dec411de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 4 client.18735 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dec3677f0 con 0x558dec40f9b0 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.523+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec40f9b0 0x558dec411de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa643fff640 10 client.18735 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa668c7c640 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec40f9b0 0x558dec411de0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565c041f8cb6aec9 server_cookie=f6d088488e084dc3 in_seq=0 out_seq=0 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa643fff640 1 -- 172.21.15.16:0/3420376194 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dec3677f0 con 0x558dec40f9b0 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa643fff640 10 client.18735 handle_command_reply: tid=0 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 client.18735 shutdown 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 msgr2=0x558dec3fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fa658011e70 tx=0x7fa658013040 comp rx=0 tx=0).stop 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 shutdown_connections 2024-09-12T01:06:43.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec40f9b0 0x558dec411de0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3ffd40 0x558dec3fd650 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/3420376194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3f4b60 0x558dec3fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa643fff640 0 client.18735 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.524+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 >> 172.21.15.16:0/3420376194 conn(0x558dec3f10f0 msgr2=0x558dec3f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.525+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 shutdown_connections 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.525+0000 7fa66d2b9740 1 -- 172.21.15.16:0/3420376194 wait complete. 2024-09-12T01:06:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.525+0000 7fa66d2b9740 20 client.18735 trim_cache size 0 max 16384 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 Processor -- start 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 -- start start 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec4176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ab5f0 con 0x558dec3f4b60 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ad130 con 0x558dec3ffd40 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641abcc0 con 0x558dec3f5480 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa663fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33724/0 (socket says 172.21.15.16:33724) 2024-09-12T01:06:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.542+0000 7fa663fff640 1 -- 172.21.15.16:0/108982434 learned_addr learned my addr 172.21.15.16:0/108982434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 -- 172.21.15.16:0/108982434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 msgr2=0x558dec3fc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 -- 172.21.15.16:0/108982434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 msgr2=0x558dec4176b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec4176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 -- 172.21.15.16:0/108982434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dec3f52b0 con 0x558dec3f4b60 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa663fff640 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa658004610 tx=0x7fa65800a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1543f2c3eb07964 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa64a7fc640 1 -- 172.21.15.16:0/108982434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65800a2d0 con 0x558dec3f4b60 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa64a7fc640 1 -- 172.21.15.16:0/108982434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658004c10 con 0x558dec3f4b60 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.543+0000 7fa64a7fc640 1 -- 172.21.15.16:0/108982434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658005380 con 0x558dec3f4b60 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- 172.21.15.16:0/108982434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 msgr2=0x558dec3f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fa658004610 tx=0x7fa65800a8d0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- 172.21.15.16:0/108982434 shutdown_connections 2024-09-12T01:06:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec4176b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f5480 0x558dec3fc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/108982434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f4b60 0x558dec3f75c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- 172.21.15.16:0/108982434 >> 172.21.15.16:0/108982434 conn(0x558dec3f10f0 msgr2=0x558dec3f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- 172.21.15.16:0/108982434 shutdown_connections 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- 172.21.15.16:0/108982434 wait complete. 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 -- start start 2024-09-12T01:06:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.544+0000 7fa66d2b9740 1 Processor -- start 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec403c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec404690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641aca60 con 0x558dec3f5480 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ac390 con 0x558dec3ffd40 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641a95f0 con 0x558dec3f4b60 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 10 client.0 _ll_register_callbacks cb 0x7fa65800aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33738/0 (socket says 172.21.15.16:33738) 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 -- 172.21.15.16:0/1336888060 learned_addr learned my addr 172.21.15.16:0/1336888060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:43.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa663fff640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec403c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa663fff640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec404690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 -- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 msgr2=0x558dec403c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec403c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 -- 172.21.15.16:0/1336888060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 msgr2=0x558dec404690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec404690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 -- 172.21.15.16:0/1336888060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dec479820 con 0x558dec3f5480 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa663fff640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec403c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa668c7c640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa654009900 tx=0x7fa65402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190920e06876bdba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa643fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654057030 con 0x558dec3f5480 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fa65403f8a0 con 0x558dec3f5480 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.545+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa658004c10 con 0x558dec3f5480 2024-09-12T01:06:43.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa654063050 con 0x558dec3f5480 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 10 client.18741 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558dec47ab40 con 0x558dec3f5480 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658004c10 con 0x558dec3f5480 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6540047a0 con 0x558dec3f5480 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 10 client.18741 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 10 client.18741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 20 client.18741 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec47af70 0x558dec47d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 4 client.18741 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558dec361d00 con 0x558dec47af70 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.546+0000 7fa663fff640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec47af70 0x558dec47d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.547+0000 7fa663fff640 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec47af70 0x558dec47d360 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80147f5d831bf873 server_cookie=3c9883c58f852ab1 in_seq=0 out_seq=0 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.547+0000 7fa643fff640 10 client.18741 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:43.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa643fff640 1 -- 172.21.15.16:0/1336888060 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558dec361d00 con 0x558dec47af70 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa643fff640 10 client.18741 handle_command_reply: tid=0 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 client.18741 shutdown 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 msgr2=0x558dec404150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa654009900 tx=0x7fa65402a410 comp rx=0 tx=0).stop 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 shutdown_connections 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558dec47af70 0x558dec47d360 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558dec3ffd40 0x558dec404690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558dec3f5480 0x558dec404150 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 --2- 172.21.15.16:0/1336888060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dec3f4b60 0x558dec403c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 >> 172.21.15.16:0/1336888060 conn(0x558dec3f10f0 msgr2=0x558dec3efa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa643fff640 0 client.18741 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 shutdown_connections 2024-09-12T01:06:43.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 1 -- 172.21.15.16:0/1336888060 wait complete. 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.548+0000 7fa66d2b9740 20 client.18741 trim_cache size 0 max 16384 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a938e228-8a6c-4340-95ba-ae267076c815", 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:06:43.552 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 msgr2=0x7fa64c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 0x7fa64c0a51b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa65c09bbf0 tx=0x7fa65c0186b0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 msgr2=0x7fa66419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00cce0 tx=0x7fa65c00a620 comp rx=0 tx=0).stop 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 shutdown_connections 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa64c0a2cf0 0x7fa64c0a51b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664106f10 0x7fa66419cca0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa664102790 0x7fa664198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 --2- 172.21.15.16:0/4136777658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa664101de0 0x7fa6641983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 >> 172.21.15.16:0/4136777658 conn(0x7fa6640fd8c0 msgr2=0x7fa6640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 shutdown_connections 2024-09-12T01:06:43.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:43.549+0000 7fa66aee0640 1 -- 172.21.15.16:0/4136777658 wait complete. 2024-09-12T01:06:44.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:06:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:44.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:44 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:44 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: pgmap v329: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:44 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 -- 172.21.15.16:0/137344821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704102790 msgr2=0x7f47041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 --2- 172.21.15.16:0/137344821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704102790 0x7f47041067e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009900 tx=0x7f46ec02a410 comp rx=0 tx=0).stop 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 -- 172.21.15.16:0/137344821 shutdown_connections 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 --2- 172.21.15.16:0/137344821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f470410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 --2- 172.21.15.16:0/137344821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704102790 0x7f47041067e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 --2- 172.21.15.16:0/137344821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704101de0 0x7f47041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 -- 172.21.15.16:0/137344821 >> 172.21.15.16:0/137344821 conn(0x7f47040fd8c0 msgr2=0x7f47040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 -- 172.21.15.16:0/137344821 shutdown_connections 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 -- 172.21.15.16:0/137344821 wait complete. 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.197+0000 7f470a213640 1 Processor -- start 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 -- start start 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 0x7f4704074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f4704074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4704111f30 con 0x7f4704101de0 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4704111db0 con 0x7f4704106f10 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f470a213640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47041120b0 con 0x7f4704102790 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33756/0 (socket says 172.21.15.16:33756) 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 -- 172.21.15.16:0/2603423473 learned_addr learned my addr 172.21.15.16:0/2603423473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f4703fff640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f4704074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f46faffd640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 0x7f4704074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 -- 172.21.15.16:0/2603423473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 msgr2=0x7f4704074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 0x7f4704074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 -- 172.21.15.16:0/2603423473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 msgr2=0x7f4704074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f4704074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f47037fe640 1 -- 172.21.15.16:0/2603423473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47041a67d0 con 0x7f4704101de0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f46faffd640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 0x7f4704074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.198+0000 7f4703fff640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f4704074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f47037fe640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c920 tx=0x7f46f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f47017fa640 1 -- 172.21.15.16:0/2603423473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007c70 con 0x7f4704101de0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f47017fa640 1 -- 172.21.15.16:0/2603423473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f4007e10 con 0x7f4704101de0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47041a6ac0 con 0x7f4704101de0 2024-09-12T01:06:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f47017fa640 1 -- 172.21.15.16:0/2603423473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4018890 con 0x7f4704101de0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.199+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47041a7000 con 0x7f4704101de0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.201+0000 7f47017fa640 1 -- 172.21.15.16:0/2603423473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f46f4018a70 con 0x7f4704101de0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.201+0000 7f470c5ec740 1 Processor -- start 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.201+0000 7f470c5ec740 1 -- start start 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2ddeda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db2dd462e0 con 0x55db2dde0dc0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db2dcedce0 con 0x55db2ddeb680 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f470c5ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db2dd422a0 con 0x55db2dde04a0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47017fa640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 0x7f46f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33770/0 (socket says 172.21.15.16:33770) 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 -- 172.21.15.16:0/3513299926 learned_addr learned my addr 172.21.15.16:0/3513299926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f46faffd640 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2ddeda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f4703fff640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 0x7f46f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f4703fff640 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47017fa640 1 -- 172.21.15.16:0/2603423473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f46f40cf6b0 con 0x7f4704101de0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 -- 172.21.15.16:0/3513299926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 msgr2=0x55db2dde0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 -- 172.21.15.16:0/3513299926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 msgr2=0x55db2ddeda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2ddeda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.202+0000 7f47037fe640 1 -- 172.21.15.16:0/3513299926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2dcd2440 con 0x55db2dde0dc0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f47037fe640 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f46f4097fd0 tx=0x7f46f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f33601ecf0b0e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f4703fff640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 0x7f46f00a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f46fc007310 tx=0x7f46fc007460 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f46f9ffb640 1 -- 172.21.15.16:0/3513299926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f409c270 con 0x55db2dde0dc0 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f46f9ffb640 1 -- 172.21.15.16:0/3513299926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f400de20 con 0x55db2dde0dc0 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f46f9ffb640 1 -- 172.21.15.16:0/3513299926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40cd020 con 0x55db2dde0dc0 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 -- 172.21.15.16:0/3513299926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 msgr2=0x55db2ddeb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f46f4097fd0 tx=0x7f46f4000bc0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 -- 172.21.15.16:0/3513299926 shutdown_connections 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2ddeda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddeb140 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 --2- 172.21.15.16:0/3513299926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 -- 172.21.15.16:0/3513299926 >> 172.21.15.16:0/3513299926 conn(0x55db2dddca30 msgr2=0x55db2dddce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 -- 172.21.15.16:0/3513299926 shutdown_connections 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.203+0000 7f470c5ec740 1 -- 172.21.15.16:0/3513299926 wait complete. 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 -- start start 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 Processor -- start 2024-09-12T01:06:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddeafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2dde87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db2ddee1f0 con 0x55db2dde0dc0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55db2dd01050 con 0x55db2dde04a0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db2ddee370 con 0x55db2ddeb680 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f470c5ec740 10 client.0 _ll_register_callbacks cb 0x55db2dd41160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33776/0 (socket says 172.21.15.16:33776) 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.204+0000 7f46faffd640 1 -- 172.21.15.16:0/2665795688 learned_addr learned my addr 172.21.15.16:0/2665795688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f47037fe640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddeafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f4703fff640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2dde87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 -- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 msgr2=0x55db2dde87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2dde87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 -- 172.21.15.16:0/2665795688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 msgr2=0x55db2ddeafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddeafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 -- 172.21.15.16:0/2665795688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2dab73c0 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f4703fff640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2dde87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46faffd640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031550 tx=0x7f46ec0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d8b4ad551c7ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec040070 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f46ec004450 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f400de20 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46ec035040 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f470c5ec740 10 client.18753 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db2ddf9bc0 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.205+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f400de20 con 0x55db2dde0dc0 2024-09-12T01:06:45.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46ec004be0 con 0x55db2dde0dc0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 10 client.18753 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 20 client.18753 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2ddfafd0 0x55db2ddfd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 4 client.18753 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db2dd53850 con 0x55db2ddfafd0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.206+0000 7f47037fe640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2ddfafd0 0x55db2ddfd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.207+0000 7f46e2ffd640 10 client.18753 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.207+0000 7f47037fe640 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2ddfafd0 0x55db2ddfd410 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5142e4b2b899034 server_cookie=b98459a45a23e289 in_seq=0 out_seq=0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.207+0000 7f46e2ffd640 1 -- 172.21.15.16:0/2665795688 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db2dd53850 con 0x55db2ddfafd0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.207+0000 7f46e2ffd640 10 client.18753 handle_command_reply: tid=0 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.207+0000 7f470c5ec740 1 client.18753 shutdown 2024-09-12T01:06:45.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 msgr2=0x55db2dde8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031550 tx=0x7f46ec0317e0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 shutdown_connections 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2ddfafd0 0x55db2ddfd410 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2dde87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2dde8290 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 --2- 172.21.15.16:0/2665795688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddeafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 >> 172.21.15.16:0/2665795688 conn(0x55db2dddca30 msgr2=0x55db2dde2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f46e2ffd640 0 client.18753 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 shutdown_connections 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 1 -- 172.21.15.16:0/2665795688 wait complete. 2024-09-12T01:06:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.208+0000 7f470c5ec740 20 client.18753 trim_cache size 0 max 16384 2024-09-12T01:06:45.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.231+0000 7f470c5ec740 1 Processor -- start 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 -- start start 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2de02ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47041af9e0 con 0x55db2dde0dc0 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b1520 con 0x55db2ddeb680 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f470c5ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b00b0 con 0x55db2dde04a0 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33790/0 (socket says 172.21.15.16:33790) 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 -- 172.21.15.16:0/1535341263 learned_addr learned my addr 172.21.15.16:0/1535341263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f47037fe640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f4703fff640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2de02ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 -- 172.21.15.16:0/1535341263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 msgr2=0x55db2dde8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 -- 172.21.15.16:0/1535341263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 msgr2=0x55db2de02ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2de02ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 -- 172.21.15.16:0/1535341263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db2dde8b60 con 0x55db2dde0dc0 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f4703fff640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2de02ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.232+0000 7f46faffd640 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0037c0 tx=0x7f46ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fafe2fb8a00ae21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f46f97fa640 1 -- 172.21.15.16:0/1535341263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec040070 con 0x55db2dde0dc0 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f46f97fa640 1 -- 172.21.15.16:0/1535341263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ec031a10 con 0x55db2dde0dc0 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f46f97fa640 1 -- 172.21.15.16:0/1535341263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0052e0 con 0x55db2dde0dc0 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 -- 172.21.15.16:0/1535341263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 msgr2=0x55db2ddf7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0037c0 tx=0x7f46ec001e70 comp rx=0 tx=0).stop 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 -- 172.21.15.16:0/1535341263 shutdown_connections 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2ddeb680 0x55db2de02ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf7450 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1535341263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2dde04a0 0x55db2dde8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 -- 172.21.15.16:0/1535341263 >> 172.21.15.16:0/1535341263 conn(0x55db2dddca30 msgr2=0x55db2dde7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 -- 172.21.15.16:0/1535341263 shutdown_connections 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.233+0000 7f470c5ec740 1 -- 172.21.15.16:0/1535341263 wait complete. 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 -- start start 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 Processor -- start 2024-09-12T01:06:45.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddf3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2ddeef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b0e50 con 0x55db2dde0dc0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47041b0780 con 0x55db2dde04a0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47041ad9e0 con 0x55db2ddeb680 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.234+0000 7f470c5ec740 10 client.0 _ll_register_callbacks cb 0x7f46ec005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33796/0 (socket says 172.21.15.16:33796) 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f4703fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2ddeef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 -- 172.21.15.16:0/1901757959 learned_addr learned my addr 172.21.15.16:0/1901757959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f47037fe640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddf3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 -- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 msgr2=0x55db2ddeef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2ddeef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 -- 172.21.15.16:0/1901757959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 msgr2=0x55db2ddf3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddf3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 -- 172.21.15.16:0/1901757959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db2ddef700 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46faffd640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031820 tx=0x7f46ec0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de26a8afff82473c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec040070 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f46ec0051b0 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db2dd4dd60 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46ec064050 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f470c5ec740 10 client.18759 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.235+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db2de65940 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55db2dd4dd60 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f46ec055030 con 0x55db2dde0dc0 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 10 client.18759 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 20 client.18759 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2de65f90 0x55db2de68380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 4 client.18759 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:06:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db2dcca210 con 0x55db2de65f90 2024-09-12T01:06:45.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.236+0000 7f47037fe640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2de65f90 0x55db2de68380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f46e2ffd640 10 client.18759 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f47037fe640 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2de65f90 0x55db2de68380 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c145d91e9e8b57 server_cookie=1f63d9542b1371c7 in_seq=0 out_seq=0 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f46e2ffd640 1 -- 172.21.15.16:0/1901757959 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db2dcca210 con 0x55db2de65f90 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f46e2ffd640 10 client.18759 handle_command_reply: tid=0 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 client.18759 shutdown 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 msgr2=0x55db2ddf3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f46ec031820 tx=0x7f46ec0044d0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 shutdown_connections 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55db2de65f90 0x55db2de68380 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2ddeb680 0x55db2ddeef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f46e2ffd640 0 client.18759 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db2dde0dc0 0x55db2ddf3ec0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 --2- 172.21.15.16:0/1901757959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55db2dde04a0 0x55db2ddf3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 >> 172.21.15.16:0/1901757959 conn(0x55db2dddca30 msgr2=0x55db2dde9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 shutdown_connections 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 1 -- 172.21.15.16:0/1901757959 wait complete. 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.237+0000 7f470c5ec740 20 client.18759 trim_cache size 0 max 16384 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:06:45.240 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 msgr2=0x7f46f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 0x7f46f00a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f46fc007310 tx=0x7f46fc007460 comp rx=0 tx=0).stop 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 msgr2=0x7f47040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c920 tx=0x7f46f400cde0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 shutdown_connections 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f46f00a2cf0 0x7f46f00a51b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4704106f10 0x7f4704074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4704102790 0x7f4704074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 --2- 172.21.15.16:0/2603423473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4704101de0 0x7f47040744f0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 >> 172.21.15.16:0/2603423473 conn(0x7f47040fd8c0 msgr2=0x7f47040febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 shutdown_connections 2024-09-12T01:06:45.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:45.238+0000 7f470a213640 1 -- 172.21.15.16:0/2603423473 wait complete. 2024-09-12T01:06:45.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:06:45.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3455320170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c102790 msgr2=0x7fe59c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3455320170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c102790 0x7fe59c1067e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009900 tx=0x7fe58c02a410 comp rx=0 tx=0).stop 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3455320170 shutdown_connections 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3455320170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c106f10 0x7fe59c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3455320170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c102790 0x7fe59c1067e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3455320170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3455320170 >> 172.21.15.16:0/3455320170 conn(0x7fe59c0fd8c0 msgr2=0x7fe59c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3455320170 shutdown_connections 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3455320170 wait complete. 2024-09-12T01:06:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 Processor -- start 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.764+0000 7fe5a3b8a640 1 -- start start 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c102790 0x7fe59c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111f70 con 0x7fe59c106f10 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111df0 con 0x7fe59c102790 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a3b8a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1120f0 con 0x7fe59c101de0 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33814/0 (socket says 172.21.15.16:33814) 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 -- 172.21.15.16:0/3910935194 learned_addr learned my addr 172.21.15.16:0/3910935194 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a1926640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a1125640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c102790 0x7fe59c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 -- 172.21.15.16:0/3910935194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 msgr2=0x7fe59c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 -- 172.21.15.16:0/3910935194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c102790 msgr2=0x7fe59c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c102790 0x7fe59c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 -- 172.21.15.16:0/3910935194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c19d4c0 con 0x7fe59c106f10 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a1926640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe5a2127640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe59400cad0 tx=0x7fe59400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.765+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400ae20 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.766+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c19d7b0 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.766+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe594005020 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.766+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400f6f0 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.766+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c1a5090 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.767+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59c1021c0 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.768+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5940051d0 con 0x7fe59c106f10 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.768+0000 7fe592ffd640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.769+0000 7fe5a1926640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:46.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.769+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fe5940cf530 con 0x7fe59c106f10 2024-09-12T01:06:46.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.769+0000 7fe5a1926640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 0x7fe5880a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe598005fd0 tx=0x7fe598005ea0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:46.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.773+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe594014030 con 0x7fe59c106f10 2024-09-12T01:06:46.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:46 smithi016 ceph-mon[30972]: pgmap v330: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-12T01:06:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.986+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe59c10a1b0 con 0x7fe59c106f10 2024-09-12T01:06:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.986+0000 7fe592ffd640 1 -- 172.21.15.16:0/3910935194 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fe59409c150 con 0x7fe59c106f10 2024-09-12T01:06:46.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:46.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:46.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.988+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 msgr2=0x7fe5880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.988+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 0x7fe5880a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe598005fd0 tx=0x7fe598005ea0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.988+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 msgr2=0x7fe59c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.988+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe59400cad0 tx=0x7fe59400cf90 comp rx=0 tx=0).stop 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 shutdown_connections 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5880a2cf0 0x7fe5880a51b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59c106f10 0x7fe59c19cde0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe59c102790 0x7fe59c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 --2- 172.21.15.16:0/3910935194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59c101de0 0x7fe59c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 >> 172.21.15.16:0/3910935194 conn(0x7fe59c0fd8c0 msgr2=0x7fe59c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 shutdown_connections 2024-09-12T01:06:46.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:46.989+0000 7fe5a3b8a640 1 -- 172.21.15.16:0/3910935194 wait complete. 2024-09-12T01:06:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:46 smithi099 ceph-mon[35654]: pgmap v330: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-12T01:06:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:46 smithi159 ceph-mon[35566]: pgmap v330: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-12T01:06:47.676 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3910935194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:47.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:06:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:48.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3910935194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:48.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3910935194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 -- 172.21.15.16:0/3620015854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c101df0 msgr2=0x7fbb6c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 --2- 172.21.15.16:0/3620015854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c101df0 0x7fbb6c1021d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009900 tx=0x7fbb5c02a410 comp rx=0 tx=0).stop 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 -- 172.21.15.16:0/3620015854 shutdown_connections 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 --2- 172.21.15.16:0/3620015854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c106f20 0x7fbb6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 --2- 172.21.15.16:0/3620015854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 --2- 172.21.15.16:0/3620015854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c101df0 0x7fbb6c1021d0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 -- 172.21.15.16:0/3620015854 >> 172.21.15.16:0/3620015854 conn(0x7fbb6c0fd8f0 msgr2=0x7fbb6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 -- 172.21.15.16:0/3620015854 shutdown_connections 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.390+0000 7fbb72326640 1 -- 172.21.15.16:0/3620015854 wait complete. 2024-09-12T01:06:48.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 Processor -- start 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 -- start start 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c101df0 0x7fbb6c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 0x7fbb6c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c111fa0 con 0x7fbb6c1027a0 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c111e20 con 0x7fbb6c101df0 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb72326640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c112120 con 0x7fbb6c106f20 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33826/0 (socket says 172.21.15.16:33826) 2024-09-12T01:06:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 -- 172.21.15.16:0/909590553 learned_addr learned my addr 172.21.15.16:0/909590553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb708c3640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 0x7fbb6c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c101df0 0x7fbb6c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 -- 172.21.15.16:0/909590553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 msgr2=0x7fbb6c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 0x7fbb6c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 -- 172.21.15.16:0/909590553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c101df0 msgr2=0x7fbb6c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c101df0 0x7fbb6c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.391+0000 7fbb63fff640 1 -- 172.21.15.16:0/909590553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6c19d4b0 con 0x7fbb6c1027a0 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb708c3640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 0x7fbb6c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb63fff640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400b4a0 tx=0x7fbb5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb69ffb640 1 -- 172.21.15.16:0/909590553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54014070 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6c19d7a0 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb69ffb640 1 -- 172.21.15.16:0/909590553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb54010af0 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb69ffb640 1 -- 172.21.15.16:0/909590553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54017d10 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.392+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6c1a5080 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.394+0000 7fbb69ffb640 1 -- 172.21.15.16:0/909590553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbb54017070 con 0x7fbb6c1027a0 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 Processor -- start 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 -- start start 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f73da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f74a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f6f6a32f0 con 0x564f6f73eb40 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564f6f64acf0 con 0x564f6f73d5d0 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb746ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f6f69f2b0 con 0x564f6f748370 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb69ffb640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 0x7fbb580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb6bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f73da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33840/0 (socket says 172.21.15.16:33840) 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.395+0000 7fbb708c3640 1 -- 172.21.15.16:0/2805721577 learned_addr learned my addr 172.21.15.16:0/2805721577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 0x7fbb580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb63fff640 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f74a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 -- 172.21.15.16:0/2805721577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 msgr2=0x564f6f74a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f74a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 -- 172.21.15.16:0/2805721577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 msgr2=0x564f6f73da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f73da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 -- 172.21.15.16:0/2805721577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6f62f3a0 con 0x564f6f73eb40 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbb64015f70 tx=0x7fbb64012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3517bd647eff8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb69ffb640 1 -- 172.21.15.16:0/909590553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fbb540ce820 con 0x7fbb6c1027a0 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb708c3640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 0x7fbb580a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009fd0 tx=0x7fbb6401d000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb627fc640 1 -- 172.21.15.16:0/2805721577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64025070 con 0x564f6f73eb40 2024-09-12T01:06:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb627fc640 1 -- 172.21.15.16:0/2805721577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64009d30 con 0x564f6f73eb40 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.396+0000 7fbb627fc640 1 -- 172.21.15.16:0/2805721577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64010400 con 0x564f6f73eb40 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 -- 172.21.15.16:0/2805721577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 msgr2=0x564f6f73d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbb64015f70 tx=0x7fbb64012820 comp rx=0 tx=0).stop 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 -- 172.21.15.16:0/2805721577 shutdown_connections 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f74a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f73da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2805721577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73eb40 0x564f6f73d090 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 -- 172.21.15.16:0/2805721577 >> 172.21.15.16:0/2805721577 conn(0x564f6f706310 msgr2=0x564f6f739c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 -- 172.21.15.16:0/2805721577 shutdown_connections 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.397+0000 7fbb746ff740 1 -- 172.21.15.16:0/2805721577 wait complete. 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 -- start start 2024-09-12T01:06:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 Processor -- start 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f747de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73eb40 0x564f6f745090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f6f74b010 con 0x564f6f748370 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564f6f65e060 con 0x564f6f73d5d0 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f6f74b190 con 0x564f6f73eb40 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb746ff740 10 client.0 _ll_register_callbacks cb 0x564f6f3b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33842/0 (socket says 172.21.15.16:33842) 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.398+0000 7fbb708c3640 1 -- 172.21.15.16:0/1385147991 learned_addr learned my addr 172.21.15.16:0/1385147991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f747de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb63fff640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73eb40 0x564f6f745090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 -- 172.21.15.16:0/1385147991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73eb40 msgr2=0x564f6f745090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73eb40 0x564f6f745090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 -- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 msgr2=0x564f6f747de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f747de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 -- 172.21.15.16:0/1385147991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6f4143c0 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb708c3640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009820 tx=0x7fbb64015720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128bd296692e37ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64025040 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fbb64004c10 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb64009d30 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb64004e10 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb746ff740 10 client.18777 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.399+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f6f756b40 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64009d30 con 0x564f6f748370 2024-09-12T01:06:48.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb6401e030 con 0x564f6f748370 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 10 client.18777 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 10 client.18777 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 20 client.18777 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f757f50 0x564f6f75a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 4 client.18777 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f6f6b0860 con 0x564f6f757f50 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f757f50 0x564f6f75a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb4b7fe640 10 client.18777 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.400+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f757f50 0x564f6f75a340 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=765b8c428f3e545f server_cookie=354d0d26d6017c3f in_seq=0 out_seq=0 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/1385147991 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f6f6b0860 con 0x564f6f757f50 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb4b7fe640 10 client.18777 handle_command_reply: tid=0 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 client.18777 shutdown 2024-09-12T01:06:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 msgr2=0x564f6f7457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009820 tx=0x7fbb64015720 comp rx=0 tx=0).stop 2024-09-12T01:06:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 shutdown_connections 2024-09-12T01:06:48.466 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f757f50 0x564f6f75a340 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f7457d0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73eb40 0x564f6f745090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 --2- 172.21.15.16:0/1385147991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73d5d0 0x564f6f747de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 >> 172.21.15.16:0/1385147991 conn(0x564f6f706310 msgr2=0x564f6f73f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.401+0000 7fbb4b7fe640 0 client.18777 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.402+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 shutdown_connections 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.402+0000 7fbb746ff740 1 -- 172.21.15.16:0/1385147991 wait complete. 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.402+0000 7fbb746ff740 20 client.18777 trim_cache size 0 max 16384 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.414+0000 7fbb746ff740 1 Processor -- start 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 -- start start 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 0x564f6f7542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f747000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ad9b0 con 0x564f6f748370 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1af4f0 con 0x564f6f73eb40 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb746ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ae080 con 0x564f6f73d5d0 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35098/0 (socket says 172.21.15.16:35098) 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 -- 172.21.15.16:0/2682918710 learned_addr learned my addr 172.21.15.16:0/2682918710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 0x564f6f7542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb63fff640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f747000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 -- 172.21.15.16:0/2682918710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 msgr2=0x564f6f7542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 0x564f6f7542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 -- 172.21.15.16:0/2682918710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 msgr2=0x564f6f747000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f747000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 -- 172.21.15.16:0/2682918710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6f739de0 con 0x564f6f748370 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 0x564f6f7542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb708c3640 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbb64000f80 tx=0x7fbb64015b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ab047a08cd9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.415+0000 7fbb61ffb640 1 -- 172.21.15.16:0/2682918710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64009d30 con 0x564f6f748370 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb61ffb640 1 -- 172.21.15.16:0/2682918710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64041020 con 0x564f6f748370 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb61ffb640 1 -- 172.21.15.16:0/2682918710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb640216a0 con 0x564f6f748370 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- 172.21.15.16:0/2682918710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 msgr2=0x564f6f750800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbb64000f80 tx=0x7fbb64015b50 comp rx=0 tx=0).stop 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- 172.21.15.16:0/2682918710 shutdown_connections 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f748370 0x564f6f750800 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f747000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 --2- 172.21.15.16:0/2682918710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f73d5d0 0x564f6f7542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- 172.21.15.16:0/2682918710 >> 172.21.15.16:0/2682918710 conn(0x564f6f706310 msgr2=0x564f6f7468f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- 172.21.15.16:0/2682918710 shutdown_connections 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- 172.21.15.16:0/2682918710 wait complete. 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.416+0000 7fbb746ff740 1 -- start start 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 Processor -- start 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f75f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f7bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1aee20 con 0x564f6f73d5d0 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ae750 con 0x564f6f73eb40 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6c1ab9b0 con 0x564f6f748370 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb746ff740 10 client.0 _ll_register_callbacks cb 0x7fbb5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35102/0 (socket says 172.21.15.16:35102) 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 -- 172.21.15.16:0/4205960726 learned_addr learned my addr 172.21.15.16:0/4205960726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb63fff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f75f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb708c3640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f7bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 -- 172.21.15.16:0/4205960726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 msgr2=0x564f6f7bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f7bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 -- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 msgr2=0x564f6f75f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f75f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.417+0000 7fbb6bfff640 1 -- 172.21.15.16:0/4205960726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6f74aad0 con 0x564f6f73d5d0 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c02a930 tx=0x7fbb5c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed5e4ba5ddfeeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c035040 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fbb5c005040 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb64041020 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb5c053050 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb746ff740 10 client.18783 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f6f7c6650 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb64041020 con 0x564f6f73d5d0 2024-09-12T01:06:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb5c0046c0 con 0x564f6f73d5d0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb746ff740 10 client.18783 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.418+0000 7fbb746ff740 10 client.18783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 20 client.18783 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f7c6dd0 0x564f6f7c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 4 client.18783 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f6f6aad70 con 0x564f6f7c6dd0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb63fff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f7c6dd0 0x564f6f7c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb63fff640 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f7c6dd0 0x564f6f7c91f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcab7ead8b68d97e server_cookie=c3054ce295e37981 in_seq=0 out_seq=0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb4b7fe640 10 client.18783 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb4b7fe640 1 -- 172.21.15.16:0/4205960726 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f6f6aad70 con 0x564f6f7c6dd0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb4b7fe640 10 client.18783 handle_command_reply: tid=0 2024-09-12T01:06:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 1 client.18783 shutdown 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 msgr2=0x564f6f75f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.419+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c02a930 tx=0x7fbb5c005dc0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 shutdown_connections 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x564f6f7c6dd0 0x564f6f7c91f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f6f748370 0x564f6f7bdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564f6f73eb40 0x564f6f75f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 --2- 172.21.15.16:0/4205960726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f6f73d5d0 0x564f6f75f450 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 >> 172.21.15.16:0/4205960726 conn(0x564f6f706310 msgr2=0x564f6f7384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb4b7fe640 0 client.18783 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 shutdown_connections 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 1 -- 172.21.15.16:0/4205960726 wait complete. 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb746ff740 20 client.18783 trim_cache size 0 max 16384 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 msgr2=0x7fbb580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 0x7fbb580a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009fd0 tx=0x7fbb6401d000 comp rx=0 tx=0).stop 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 msgr2=0x7fbb6c198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400b4a0 tx=0x7fbb5400b960 comp rx=0 tx=0).stop 2024-09-12T01:06:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 shutdown_connections 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fbb580a2cf0 0x7fbb580a51b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb6c106f20 0x7fbb6c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6c1027a0 0x7fbb6c198a40 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 --2- 172.21.15.16:0/909590553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb6c101df0 0x7fbb6c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 >> 172.21.15.16:0/909590553 conn(0x7fbb6c0fd8f0 msgr2=0x7fbb6c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 shutdown_connections 2024-09-12T01:06:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:48.420+0000 7fbb72326640 1 -- 172.21.15.16:0/909590553 wait complete. 2024-09-12T01:06:48.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:48 smithi016 ceph-mon[30972]: pgmap v331: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:06:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:48 smithi099 ceph-mon[35654]: pgmap v331: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:06:49.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:48 smithi159 ceph-mon[35566]: pgmap v331: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-12T01:06:49.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:06:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:49.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:06:50.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 -- 172.21.15.16:0/2355146081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c102790 msgr2=0x7f310c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 --2- 172.21.15.16:0/2355146081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c102790 0x7f310c1067e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f802a410 comp rx=0 tx=0).stop 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 -- 172.21.15.16:0/2355146081 shutdown_connections 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 --2- 172.21.15.16:0/2355146081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 0x7f310c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 --2- 172.21.15.16:0/2355146081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c102790 0x7f310c1067e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 --2- 172.21.15.16:0/2355146081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c101de0 0x7f310c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 -- 172.21.15.16:0/2355146081 >> 172.21.15.16:0/2355146081 conn(0x7f310c0fd8c0 msgr2=0x7f310c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 -- 172.21.15.16:0/2355146081 shutdown_connections 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.026+0000 7f3112e35640 1 -- 172.21.15.16:0/2355146081 wait complete. 2024-09-12T01:06:50.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 Processor -- start 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 -- start start 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c102790 0x7f310c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 0x7f310c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f310c111f90 con 0x7f310c101de0 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f310c111e10 con 0x7f310c102790 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3112e35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f310c112110 con 0x7f310c106f10 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35108/0 (socket says 172.21.15.16:35108) 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f3110bd1640 1 -- 172.21.15.16:0/2278775580 learned_addr learned my addr 172.21.15.16:0/2278775580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.027+0000 7f31113d2640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 0x7f310c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 -- 172.21.15.16:0/2278775580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 msgr2=0x7f310c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 0x7f310c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 -- 172.21.15.16:0/2278775580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c102790 msgr2=0x7f310c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f310bfff640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c102790 0x7f310c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c102790 0x7f310c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 -- 172.21.15.16:0/2278775580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c19d520 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3110bd1640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f310000c890 tx=0x7f310000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3109ffb640 1 -- 172.21.15.16:0/2278775580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100004220 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3109ffb640 1 -- 172.21.15.16:0/2278775580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31000043c0 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310c19d810 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3109ffb640 1 -- 172.21.15.16:0/2278775580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000049e0 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.028+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310c1a50f0 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.030+0000 7f3109ffb640 1 -- 172.21.15.16:0/2278775580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f31000079b0 con 0x7f310c101de0 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 Processor -- start 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 -- start start 2024-09-12T01:06:50.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f3109ffb640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 0x7f30fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a63ad762f0 con 0x55a63ae11b40 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a63ad1dcf0 con 0x55a63ae1b370 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f311520e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a63ad722b0 con 0x55a63ae105d0 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f31113d2640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 0x7f30fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f3109ffb640 1 -- 172.21.15.16:0/2278775580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f31000cf5c0 con 0x7f310c101de0 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35116/0 (socket says 172.21.15.16:35116) 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f3110bd1640 1 -- 172.21.15.16:0/177139424 learned_addr learned my addr 172.21.15.16:0/177139424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f310bfff640 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.031+0000 7f31113d2640 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 -- 172.21.15.16:0/177139424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 msgr2=0x55a63ae10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 -- 172.21.15.16:0/177139424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 msgr2=0x55a63ae1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 -- 172.21.15.16:0/177139424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63ad023a0 con 0x55a63ae11b40 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f31113d2640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 0x7f30fc0a51b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f31040070b0 tx=0x7f310400cb80 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f3110bd1640 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3100000c00 tx=0x7f3100020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d719a5a250e8a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f30f6ffd640 1 -- 172.21.15.16:0/177139424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310001d030 con 0x55a63ae11b40 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f30f6ffd640 1 -- 172.21.15.16:0/177139424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31000a55b0 con 0x55a63ae11b40 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.032+0000 7f30f6ffd640 1 -- 172.21.15.16:0/177139424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000a46f0 con 0x55a63ae11b40 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- 172.21.15.16:0/177139424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 msgr2=0x55a63ae10090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3100000c00 tx=0x7f3100020000 comp rx=0 tx=0).stop 2024-09-12T01:06:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- 172.21.15.16:0/177139424 shutdown_connections 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 --2- 172.21.15.16:0/177139424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae10090 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- 172.21.15.16:0/177139424 >> 172.21.15.16:0/177139424 conn(0x55a63add9310 msgr2=0x55a63ae0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- 172.21.15.16:0/177139424 shutdown_connections 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- 172.21.15.16:0/177139424 wait complete. 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 -- start start 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.033+0000 7f311520e740 1 Processor -- start 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae18080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae1b370 0x55a63ae18810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a63ae1e070 con 0x55a63ae105d0 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a63ad31060 con 0x55a63ae11b40 2024-09-12T01:06:50.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a63ae1e1f0 con 0x55a63ae1b370 2024-09-12T01:06:50.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 10 client.0 _ll_register_callbacks cb 0x55a63aa848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35124/0 (socket says 172.21.15.16:35124) 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 -- 172.21.15.16:0/807177503 learned_addr learned my addr 172.21.15.16:0/807177503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f31113d2640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae1b370 0x55a63ae18810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f310bfff640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae18080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 -- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae1b370 msgr2=0x55a63ae18810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae1b370 0x55a63ae18810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 -- 172.21.15.16:0/807177503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 msgr2=0x55a63ae18080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae18080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 -- 172.21.15.16:0/807177503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63aae73c0 con 0x55a63ae105d0 2024-09-12T01:06:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f3110bd1640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2de5ee6dedce89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f30ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310001d040 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f310009cfa0 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31000a55b0 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f31000cd140 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 10 client.18795 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.034+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a63ae29b90 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31000a55b0 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f31000a4260 con 0x55a63ae105d0 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 10 client.18795 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 10 client.18795 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 20 client.18795 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae2aee0 0x55a63ae2d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 4 client.18795 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a63ad83860 con 0x55a63ae2aee0 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.035+0000 7f310bfff640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae2aee0 0x55a63ae2d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f310bfff640 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae2aee0 0x55a63ae2d2d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed41546a8318abc9 server_cookie=4524b867c6fb91f0 in_seq=0 out_seq=0 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f30ebfff640 10 client.18795 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f30ebfff640 1 -- 172.21.15.16:0/807177503 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a63ad83860 con 0x55a63ae2aee0 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f30ebfff640 10 client.18795 handle_command_reply: tid=0 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 client.18795 shutdown 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 msgr2=0x55a63ae1add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100007ed0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 shutdown_connections 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae1b370 0x55a63ae18810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae18080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae2aee0 0x55a63ae2d2d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 --2- 172.21.15.16:0/807177503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae105d0 0x55a63ae1add0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f30ebfff640 0 client.18795 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.036+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 >> 172.21.15.16:0/807177503 conn(0x55a63add9310 msgr2=0x55a63ae12990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.037+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 shutdown_connections 2024-09-12T01:06:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.037+0000 7f311520e740 1 -- 172.21.15.16:0/807177503 wait complete. 2024-09-12T01:06:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.037+0000 7f311520e740 20 client.18795 trim_cache size 0 max 16384 2024-09-12T01:06:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 Processor -- start 2024-09-12T01:06:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 -- start start 2024-09-12T01:06:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1ad900 con 0x55a63ae11b40 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1af440 con 0x55a63ae1b370 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f311520e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1adfd0 con 0x55a63ae105d0 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35130/0 (socket says 172.21.15.16:35130) 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 -- 172.21.15.16:0/3102836394 learned_addr learned my addr 172.21.15.16:0/3102836394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f3110bd1640 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 -- 172.21.15.16:0/3102836394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 msgr2=0x55a63ae26b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae26b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 -- 172.21.15.16:0/3102836394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 msgr2=0x55a63ae13660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.050+0000 7f310bfff640 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f310bfff640 1 -- 172.21.15.16:0/3102836394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63ae165d0 con 0x55a63ae11b40 2024-09-12T01:06:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f310bfff640 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f30f8033000 tx=0x7f30f80040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b84568bcc8c1d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f30f67fc640 1 -- 172.21.15.16:0/3102836394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8035040 con 0x55a63ae11b40 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f30f67fc640 1 -- 172.21.15.16:0/3102836394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8005520 con 0x55a63ae11b40 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f30f67fc640 1 -- 172.21.15.16:0/3102836394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8004d70 con 0x55a63ae11b40 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 -- 172.21.15.16:0/3102836394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 msgr2=0x55a63ae13120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f30f8033000 tx=0x7f30f80040f0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 -- 172.21.15.16:0/3102836394 shutdown_connections 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae1b370 0x55a63ae13660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae11b40 0x55a63ae13120 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 --2- 172.21.15.16:0/3102836394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae26b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.051+0000 7f311520e740 1 -- 172.21.15.16:0/3102836394 >> 172.21.15.16:0/3102836394 conn(0x55a63add9310 msgr2=0x55a63ae26480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- 172.21.15.16:0/3102836394 shutdown_connections 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- 172.21.15.16:0/3102836394 wait complete. 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- start start 2024-09-12T01:06:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 Processor -- start 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1aed70 con 0x55a63ae1b370 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1ae6a0 con 0x55a63ae11b40 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f310c1ab900 con 0x55a63ae105d0 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.052+0000 7f311520e740 10 client.0 _ll_register_callbacks cb 0x7f30f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f3110bd1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35134/0 (socket says 172.21.15.16:35134) 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 -- 172.21.15.16:0/396047053 learned_addr learned my addr 172.21.15.16:0/396047053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f310bfff640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 -- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 msgr2=0x55a63ae2e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae2e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 -- 172.21.15.16:0/396047053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 msgr2=0x55a63ae2e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae2e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 -- 172.21.15.16:0/396047053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63ae94820 con 0x55a63ae1b370 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f3110bd1640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae2e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f31113d2640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3104007b90 tx=0x7f3104004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75eabae333b5959 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f30ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310401d7d0 con 0x55a63ae1b370 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f310401d970 con 0x55a63ae1b370 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.053+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8005520 con 0x55a63ae1b370 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f310401db70 con 0x55a63ae1b370 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 10 client.18801 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a63ae95b00 con 0x55a63ae1b370 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8005520 con 0x55a63ae1b370 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f310402f400 con 0x55a63ae1b370 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 10 client.18801 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 10 client.18801 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 20 client.18801 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae962a0 0x55a63ae986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 4 client.18801 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a63ad7dd70 con 0x55a63ae962a0 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.054+0000 7f3110bd1640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae962a0 0x55a63ae986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f3110bd1640 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae962a0 0x55a63ae986c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6539781fab419ca server_cookie=dfb075947530705d in_seq=0 out_seq=0 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f30ebfff640 10 client.18801 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f30ebfff640 1 -- 172.21.15.16:0/396047053 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a63ad7dd70 con 0x55a63ae962a0 2024-09-12T01:06:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f30ebfff640 10 client.18801 handle_command_reply: tid=0 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 client.18801 shutdown 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 msgr2=0x55a63ae2eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3104007b90 tx=0x7f3104004b50 comp rx=0 tx=0).stop 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 shutdown_connections 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55a63ae962a0 0x55a63ae986c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a63ae1b370 0x55a63ae2eb00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a63ae11b40 0x55a63ae2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f30ebfff640 0 client.18801 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 --2- 172.21.15.16:0/396047053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a63ae105d0 0x55a63ae2e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 >> 172.21.15.16:0/396047053 conn(0x55a63add9310 msgr2=0x55a63ae26ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 shutdown_connections 2024-09-12T01:06:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 1 -- 172.21.15.16:0/396047053 wait complete. 2024-09-12T01:06:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.055+0000 7f311520e740 20 client.18801 trim_cache size 0 max 16384 2024-09-12T01:06:50.070 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 msgr2=0x7f30fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 0x7f30fc0a51b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f31040070b0 tx=0x7f310400cb80 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 msgr2=0x7f310c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f310000c890 tx=0x7f310000cd50 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 shutdown_connections 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30fc0a2cf0 0x7f30fc0a51b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f310c106f10 0x7f310c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f310c102790 0x7f310c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 --2- 172.21.15.16:0/2278775580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c101de0 0x7f310c198570 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 >> 172.21.15.16:0/2278775580 conn(0x7f310c0fd8c0 msgr2=0x7f310c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 shutdown_connections 2024-09-12T01:06:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:50.056+0000 7f3112e35640 1 -- 172.21.15.16:0/2278775580 wait complete. 2024-09-12T01:06:50.803 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:50 smithi016 ceph-mon[30972]: pgmap v332: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T01:06:50.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:06:50.868 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:50 smithi099 ceph-mon[35654]: pgmap v332: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T01:06:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:50 smithi159 ceph-mon[35566]: pgmap v332: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T01:06:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 -- 172.21.15.16:0/4152745618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 msgr2=0x7fda38103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 --2- 172.21.15.16:0/4152745618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38103110 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fda28009cd0 tx=0x7fda2802a910 comp rx=0 tx=0).stop 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 -- 172.21.15.16:0/4152745618 shutdown_connections 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 --2- 172.21.15.16:0/4152745618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381082a0 0x7fda3810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 --2- 172.21.15.16:0/4152745618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381036e0 0x7fda38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 --2- 172.21.15.16:0/4152745618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38103110 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 -- 172.21.15.16:0/4152745618 >> 172.21.15.16:0/4152745618 conn(0x7fda380fc7d0 msgr2=0x7fda380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.348+0000 7fda3d011640 1 -- 172.21.15.16:0/4152745618 shutdown_connections 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.349+0000 7fda3d011640 1 -- 172.21.15.16:0/4152745618 wait complete. 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.349+0000 7fda3d011640 1 Processor -- start 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.349+0000 7fda3d011640 1 -- start start 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381036e0 0x7fda38100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381082a0 0x7fda38101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda381120e0 con 0x7fda38102d30 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda38111f60 con 0x7fda381036e0 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda38112260 con 0x7fda381082a0 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35160/0 (socket says 172.21.15.16:35160) 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 -- 172.21.15.16:0/749090721 learned_addr learned my addr 172.21.15.16:0/749090721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3659c640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381036e0 0x7fda38100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3759e640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381082a0 0x7fda38101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 -- 172.21.15.16:0/749090721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381082a0 msgr2=0x7fda38101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381082a0 0x7fda38101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 -- 172.21.15.16:0/749090721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381036e0 msgr2=0x7fda38100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381036e0 0x7fda38100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 -- 172.21.15.16:0/749090721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda381a6ae0 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda36d9d640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fda28033040 tx=0x7fda28003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28031660 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda28031800 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.350+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda381a6d70 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.351+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2802fc00 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.351+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda381a7270 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.351+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda38103110 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.353+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda280046b0 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.354+0000 7fda2ffff640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 0x7fda240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.354+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fda280f0820 con 0x7fda38102d30 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.354+0000 7fda3659c640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 0x7fda240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.355+0000 7fda3659c640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 0x7fda240a5230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fda38102420 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.357+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda280bd550 con 0x7fda38102d30 2024-09-12T01:06:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.581+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3810a2b0 con 0x7fda38102d30 2024-09-12T01:06:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.582+0000 7fda2ffff640 1 -- 172.21.15.16:0/749090721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fda280bce50 con 0x7fda38102d30 2024-09-12T01:06:51.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:06:51.581 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:06:51.581 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 msgr2=0x7fda240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 0x7fda240a5230 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fda38102420 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 msgr2=0x7fda38100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fda28033040 tx=0x7fda28003f10 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 shutdown_connections 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fda240a2d70 0x7fda240a5230 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda381082a0 0x7fda38101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda381036e0 0x7fda38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 --2- 172.21.15.16:0/749090721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda38102d30 0x7fda38100990 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 >> 172.21.15.16:0/749090721 conn(0x7fda380fc7d0 msgr2=0x7fda38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 shutdown_connections 2024-09-12T01:06:51.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:51.584+0000 7fda3d011640 1 -- 172.21.15.16:0/749090721 wait complete. 2024-09-12T01:06:51.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:51 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/749090721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:52.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:51 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/749090721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:52.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:51 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/749090721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:06:52.344 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 233533b6 --release --await' 2024-09-12T01:06:52.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 233533b6 --release --await 2024-09-12T01:06:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 -- 172.21.15.16:0/3290773233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184100e20 msgr2=0x7fd184101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 --2- 172.21.15.16:0/3290773233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184100e20 0x7fd184101200 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd18002a480 comp rx=0 tx=0).stop 2024-09-12T01:06:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 -- 172.21.15.16:0/3290773233 shutdown_connections 2024-09-12T01:06:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 --2- 172.21.15.16:0/3290773233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184102ef0 0x7fd18410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 --2- 172.21.15.16:0/3290773233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102570 0x7fd1841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 --2- 172.21.15.16:0/3290773233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184100e20 0x7fd184101200 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 -- 172.21.15.16:0/3290773233 >> 172.21.15.16:0/3290773233 conn(0x7fd1840fc7d0 msgr2=0x7fd1840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 -- 172.21.15.16:0/3290773233 shutdown_connections 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.520+0000 7fd18b509640 1 -- 172.21.15.16:0/3290773233 wait complete. 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 Processor -- start 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 -- start start 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184100e20 0x7fd1841973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102ef0 0x7fd18419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd184110f80 con 0x7fd184102570 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd184110e00 con 0x7fd184102ef0 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd18b509640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd184111100 con 0x7fd184100e20 2024-09-12T01:06:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd188aa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd188aa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35164/0 (socket says 172.21.15.16:35164) 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd188aa4640 1 -- 172.21.15.16:0/3034999214 learned_addr learned my addr 172.21.15.16:0/3034999214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd1892a5640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184100e20 0x7fd1841973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.521+0000 7fd189aa6640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102ef0 0x7fd18419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 -- 172.21.15.16:0/3034999214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184100e20 msgr2=0x7fd1841973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184100e20 0x7fd1841973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 -- 172.21.15.16:0/3034999214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102ef0 msgr2=0x7fd18419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102ef0 0x7fd18419bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 -- 172.21.15.16:0/3034999214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18419c360 con 0x7fd184102570 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd188aa4640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd17400e9f0 tx=0x7fd17400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd17a7fc640 1 -- 172.21.15.16:0/3034999214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17400cd80 con 0x7fd184102570 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd17a7fc640 1 -- 172.21.15.16:0/3034999214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd174010910 con 0x7fd184102570 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18419c650 con 0x7fd184102570 2024-09-12T01:06:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd17a7fc640 1 -- 172.21.15.16:0/3034999214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174018990 con 0x7fd184102570 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.522+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841a3f30 con 0x7fd184102570 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.524+0000 7fd17a7fc640 1 -- 172.21.15.16:0/3034999214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd174010ac0 con 0x7fd184102570 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.525+0000 7fd17a7fc640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 0x7fd1700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.525+0000 7fd1892a5640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 0x7fd1700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.525+0000 7fd17a7fc640 1 -- 172.21.15.16:0/3034999214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd1740cf6c0 con 0x7fd184102570 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd1892a5640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 0x7fd1700a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd180009230 tx=0x7fd1800058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 Processor -- start 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 -- start start 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc36365090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc36365a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc362cb2f0 con 0x55cc36370370 2024-09-12T01:06:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc36272cf0 con 0x55cc36366b40 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cc362c72b0 con 0x55cc363655d0 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd189aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd189aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35176/0 (socket says 172.21.15.16:35176) 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.526+0000 7fd189aa6640 1 -- 172.21.15.16:0/3854069651 learned_addr learned my addr 172.21.15.16:0/3854069651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd188aa4640 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc36365a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd1892a5640 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc36365090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 -- 172.21.15.16:0/3854069651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 msgr2=0x55cc36365a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc36365a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 -- 172.21.15.16:0/3854069651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 msgr2=0x55cc36365090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc36365090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 -- 172.21.15.16:0/3854069651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc362573a0 con 0x55cc36370370 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd189aa6640 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c460 tx=0x7fd17c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9857cbb6f6a1efb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd16f7fe640 1 -- 172.21.15.16:0/3854069651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004d20 con 0x55cc36370370 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd16f7fe640 1 -- 172.21.15.16:0/3854069651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c012a50 con 0x55cc36370370 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.527+0000 7fd16f7fe640 1 -- 172.21.15.16:0/3854069651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005650 con 0x55cc36370370 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- 172.21.15.16:0/3854069651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 msgr2=0x55cc36372760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c460 tx=0x7fd17c00c920 comp rx=0 tx=0).stop 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- 172.21.15.16:0/3854069651 shutdown_connections 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc36372760 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc36365a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/3854069651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc36365090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- 172.21.15.16:0/3854069651 >> 172.21.15.16:0/3854069651 conn(0x55cc3632e310 msgr2=0x55cc36361c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- 172.21.15.16:0/3854069651 shutdown_connections 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- 172.21.15.16:0/3854069651 wait complete. 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 -- start start 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.528+0000 7fd18d8e2740 1 Processor -- start 2024-09-12T01:06:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc3636fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc3636cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc36373070 con 0x55cc36370370 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc36286060 con 0x55cc36366b40 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cc363731f0 con 0x55cc363655d0 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd18d8e2740 10 client.0 _ll_register_callbacks cb 0x55cc35fd98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 -- 172.21.15.16:0/2922965098 learned_addr learned my addr 172.21.15.16:0/2922965098 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd188aa4640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc3636cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc3636fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 -- 172.21.15.16:0/2922965098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 msgr2=0x55cc3636fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc3636fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 -- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 msgr2=0x55cc3636cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.529+0000 7fd189aa6640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc3636cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd189aa6640 1 -- 172.21.15.16:0/2922965098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc3603c3c0 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc3636fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd189aa6640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd17c000f80 tx=0x7fd17c012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8985c75fb5b3d8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c028020 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd17c003970 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17c012a50 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd17c033050 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd18d8e2740 10 client.18819 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc3637eac0 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.530+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c012a50 con 0x55cc36370370 2024-09-12T01:06:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd17c019070 con 0x55cc36370370 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 10 client.18819 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 10 client.18819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 20 client.18819 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc3637ff20 0x55cc36382310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 4 client.18819 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc362d8860 con 0x55cc3637ff20 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc3637ff20 0x55cc36382310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc3637ff20 0x55cc36382310 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10a4ab2e933299aa server_cookie=34f16bd5d7f86777 in_seq=0 out_seq=0 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.531+0000 7fd16cff9640 10 client.18819 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd16cff9640 1 -- 172.21.15.16:0/2922965098 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc362d8860 con 0x55cc3637ff20 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd16cff9640 10 client.18819 handle_command_reply: tid=0 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 client.18819 shutdown 2024-09-12T01:06:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 msgr2=0x55cc3636d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd17c000f80 tx=0x7fd17c012e90 comp rx=0 tx=0).stop 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 shutdown_connections 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc3637ff20 0x55cc36382310 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc36370370 0x55cc3636d570 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc3636cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd16cff9640 0 client.18819 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2922965098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc363655d0 0x55cc3636fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.532+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 >> 172.21.15.16:0/2922965098 conn(0x55cc3632e310 msgr2=0x55cc36367960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.533+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 shutdown_connections 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.533+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2922965098 wait complete. 2024-09-12T01:06:52.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.533+0000 7fd18d8e2740 20 client.18819 trim_cache size 0 max 16384 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 Processor -- start 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 -- start start 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36366b40 0x55cc3636be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36370370 0x55cc36387ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841ab640 con 0x55cc363655d0 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841ad180 con 0x55cc36370370 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841abd10 con 0x55cc36366b40 2024-09-12T01:06:52.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd1892a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd1892a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35198/0 (socket says 172.21.15.16:35198) 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.546+0000 7fd1892a5640 1 -- 172.21.15.16:0/2325882963 learned_addr learned my addr 172.21.15.16:0/2325882963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd189aa6640 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36370370 0x55cc36387ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd188aa4640 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36366b40 0x55cc3636be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 -- 172.21.15.16:0/2325882963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36366b40 msgr2=0x55cc3636be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36366b40 0x55cc3636be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 -- 172.21.15.16:0/2325882963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36370370 msgr2=0x55cc36387ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36370370 0x55cc36387ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 -- 172.21.15.16:0/2325882963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc363828a0 con 0x55cc363655d0 2024-09-12T01:06:52.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd1892a5640 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd180054fd0 tx=0x7fd18003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52aac4d9485341b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd16effd640 1 -- 172.21.15.16:0/2325882963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180035050 con 0x55cc363655d0 2024-09-12T01:06:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd16effd640 1 -- 172.21.15.16:0/2325882963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18005a070 con 0x55cc363655d0 2024-09-12T01:06:52.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.547+0000 7fd16effd640 1 -- 172.21.15.16:0/2325882963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180031a80 con 0x55cc363655d0 2024-09-12T01:06:52.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2325882963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 msgr2=0x55cc3637c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd180054fd0 tx=0x7fd18003e000 comp rx=0 tx=0).stop 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2325882963 shutdown_connections 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36370370 0x55cc36387ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36366b40 0x55cc3636be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/2325882963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc3637c310 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2325882963 >> 172.21.15.16:0/2325882963 conn(0x55cc3632e310 msgr2=0x55cc3636b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2325882963 shutdown_connections 2024-09-12T01:06:52.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- 172.21.15.16:0/2325882963 wait complete. 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 -- start start 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.548+0000 7fd18d8e2740 1 Processor -- start 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc363789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36370370 0x55cc36375450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841acab0 con 0x55cc363655d0 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841ac3e0 con 0x55cc36366b40 2024-09-12T01:06:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841a9640 con 0x55cc36370370 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd18d8e2740 10 client.0 _ll_register_callbacks cb 0x7fd174097810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35208/0 (socket says 172.21.15.16:35208) 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 -- 172.21.15.16:0/717034068 learned_addr learned my addr 172.21.15.16:0/717034068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd188aa4640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc363789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd189aa6640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36370370 0x55cc36375450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 -- 172.21.15.16:0/717034068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36370370 msgr2=0x55cc36375450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36370370 0x55cc36375450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 -- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 msgr2=0x55cc363789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc363789c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.549+0000 7fd1892a5640 1 -- 172.21.15.16:0/717034068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc363614f0 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd1892a5640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd180057040 tx=0x7fd180031540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034ea3803291e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180035040 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd180031a00 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18005a070 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1800040e0 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd18d8e2740 10 client.18825 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc36362960 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18005a070 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.550+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd180071480 con 0x55cc363655d0 2024-09-12T01:06:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 10 client.18825 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 10 client.18825 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 20 client.18825 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc363eb4e0 0x55cc363ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 4 client.18825 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc362d2d70 con 0x55cc363eb4e0 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd188aa4640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc363eb4e0 0x55cc363ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd188aa4640 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc363eb4e0 0x55cc363ed8d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9fba22706b9a03c server_cookie=b3cae6c14cbe7e0c in_seq=0 out_seq=0 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.551+0000 7fd16cff9640 10 client.18825 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd16cff9640 1 -- 172.21.15.16:0/717034068 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc362d2d70 con 0x55cc363eb4e0 2024-09-12T01:06:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd16cff9640 10 client.18825 handle_command_reply: tid=0 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 client.18825 shutdown 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 msgr2=0x55cc36378480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd180057040 tx=0x7fd180031540 comp rx=0 tx=0).stop 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 shutdown_connections 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55cc363eb4e0 0x55cc363ed8d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc36370370 0x55cc36375450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc36366b40 0x55cc363789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 --2- 172.21.15.16:0/717034068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc363655d0 0x55cc36378480 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 >> 172.21.15.16:0/717034068 conn(0x55cc3632e310 msgr2=0x55cc36360d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd16cff9640 0 client.18825 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:06:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 shutdown_connections 2024-09-12T01:06:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 1 -- 172.21.15.16:0/717034068 wait complete. 2024-09-12T01:06:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.552+0000 7fd18d8e2740 20 client.18825 trim_cache size 0 max 16384 2024-09-12T01:06:52.559 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 msgr2=0x7fd1700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 0x7fd1700a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd180009230 tx=0x7fd1800058d0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 msgr2=0x7fd1841978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd17400e9f0 tx=0x7fd17400eeb0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 shutdown_connections 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd1700a2d70 0x7fd1700a5230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184102ef0 0x7fd18419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd184102570 0x7fd1841978f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 --2- 172.21.15.16:0/3034999214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd184100e20 0x7fd1841973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 >> 172.21.15.16:0/3034999214 conn(0x7fd1840fc7d0 msgr2=0x7fd1840fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 shutdown_connections 2024-09-12T01:06:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:52.553+0000 7fd18b509640 1 -- 172.21.15.16:0/3034999214 wait complete. 2024-09-12T01:06:53.011 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:52 smithi016 ceph-mon[30972]: pgmap v333: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:52 smithi099 ceph-mon[35654]: pgmap v333: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:53.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:52 smithi159 ceph-mon[35566]: pgmap v333: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:53.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:06:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 -- 172.21.15.16:0/4126029407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 msgr2=0x7f590c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 --2- 172.21.15.16:0/4126029407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c100530 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f5900009900 tx=0x7f590002a410 comp rx=0 tx=0).stop 2024-09-12T01:06:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 -- 172.21.15.16:0/4126029407 shutdown_connections 2024-09-12T01:06:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 --2- 172.21.15.16:0/4126029407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c100a70 0x7f590c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 --2- 172.21.15.16:0/4126029407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c100530 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 --2- 172.21.15.16:0/4126029407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c106ba0 0x7f590c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 -- 172.21.15.16:0/4126029407 >> 172.21.15.16:0/4126029407 conn(0x7f590c077210 msgr2=0x7f590c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 -- 172.21.15.16:0/4126029407 shutdown_connections 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.932+0000 7f5911f27640 1 -- 172.21.15.16:0/4126029407 wait complete. 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 Processor -- start 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 -- start start 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c100a70 0x7f590c1017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 0x7f590c101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1120b0 con 0x7f590c1000d0 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f590c111f30 con 0x7f590c106ba0 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f5911f27640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f590c112230 con 0x7f590c100a70 2024-09-12T01:06:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35224/0 (socket says 172.21.15.16:35224) 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f590b7fe640 1 -- 172.21.15.16:0/2519384076 learned_addr learned my addr 172.21.15.16:0/2519384076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.933+0000 7f590affd640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c100a70 0x7f590c1017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590bfff640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 0x7f590c101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 -- 172.21.15.16:0/2519384076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c100a70 msgr2=0x7f590c1017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c100a70 0x7f590c1017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 -- 172.21.15.16:0/2519384076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 msgr2=0x7f590c101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 0x7f590c101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 -- 172.21.15.16:0/2519384076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c1a6a10 con 0x7f590c1000d0 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590bfff640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 0x7f590c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f590b7fe640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00b4a0 tx=0x7f58fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f5908ff9640 1 -- 172.21.15.16:0/2519384076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc014070 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590c1a6d00 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f5908ff9640 1 -- 172.21.15.16:0/2519384076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc010af0 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f5908ff9640 1 -- 172.21.15.16:0/2519384076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc017d10 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.934+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590c1a71c0 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.936+0000 7f5908ff9640 1 -- 172.21.15.16:0/2519384076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f58fc017070 con 0x7f590c1000d0 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 Processor -- start 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 -- start start 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 0x55d4907c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d490724620 con 0x55d4907ca240 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d4906cbe50 con 0x55d4907bf980 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5914300740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4907205e0 con 0x55d4907bf060 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5908ff9640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 0x7f58f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35228/0 (socket says 172.21.15.16:35228) 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590affd640 1 -- 172.21.15.16:0/796062476 learned_addr learned my addr 172.21.15.16:0/796062476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590bfff640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590b7fe640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 0x55d4907c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f590bfff640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 0x7f58f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.937+0000 7f5908ff9640 1 -- 172.21.15.16:0/2519384076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f58fc0ce6b0 con 0x7f590c1000d0 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 -- 172.21.15.16:0/796062476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 msgr2=0x55d4907bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 -- 172.21.15.16:0/796062476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 msgr2=0x55d4907c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 0x55d4907c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 -- 172.21.15.16:0/796062476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4906bcd40 con 0x55d4907ca240 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590b7fe640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 0x55d4907c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590affd640 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5900009900 tx=0x7f5900004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fae7da2d2de2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f590bfff640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 0x7f58f40a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5904005fd0 tx=0x7f5904005f40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f58f1ffb640 1 -- 172.21.15.16:0/796062476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900035030 con 0x55d4907ca240 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f58f1ffb640 1 -- 172.21.15.16:0/796062476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900040070 con 0x55d4907ca240 2024-09-12T01:06:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.938+0000 7f58f1ffb640 1 -- 172.21.15.16:0/796062476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900031cb0 con 0x55d4907ca240 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- 172.21.15.16:0/796062476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 msgr2=0x55d4907cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f5900009900 tx=0x7f5900004200 comp rx=0 tx=0).stop 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- 172.21.15.16:0/796062476 shutdown_connections 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907cc630 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907bf980 0x55d4907c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 --2- 172.21.15.16:0/796062476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- 172.21.15.16:0/796062476 >> 172.21.15.16:0/796062476 conn(0x55d4907bb5f0 msgr2=0x55d4907bb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- 172.21.15.16:0/796062476 shutdown_connections 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- 172.21.15.16:0/796062476 wait complete. 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 -- start start 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.939+0000 7f5914300740 1 Processor -- start 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d4907cce10 con 0x55d4907bf980 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d4906acf10 con 0x55d4907ca240 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4907ccf90 con 0x55d4907bf060 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 10 client.0 _ll_register_callbacks cb 0x55d4906d0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35236/0 (socket says 172.21.15.16:35236) 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 -- 172.21.15.16:0/1953674503 learned_addr learned my addr 172.21.15.16:0/1953674503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590bfff640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590b7fe640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 -- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 msgr2=0x55d4907c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 -- 172.21.15.16:0/1953674503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 msgr2=0x55d4907c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 -- 172.21.15.16:0/1953674503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d490433580 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f590affd640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5900031c60 tx=0x7f5900005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e68ef931d07d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f58eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900035070 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f5900003a00 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.940+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5900040070 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.941+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5900031e40 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.941+0000 7f5914300740 10 client.18837 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.941+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d4904fd960 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.941+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900040070 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.941+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5900045470 con 0x55d4907bf980 2024-09-12T01:06:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 10 client.18837 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 10 client.18837 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 20 client.18837 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d4907d9c40 0x55d4907dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 4 client.18837 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d490731b90 con 0x55d4907d9c40 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f590b7fe640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d4907d9c40 0x55d4907dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f590b7fe640 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d4907d9c40 0x55d4907dc030 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4759a85901118483 server_cookie=c523432a8793847d in_seq=0 out_seq=0 2024-09-12T01:06:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.942+0000 7f58eaffd640 10 client.18837 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f58eaffd640 1 -- 172.21.15.16:0/1953674503 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d490731b90 con 0x55d4907d9c40 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f58eaffd640 10 client.18837 handle_command_reply: tid=0 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 client.18837 shutdown 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 msgr2=0x55d4907c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f5900031c60 tx=0x7f5900005c00 comp rx=0 tx=0).stop 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 shutdown_connections 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d4907d9c40 0x55d4907dc030 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f58eaffd640 0 client.18837 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907bf980 0x55d4907c7660 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 --2- 172.21.15.16:0/1953674503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.943+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 >> 172.21.15.16:0/1953674503 conn(0x55d4907bb5f0 msgr2=0x55d4907c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.944+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 shutdown_connections 2024-09-12T01:06:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.944+0000 7f5914300740 1 -- 172.21.15.16:0/1953674503 wait complete. 2024-09-12T01:06:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.944+0000 7f5914300740 20 client.18837 trim_cache size 0 max 16384 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 Processor -- start 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 -- start start 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 0x55d4907c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1afa20 con 0x55d4907ca240 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b1560 con 0x55d4907c7d30 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f5914300740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b00f0 con 0x55d4907bf060 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35244/0 (socket says 172.21.15.16:35244) 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 -- 172.21.15.16:0/1504029334 learned_addr learned my addr 172.21.15.16:0/1504029334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590b7fe640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590bfff640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 0x55d4907c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 -- 172.21.15.16:0/1504029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 msgr2=0x55d4907c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 -- 172.21.15.16:0/1504029334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 msgr2=0x55d4907c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 0x55d4907c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590affd640 1 -- 172.21.15.16:0/1504029334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4907cce90 con 0x55d4907ca240 2024-09-12T01:06:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590b7fe640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.961+0000 7f590bfff640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 0x55d4907c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f590affd640 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5900045840 tx=0x7f590002aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2b68e1db3bed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f58f17fa640 1 -- 172.21.15.16:0/1504029334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900035040 con 0x55d4907ca240 2024-09-12T01:06:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f58f17fa640 1 -- 172.21.15.16:0/1504029334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900040030 con 0x55d4907ca240 2024-09-12T01:06:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f58f17fa640 1 -- 172.21.15.16:0/1504029334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900003a60 con 0x55d4907ca240 2024-09-12T01:06:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 -- 172.21.15.16:0/1504029334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 msgr2=0x55d4907c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f5900045840 tx=0x7f590002aba0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 -- 172.21.15.16:0/1504029334 shutdown_connections 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907c7d30 0x55d4907c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907ca240 0x55d4907c4380 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 --2- 172.21.15.16:0/1504029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.962+0000 7f5914300740 1 -- 172.21.15.16:0/1504029334 >> 172.21.15.16:0/1504029334 conn(0x55d4907bb5f0 msgr2=0x55d4907c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- 172.21.15.16:0/1504029334 shutdown_connections 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- 172.21.15.16:0/1504029334 wait complete. 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- start start 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 Processor -- start 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b0e90 con 0x55d4907c7d30 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b07c0 con 0x55d4907ca240 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.963+0000 7f5914300740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1ada20 con 0x55d4907bf060 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f5914300740 10 client.0 _ll_register_callbacks cb 0x7f5900031540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35260/0 (socket says 172.21.15.16:35260) 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 -- 172.21.15.16:0/4239603686 learned_addr learned my addr 172.21.15.16:0/4239603686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590b7fe640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590bfff640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 -- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 msgr2=0x55d4907df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 -- 172.21.15.16:0/4239603686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 msgr2=0x55d4907dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907dda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 -- 172.21.15.16:0/4239603686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4907de1f0 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f590affd640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f590002a930 tx=0x7f5900000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356c15f6c07c787f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f58eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900035070 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f5900045d60 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5900040030 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.964+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59000040d0 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 10 client.18843 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d490845260 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5900040030 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5900066030 con 0x55d4907c7d30 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 10 client.18843 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 10 client.18843 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:06:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 20 client.18843 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d490845710 0x55d490847b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 4 client.18843 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "233533b6", "release": true, "await": true}] 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "233533b6", "release": true, "await": true}) -- 0x55d4907b6950 con 0x55d490845710 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.965+0000 7f590b7fe640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d490845710 0x55d490847b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.966+0000 7f590b7fe640 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d490845710 0x55d490847b30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2a740f87abd753 server_cookie=77f1b2445a185894 in_seq=0 out_seq=0 2024-09-12T01:06:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.966+0000 7f58eaffd640 10 client.18843 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f58eaffd640 1 -- 172.21.15.16:0/4239603686 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d4907b6950 con 0x55d490845710 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f58eaffd640 10 client.18843 handle_command_reply: tid=0 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 client.18843 shutdown 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 msgr2=0x55d4907df960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f590002a930 tx=0x7f5900000bc0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 shutdown_connections 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d490845710 0x55d490847b30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4907ca240 0x55d4907dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4907c7d30 0x55d4907df960 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 --2- 172.21.15.16:0/4239603686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4907bf060 0x55d4907df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f58eaffd640 0 client.18843 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 >> 172.21.15.16:0/4239603686 conn(0x55d4907bb5f0 msgr2=0x55d4907b9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 shutdown_connections 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 1 -- 172.21.15.16:0/4239603686 wait complete. 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.969+0000 7f5914300740 20 client.18843 trim_cache size 0 max 16384 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 44, 2024-09-12T01:06:53.970 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "233533b6": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 44, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 142.6, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:06:53.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 msgr2=0x7f58f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 0x7f58f40a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5904005fd0 tx=0x7f5904005f40 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 msgr2=0x7f590c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00b4a0 tx=0x7f58fc00b960 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 shutdown_connections 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58f40a2cf0 0x7f58f40a51b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f590c106ba0 0x7f590c101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f590c100a70 0x7f590c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 --2- 172.21.15.16:0/2519384076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f590c1000d0 0x7f590c1012b0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 >> 172.21.15.16:0/2519384076 conn(0x7f590c077210 msgr2=0x7f590c104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 shutdown_connections 2024-09-12T01:06:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:53.970+0000 7f5911f27640 1 -- 172.21.15.16:0/2519384076 wait complete. 2024-09-12T01:06:54.195 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:54.614 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '233533b6', total seconds elapsed: 27.5 2024-09-12T01:06:54.875 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:54 smithi016 ceph-mon[30972]: pgmap v334: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:54.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:54 smithi159 ceph-mon[35566]: pgmap v334: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:54 smithi099 ceph-mon[35654]: pgmap v334: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:06:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- 172.21.15.16:0/2340226622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 msgr2=0x7fef181089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/2340226622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef181089f0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef0402a410 comp rx=0 tx=0).stop 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- 172.21.15.16:0/2340226622 shutdown_connections 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/2340226622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 0x7fef18110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/2340226622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef181089f0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/2340226622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 0x7fef181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- 172.21.15.16:0/2340226622 >> 172.21.15.16:0/2340226622 conn(0x7fef180ffa90 msgr2=0x7fef18101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- 172.21.15.16:0/2340226622 shutdown_connections 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- 172.21.15.16:0/2340226622 wait complete. 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 Processor -- start 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.092+0000 7fef1f1f5640 1 -- start start 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 0x7fef1819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 0x7fef1819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef18114120 con 0x7fef181049a0 2024-09-12T01:06:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef18113fa0 con 0x7fef18103ff0 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef181142a0 con 0x7fef18109120 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35286/0 (socket says 172.21.15.16:35286) 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 -- 172.21.15.16:0/708335115 learned_addr learned my addr 172.21.15.16:0/708335115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1d792640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 0x7fef1819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1cf91640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 0x7fef1819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 -- 172.21.15.16:0/708335115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 msgr2=0x7fef1819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 0x7fef1819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 -- 172.21.15.16:0/708335115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 msgr2=0x7fef1819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 0x7fef1819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 -- 172.21.15.16:0/708335115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1819f6d0 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef17fff640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fef04006010 tx=0x7fef040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef15ffb640 1 -- 172.21.15.16:0/708335115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0403f030 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef15ffb640 1 -- 172.21.15.16:0/708335115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef04003ce0 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1819f960 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.093+0000 7fef15ffb640 1 -- 172.21.15.16:0/708335115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04035070 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.094+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a72a0 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef15ffb640 1 -- 172.21.15.16:0/708335115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef04042030 con 0x7fef181049a0 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 Processor -- start 2024-09-12T01:06:55.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 -- start start 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067def760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b067d482f0 con 0x55b067de25d0 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b067cefcf0 con 0x55b067ded370 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.096+0000 7fef215ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b067d442b0 con 0x55b067de3b40 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef15ffb640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 0x7fef080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35288/0 (socket says 172.21.15.16:35288) 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 -- 172.21.15.16:0/3419354303 learned_addr learned my addr 172.21.15.16:0/3419354303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1d792640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef17fff640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067def760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1d792640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 0x7fef080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef15ffb640 1 -- 172.21.15.16:0/708335115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fef040efa80 con 0x7fef181049a0 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 -- 172.21.15.16:0/3419354303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 msgr2=0x55b067de2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 -- 172.21.15.16:0/3419354303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 msgr2=0x55b067def760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067def760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 -- 172.21.15.16:0/3419354303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b067cd43a0 con 0x55b067de25d0 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1d792640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef17fff640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067def760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1cf91640 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00bcb0 tx=0x7fef0c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492db7af8c3a8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef1d792640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 0x7fef080a51b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef100096a0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef02ffd640 1 -- 172.21.15.16:0/3419354303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c015070 con 0x55b067de25d0 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef02ffd640 1 -- 172.21.15.16:0/3419354303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0c01a940 con 0x55b067de25d0 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.097+0000 7fef02ffd640 1 -- 172.21.15.16:0/3419354303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c011a60 con 0x55b067de25d0 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 -- 172.21.15.16:0/3419354303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 msgr2=0x55b067de2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00bcb0 tx=0x7fef0c00f730 comp rx=0 tx=0).stop 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 -- 172.21.15.16:0/3419354303 shutdown_connections 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067def760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067de2a10 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 --2- 172.21.15.16:0/3419354303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 -- 172.21.15.16:0/3419354303 >> 172.21.15.16:0/3419354303 conn(0x55b067dab310 msgr2=0x55b067ddec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 -- 172.21.15.16:0/3419354303 shutdown_connections 2024-09-12T01:06:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.098+0000 7fef215ce740 1 -- 172.21.15.16:0/3419354303 wait complete. 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 -- start start 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 Processor -- start 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067dea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067dea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b067df0070 con 0x55b067de25d0 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b067d03060 con 0x55b067ded370 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b067df01f0 con 0x55b067de3b40 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef215ce740 10 client.0 _ll_register_callbacks cb 0x55b067a568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef1cf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef1cf91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35298/0 (socket says 172.21.15.16:35298) 2024-09-12T01:06:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef1cf91640 1 -- 172.21.15.16:0/1182130278 learned_addr learned my addr 172.21.15.16:0/1182130278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef17fff640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067dea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.099+0000 7fef1d792640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067dea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 -- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 msgr2=0x55b067dea030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067dea030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 -- 172.21.15.16:0/1182130278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 msgr2=0x55b067dea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067dea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 -- 172.21.15.16:0/1182130278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b067ab93c0 con 0x55b067de25d0 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1d792640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067dea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fef0c000c00 tx=0x7fef0c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8988022dda7083ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7feef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c028060 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0c01a940 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fef0c01a6f0 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef0c015070 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef215ce740 10 client.18855 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b067dfbb40 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.100+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0c01a940 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fef0c0119b0 con 0x55b067de25d0 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 10 client.18855 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 20 client.18855 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067dfcef0 0x55b067dff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 4 client.18855 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b067d55860 con 0x55b067dfcef0 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef17fff640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067dfcef0 0x55b067dff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7fef17fff640 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067dfcef0 0x55b067dff2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7eefde56701e52f4 server_cookie=fcc354d084e10b2d in_seq=0 out_seq=0 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.101+0000 7feef7fff640 10 client.18855 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7feef7fff640 1 -- 172.21.15.16:0/1182130278 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b067d55860 con 0x55b067dfcef0 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7feef7fff640 10 client.18855 handle_command_reply: tid=0 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 client.18855 shutdown 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 msgr2=0x55b067decd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fef0c000c00 tx=0x7fef0c00bfd0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 shutdown_connections 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067dfcef0 0x55b067dff2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067ded370 0x55b067dea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067dea030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7feef7fff640 0 client.18855 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 --2- 172.21.15.16:0/1182130278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067de25d0 0x55b067decd80 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 >> 172.21.15.16:0/1182130278 conn(0x55b067dab310 msgr2=0x55b067de4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 shutdown_connections 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 1 -- 172.21.15.16:0/1182130278 wait complete. 2024-09-12T01:06:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.102+0000 7fef215ce740 20 client.18855 trim_cache size 0 max 16384 2024-09-12T01:06:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 Processor -- start 2024-09-12T01:06:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 -- start start 2024-09-12T01:06:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 0x55b067df9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef181afa90 con 0x55b067ded370 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef181b15d0 con 0x55b067de25d0 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef215ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef181b0160 con 0x55b067de3b40 2024-09-12T01:06:55.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35308/0 (socket says 172.21.15.16:35308) 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 -- 172.21.15.16:0/2781972617 learned_addr learned my addr 172.21.15.16:0/2781972617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef17fff640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1cf91640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 0x55b067df9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 -- 172.21.15.16:0/2781972617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 msgr2=0x55b067de4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 -- 172.21.15.16:0/2781972617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 msgr2=0x55b067df9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 0x55b067df9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1d792640 1 -- 172.21.15.16:0/2781972617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b067de8580 con 0x55b067ded370 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef17fff640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.116+0000 7fef1cf91640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 0x55b067df9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:55.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef1d792640 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fef10003c90 tx=0x7fef10009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447ada2129233d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef027fc640 1 -- 172.21.15.16:0/2781972617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1000d580 con 0x55b067ded370 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef027fc640 1 -- 172.21.15.16:0/2781972617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1001a070 con 0x55b067ded370 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef027fc640 1 -- 172.21.15.16:0/2781972617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10010bf0 con 0x55b067ded370 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 -- 172.21.15.16:0/2781972617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 msgr2=0x55b067de4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fef10003c90 tx=0x7fef10009920 comp rx=0 tx=0).stop 2024-09-12T01:06:55.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 -- 172.21.15.16:0/2781972617 shutdown_connections 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067de4d60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de3b40 0x55b067de4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 --2- 172.21.15.16:0/2781972617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de25d0 0x55b067df9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 -- 172.21.15.16:0/2781972617 >> 172.21.15.16:0/2781972617 conn(0x55b067dab310 msgr2=0x55b067de4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.117+0000 7fef215ce740 1 -- 172.21.15.16:0/2781972617 shutdown_connections 2024-09-12T01:06:55.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 -- 172.21.15.16:0/2781972617 wait complete. 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 -- start start 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 Processor -- start 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de25d0 0x55b067df1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de3b40 0x55b067df1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef181b0f00 con 0x55b067ded370 2024-09-12T01:06:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef181b0830 con 0x55b067de3b40 2024-09-12T01:06:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef181ada90 con 0x55b067de25d0 2024-09-12T01:06:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.118+0000 7fef215ce740 10 client.0 _ll_register_callbacks cb 0x7fef0c00bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35320/0 (socket says 172.21.15.16:35320) 2024-09-12T01:06:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 -- 172.21.15.16:0/1554529392 learned_addr learned my addr 172.21.15.16:0/1554529392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de25d0 0x55b067df1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef17fff640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de3b40 0x55b067df1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 -- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de25d0 msgr2=0x55b067df1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de25d0 0x55b067df1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 -- 172.21.15.16:0/1554529392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de3b40 msgr2=0x55b067df1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:55.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de3b40 0x55b067df1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 -- 172.21.15.16:0/1554529392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b067df2370 con 0x55b067ded370 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef1d792640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fef100049d0 tx=0x7fef10003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8cc3dd12eb567e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7feef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10010ac0 con 0x55b067ded370 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fef10010c60 con 0x55b067ded370 2024-09-12T01:06:55.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1001a070 con 0x55b067ded370 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.119+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef10010e60 con 0x55b067ded370 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 10 client.18861 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b067e67e40 con 0x55b067ded370 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1001a070 con 0x55b067ded370 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fef1001d070 con 0x55b067ded370 2024-09-12T01:06:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 10 client.18861 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 10 client.18861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 20 client.18861 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067e684b0 0x55b067e6a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 4 client.18861 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b067d4fd70 con 0x55b067e684b0 2024-09-12T01:06:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.120+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067e684b0 0x55b067e6a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef1cf91640 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067e684b0 0x55b067e6a8a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d965fb93073a60fa server_cookie=a289e5990c5c5d24 in_seq=0 out_seq=0 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7feef7fff640 10 client.18861 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7feef7fff640 1 -- 172.21.15.16:0/1554529392 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b067d4fd70 con 0x55b067e684b0 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7feef7fff640 10 client.18861 handle_command_reply: tid=0 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 client.18861 shutdown 2024-09-12T01:06:55.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 msgr2=0x55b067e66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fef100049d0 tx=0x7fef10003d80 comp rx=0 tx=0).stop 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 shutdown_connections 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b067ded370 0x55b067e66880 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b067e684b0 0x55b067e6a8a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7feef7fff640 0 client.18861 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b067de3b40 0x55b067df1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 --2- 172.21.15.16:0/1554529392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b067de25d0 0x55b067df1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 >> 172.21.15.16:0/1554529392 conn(0x55b067dab310 msgr2=0x55b067de4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 shutdown_connections 2024-09-12T01:06:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 1 -- 172.21.15.16:0/1554529392 wait complete. 2024-09-12T01:06:55.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.121+0000 7fef215ce740 20 client.18861 trim_cache size 0 max 16384 2024-09-12T01:06:55.142 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 msgr2=0x7fef080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 0x7fef080a51b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef100096a0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 msgr2=0x7fef1819ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fef04006010 tx=0x7fef040042f0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 shutdown_connections 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fef080a2cf0 0x7fef080a51b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef18109120 0x7fef1819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef181049a0 0x7fef1819ac60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 --2- 172.21.15.16:0/708335115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef18103ff0 0x7fef1819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 >> 172.21.15.16:0/708335115 conn(0x7fef180ffa90 msgr2=0x7fef18100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 shutdown_connections 2024-09-12T01:06:55.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:55.122+0000 7fef1f1f5640 1 -- 172.21.15.16:0/708335115 wait complete. 2024-09-12T01:06:55.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:06:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 -- 172.21.15.16:0/151549843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 msgr2=0x7f7b1010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 --2- 172.21.15.16:0/151549843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1010da40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009ca0 tx=0x7f7b0802a950 comp rx=0 tx=0).stop 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 -- 172.21.15.16:0/151549843 shutdown_connections 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 --2- 172.21.15.16:0/151549843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1010da40 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 --2- 172.21.15.16:0/151549843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10102570 0x7f7b101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 --2- 172.21.15.16:0/151549843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10100e20 0x7f7b10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 -- 172.21.15.16:0/151549843 >> 172.21.15.16:0/151549843 conn(0x7f7b100fc7d0 msgr2=0x7f7b100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 -- 172.21.15.16:0/151549843 shutdown_connections 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 -- 172.21.15.16:0/151549843 wait complete. 2024-09-12T01:06:56.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.594+0000 7f7b1749a640 1 Processor -- start 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 -- start start 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10100e20 0x7f7b101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10102570 0x7f7b10198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110f80 con 0x7f7b10102ef0 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110e00 con 0x7f7b10100e20 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b1749a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10111100 con 0x7f7b10102570 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35344/0 (socket says 172.21.15.16:35344) 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 -- 172.21.15.16:0/414499531 learned_addr learned my addr 172.21.15.16:0/414499531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15236640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10100e20 0x7f7b101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b14a35640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10102570 0x7f7b10198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 -- 172.21.15.16:0/414499531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10102570 msgr2=0x7f7b10198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10102570 0x7f7b10198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 -- 172.21.15.16:0/414499531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10100e20 msgr2=0x7f7b101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10100e20 0x7f7b101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 -- 172.21.15.16:0/414499531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1019d3a0 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.595+0000 7f7b15a37640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7b0802ae70 tx=0x7f7b0802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.596+0000 7f7b067fc640 1 -- 172.21.15.16:0/414499531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08034040 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.596+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1019d630 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.596+0000 7f7b067fc640 1 -- 172.21.15.16:0/414499531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b08030920 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.596+0000 7f7b067fc640 1 -- 172.21.15.16:0/414499531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080047f0 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.596+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a4f70 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.598+0000 7f7b067fc640 1 -- 172.21.15.16:0/414499531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b08037020 con 0x7f7b10102ef0 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.598+0000 7f7b19873740 1 Processor -- start 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.598+0000 7f7b19873740 1 -- start start 2024-09-12T01:06:56.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92ba5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92bb4780 0x55ac92bb6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac92b0b2f0 con 0x55ac92ba55d0 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ac92ab2cf0 con 0x55ac92bb4780 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b19873740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac92b072b0 con 0x55ac92ba6b40 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b067fc640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35350/0 (socket says 172.21.15.16:35350) 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 -- 172.21.15.16:0/522570424 learned_addr learned my addr 172.21.15.16:0/522570424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b14a35640 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92bb4780 0x55ac92bb6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b14a35640 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92ba5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b067fc640 1 -- 172.21.15.16:0/414499531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f7b080f0110 con 0x7f7b10102ef0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15a37640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 -- 172.21.15.16:0/522570424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 msgr2=0x55ac92ba5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92ba5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 -- 172.21.15.16:0/522570424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92bb4780 msgr2=0x55ac92bb6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92bb4780 0x55ac92bb6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 -- 172.21.15.16:0/522570424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac92a973a0 con 0x55ac92ba55d0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.599+0000 7f7b15236640 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00b760 tx=0x7f7b0c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa700777b83863b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b15a37640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b080edac0 tx=0x7f7b0802a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7afb7fe640 1 -- 172.21.15.16:0/522570424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c003dd0 con 0x55ac92ba55d0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7afb7fe640 1 -- 172.21.15.16:0/522570424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c015070 con 0x55ac92ba55d0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7afb7fe640 1 -- 172.21.15.16:0/522570424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c011440 con 0x55ac92ba55d0 2024-09-12T01:06:56.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 -- 172.21.15.16:0/522570424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 msgr2=0x55ac92ba5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00b760 tx=0x7f7b0c00bc20 comp rx=0 tx=0).stop 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 -- 172.21.15.16:0/522570424 shutdown_connections 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92bb4780 0x55ac92bb6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba55d0 0x55ac92ba5a10 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 --2- 172.21.15.16:0/522570424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92ba5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 -- 172.21.15.16:0/522570424 >> 172.21.15.16:0/522570424 conn(0x55ac92b6e310 msgr2=0x55ac92ba1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.600+0000 7f7b19873740 1 -- 172.21.15.16:0/522570424 shutdown_connections 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 -- 172.21.15.16:0/522570424 wait complete. 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 -- start start 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 Processor -- start 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92bb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92bb4780 0x55ac92bb1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac92bb7480 con 0x55ac92ba6b40 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ac92ac6060 con 0x55ac92ba55d0 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac92bb7600 con 0x55ac92bb4780 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b19873740 10 client.0 _ll_register_callbacks cb 0x55ac928198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b15236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b15236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35358/0 (socket says 172.21.15.16:35358) 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.601+0000 7f7b15236640 1 -- 172.21.15.16:0/37882942 learned_addr learned my addr 172.21.15.16:0/37882942 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b14a35640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92bb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b14a35640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92bb4780 0x55ac92bb1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 -- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92bb4780 msgr2=0x55ac92bb1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92bb4780 0x55ac92bb1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 -- 172.21.15.16:0/37882942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 msgr2=0x55ac92bb4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92bb4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 -- 172.21.15.16:0/37882942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9287c3c0 con 0x55ac92ba6b40 2024-09-12T01:06:56.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b15236640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0039c0 tx=0x7f7b0c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bada3fcfb1c8e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c028060 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f7b0c011440 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0c015070 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c011640 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b19873740 10 client.18873 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.602+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac92bc2f20 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c015070 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b0c022070 con 0x55ac92ba6b40 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 10 client.18873 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 20 client.18873 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:56.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92bc4320 0x55ac92bc6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 4 client.18873 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac92b18860 con 0x55ac92bc4320 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b14a35640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92bc4320 0x55ac92bc6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7b14a35640 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92bc4320 0x55ac92bc6710 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb2643454d99cd4 server_cookie=aaab463a7a74f8aa in_seq=0 out_seq=0 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.603+0000 7f7af8ff9640 10 client.18873 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7af8ff9640 1 -- 172.21.15.16:0/37882942 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac92b18860 con 0x55ac92bc4320 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7af8ff9640 10 client.18873 handle_command_reply: tid=0 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 client.18873 shutdown 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 msgr2=0x55ac92bb1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0039c0 tx=0x7f7b0c0048f0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 shutdown_connections 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92bc4320 0x55ac92bc6710 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92bb4780 0x55ac92bb1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92ba6b40 0x55ac92bb1420 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7af8ff9640 0 client.18873 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 --2- 172.21.15.16:0/37882942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92bb4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.604+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 >> 172.21.15.16:0/37882942 conn(0x55ac92b6e310 msgr2=0x55ac92ba7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.605+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 shutdown_connections 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.605+0000 7f7b19873740 1 -- 172.21.15.16:0/37882942 wait complete. 2024-09-12T01:06:56.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.605+0000 7f7b19873740 20 client.18873 trim_cache size 0 max 16384 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 Processor -- start 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 -- start start 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92ba1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92bc0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101ab4d0 con 0x55ac92bb4780 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101ad010 con 0x55ac92ba55d0 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b19873740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101abba0 con 0x55ac92ba6b40 2024-09-12T01:06:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 -- 172.21.15.16:0/3067490031 learned_addr learned my addr 172.21.15.16:0/3067490031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92ba1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b15236640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92bc0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 -- 172.21.15.16:0/3067490031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 msgr2=0x55ac92bc0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92bc0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 -- 172.21.15.16:0/3067490031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 msgr2=0x55ac92ba1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92ba1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b14a35640 1 -- 172.21.15.16:0/3067490031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac92ba2960 con 0x55ac92bb4780 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.618+0000 7f7b15236640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92bc0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.619+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000a700 tx=0x7f7b00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2001768c004e740 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.619+0000 7f7afaffd640 1 -- 172.21.15.16:0/3067490031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0001a040 con 0x55ac92bb4780 2024-09-12T01:06:56.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.619+0000 7f7afaffd640 1 -- 172.21.15.16:0/3067490031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0000fe80 con 0x55ac92bb4780 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.619+0000 7f7afaffd640 1 -- 172.21.15.16:0/3067490031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004e20 con 0x55ac92bb4780 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- 172.21.15.16:0/3067490031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 msgr2=0x55ac92bcbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000a700 tx=0x7f7b00004730 comp rx=0 tx=0).stop 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- 172.21.15.16:0/3067490031 shutdown_connections 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92bcbd60 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba6b40 0x55ac92bc0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- 172.21.15.16:0/3067490031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba55d0 0x55ac92ba1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- 172.21.15.16:0/3067490031 >> 172.21.15.16:0/3067490031 conn(0x55ac92b6e310 msgr2=0x55ac92ba1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- 172.21.15.16:0/3067490031 shutdown_connections 2024-09-12T01:06:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- 172.21.15.16:0/3067490031 wait complete. 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 -- start start 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 Processor -- start 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba55d0 0x55ac92bae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.620+0000 7f7b19873740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 0x55ac92baea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101ac940 con 0x55ac92bb4780 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101ac270 con 0x55ac92ba6b40 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101a94d0 con 0x55ac92ba55d0 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 10 client.0 _ll_register_callbacks cb 0x7f7b0000a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35388/0 (socket says 172.21.15.16:35388) 2024-09-12T01:06:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 -- 172.21.15.16:0/3465460428 learned_addr learned my addr 172.21.15.16:0/3465460428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b15236640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 0x55ac92baea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba55d0 0x55ac92bae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 -- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba55d0 msgr2=0x55ac92bae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba55d0 0x55ac92bae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 -- 172.21.15.16:0/3465460428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 msgr2=0x55ac92baea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 0x55ac92baea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 -- 172.21.15.16:0/3465460428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac92bc0c60 con 0x55ac92bb4780 2024-09-12T01:06:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b15236640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 0x55ac92baea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b14a35640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x55ac92baf6a0 tx=0x7f7b0000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66953dfe85ded889 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000047a0 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f7b00023480 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.621+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac92b12d70 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b00023680 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 10 client.18879 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac92c2ee50 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ac92b12d70 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b00037040 con 0x55ac92bb4780 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 10 client.18879 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 20 client.18879 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92c2f4b0 0x55ac92c318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 4 client.18879 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac92a8f170 con 0x55ac92c2f4b0 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.622+0000 7f7b15236640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92c2f4b0 0x55ac92c318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b15236640 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92c2f4b0 0x55ac92c318a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf1928211d04143 server_cookie=e60364f939aa590c in_seq=0 out_seq=0 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7af8ff9640 10 client.18879 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7af8ff9640 1 -- 172.21.15.16:0/3465460428 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac92a8f170 con 0x55ac92c2f4b0 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7af8ff9640 10 client.18879 handle_command_reply: tid=0 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 client.18879 shutdown 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 msgr2=0x55ac92baef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x55ac92baf6a0 tx=0x7f7b0000a2d0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 shutdown_connections 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ac92c2f4b0 0x55ac92c318a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac92bb4780 0x55ac92baef90 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ac92ba6b40 0x55ac92baea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7af8ff9640 0 client.18879 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 --2- 172.21.15.16:0/3465460428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac92ba55d0 0x55ac92bae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 >> 172.21.15.16:0/3465460428 conn(0x55ac92b6e310 msgr2=0x55ac92bb2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 shutdown_connections 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 1 -- 172.21.15.16:0/3465460428 wait complete. 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.623+0000 7f7b19873740 20 client.18879 trim_cache size 0 max 16384 2024-09-12T01:06:56.630 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:06:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 msgr2=0x7f7afc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7b080edac0 tx=0x7f7b0802a950 comp rx=0 tx=0).stop 2024-09-12T01:06:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 msgr2=0x7f7b1019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:56.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7b0802ae70 tx=0x7f7b0802f5d0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 shutdown_connections 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102ef0 0x7f7b1019ccc0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b10102570 0x7f7b10198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 --2- 172.21.15.16:0/414499531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b10100e20 0x7f7b101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 >> 172.21.15.16:0/414499531 conn(0x7f7b100fc7d0 msgr2=0x7f7b1006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 shutdown_connections 2024-09-12T01:06:56.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:56.624+0000 7f7b1749a640 1 -- 172.21.15.16:0/414499531 wait complete. 2024-09-12T01:06:56.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:56 smithi016 ceph-mon[30972]: pgmap v335: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-12T01:06:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:56 smithi099 ceph-mon[35654]: pgmap v335: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-12T01:06:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:56 smithi159 ceph-mon[35566]: pgmap v335: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-12T01:06:58.490 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:06:58.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:06:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:06:58.880 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:58 smithi016 ceph-mon[30972]: pgmap v336: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T01:06:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:58 smithi099 ceph-mon[35654]: pgmap v336: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T01:06:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:58 smithi159 ceph-mon[35566]: pgmap v336: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-09-12T01:06:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- 172.21.15.16:0/3811547383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8410102d30 msgr2=0x7f8410103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 --2- 172.21.15.16:0/3811547383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8410102d30 0x7f8410103110 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f8400009cd0 tx=0x7f840002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- 172.21.15.16:0/3811547383 shutdown_connections 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 --2- 172.21.15.16:0/3811547383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101082a0 0x7f841010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 --2- 172.21.15.16:0/3811547383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84101036e0 0x7f8410107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 --2- 172.21.15.16:0/3811547383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8410102d30 0x7f8410103110 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- 172.21.15.16:0/3811547383 >> 172.21.15.16:0/3811547383 conn(0x7f84100fc7d0 msgr2=0x7f84100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- 172.21.15.16:0/3811547383 shutdown_connections 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- 172.21.15.16:0/3811547383 wait complete. 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 Processor -- start 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.271+0000 7f84156fc640 1 -- start start 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8410102d30 0x7f84101985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101036e0 0x7f8410198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84101120a0 con 0x7f84101082a0 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8410111f20 con 0x7f84101036e0 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f84156fc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8410112220 con 0x7f8410102d30 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45918/0 (socket says 172.21.15.16:45918) 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 -- 172.21.15.16:0/3253404927 learned_addr learned my addr 172.21.15.16:0/3253404927 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840e7fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101036e0 0x7f8410198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840effd640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8410102d30 0x7f84101985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 -- 172.21.15.16:0/3253404927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8410102d30 msgr2=0x7f84101985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8410102d30 0x7f84101985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 -- 172.21.15.16:0/3253404927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101036e0 msgr2=0x7f8410198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101036e0 0x7f8410198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 -- 172.21.15.16:0/3253404927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841019d550 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.272+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f840800c970 tx=0x7f840800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.273+0000 7f83f7fff640 1 -- 172.21.15.16:0/3253404927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408007bb0 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.273+0000 7f83f7fff640 1 -- 172.21.15.16:0/3253404927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8408007d50 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.273+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841019d840 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.273+0000 7f83f7fff640 1 -- 172.21.15.16:0/3253404927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408018800 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.273+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a5120 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.275+0000 7f83f7fff640 1 -- 172.21.15.16:0/3253404927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f840800fa50 con 0x7f84101082a0 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.275+0000 7f8417ad5740 1 Processor -- start 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.275+0000 7f8417ad5740 1 -- start start 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.275+0000 7f83f7fff640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.275+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b646fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 0x5591b6472540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f8417ad5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5591b63c6280 con 0x5591b6460a60 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f8417ad5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591b636db10 con 0x5591b6470150 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f8417ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591b63c2240 con 0x5591b6461380 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840e7fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f83f7fff640 1 -- 172.21.15.16:0/3253404927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f84080cf410 con 0x7f84101082a0 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45930/0 (socket says 172.21.15.16:45930) 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 -- 172.21.15.16:0/786438066 learned_addr learned my addr 172.21.15.16:0/786438066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840f7fe640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 0x5591b6472540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840e7fc640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b646fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 -- 172.21.15.16:0/786438066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 msgr2=0x5591b646fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b646fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 -- 172.21.15.16:0/786438066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 msgr2=0x5591b6472540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 0x5591b6472540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 -- 172.21.15.16:0/786438066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591b637a1f0 con 0x5591b6460a60 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840f7fe640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 0x5591b6472540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840e7fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84100fedc0 tx=0x7f84040041e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.276+0000 7f840effd640 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8400005cc0 tx=0x7f8400031580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110a9fd05cda36e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f83f57fa640 1 -- 172.21.15.16:0/786438066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400031ac0 con 0x5591b6460a60 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f83f57fa640 1 -- 172.21.15.16:0/786438066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8400036040 con 0x5591b6460a60 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f83f57fa640 1 -- 172.21.15.16:0/786438066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400004830 con 0x5591b6460a60 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 -- 172.21.15.16:0/786438066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 msgr2=0x5591b6460e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f8400005cc0 tx=0x7f8400031580 comp rx=0 tx=0).stop 2024-09-12T01:06:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 -- 172.21.15.16:0/786438066 shutdown_connections 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6470150 0x5591b6472540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b646fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 --2- 172.21.15.16:0/786438066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6460a60 0x5591b6460e40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 -- 172.21.15.16:0/786438066 >> 172.21.15.16:0/786438066 conn(0x5591b645d0f0 msgr2=0x5591b645d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 -- 172.21.15.16:0/786438066 shutdown_connections 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.277+0000 7f8417ad5740 1 -- 172.21.15.16:0/786438066 wait complete. 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 -- start start 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 Processor -- start 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b646d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5591b6472d20 con 0x5591b6470150 2024-09-12T01:06:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591b6380f10 con 0x5591b6461380 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591b6472ea0 con 0x5591b6460a60 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f8417ad5740 10 client.0 _ll_register_callbacks cb 0x5591b6349a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45942/0 (socket says 172.21.15.16:45942) 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f840f7fe640 1 -- 172.21.15.16:0/4106609305 learned_addr learned my addr 172.21.15.16:0/4106609305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f840e7fc640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b646d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.278+0000 7f840effd640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 -- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 msgr2=0x5591b646d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 -- 172.21.15.16:0/4106609305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 msgr2=0x5591b646d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b646d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 -- 172.21.15.16:0/4106609305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591b61376a0 con 0x5591b6470150 2024-09-12T01:06:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f840f7fe640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f840809b5d0 tx=0x7f840809bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3c9661fe7b753f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080a4540 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f84080860d0 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400036040 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84080862d0 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f8417ad5740 10 client.18891 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591b647e960 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8400036040 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.279+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84080cf050 con 0x5591b6470150 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 10 client.18891 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 20 client.18891 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b647fcb0 0x5591b6482100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 4 client.18891 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591b63d37f0 con 0x5591b647fcb0 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f840effd640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b647fcb0 0x5591b6482100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f840effd640 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b647fcb0 0x5591b6482100 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0611d82ae5bf1d server_cookie=1075b4eefd9cec9f in_seq=0 out_seq=0 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.280+0000 7f83f2ff5640 10 client.18891 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f83f2ff5640 1 -- 172.21.15.16:0/4106609305 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591b63d37f0 con 0x5591b647fcb0 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f83f2ff5640 10 client.18891 handle_command_reply: tid=0 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 client.18891 shutdown 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 msgr2=0x5591b646dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f840809b5d0 tx=0x7f840809bfd0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 shutdown_connections 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b647fcb0 0x5591b6482100 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b646dab0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b646d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 --2- 172.21.15.16:0/4106609305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 >> 172.21.15.16:0/4106609305 conn(0x5591b645d0f0 msgr2=0x5591b6462e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f83f2ff5640 0 client.18891 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 shutdown_connections 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 1 -- 172.21.15.16:0/4106609305 wait complete. 2024-09-12T01:06:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.281+0000 7f8417ad5740 20 client.18891 trim_cache size 0 max 16384 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 Processor -- start 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 -- start start 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b64879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84101ab680 con 0x5591b6470150 2024-09-12T01:06:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84101ad1c0 con 0x5591b6461380 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f8417ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84101abd50 con 0x5591b6460a60 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45958/0 (socket says 172.21.15.16:45958) 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 -- 172.21.15.16:0/3746090276 learned_addr learned my addr 172.21.15.16:0/3746090276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840e7fc640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b64879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 -- 172.21.15.16:0/3746090276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 msgr2=0x5591b646ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840effd640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 -- 172.21.15.16:0/3746090276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 msgr2=0x5591b64879d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b64879d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840f7fe640 1 -- 172.21.15.16:0/3746090276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591b6458830 con 0x5591b6470150 2024-09-12T01:06:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.299+0000 7f840e7fc640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b64879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:06:59.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f840f7fe640 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f840809c180 tx=0x7f8408099de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d93ff7cc8853120 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f83f4ff9640 1 -- 172.21.15.16:0/3746090276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080ccd80 con 0x5591b6470150 2024-09-12T01:06:59.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f83f4ff9640 1 -- 172.21.15.16:0/3746090276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84080ccf20 con 0x5591b6470150 2024-09-12T01:06:59.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f83f4ff9640 1 -- 172.21.15.16:0/3746090276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840809a9f0 con 0x5591b6470150 2024-09-12T01:06:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 -- 172.21.15.16:0/3746090276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 msgr2=0x5591b6487f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f840809c180 tx=0x7f8408099de0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 -- 172.21.15.16:0/3746090276 shutdown_connections 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6487f10 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6461380 0x5591b64879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 --2- 172.21.15.16:0/3746090276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6460a60 0x5591b646ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 -- 172.21.15.16:0/3746090276 >> 172.21.15.16:0/3746090276 conn(0x5591b645d0f0 msgr2=0x5591b646e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 -- 172.21.15.16:0/3746090276 shutdown_connections 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.300+0000 7f8417ad5740 1 -- 172.21.15.16:0/3746090276 wait complete. 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 -- start start 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 Processor -- start 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6460a60 0x5591b6478310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6475100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84101acaf0 con 0x5591b6470150 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84101ac420 con 0x5591b6460a60 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84101a9680 con 0x5591b6461380 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f8417ad5740 10 client.0 _ll_register_callbacks cb 0x7f840809b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f840e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f840e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:50398/0 (socket says 172.21.15.16:50398) 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f840e7fc640 1 -- 172.21.15.16:0/982367060 learned_addr learned my addr 172.21.15.16:0/982367060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.301+0000 7f840f7fe640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6475100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840effd640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6460a60 0x5591b6478310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 -- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 msgr2=0x5591b6478850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 -- 172.21.15.16:0/982367060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6460a60 msgr2=0x5591b6478310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6460a60 0x5591b6478310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 -- 172.21.15.16:0/982367060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591b64611b0 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840e7fc640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f840f7fe640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6475100 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f840800c940 tx=0x7f840800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09b6e68f67015a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f83f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080ccd80 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f8408097830 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84080ccf20 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84080a4540 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f8417ad5740 10 client.18897 fetch_fsmap learned FSMap version 28 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.302+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591b64eb200 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84080ccf20 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84080cf040 con 0x5591b6470150 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 10 client.18897 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:06:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 20 client.18897 populate_metadata read hostname 'smithi016' 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b64eb6d0 0x5591b64edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 4 client.18897 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591b63cdd00 con 0x5591b64eb6d0 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f840effd640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b64eb6d0 0x5591b64edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f840effd640 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b64eb6d0 0x5591b64edac0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ac1bfaf4134b94 server_cookie=bcfbaf346b6b51b7 in_seq=0 out_seq=0 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.303+0000 7f83f2ff5640 10 client.18897 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.304+0000 7f83f2ff5640 1 -- 172.21.15.16:0/982367060 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591b63cdd00 con 0x5591b64eb6d0 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.304+0000 7f83f2ff5640 10 client.18897 handle_command_reply: tid=0 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.304+0000 7f8417ad5740 1 client.18897 shutdown 2024-09-12T01:06:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 msgr2=0x5591b6475100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6475100 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f840800c940 tx=0x7f840800fd40 comp rx=0 tx=0).stop 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 shutdown_connections 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5591b64eb6d0 0x5591b64edac0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591b6470150 0x5591b6475100 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f83f2ff5640 0 client.18897 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591b6461380 0x5591b6478850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 --2- 172.21.15.16:0/982367060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591b6460a60 0x5591b6478310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 >> 172.21.15.16:0/982367060 conn(0x5591b645d0f0 msgr2=0x5591b646d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 shutdown_connections 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 1 -- 172.21.15.16:0/982367060 wait complete. 2024-09-12T01:06:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f8417ad5740 20 client.18897 trim_cache size 0 max 16384 2024-09-12T01:06:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:06:59.309 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:06:59.309 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c9f82a47-d157-490b-a95a-2861a60936da", 2024-09-12T01:06:59.309 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:06:59.309 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 msgr2=0x7f83f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f84100fedc0 tx=0x7f84040041e0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 msgr2=0x7f841019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f840800c970 tx=0x7f840800ce30 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 shutdown_connections 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84101082a0 0x7f841019ce70 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84101036e0 0x7f8410198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 --2- 172.21.15.16:0/3253404927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8410102d30 0x7f84101985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.305+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 >> 172.21.15.16:0/3253404927 conn(0x7f84100fc7d0 msgr2=0x7f8410105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.306+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 shutdown_connections 2024-09-12T01:06:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:06:59.306+0000 7f84156fc640 1 -- 172.21.15.16:0/3253404927 wait complete. 2024-09-12T01:06:59.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:59 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:06:59.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:59 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:59.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:59 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:06:59.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:59 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:06:59.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:06:59 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:07:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:59 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:07:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:59 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:59 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:07:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:59 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:00.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:06:59 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:07:00.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:07:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:59 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:07:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:59 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:59 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:07:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:59 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:06:59 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:07:00.299 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:00.725 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:00 smithi016 ceph-mon[30972]: pgmap v337: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-12T01:07:00.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 -- 172.21.15.16:0/682713780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102570 msgr2=0x7f21e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 --2- 172.21.15.16:0/682713780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102570 0x7f21e01029b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009900 tx=0x7f21c802a480 comp rx=0 tx=0).stop 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 -- 172.21.15.16:0/682713780 shutdown_connections 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 --2- 172.21.15.16:0/682713780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0102ef0 0x7f21e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 --2- 172.21.15.16:0/682713780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102570 0x7f21e01029b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 --2- 172.21.15.16:0/682713780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e0100e20 0x7f21e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 -- 172.21.15.16:0/682713780 >> 172.21.15.16:0/682713780 conn(0x7f21e00fc7d0 msgr2=0x7f21e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 -- 172.21.15.16:0/682713780 shutdown_connections 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.822+0000 7f21e52d1640 1 -- 172.21.15.16:0/682713780 wait complete. 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 Processor -- start 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 -- start start 2024-09-12T01:07:00.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0100e20 0x7f21e01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e0102570 0x7f21e0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0110f80 con 0x7f21e0102ef0 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0110e00 con 0x7f21e0102570 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21e52d1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0111100 con 0x7f21e0100e20 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45990/0 (socket says 172.21.15.16:45990) 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21df7fe640 1 -- 172.21.15.16:0/2989792217 learned_addr learned my addr 172.21.15.16:0/2989792217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21deffd640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0100e20 0x7f21e01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21df7fe640 1 -- 172.21.15.16:0/2989792217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0100e20 msgr2=0x7f21e01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.823+0000 7f21df7fe640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0100e20 0x7f21e01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21df7fe640 1 -- 172.21.15.16:0/2989792217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e0102570 msgr2=0x7f21e0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21df7fe640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e0102570 0x7f21e0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21df7fe640 1 -- 172.21.15.16:0/2989792217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e019d480 con 0x7f21e0102ef0 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21df7fe640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f21d800eb10 tx=0x7f21d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21dcff9640 1 -- 172.21.15.16:0/2989792217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8004b20 con 0x7f21e0102ef0 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e019d770 con 0x7f21e0102ef0 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21dcff9640 1 -- 172.21.15.16:0/2989792217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d8004cc0 con 0x7f21e0102ef0 2024-09-12T01:07:00.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21dcff9640 1 -- 172.21.15.16:0/2989792217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8010760 con 0x7f21e0102ef0 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.824+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01a5050 con 0x7f21e0102ef0 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.827+0000 7f21e76aa740 1 Processor -- start 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.827+0000 7f21dcff9640 1 -- 172.21.15.16:0/2989792217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f21d8005020 con 0x7f21e0102ef0 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.827+0000 7f21e76aa740 1 -- start start 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 0x561fc4ef4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4eff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eff680 0x561fc4f01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561fc4e5a2e0 con 0x561fc4eff680 2024-09-12T01:07:00.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561fc4e01ce0 con 0x561fc4ef4dc0 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21e76aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561fc4e562a0 con 0x561fc4ef44a0 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 0x561fc4ef4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21deffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4eff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 0x561fc4ef4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:50414/0 (socket says 172.21.15.16:50414) 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21d7fff640 1 -- 172.21.15.16:0/1099211335 learned_addr learned my addr 172.21.15.16:0/1099211335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21d7fff640 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eff680 0x561fc4f01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21deffd640 1 -- 172.21.15.16:0/1099211335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 msgr2=0x561fc4ef4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21deffd640 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 0x561fc4ef4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21deffd640 1 -- 172.21.15.16:0/1099211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eff680 msgr2=0x561fc4f01a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.828+0000 7f21deffd640 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eff680 0x561fc4f01a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21deffd640 1 -- 172.21.15.16:0/1099211335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc4de6440 con 0x561fc4ef4dc0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21dcff9640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 0x7f21cc0a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21deffd640 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4eff140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009c20 tx=0x7f21d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e07e57a18b6134f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21dcff9640 1 -- 172.21.15.16:0/2989792217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f21d80cfb60 con 0x7f21e0102ef0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21d7fff640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 0x7f21cc0a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21d57fa640 1 -- 172.21.15.16:0/1099211335 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0004050 con 0x561fc4ef4dc0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21d57fa640 1 -- 172.21.15.16:0/1099211335 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d0005500 con 0x561fc4ef4dc0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.829+0000 7f21d57fa640 1 -- 172.21.15.16:0/1099211335 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00035f0 con 0x561fc4ef4dc0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 -- 172.21.15.16:0/1099211335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 msgr2=0x561fc4eff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21d7fff640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 0x7f21cc0a9640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x561fc4ef1db0 tx=0x7f21c8000bc0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4eff140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009c20 tx=0x7f21d0011810 comp rx=0 tx=0).stop 2024-09-12T01:07:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 -- 172.21.15.16:0/1099211335 shutdown_connections 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eff680 0x561fc4f01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4eff140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 --2- 172.21.15.16:0/1099211335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef44a0 0x561fc4ef4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 -- 172.21.15.16:0/1099211335 >> 172.21.15.16:0/1099211335 conn(0x561fc4ef0a30 msgr2=0x561fc4ef0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 -- 172.21.15.16:0/1099211335 shutdown_connections 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.830+0000 7f21e76aa740 1 -- 172.21.15.16:0/1099211335 wait complete. 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 -- start start 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 Processor -- start 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4efc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4efc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561fc4f02250 con 0x561fc4ef44a0 2024-09-12T01:07:00.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561fc4e15050 con 0x561fc4ef4dc0 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561fc4f023d0 con 0x561fc4eff680 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21e76aa740 10 client.0 _ll_register_callbacks cb 0x561fc4e55160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46006/0 (socket says 172.21.15.16:46006) 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4efc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21deffd640 1 -- 172.21.15.16:0/3503510185 learned_addr learned my addr 172.21.15.16:0/3503510185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.831+0000 7f21df7fe640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4efc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 -- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 msgr2=0x561fc4efc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4efc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 -- 172.21.15.16:0/3503510185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 msgr2=0x561fc4efc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4efc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 -- 172.21.15.16:0/3503510185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc4bcb3c0 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4efc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21deffd640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f21d0004e50 tx=0x7f21d0004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01e76f1dfc6811 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d002a060 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d0005500 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.832+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f21d0004480 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21d0005230 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 10 client.18909 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fc4f0dc20 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d0005500 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21d0018070 con 0x561fc4ef44a0 2024-09-12T01:07:00.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 10 client.18909 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 10 client.18909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 20 client.18909 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f0f030 0x561fc4f11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 4 client.18909 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc4e67850 con 0x561fc4f0f030 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.833+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f0f030 0x561fc4f11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.834+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f0f030 0x561fc4f11420 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719c802f6cd71903 server_cookie=b40ef567b3ebfb3 in_seq=0 out_seq=0 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.834+0000 7f21c67fc640 10 client.18909 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21c67fc640 1 -- 172.21.15.16:0/3503510185 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fc4e67850 con 0x561fc4f0f030 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21c67fc640 10 client.18909 handle_command_reply: tid=0 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 client.18909 shutdown 2024-09-12T01:07:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 msgr2=0x561fc4efef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f21d0004e50 tx=0x7f21d0004e80 comp rx=0 tx=0).stop 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 shutdown_connections 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f0f030 0x561fc4f11420 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4efc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4ef4dc0 0x561fc4efc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3503510185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef44a0 0x561fc4efef80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 >> 172.21.15.16:0/3503510185 conn(0x561fc4ef0a30 msgr2=0x561fc4ef6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21c67fc640 0 client.18909 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 shutdown_connections 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 1 -- 172.21.15.16:0/3503510185 wait complete. 2024-09-12T01:07:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.835+0000 7f21e76aa740 20 client.18909 trim_cache size 0 max 16384 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.854+0000 7f21e76aa740 1 Processor -- start 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 -- start start 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef4dc0 0x561fc4efe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 0x561fc4f16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ab750 con 0x561fc4eefb70 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ad290 con 0x561fc4eff680 2024-09-12T01:07:00.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21e76aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01abe20 con 0x561fc4ef4dc0 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46020/0 (socket says 172.21.15.16:46020) 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21df7fe640 1 -- 172.21.15.16:0/226909331 learned_addr learned my addr 172.21.15.16:0/226909331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21d7fff640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 0x561fc4f16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21deffd640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef4dc0 0x561fc4efe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.855+0000 7f21df7fe640 1 -- 172.21.15.16:0/226909331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef4dc0 msgr2=0x561fc4efe1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21df7fe640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef4dc0 0x561fc4efe1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21df7fe640 1 -- 172.21.15.16:0/226909331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 msgr2=0x561fc4f16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21df7fe640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 0x561fc4f16be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21df7fe640 1 -- 172.21.15.16:0/226909331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc4ef04d0 con 0x561fc4eefb70 2024-09-12T01:07:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21d7fff640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 0x561fc4f16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21df7fe640 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f21d8098a50 tx=0x7f21d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a93a0aef9e7c590 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21d4ff9640 1 -- 172.21.15.16:0/226909331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d809c080 con 0x561fc4eefb70 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21d4ff9640 1 -- 172.21.15.16:0/226909331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d80a0430 con 0x561fc4eefb70 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.856+0000 7f21d4ff9640 1 -- 172.21.15.16:0/226909331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d80986f0 con 0x561fc4eefb70 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- 172.21.15.16:0/226909331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 msgr2=0x561fc4f17120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f21d8098a50 tx=0x7f21d8004a40 comp rx=0 tx=0).stop 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- 172.21.15.16:0/226909331 shutdown_connections 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4eefb70 0x561fc4f17120 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eff680 0x561fc4f16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- 172.21.15.16:0/226909331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4ef4dc0 0x561fc4efe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- 172.21.15.16:0/226909331 >> 172.21.15.16:0/226909331 conn(0x561fc4ef0a30 msgr2=0x561fc4efda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- 172.21.15.16:0/226909331 shutdown_connections 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- 172.21.15.16:0/226909331 wait complete. 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 -- start start 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 Processor -- start 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eefb70 0x561fc4f07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.857+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4f05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01acbc0 con 0x561fc4ef4dc0 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01ac4f0 con 0x561fc4eefb70 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a9770 con 0x561fc4eff680 2024-09-12T01:07:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 10 client.0 _ll_register_callbacks cb 0x7f21d80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46034/0 (socket says 172.21.15.16:46034) 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 -- 172.21.15.16:0/3539275960 learned_addr learned my addr 172.21.15.16:0/3539275960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21deffd640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eefb70 0x561fc4f07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21df7fe640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4f05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 -- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 msgr2=0x561fc4f05460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4f05460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 -- 172.21.15.16:0/3539275960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eefb70 msgr2=0x561fc4f07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eefb70 0x561fc4f07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 -- 172.21.15.16:0/3539275960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc4f05aa0 con 0x561fc4ef4dc0 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21df7fe640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4f05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21d7fff640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a9a0 tx=0x7f21c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e7c51530db6b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8045070 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d80a0430 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.858+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f21c804b9d0 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21c8060050 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 10 client.18915 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fc4f79a10 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d80a0430 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21c803a040 con 0x561fc4ef4dc0 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 10 client.18915 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 10 client.18915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 20 client.18915 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f7a1f0 0x561fc4f7c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 4 client.18915 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fc4e61d60 con 0x561fc4f7a1f0 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.859+0000 7f21deffd640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f7a1f0 0x561fc4f7c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21c67fc640 10 client.18915 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21deffd640 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f7a1f0 0x561fc4f7c610 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1bc976f62059ed4 server_cookie=6a8b2e4584eaa61f in_seq=0 out_seq=0 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21c67fc640 1 -- 172.21.15.16:0/3539275960 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fc4e61d60 con 0x561fc4f7a1f0 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21c67fc640 10 client.18915 handle_command_reply: tid=0 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 client.18915 shutdown 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 msgr2=0x561fc4f04f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f21c802a9a0 tx=0x7f21c8005fd0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 shutdown_connections 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561fc4f7a1f0 0x561fc4f7c610 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561fc4eff680 0x561fc4f05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561fc4ef4dc0 0x561fc4f04f20 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21c67fc640 0 client.18915 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 --2- 172.21.15.16:0/3539275960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561fc4eefb70 0x561fc4f07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 >> 172.21.15.16:0/3539275960 conn(0x561fc4ef0a30 msgr2=0x561fc4efe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 shutdown_connections 2024-09-12T01:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 1 -- 172.21.15.16:0/3539275960 wait complete. 2024-09-12T01:07:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.860+0000 7f21e76aa740 20 client.18915 trim_cache size 0 max 16384 2024-09-12T01:07:00.875 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:00.875 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:07:00.875 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:07:00.876 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 msgr2=0x7f21cc0a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 0x7f21cc0a9640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x561fc4ef1db0 tx=0x7f21c8000bc0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 msgr2=0x7f21e019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f21d800eb10 tx=0x7f21d800c2d0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 shutdown_connections 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f21cc0a7180 0x7f21cc0a9640 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e0102ef0 0x7f21e019cda0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e0102570 0x7f21e0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 --2- 172.21.15.16:0/2989792217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f21e0100e20 0x7f21e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 >> 172.21.15.16:0/2989792217 conn(0x7f21e00fc7d0 msgr2=0x7f21e006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 shutdown_connections 2024-09-12T01:07:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:00.861+0000 7f21e52d1640 1 -- 172.21.15.16:0/2989792217 wait complete. 2024-09-12T01:07:01.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:00 smithi099 ceph-mon[35654]: pgmap v337: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-12T01:07:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:00 smithi159 ceph-mon[35566]: pgmap v337: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-12T01:07:01.534 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:07:01.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:07:01.791 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:02.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 -- 172.21.15.16:0/1627426299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 msgr2=0x7f33081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 --2- 172.21.15.16:0/1627426299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f33081067e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009900 tx=0x7f32fc02a410 comp rx=0 tx=0).stop 2024-09-12T01:07:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 -- 172.21.15.16:0/1627426299 shutdown_connections 2024-09-12T01:07:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 --2- 172.21.15.16:0/1627426299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 0x7f330810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 --2- 172.21.15.16:0/1627426299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f33081067e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 --2- 172.21.15.16:0/1627426299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 0x7f33081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 -- 172.21.15.16:0/1627426299 >> 172.21.15.16:0/1627426299 conn(0x7f33080fd8c0 msgr2=0x7f33080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 -- 172.21.15.16:0/1627426299 shutdown_connections 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.310+0000 7f330cbc4640 1 -- 172.21.15.16:0/1627426299 wait complete. 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 Processor -- start 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 -- start start 2024-09-12T01:07:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 0x7f3308198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 0x7f330819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111f70 con 0x7f3308102790 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111df0 con 0x7f3308106f10 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330cbc4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33081120f0 con 0x7f3308101de0 2024-09-12T01:07:02.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f3305d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f3305d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46050/0 (socket says 172.21.15.16:46050) 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f3305d9b640 1 -- 172.21.15.16:0/915023405 learned_addr learned my addr 172.21.15.16:0/915023405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f330659c640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 0x7f3308198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.311+0000 7f3306d9d640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 0x7f330819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 -- 172.21.15.16:0/915023405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 msgr2=0x7f3308198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:02.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 0x7f3308198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 -- 172.21.15.16:0/915023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 msgr2=0x7f330819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:02.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 0x7f330819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 -- 172.21.15.16:0/915023405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330819d400 con 0x7f3308102790 2024-09-12T01:07:02.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f3305d9b640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f32fc006010 tx=0x7f32fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc03f030 con 0x7f3308102790 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32fc003ce0 con 0x7f3308102790 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330819d690 con 0x7f3308102790 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc035070 con 0x7f3308102790 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.312+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a4fd0 con 0x7f3308102790 2024-09-12T01:07:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.313+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33081021c0 con 0x7f3308102790 2024-09-12T01:07:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.314+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f32fc042030 con 0x7f3308102790 2024-09-12T01:07:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.315+0000 7f32f77fe640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 0x7f32f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.315+0000 7f330659c640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 0x7f32f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:02.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.315+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7f32fc0ef930 con 0x7f3308102790 2024-09-12T01:07:02.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.316+0000 7f330659c640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 0x7f32f00a5230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32f800bbe0 tx=0x7f32f800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:02.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.319+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32fc0bbfc0 con 0x7f3308102790 2024-09-12T01:07:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.532+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f330810a1b0 con 0x7f3308102790 2024-09-12T01:07:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.533+0000 7f32f77fe640 1 -- 172.21.15.16:0/915023405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f32fc0c02c0 con 0x7f3308102790 2024-09-12T01:07:02.532 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:07:02.532 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:07:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:07:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 msgr2=0x7f32f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 0x7f32f00a5230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32f800bbe0 tx=0x7f32f800b390 comp rx=0 tx=0).stop 2024-09-12T01:07:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 msgr2=0x7f3308198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f32fc006010 tx=0x7f32fc0042f0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 shutdown_connections 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f32f00a2d70 0x7f32f00a5230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3308106f10 0x7f330819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.535+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3308102790 0x7f3308198990 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.536+0000 7f330cbc4640 1 --2- 172.21.15.16:0/915023405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3308101de0 0x7f3308198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.536+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 >> 172.21.15.16:0/915023405 conn(0x7f33080fd8c0 msgr2=0x7f33080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.536+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 shutdown_connections 2024-09-12T01:07:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:02.536+0000 7f330cbc4640 1 -- 172.21.15.16:0/915023405 wait complete. 2024-09-12T01:07:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:02 smithi016 ceph-mon[30972]: pgmap v338: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:02.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:02 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/915023405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:02 smithi099 ceph-mon[35654]: pgmap v338: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:02 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/915023405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:02 smithi159 ceph-mon[35566]: pgmap v338: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:02 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/915023405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:03.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:07:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 -- 172.21.15.16:0/2622363678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 msgr2=0x7fd73c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 --2- 172.21.15.16:0/2622363678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c10e610 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ca0 tx=0x7fd73402a950 comp rx=0 tx=0).stop 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 -- 172.21.15.16:0/2622363678 shutdown_connections 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 --2- 172.21.15.16:0/2622363678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c10e610 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 --2- 172.21.15.16:0/2622363678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 --2- 172.21.15.16:0/2622363678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 0x7fd73c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 -- 172.21.15.16:0/2622363678 >> 172.21.15.16:0/2622363678 conn(0x7fd73c0fc7a0 msgr2=0x7fd73c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 -- 172.21.15.16:0/2622363678 shutdown_connections 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.935+0000 7fd742bda640 1 -- 172.21.15.16:0/2622363678 wait complete. 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 Processor -- start 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 -- start start 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 0x7fd73c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111f70 con 0x7fd73c10c170 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111df0 con 0x7fd73c103660 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd742bda640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1120f0 con 0x7fd73c102d40 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd741177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd741177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46058/0 (socket says 172.21.15.16:46058) 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd741177640 1 -- 172.21.15.16:0/3876897509 learned_addr learned my addr 172.21.15.16:0/3876897509 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.936+0000 7fd740976640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 0x7fd73c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 -- 172.21.15.16:0/3876897509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 msgr2=0x7fd73c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 0x7fd73c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 -- 172.21.15.16:0/3876897509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 msgr2=0x7fd73c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 -- 172.21.15.16:0/3876897509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c19d3c0 con 0x7fd73c10c170 2024-09-12T01:07:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd741177640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd73402ae70 tx=0x7fd73402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd739ffb640 1 -- 172.21.15.16:0/3876897509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734034040 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd739ffb640 1 -- 172.21.15.16:0/3876897509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd734030920 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c19d650 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd739ffb640 1 -- 172.21.15.16:0/3876897509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734045480 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.937+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1a4f90 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd739ffb640 1 -- 172.21.15.16:0/3876897509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd734037020 con 0x7fd73c10c170 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 Processor -- start 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 -- start start 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a398382f0 con 0x561a398d3b40 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a397dfcf0 con 0x561a398e1780 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd744fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a398342b0 con 0x561a398d25d0 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46066/0 (socket says 172.21.15.16:46066) 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd739ffb640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 0x7fd72c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd73bfff640 1 -- 172.21.15.16:0/1137294603 learned_addr learned my addr 172.21.15.16:0/1137294603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd740976640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.940+0000 7fd741177640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 0x7fd72c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd739ffb640 1 -- 172.21.15.16:0/3876897509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd7340f02a0 con 0x7fd73c10c170 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 -- 172.21.15.16:0/1137294603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 msgr2=0x561a398d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 -- 172.21.15.16:0/1137294603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 msgr2=0x561a398e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 -- 172.21.15.16:0/1137294603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a397c43a0 con 0x561a398d3b40 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd740976640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd728008550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1242b4556d077e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd741177640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 0x7fd72c0a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7340edff0 tx=0x7fd73408f940 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd726ffd640 1 -- 172.21.15.16:0/1137294603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728018030 con 0x561a398d3b40 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd726ffd640 1 -- 172.21.15.16:0/1137294603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728005500 con 0x561a398d3b40 2024-09-12T01:07:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.941+0000 7fd726ffd640 1 -- 172.21.15.16:0/1137294603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72800c750 con 0x561a398d3b40 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- 172.21.15.16:0/1137294603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 msgr2=0x561a398d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd728008550 comp rx=0 tx=0).stop 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- 172.21.15.16:0/1137294603 shutdown_connections 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1137294603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d3b40 0x561a398d2090 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- 172.21.15.16:0/1137294603 >> 172.21.15.16:0/1137294603 conn(0x561a3989b310 msgr2=0x561a398cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- 172.21.15.16:0/1137294603 shutdown_connections 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- 172.21.15.16:0/1137294603 wait complete. 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 -- start start 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.942+0000 7fd744fb3740 1 Processor -- start 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 0x561a398de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a398e4480 con 0x561a398d25d0 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a397f3060 con 0x561a398e1780 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a398e4600 con 0x561a398d3b40 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd744fb3740 10 client.0 _ll_register_callbacks cb 0x561a395468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46074/0 (socket says 172.21.15.16:46074) 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 -- 172.21.15.16:0/1280775514 learned_addr learned my addr 172.21.15.16:0/1280775514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd740976640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 0x561a398de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 -- 172.21.15.16:0/1280775514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 msgr2=0x561a398de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 0x561a398de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 -- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 msgr2=0x561a398de9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398de9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 -- 172.21.15.16:0/1280775514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a395a93c0 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd740976640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 0x561a398de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd73bfff640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fd728005940 tx=0x7fd7280056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29efa6fb41b963de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.943+0000 7fd71bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72800c750 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd72800c8f0 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd728005500 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd728030050 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 10 client.18933 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a398efee0 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728005500 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd728029060 con 0x561a398d25d0 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 10 client.18933 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 10 client.18933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:07:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 20 client.18933 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a398f1280 0x561a398f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 4 client.18933 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.944+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a39845860 con 0x561a398f1280 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.945+0000 7fd740976640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a398f1280 0x561a398f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.945+0000 7fd71bfff640 10 client.18933 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.945+0000 7fd740976640 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a398f1280 0x561a398f3670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea89d1ddd23ec53d server_cookie=fc020cd0652a3a67 in_seq=0 out_seq=0 2024-09-12T01:07:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd71bfff640 1 -- 172.21.15.16:0/1280775514 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a39845860 con 0x561a398f1280 2024-09-12T01:07:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd71bfff640 10 client.18933 handle_command_reply: tid=0 2024-09-12T01:07:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 client.18933 shutdown 2024-09-12T01:07:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 msgr2=0x561a398e1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fd728005940 tx=0x7fd7280056b0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 shutdown_connections 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a398f1280 0x561a398f3670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398e1780 0x561a398de9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d3b40 0x561a398de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 --2- 172.21.15.16:0/1280775514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398e1140 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd71bfff640 0 client.18933 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 >> 172.21.15.16:0/1280775514 conn(0x561a3989b310 msgr2=0x561a398d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 shutdown_connections 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 1 -- 172.21.15.16:0/1280775514 wait complete. 2024-09-12T01:07:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.946+0000 7fd744fb3740 20 client.18933 trim_cache size 0 max 16384 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.959+0000 7fd744fb3740 1 Processor -- start 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.959+0000 7fd744fb3740 1 -- start start 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ab620 con 0x561a398e1780 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ad160 con 0x561a398d3b40 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd744fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1abcf0 con 0x561a398d25d0 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46076/0 (socket says 172.21.15.16:46076) 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 -- 172.21.15.16:0/3175036218 learned_addr learned my addr 172.21.15.16:0/3175036218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd740976640 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 -- 172.21.15.16:0/3175036218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 msgr2=0x561a398ed720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398ed720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 -- 172.21.15.16:0/3175036218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 msgr2=0x561a398ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 -- 172.21.15.16:0/3175036218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a398d3750 con 0x561a398e1780 2024-09-12T01:07:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd73bfff640 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd728008590 tx=0x7fd728008b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a83b84e116493fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.960+0000 7fd7267fc640 1 -- 172.21.15.16:0/3175036218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728018070 con 0x561a398e1780 2024-09-12T01:07:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd7267fc640 1 -- 172.21.15.16:0/3175036218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728019040 con 0x561a398e1780 2024-09-12T01:07:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd7267fc640 1 -- 172.21.15.16:0/3175036218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280120c0 con 0x561a398e1780 2024-09-12T01:07:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 -- 172.21.15.16:0/3175036218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 msgr2=0x561a398f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd728008590 tx=0x7fd728008b40 comp rx=0 tx=0).stop 2024-09-12T01:07:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 -- 172.21.15.16:0/3175036218 shutdown_connections 2024-09-12T01:07:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398e1780 0x561a398f8db0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 --2- 172.21.15.16:0/3175036218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398d25d0 0x561a398ed720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 -- 172.21.15.16:0/3175036218 >> 172.21.15.16:0/3175036218 conn(0x561a3989b310 msgr2=0x561a398dd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 -- 172.21.15.16:0/3175036218 shutdown_connections 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.961+0000 7fd744fb3740 1 -- 172.21.15.16:0/3175036218 wait complete. 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 -- start start 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 Processor -- start 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398e1780 0x561a398e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1aca90 con 0x561a398d25d0 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ac3c0 con 0x561a398d3b40 2024-09-12T01:07:03.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1a9620 con 0x561a398e1780 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd744fb3740 10 client.0 _ll_register_callbacks cb 0x7fd728008ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46078/0 (socket says 172.21.15.16:46078) 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd73bfff640 1 -- 172.21.15.16:0/2611283830 learned_addr learned my addr 172.21.15.16:0/2611283830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd73bfff640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398e1780 0x561a398e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.962+0000 7fd740976640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 -- 172.21.15.16:0/2611283830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398e1780 msgr2=0x561a398e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398e1780 0x561a398e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 -- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 msgr2=0x561a398e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 -- 172.21.15.16:0/2611283830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a398e6160 con 0x561a398d25d0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd73bfff640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd728008700 tx=0x7fd728005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478dc7b1bc2cddcd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd71bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728018070 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8521+0+0 (secure 0 0 0) 0x7fd728004020 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd728019040 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7280056b0 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd744fb3740 10 client.18939 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561a3995be20 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.963+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728019040 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd728029050 con 0x561a398d25d0 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 10 client.18939 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 10 client.18939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 20 client.18939 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a3995c420 0x561a3995e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 4 client.18939 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a3983fd70 con 0x561a3995c420 2024-09-12T01:07:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd740976640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a3995c420 0x561a3995e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd71bfff640 10 client.18939 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd740976640 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a3995c420 0x561a3995e810 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b13fc966e095a9d server_cookie=e9c1d23d409fbe8b in_seq=0 out_seq=0 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd71bfff640 1 -- 172.21.15.16:0/2611283830 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a3983fd70 con 0x561a3995c420 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd71bfff640 10 client.18939 handle_command_reply: tid=0 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.964+0000 7fd744fb3740 1 client.18939 shutdown 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 msgr2=0x561a398ea080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd728008700 tx=0x7fd728005610 comp rx=0 tx=0).stop 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 shutdown_connections 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x561a3995c420 0x561a3995e810 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a398e1780 0x561a398e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a398d3b40 0x561a398e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 --2- 172.21.15.16:0/2611283830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a398d25d0 0x561a398ea080 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd71bfff640 0 client.18939 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:03.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 >> 172.21.15.16:0/2611283830 conn(0x561a3989b310 msgr2=0x561a398cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 shutdown_connections 2024-09-12T01:07:03.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 1 -- 172.21.15.16:0/2611283830 wait complete. 2024-09-12T01:07:03.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd744fb3740 20 client.18939 trim_cache size 0 max 16384 2024-09-12T01:07:03.969 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:03.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 msgr2=0x7fd72c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 0x7fd72c0a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7340edff0 tx=0x7fd73408f940 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 msgr2=0x7fd73c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.965+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd73402ae70 tx=0x7fd73402f5b0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 shutdown_connections 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd72c0a2cf0 0x7fd72c0a51b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd73c10c170 0x7fd73c19cce0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd73c103660 0x7fd73c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 --2- 172.21.15.16:0/3876897509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd73c102d40 0x7fd73c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 >> 172.21.15.16:0/3876897509 conn(0x7fd73c0fc7a0 msgr2=0x7fd73c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 shutdown_connections 2024-09-12T01:07:03.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:03.966+0000 7fd742bda640 1 -- 172.21.15.16:0/3876897509 wait complete. 2024-09-12T01:07:04.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:07:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:04 smithi016 ceph-mon[30972]: pgmap v339: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:04 smithi099 ceph-mon[35654]: pgmap v339: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:04 smithi159 ceph-mon[35566]: pgmap v339: 97 pgs: 97 active+clean; 14 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-12T01:07:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 -- 172.21.15.16:0/1343290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101de0 msgr2=0x7fe9941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 --2- 172.21.15.16:0/1343290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101de0 0x7fe9941021c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe984009900 tx=0x7fe98402a410 comp rx=0 tx=0).stop 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 -- 172.21.15.16:0/1343290252 shutdown_connections 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 --2- 172.21.15.16:0/1343290252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994106f10 0x7fe99410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 --2- 172.21.15.16:0/1343290252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994102790 0x7fe9941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 --2- 172.21.15.16:0/1343290252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994101de0 0x7fe9941021c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 -- 172.21.15.16:0/1343290252 >> 172.21.15.16:0/1343290252 conn(0x7fe9940fd8c0 msgr2=0x7fe9940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 -- 172.21.15.16:0/1343290252 shutdown_connections 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.363+0000 7fe999255640 1 -- 172.21.15.16:0/1343290252 wait complete. 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 Processor -- start 2024-09-12T01:07:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 -- start start 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994101de0 0x7fe994198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994102790 0x7fe994198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe994111f70 con 0x7fe994106f10 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe994111df0 con 0x7fe994101de0 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.364+0000 7fe999255640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941120f0 con 0x7fe994102790 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46104/0 (socket says 172.21.15.16:46104) 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 -- 172.21.15.16:0/327913738 learned_addr learned my addr 172.21.15.16:0/327913738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99259c640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994102790 0x7fe994198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe992d9d640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994101de0 0x7fe994198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 -- 172.21.15.16:0/327913738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994102790 msgr2=0x7fe994198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994102790 0x7fe994198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 -- 172.21.15.16:0/327913738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994101de0 msgr2=0x7fe994198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994101de0 0x7fe994198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 -- 172.21.15.16:0/327913738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99419d3e0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe99359e640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eaa0 tx=0x7fe98c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe97bfff640 1 -- 172.21.15.16:0/327913738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cdb0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99419d6d0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe97bfff640 1 -- 172.21.15.16:0/327913738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c005020 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe97bfff640 1 -- 172.21.15.16:0/327913738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0116d0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.365+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a4fb0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe97bfff640 1 -- 172.21.15.16:0/327913738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe98c0054d0 con 0x7fe994106f10 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 Processor -- start 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 -- start start 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff42b40 0x559bbff41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe97bfff640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 0x7fe97c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bbfea72f0 con 0x559bbff4c370 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559bbfe4ecf0 con 0x559bbff415d0 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99b62e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559bbfea32b0 con 0x559bbff42b40 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46120/0 (socket says 172.21.15.16:46120) 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99259c640 1 -- 172.21.15.16:0/64804726 learned_addr learned my addr 172.21.15.16:0/64804726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.368+0000 7fe99359e640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 0x7fe97c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff42b40 0x559bbff41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe97bfff640 1 -- 172.21.15.16:0/327913738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fe98c0cf660 con 0x7fe994106f10 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe992d9d640 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 -- 172.21.15.16:0/64804726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff42b40 msgr2=0x559bbff41090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff42b40 0x559bbff41090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 -- 172.21.15.16:0/64804726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 msgr2=0x559bbff41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 -- 172.21.15.16:0/64804726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbfe333a0 con 0x559bbff4c370 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99359e640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 0x7fe97c0a51b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0cce30 tx=0x7fe98c01eb40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe99259c640 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b6c0 tx=0x7fe98800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412ac296b7f08c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe9797fa640 1 -- 172.21.15.16:0/64804726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988012d00 con 0x559bbff4c370 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe9797fa640 1 -- 172.21.15.16:0/64804726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988010770 con 0x559bbff4c370 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.369+0000 7fe9797fa640 1 -- 172.21.15.16:0/64804726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988003850 con 0x559bbff4c370 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- 172.21.15.16:0/64804726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 msgr2=0x559bbff4e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b6c0 tx=0x7fe98800b650 comp rx=0 tx=0).stop 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- 172.21.15.16:0/64804726 shutdown_connections 2024-09-12T01:07:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff4e760 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 --2- 172.21.15.16:0/64804726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff42b40 0x559bbff41090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- 172.21.15.16:0/64804726 >> 172.21.15.16:0/64804726 conn(0x559bbff0a310 msgr2=0x559bbff3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- 172.21.15.16:0/64804726 shutdown_connections 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- 172.21.15.16:0/64804726 wait complete. 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 -- start start 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.370+0000 7fe99b62e740 1 Processor -- start 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff4be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff4c370 0x559bbff49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559bbff4f070 con 0x559bbff42b40 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559bbfe62060 con 0x559bbff415d0 2024-09-12T01:07:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559bbff4f1f0 con 0x559bbff4c370 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99b62e740 10 client.0 _ll_register_callbacks cb 0x559bbfbb58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46130/0 (socket says 172.21.15.16:46130) 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 -- 172.21.15.16:0/2294678507 learned_addr learned my addr 172.21.15.16:0/2294678507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99259c640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff4be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe99259c640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff4c370 0x559bbff49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 -- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff4c370 msgr2=0x559bbff49840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff4c370 0x559bbff49840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 -- 172.21.15.16:0/2294678507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 msgr2=0x559bbff4be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff4be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 -- 172.21.15.16:0/2294678507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbfc183c0 con 0x559bbff42b40 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe992d9d640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe984009900 tx=0x7fe9840317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22738f2717cff78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe976ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.371+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984035070 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fe984004040 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988010770 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe984004240 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 10 client.18951 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bbff5abc0 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988010770 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe98404b070 con 0x559bbff42b40 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 10 client.18951 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 10 client.18951 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 20 client.18951 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbff5c080 0x559bbff5e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 4 client.18951 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.372+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bbfeb4860 con 0x559bbff5c080 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.373+0000 7fe99259c640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbff5c080 0x559bbff5e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.373+0000 7fe976ff5640 10 client.18951 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.373+0000 7fe99259c640 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbff5c080 0x559bbff5e470 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ef5bc18324949fe server_cookie=7556a5a6c1d217e1 in_seq=0 out_seq=0 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe976ff5640 1 -- 172.21.15.16:0/2294678507 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bbfeb4860 con 0x559bbff5c080 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe976ff5640 10 client.18951 handle_command_reply: tid=0 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 client.18951 shutdown 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 msgr2=0x559bbff490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe984009900 tx=0x7fe9840317e0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 shutdown_connections 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff4c370 0x559bbff49840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbff5c080 0x559bbff5e470 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff490b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2294678507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff415d0 0x559bbff4be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 >> 172.21.15.16:0/2294678507 conn(0x559bbff0a310 msgr2=0x559bbff43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe976ff5640 0 client.18951 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 shutdown_connections 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 1 -- 172.21.15.16:0/2294678507 wait complete. 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.374+0000 7fe99b62e740 20 client.18951 trim_cache size 0 max 16384 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 Processor -- start 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 -- start start 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff42b40 0x559bbff63d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ad940 con 0x559bbff4c370 2024-09-12T01:07:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941af480 con 0x559bbff42b40 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99b62e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ae010 con 0x559bbff415d0 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.387+0000 7fe99259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46146/0 (socket says 172.21.15.16:46146) 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 -- 172.21.15.16:0/2371986250 learned_addr learned my addr 172.21.15.16:0/2371986250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe992d9d640 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff42b40 0x559bbff63d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 -- 172.21.15.16:0/2371986250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 msgr2=0x559bbff3de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff3de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 -- 172.21.15.16:0/2371986250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff42b40 msgr2=0x559bbff63d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff42b40 0x559bbff63d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 -- 172.21.15.16:0/2371986250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbff4a510 con 0x559bbff4c370 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe99259c640 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b6c0 tx=0x7fe988023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9bbdd9cb2d3921 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe978ff9640 1 -- 172.21.15.16:0/2371986250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988012050 con 0x559bbff4c370 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe978ff9640 1 -- 172.21.15.16:0/2371986250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988016030 con 0x559bbff4c370 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.388+0000 7fe978ff9640 1 -- 172.21.15.16:0/2371986250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98802d5b0 con 0x559bbff4c370 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- 172.21.15.16:0/2371986250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 msgr2=0x559bbff642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b6c0 tx=0x7fe988023040 comp rx=0 tx=0).stop 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- 172.21.15.16:0/2371986250 shutdown_connections 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff4c370 0x559bbff642c0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff42b40 0x559bbff63d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 --2- 172.21.15.16:0/2371986250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff3de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- 172.21.15.16:0/2371986250 >> 172.21.15.16:0/2371986250 conn(0x559bbff0a310 msgr2=0x559bbff3d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- 172.21.15.16:0/2371986250 shutdown_connections 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- 172.21.15.16:0/2371986250 wait complete. 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 -- start start 2024-09-12T01:07:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.389+0000 7fe99b62e740 1 Processor -- start 2024-09-12T01:07:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff54480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff4c370 0x559bbff456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941aedb0 con 0x559bbff42b40 2024-09-12T01:07:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ae6e0 con 0x559bbff4c370 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941ab940 con 0x559bbff415d0 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 10 client.0 _ll_register_callbacks cb 0x7fe984009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46154/0 (socket says 172.21.15.16:46154) 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 -- 172.21.15.16:0/728767480 learned_addr learned my addr 172.21.15.16:0/728767480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99259c640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff54480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99259c640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff4c370 0x559bbff456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 -- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 msgr2=0x559bbff54480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff54480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 -- 172.21.15.16:0/728767480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff4c370 msgr2=0x559bbff456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff4c370 0x559bbff456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 -- 172.21.15.16:0/728767480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbff45c20 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe992d9d640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe984009850 tx=0x7fe984031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d838a6cd7e293b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe976ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984051030 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fe984031850 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.390+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988016030 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe984035040 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 10 client.18957 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bbffc6530 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988016030 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe98404b040 con 0x559bbff42b40 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 10 client.18957 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 10 client.18957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:07:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 20 client.18957 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbffc6cb0 0x559bbffc90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 4 client.18957 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bbfeaed70 con 0x559bbffc6cb0 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.391+0000 7fe99259c640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbffc6cb0 0x559bbffc90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99259c640 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbffc6cb0 0x559bbffc90d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=363df27ee0ffd0a6 server_cookie=406165f7f35668ad in_seq=0 out_seq=0 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe976ff5640 10 client.18957 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe976ff5640 1 -- 172.21.15.16:0/728767480 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bbfeaed70 con 0x559bbffc6cb0 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe976ff5640 10 client.18957 handle_command_reply: tid=0 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 client.18957 shutdown 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 msgr2=0x559bbff549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe984009850 tx=0x7fe984031490 comp rx=0 tx=0).stop 2024-09-12T01:07:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 shutdown_connections 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559bbff4c370 0x559bbff456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559bbffc6cb0 0x559bbffc90d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559bbff42b40 0x559bbff549c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 --2- 172.21.15.16:0/728767480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbff415d0 0x559bbff54480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe976ff5640 0 client.18957 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 >> 172.21.15.16:0/728767480 conn(0x559bbff0a310 msgr2=0x559bbff479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 shutdown_connections 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 1 -- 172.21.15.16:0/728767480 wait complete. 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.392+0000 7fe99b62e740 20 client.18957 trim_cache size 0 max 16384 2024-09-12T01:07:05.406 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 msgr2=0x7fe97c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 0x7fe97c0a51b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0cce30 tx=0x7fe98c01eb40 comp rx=0 tx=0).stop 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 msgr2=0x7fe99419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00eaa0 tx=0x7fe98c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 shutdown_connections 2024-09-12T01:07:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe97c0a2cf0 0x7fe97c0a51b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe994106f10 0x7fe99419cd00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe994102790 0x7fe994198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 --2- 172.21.15.16:0/327913738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe994101de0 0x7fe994198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 >> 172.21.15.16:0/327913738 conn(0x7fe9940fd8c0 msgr2=0x7fe9940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 shutdown_connections 2024-09-12T01:07:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:05.393+0000 7fe999255640 1 -- 172.21.15.16:0/327913738 wait complete. 2024-09-12T01:07:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:05 smithi099 ceph-mon[35654]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:313/600: open fd = 3 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:314/600: open fd = 3 2024-09-12T01:07:06.070 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:315/600: open fd = 3 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:316/600: open fd = 3 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.071 INFO:tasks.workunit.client.0.smithi016.stdout:317/600: open fd = 3 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:318/600: open fd = 3 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:319/600: open fd = 3 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.072 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:320/600: open fd = 3 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:321/600: open fd = 3 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:322/600: open fd = 3 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.073 INFO:tasks.workunit.client.0.smithi016.stdout:323/600: open fd = 3 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:324/600: open fd = 3 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:325/600: open fd = 3 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:326/600: open fd = 3 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.074 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:327/600: open fd = 3 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:328/600: open fd = 3 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:329/600: open fd = 3 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:330/600: open fd = 3 2024-09-12T01:07:06.075 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:331/600: open fd = 3 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:332/600: open fd = 3 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:333/600: open fd = 3 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.076 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:334/600: open fd = 3 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:335/600: open fd = 3 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:336/600: open fd = 3 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:337/600: open fd = 3 2024-09-12T01:07:06.077 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:338/600: open fd = 3 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:339/600: open fd = 3 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:340/600: open fd = 3 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.078 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:341/600: open fd = 3 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:342/600: open fd = 3 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:343/600: open fd = 3 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:344/600: open fd = 3 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.079 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:345/600: open fd = 3 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:346/600: open fd = 3 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:347/600: open fd = 3 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.080 INFO:tasks.workunit.client.0.smithi016.stdout:348/600: open fd = 3 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:349/600: open fd = 3 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:350/600: open fd = 3 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:351/600: open fd = 3 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.081 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:352/600: open fd = 3 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:353/600: open fd = 3 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:354/600: open fd = 3 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:355/600: open fd = 3 2024-09-12T01:07:06.082 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:356/600: open fd = 3 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:357/600: open fd = 3 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:358/600: open fd = 3 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.083 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:359/600: open fd = 3 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:360/600: open fd = 3 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:361/600: open fd = 3 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:362/600: open fd = 3 2024-09-12T01:07:06.084 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:363/600: open fd = 3 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:364/600: open fd = 3 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:365/600: open fd = 3 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.085 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:366/600: open fd = 3 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:367/600: open fd = 3 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:368/600: open fd = 3 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.086 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:369/600: open fd = 3 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:370/600: open fd = 3 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:371/600: open fd = 3 2024-09-12T01:07:06.087 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:372/600: open fd = 3 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:373/600: open fd = 3 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:374/600: open fd = 3 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:07:06.088 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:07:06.089 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:05 smithi016 ceph-mon[30972]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:07:06.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:07:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:05 smithi159 ceph-mon[35566]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T01:07:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:06.761 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:06 smithi016 ceph-mon[30972]: pgmap v341: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2235868424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8102d40 msgr2=0x7fb0e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2235868424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8102d40 0x7fb0e8103120 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d802a480 comp rx=0 tx=0).stop 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2235868424 shutdown_connections 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2235868424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 0x7fb0e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2235868424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8103660 0x7fb0e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2235868424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8102d40 0x7fb0e8103120 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2235868424 >> 172.21.15.16:0/2235868424 conn(0x7fb0e80fc7a0 msgr2=0x7fb0e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2235868424 shutdown_connections 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.848+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2235868424 wait complete. 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 Processor -- start 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 -- start start 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8102d40 0x7fb0e81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 0x7fb0e819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8111fb0 con 0x7fb0e8103660 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8111e30 con 0x7fb0e810c170 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0eccd2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8112130 con 0x7fb0e8102d40 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46176/0 (socket says 172.21.15.16:46176) 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2890873640 learned_addr learned my addr 172.21.15.16:0/2890873640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8102d40 0x7fb0e81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.849+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 0x7fb0e819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2890873640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8102d40 msgr2=0x7fb0e81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8102d40 0x7fb0e81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2890873640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 msgr2=0x7fb0e819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 0x7fb0e819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2890873640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819d460 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b3f0 tx=0x7fb0d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0dffff640 1 -- 172.21.15.16:0/2890873640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0015020 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819d750 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0dffff640 1 -- 172.21.15.16:0/2890873640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0011d10 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0dffff640 1 -- 172.21.15.16:0/2890873640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000f2e0 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.850+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a5030 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.852+0000 7fb0dffff640 1 -- 172.21.15.16:0/2890873640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb0d0018020 con 0x7fb0e8103660 2024-09-12T01:07:06.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 Processor -- start 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 -- start start 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c38ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565c38552f0 con 0x5565c38fe780 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565c37fccf0 con 0x5565c38ef5d0 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565c38512b0 con 0x5565c38f0b40 2024-09-12T01:07:06.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0dffff640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 0x7fb0d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46178/0 (socket says 172.21.15.16:46178) 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2279458045 learned_addr learned my addr 172.21.15.16:0/2279458045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c38ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.853+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0dffff640 1 -- 172.21.15.16:0/2890873640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fb0d00ceb20 con 0x7fb0e8103660 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 0x7fb0d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2279458045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 msgr2=0x5565c38ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c38ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2279458045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 msgr2=0x5565c38efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2279458045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c37e13a0 con 0x5565c38fe780 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b3c0 tx=0x7fb0d00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef21ba0061f1cd12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0dcd99640 1 -- 172.21.15.16:0/2279458045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d009b820 con 0x5565c38fe780 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0dcd99640 1 -- 172.21.15.16:0/2279458045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d00d3040 con 0x5565c38fe780 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 0x7fb0d40a51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005c40 tx=0x7fb0e0009500 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.854+0000 7fb0dcd99640 1 -- 172.21.15.16:0/2279458045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00cc4c0 con 0x5565c38fe780 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2279458045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 msgr2=0x5565c3900b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b3c0 tx=0x7fb0d00047d0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2279458045 shutdown_connections 2024-09-12T01:07:06.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38fe780 0x5565c3900b70 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2279458045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c38ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2279458045 >> 172.21.15.16:0/2279458045 conn(0x5565c38b8310 msgr2=0x5565c38ebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2279458045 shutdown_connections 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2279458045 wait complete. 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 -- start start 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.855+0000 7fb0ef0ab740 1 Processor -- start 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c38fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565c39013c0 con 0x5565c38f0b40 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5565c3810060 con 0x5565c38ef5d0 2024-09-12T01:07:06.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565c3901540 con 0x5565c38fe780 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 10 client.0 _ll_register_callbacks cb 0x5565c35638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46188/0 (socket says 172.21.15.16:46188) 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2599801707 learned_addr learned my addr 172.21.15.16:0/2599801707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c38fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2599801707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 msgr2=0x5565c38fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c38fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 msgr2=0x5565c38fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38fe230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 -- 172.21.15.16:0/2599801707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c35c63c0 con 0x5565c38f0b40 2024-09-12T01:07:06.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000bdb0 tx=0x7fb0d00cc6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b555f27f6631fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00973c0 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fb0d0097790 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.856+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d00d3040 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0d009b820 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 10 client.18969 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565c39013c0 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d00d3040 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d009a570 con 0x5565c38f0b40 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 10 client.18969 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 10 client.18969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:07:06.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 20 client.18969 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c390e0b0 0x5565c39104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 4 client.18969 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565c3862860 con 0x5565c390e0b0 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.857+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c390e0b0 0x5565c39104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.858+0000 7fb0e659c640 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c390e0b0 0x5565c39104a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39976fc3a3f85908 server_cookie=4912fb5bd61dcc80 in_seq=0 out_seq=0 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.858+0000 7fb0cdffb640 10 client.18969 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.858+0000 7fb0cdffb640 1 -- 172.21.15.16:0/2599801707 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565c3862860 con 0x5565c390e0b0 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.858+0000 7fb0cdffb640 10 client.18969 handle_command_reply: tid=0 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 client.18969 shutdown 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 msgr2=0x5565c38fb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000bdb0 tx=0x7fb0d00cc6f0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 shutdown_connections 2024-09-12T01:07:06.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c390e0b0 0x5565c39104a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c38fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38f0b40 0x5565c38fb4e0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0cdffb640 0 client.18969 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/2599801707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38ef5d0 0x5565c38fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 >> 172.21.15.16:0/2599801707 conn(0x5565c38b8310 msgr2=0x5565c38f1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 shutdown_connections 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/2599801707 wait complete. 2024-09-12T01:07:06.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.859+0000 7fb0ef0ab740 20 client.18969 trim_cache size 0 max 16384 2024-09-12T01:07:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 Processor -- start 2024-09-12T01:07:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 -- start start 2024-09-12T01:07:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c3915b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 0x5565c3916040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ab800 con 0x5565c38ef5d0 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ad340 con 0x5565c38fe780 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81abed0 con 0x5565c38f0b40 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.872+0000 7fb0e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46200/0 (socket says 172.21.15.16:46200) 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 -- 172.21.15.16:0/845396679 learned_addr learned my addr 172.21.15.16:0/845396679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c3915b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 0x5565c3916040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 -- 172.21.15.16:0/845396679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 msgr2=0x5565c3915b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c3915b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 -- 172.21.15.16:0/845396679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 msgr2=0x5565c3916040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 0x5565c3916040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 -- 172.21.15.16:0/845396679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565c390a660 con 0x5565c38ef5d0 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 0x5565c3916040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0e659c640 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802a9a0 tx=0x7fb0d80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab749e123cb33bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0cffff640 1 -- 172.21.15.16:0/845396679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8049070 con 0x5565c38ef5d0 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0cffff640 1 -- 172.21.15.16:0/845396679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d8005500 con 0x5565c38ef5d0 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.873+0000 7fb0cffff640 1 -- 172.21.15.16:0/845396679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8035040 con 0x5565c38ef5d0 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/845396679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 msgr2=0x5565c39104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802a9a0 tx=0x7fb0d80058d0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/845396679 shutdown_connections 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38fe780 0x5565c3916040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38f0b40 0x5565c3915b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/845396679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c39104a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/845396679 >> 172.21.15.16:0/845396679 conn(0x5565c38b8310 msgr2=0x5565c38e6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/845396679 shutdown_connections 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/845396679 wait complete. 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 -- start start 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.874+0000 7fb0ef0ab740 1 Processor -- start 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38f0b40 0x5565c38f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c39057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81acc70 con 0x5565c38ef5d0 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81ac5a0 con 0x5565c38f0b40 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81a9800 con 0x5565c38fe780 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 10 client.0 _ll_register_callbacks cb 0x7fb0d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46204/0 (socket says 172.21.15.16:46204) 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 -- 172.21.15.16:0/1266909490 learned_addr learned my addr 172.21.15.16:0/1266909490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38f0b40 0x5565c38f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c39057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 -- 172.21.15.16:0/1266909490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 msgr2=0x5565c39057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c39057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 -- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38f0b40 msgr2=0x5565c38f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38f0b40 0x5565c38f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 -- 172.21.15.16:0/1266909490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565c38f3720 con 0x5565c38ef5d0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e6d9d640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c39057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0e659c640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d043e2def175c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8049050 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fb0d8004240 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.875+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d8005500 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0d8035040 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 10 client.18975 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565c3906da0 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d8005500 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0d805e5a0 con 0x5565c38ef5d0 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 10 client.18975 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 10 client.18975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 20 client.18975 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c397d2c0 0x5565c397f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 4 client.18975 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565c385cd70 con 0x5565c397d2c0 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.876+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c397d2c0 0x5565c397f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ddd9b640 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c397d2c0 0x5565c397f6b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58caec508196e272 server_cookie=69077c95b0ec37b1 in_seq=0 out_seq=0 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0cdffb640 10 client.18975 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0cdffb640 1 -- 172.21.15.16:0/1266909490 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565c385cd70 con 0x5565c397d2c0 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0cdffb640 10 client.18975 handle_command_reply: tid=0 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 client.18975 shutdown 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 msgr2=0x5565c38f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d8031b60 comp rx=0 tx=0).stop 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 shutdown_connections 2024-09-12T01:07:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5565c397d2c0 0x5565c397f6b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0cdffb640 0 client.18975 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565c38fe780 0x5565c39057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5565c38f0b40 0x5565c38f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 --2- 172.21.15.16:0/1266909490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565c38ef5d0 0x5565c38f2950 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 >> 172.21.15.16:0/1266909490 conn(0x5565c38b8310 msgr2=0x5565c38fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 shutdown_connections 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 1 -- 172.21.15.16:0/1266909490 wait complete. 2024-09-12T01:07:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.877+0000 7fb0ef0ab740 20 client.18975 trim_cache size 0 max 16384 2024-09-12T01:07:06.886 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 msgr2=0x7fb0d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 0x7fb0d40a51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005c40 tx=0x7fb0e0009500 comp rx=0 tx=0).stop 2024-09-12T01:07:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 msgr2=0x7fb0e81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000b3f0 tx=0x7fb0d000b8b0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 shutdown_connections 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0d40a2cf0 0x7fb0d40a51b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0e810c170 0x7fb0e819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e8103660 0x7fb0e81989f0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 --2- 172.21.15.16:0/2890873640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0e8102d40 0x7fb0e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 >> 172.21.15.16:0/2890873640 conn(0x7fb0e80fc7a0 msgr2=0x7fb0e80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 shutdown_connections 2024-09-12T01:07:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:06.878+0000 7fb0eccd2640 1 -- 172.21.15.16:0/2890873640 wait complete. 2024-09-12T01:07:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:06 smithi099 ceph-mon[35654]: pgmap v341: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:06 smithi159 ceph-mon[35566]: pgmap v341: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:07.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:07:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 -- 172.21.15.16:0/1917123531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c1410c180 msgr2=0x7f7c1410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 --2- 172.21.15.16:0/1917123531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c1410c180 0x7f7c1410e600 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c02a950 comp rx=0 tx=0).stop 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 -- 172.21.15.16:0/1917123531 shutdown_connections 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 --2- 172.21.15.16:0/1917123531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c1410c180 0x7f7c1410e600 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 --2- 172.21.15.16:0/1917123531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c141036c0 0x7f7c1410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 --2- 172.21.15.16:0/1917123531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 0x7f7c14103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 -- 172.21.15.16:0/1917123531 >> 172.21.15.16:0/1917123531 conn(0x7f7c140fc7a0 msgr2=0x7f7c140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 -- 172.21.15.16:0/1917123531 shutdown_connections 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.363+0000 7f7c1b973640 1 -- 172.21.15.16:0/1917123531 wait complete. 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 Processor -- start 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 -- start start 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 0x7f7c14100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c1410c180 0x7f7c14101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14111f60 con 0x7f7c141036c0 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14111de0 con 0x7f7c14102da0 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1b973640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141120e0 con 0x7f7c1410c180 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c18f0e640 1 -- 172.21.15.16:0/3281985406 learned_addr learned my addr 172.21.15.16:0/3281985406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c19f10640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c1410c180 0x7f7c14101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.364+0000 7f7c1970f640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 0x7f7c14100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 -- 172.21.15.16:0/3281985406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c1410c180 msgr2=0x7f7c14101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c1410c180 0x7f7c14101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 -- 172.21.15.16:0/3281985406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 msgr2=0x7f7c14100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 0x7f7c14100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 -- 172.21.15.16:0/3281985406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141a6900 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d770 tx=0x7f7c0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c0a7fc640 1 -- 172.21.15.16:0/3281985406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04017070 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c0a7fc640 1 -- 172.21.15.16:0/3281985406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04005500 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141a6bf0 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c0a7fc640 1 -- 172.21.15.16:0/3281985406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004e00 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.365+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a70e0 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.367+0000 7f7c0a7fc640 1 -- 172.21.15.16:0/3281985406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c04004fa0 con 0x7f7c141036c0 2024-09-12T01:07:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 Processor -- start 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 -- start start 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 0x55742d63f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d63fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55742d5a52f0 con 0x55742d64a370 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55742d54ccf0 con 0x55742d640b40 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55742d5a12b0 con 0x55742d63f5d0 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c0a7fc640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 0x7f7c000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46242/0 (socket says 172.21.15.16:46242) 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2282278600 learned_addr learned my addr 172.21.15.16:0/2282278600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.368+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 0x55742d63f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d63fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c19f10640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 0x7f7c000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c0a7fc640 1 -- 172.21.15.16:0/3281985406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7c040ceda0 con 0x7f7c141036c0 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2282278600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 msgr2=0x55742d63fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d63fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2282278600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 msgr2=0x55742d63f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 0x55742d63f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2282278600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742d5313a0 con 0x55742d64a370 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 0x55742d63f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009610 tx=0x7f7c040048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269f9bc9d6dbcfa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7c19f10640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 0x7f7c000a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009430 tx=0x7f7c0c005c50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7bff7fe640 1 -- 172.21.15.16:0/2282278600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04020070 con 0x55742d64a370 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7bff7fe640 1 -- 172.21.15.16:0/2282278600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0409b340 con 0x55742d64a370 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.369+0000 7f7bff7fe640 1 -- 172.21.15.16:0/2282278600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040979b0 con 0x55742d64a370 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2282278600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 msgr2=0x55742d64c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009610 tx=0x7f7c040048a0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2282278600 shutdown_connections 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d64c760 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d63fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2282278600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d640b40 0x55742d63f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2282278600 >> 172.21.15.16:0/2282278600 conn(0x55742d608310 msgr2=0x55742d63bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2282278600 shutdown_connections 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.370+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2282278600 wait complete. 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 -- start start 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 Processor -- start 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d649d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55742d64cfb0 con 0x55742d64a370 2024-09-12T01:07:08.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55742d560060 con 0x55742d63f5d0 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55742d64d130 con 0x55742d640b40 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c1dd4c740 10 client.0 _ll_register_callbacks cb 0x55742d2b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c19f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c19f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46250/0 (socket says 172.21.15.16:46250) 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.371+0000 7f7c19f10640 1 -- 172.21.15.16:0/2868557417 learned_addr learned my addr 172.21.15.16:0/2868557417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d649d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 -- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 msgr2=0x55742d646fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 -- 172.21.15.16:0/2868557417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 msgr2=0x55742d649d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d649d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 -- 172.21.15.16:0/2868557417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742d3163c0 con 0x55742d64a370 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d649d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009a60 tx=0x7f7c0c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d576c58b7b949aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7bfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c04f030 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7c0c004250 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c0409b340 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c0c034040 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c1dd4c740 10 client.18987 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55742d658a90 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.372+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0409b340 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c0c04bb30 con 0x55742d64a370 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 10 client.18987 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 10 client.18987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 20 client.18987 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d659ea0 0x55742d65c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 4 client.18987 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55742d5b2860 con 0x55742d659ea0 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.373+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d659ea0 0x55742d65c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.374+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d659ea0 0x55742d65c2d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c8612e09a6c6004 server_cookie=6b34c6eb7f0db8c9 in_seq=0 out_seq=0 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.374+0000 7f7bfcff9640 10 client.18987 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.374+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2868557417 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55742d5b2860 con 0x55742d659ea0 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.374+0000 7f7bfcff9640 10 client.18987 handle_command_reply: tid=0 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.374+0000 7f7c1dd4c740 1 client.18987 shutdown 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 msgr2=0x55742d6475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009a60 tx=0x7f7c0c0095c0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 shutdown_connections 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6475d0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d659ea0 0x55742d65c2d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2868557417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d649d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7bfcff9640 0 client.18987 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 >> 172.21.15.16:0/2868557417 conn(0x55742d608310 msgr2=0x55742d641990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 shutdown_connections 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2868557417 wait complete. 2024-09-12T01:07:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.375+0000 7f7c1dd4c740 20 client.18987 trim_cache size 0 max 16384 2024-09-12T01:07:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 Processor -- start 2024-09-12T01:07:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 -- start start 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d656280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141afa30 con 0x55742d64a370 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141b1570 con 0x55742d63f5d0 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141b0100 con 0x55742d640b40 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46256/0 (socket says 172.21.15.16:46256) 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 -- 172.21.15.16:0/2234287769 learned_addr learned my addr 172.21.15.16:0/2234287769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d656280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 -- 172.21.15.16:0/2234287769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 msgr2=0x55742d646fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 -- 172.21.15.16:0/2234287769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 msgr2=0x55742d656280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.389+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d656280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7c19f10640 1 -- 172.21.15.16:0/2234287769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742d63c960 con 0x55742d64a370 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009bf0 tx=0x7f7c0c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2931f9fefb606e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7bfeffd640 1 -- 172.21.15.16:0/2234287769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c060020 con 0x55742d64a370 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7bfeffd640 1 -- 172.21.15.16:0/2234287769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c056110 con 0x55742d64a370 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.390+0000 7f7bfeffd640 1 -- 172.21.15.16:0/2234287769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c04f030 con 0x55742d64a370 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2234287769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 msgr2=0x55742d6474e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009bf0 tx=0x7f7c0c02f3a0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2234287769 shutdown_connections 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d64a370 0x55742d6474e0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d640b40 0x55742d646fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2234287769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d63f5d0 0x55742d656280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2234287769 >> 172.21.15.16:0/2234287769 conn(0x55742d608310 msgr2=0x55742d6422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2234287769 shutdown_connections 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2234287769 wait complete. 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 -- start start 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.391+0000 7f7c1dd4c740 1 Processor -- start 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d660be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d64a370 0x55742d661660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141b0ea0 con 0x55742d640b40 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141b07d0 con 0x55742d64a370 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c141ada30 con 0x55742d63f5d0 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 10 client.0 _ll_register_callbacks cb 0x7f7c0c02ff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46262/0 (socket says 172.21.15.16:46262) 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2802146203 learned_addr learned my addr 172.21.15.16:0/2802146203 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c19f10640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d64a370 0x55742d661660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d660be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 msgr2=0x55742d660be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d660be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2802146203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d64a370 msgr2=0x55742d661660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d64a370 0x55742d661660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 -- 172.21.15.16:0/2802146203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742d6513f0 con 0x55742d640b40 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d660be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c18f0e640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7c040cc570 tx=0x7f7c04097660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9022c2a37651eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7bfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04020070 con 0x55742d640b40 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7f7c040cc740 con 0x55742d640b40 2024-09-12T01:07:08.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.392+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c0c056110 con 0x55742d640b40 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c040a3f00 con 0x55742d640b40 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 10 client.18993 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55742d6526d0 con 0x55742d640b40 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0c056110 con 0x55742d640b40 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c040bb170 con 0x55742d640b40 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 10 client.18993 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 10 client.18993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 20 client.18993 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d652a60 0x55742d6c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 4 client.18993 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:08.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55742d5acd70 con 0x55742d652a60 2024-09-12T01:07:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.393+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d652a60 0x55742d6c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:08.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7bfcff9640 10 client.18993 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1970f640 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d652a60 0x55742d6c6bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d10e9ad26775df47 server_cookie=9a87f3253cbae365 in_seq=0 out_seq=0 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7bfcff9640 1 -- 172.21.15.16:0/2802146203 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55742d5acd70 con 0x55742d652a60 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7bfcff9640 10 client.18993 handle_command_reply: tid=0 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 client.18993 shutdown 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 msgr2=0x55742d661120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7c040cc570 tx=0x7f7c04097660 comp rx=0 tx=0).stop 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 shutdown_connections 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55742d652a60 0x55742d6c6bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55742d64a370 0x55742d661660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55742d640b40 0x55742d661120 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 --2- 172.21.15.16:0/2802146203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742d63f5d0 0x55742d660be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7bfcff9640 0 client.18993 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:08.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 >> 172.21.15.16:0/2802146203 conn(0x55742d608310 msgr2=0x55742d6422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 shutdown_connections 2024-09-12T01:07:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.394+0000 7f7c1dd4c740 1 -- 172.21.15.16:0/2802146203 wait complete. 2024-09-12T01:07:08.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1dd4c740 20 client.18993 trim_cache size 0 max 16384 2024-09-12T01:07:08.399 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 msgr2=0x7f7c000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 0x7f7c000a51b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009430 tx=0x7f7c0c005c50 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 msgr2=0x7f7c14100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400d770 tx=0x7f7c0400dc30 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 shutdown_connections 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7c000a2cf0 0x7f7c000a51b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c1410c180 0x7f7c14101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c141036c0 0x7f7c14100f40 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 --2- 172.21.15.16:0/3281985406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c14102da0 0x7f7c14100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 >> 172.21.15.16:0/3281985406 conn(0x7f7c140fc7a0 msgr2=0x7f7c140fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 shutdown_connections 2024-09-12T01:07:08.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:08.395+0000 7f7c1b973640 1 -- 172.21.15.16:0/3281985406 wait complete. 2024-09-12T01:07:09.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:08 smithi016 ceph-mon[30972]: pgmap v342: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:08 smithi099 ceph-mon[35654]: pgmap v342: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:08 smithi159 ceph-mon[35566]: pgmap v342: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 110 KiB/s wr, 35 op/s 2024-09-12T01:07:09.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:07:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.880+0000 7fdc1c699640 1 -- 172.21.15.16:0/3191552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 msgr2=0x7fdc1410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.880+0000 7fdc1c699640 1 --2- 172.21.15.16:0/3191552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1410e5f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009ca0 tx=0x7fdc0c02a950 comp rx=0 tx=0).stop 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 -- 172.21.15.16:0/3191552344 shutdown_connections 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 --2- 172.21.15.16:0/3191552344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1410e5f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 --2- 172.21.15.16:0/3191552344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 0x7fdc1410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 --2- 172.21.15.16:0/3191552344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 0x7fdc14103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 -- 172.21.15.16:0/3191552344 >> 172.21.15.16:0/3191552344 conn(0x7fdc140fc780 msgr2=0x7fdc140feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 -- 172.21.15.16:0/3191552344 shutdown_connections 2024-09-12T01:07:09.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 -- 172.21.15.16:0/3191552344 wait complete. 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 Processor -- start 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.881+0000 7fdc1c699640 1 -- start start 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 0x7fdc14198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 0x7fdc141989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14111fb0 con 0x7fdc1410c180 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14111e30 con 0x7fdc141036c0 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1c699640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14112130 con 0x7fdc14102da0 2024-09-12T01:07:09.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39790/0 (socket says 172.21.15.16:39790) 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 -- 172.21.15.16:0/2460137828 learned_addr learned my addr 172.21.15.16:0/2460137828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1a435640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 0x7fdc14198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc19c34640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 0x7fdc141989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 -- 172.21.15.16:0/2460137828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 msgr2=0x7fdc14198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 0x7fdc14198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 -- 172.21.15.16:0/2460137828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 msgr2=0x7fdc141989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 0x7fdc141989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 -- 172.21.15.16:0/2460137828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1419d430 con 0x7fdc1410c180 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc1ac36640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c02ae70 tx=0x7fdc0c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.882+0000 7fdc0b7fe640 1 -- 172.21.15.16:0/2460137828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c034040 con 0x7fdc1410c180 2024-09-12T01:07:09.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.883+0000 7fdc0b7fe640 1 -- 172.21.15.16:0/2460137828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0c030920 con 0x7fdc1410c180 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.883+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1419d6c0 con 0x7fdc1410c180 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.883+0000 7fdc0b7fe640 1 -- 172.21.15.16:0/2460137828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c045480 con 0x7fdc1410c180 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.883+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc141a5000 con 0x7fdc1410c180 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.885+0000 7fdc0b7fe640 1 -- 172.21.15.16:0/2460137828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc0c037020 con 0x7fdc1410c180 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.885+0000 7fdc1ea72740 1 Processor -- start 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.885+0000 7fdc1ea72740 1 -- start start 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bc3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bc3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559457b292f0 con 0x559457bce370 2024-09-12T01:07:09.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559457ad0cf0 con 0x559457bc35d0 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559457b252b0 con 0x559457bc4b40 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39796/0 (socket says 172.21.15.16:39796) 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 -- 172.21.15.16:0/4039815318 learned_addr learned my addr 172.21.15.16:0/4039815318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc0b7fe640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 0x7fdc000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bc3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1a435640 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bc3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 -- 172.21.15.16:0/4039815318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 msgr2=0x559457bc3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bc3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 -- 172.21.15.16:0/4039815318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 msgr2=0x559457bc3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bc3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 -- 172.21.15.16:0/4039815318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559457ab53a0 con 0x559457bce370 2024-09-12T01:07:09.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc1ac36640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 0x7fdc000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc0b7fe640 1 -- 172.21.15.16:0/2460137828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fdc0c0f0290 con 0x7fdc1410c180 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.886+0000 7fdc19c34640 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400ad80 tx=0x7fdc0400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e05b356d86c1f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc08ff9640 1 -- 172.21.15.16:0/4039815318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0401b070 con 0x559457bce370 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc08ff9640 1 -- 172.21.15.16:0/4039815318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0400fdf0 con 0x559457bce370 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc08ff9640 1 -- 172.21.15.16:0/4039815318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc040146e0 con 0x559457bce370 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc1ea72740 1 -- 172.21.15.16:0/4039815318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 msgr2=0x559457bd0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc1ac36640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 0x7fdc000a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0edfb0 tx=0x7fdc0c0b1d20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400ad80 tx=0x7fdc0400a490 comp rx=0 tx=0).stop 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.887+0000 7fdc1ea72740 1 -- 172.21.15.16:0/4039815318 shutdown_connections 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bce370 0x559457bd0760 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bc3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/4039815318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bc3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- 172.21.15.16:0/4039815318 >> 172.21.15.16:0/4039815318 conn(0x559457b8c310 msgr2=0x559457bbfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- 172.21.15.16:0/4039815318 shutdown_connections 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- 172.21.15.16:0/4039815318 wait complete. 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- start start 2024-09-12T01:07:09.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 Processor -- start 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 0x559457bcddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bcb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559457bd1070 con 0x559457bc4b40 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559457ae4060 con 0x559457bce370 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559457bd11f0 con 0x559457bc35d0 2024-09-12T01:07:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1ea72740 10 client.0 _ll_register_callbacks cb 0x5594578378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1a435640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1a435640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39804/0 (socket says 172.21.15.16:39804) 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.888+0000 7fdc1a435640 1 -- 172.21.15.16:0/1452951703 learned_addr learned my addr 172.21.15.16:0/1452951703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc19c34640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 0x559457bcddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc19c34640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bcb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 -- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 msgr2=0x559457bcddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 0x559457bcddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 -- 172.21.15.16:0/1452951703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 msgr2=0x559457bcb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bcb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 -- 172.21.15.16:0/1452951703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55945789a3c0 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc19c34640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 0x559457bcddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1a435640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000b960 tx=0x7fdc1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee719e1890c2034d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10003ea0 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fdc10005220 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0400fdf0 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc10005460 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1ea72740 10 client.19005 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.889+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559457bdcbe0 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0400fdf0 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdc10021070 con 0x559457bc4b40 2024-09-12T01:07:09.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 10 client.19005 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 20 client.19005 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457bde0f0 0x559457be0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 4 client.19005 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559457b36860 con 0x559457bde0f0 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.890+0000 7fdc19c34640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457bde0f0 0x559457be0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc19c34640 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457bde0f0 0x559457be0510 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a196bfced9e0755 server_cookie=790be097115df50e in_seq=0 out_seq=0 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdbfdffb640 10 client.19005 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdbfdffb640 1 -- 172.21.15.16:0/1452951703 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559457b36860 con 0x559457bde0f0 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdbfdffb640 10 client.19005 handle_command_reply: tid=0 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc1ea72740 1 client.19005 shutdown 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 msgr2=0x559457bcb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000b960 tx=0x7fdc1000be20 comp rx=0 tx=0).stop 2024-09-12T01:07:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 shutdown_connections 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.891+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457bde0f0 0x559457be0510 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bcb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457bcb070 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/1452951703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc35d0 0x559457bcddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 >> 172.21.15.16:0/1452951703 conn(0x559457b8c310 msgr2=0x559457bc5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdbfdffb640 0 client.19005 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 shutdown_connections 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 1 -- 172.21.15.16:0/1452951703 wait complete. 2024-09-12T01:07:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.892+0000 7fdc1ea72740 20 client.19005 trim_cache size 0 max 16384 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 Processor -- start 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 -- start start 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bcb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bce370 0x559457be6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141ad980 con 0x559457bc4b40 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141af4c0 con 0x559457bc35d0 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141ae050 con 0x559457bce370 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39814/0 (socket says 172.21.15.16:39814) 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 -- 172.21.15.16:0/885453172 learned_addr learned my addr 172.21.15.16:0/885453172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc19c34640 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bcb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc19c34640 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bce370 0x559457be6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 -- 172.21.15.16:0/885453172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bce370 msgr2=0x559457be6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bce370 0x559457be6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 -- 172.21.15.16:0/885453172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 msgr2=0x559457bcb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bcb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.906+0000 7fdc1a435640 1 -- 172.21.15.16:0/885453172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559457bcbb70 con 0x559457bc4b40 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.907+0000 7fdc1a435640 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdc10011bb0 tx=0x7fdc100058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a52b2e6ad74aae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.907+0000 7fdbfffff640 1 -- 172.21.15.16:0/885453172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10027040 con 0x559457bc4b40 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.907+0000 7fdbfffff640 1 -- 172.21.15.16:0/885453172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10003ea0 con 0x559457bc4b40 2024-09-12T01:07:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.907+0000 7fdbfffff640 1 -- 172.21.15.16:0/885453172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10023c70 con 0x559457bc4b40 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- 172.21.15.16:0/885453172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 msgr2=0x559457be5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdc10011bb0 tx=0x7fdc100058f0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- 172.21.15.16:0/885453172 shutdown_connections 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bce370 0x559457be6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc4b40 0x559457be5dc0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/885453172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bc35d0 0x559457bcb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- 172.21.15.16:0/885453172 >> 172.21.15.16:0/885453172 conn(0x559457b8c310 msgr2=0x559457bca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- 172.21.15.16:0/885453172 shutdown_connections 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- 172.21.15.16:0/885453172 wait complete. 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- start start 2024-09-12T01:07:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 Processor -- start 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bd6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bc78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141aedf0 con 0x559457bc35d0 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141ae720 con 0x559457bce370 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc141ab980 con 0x559457bc4b40 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc1ea72740 10 client.0 _ll_register_callbacks cb 0x7fdc100041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.908+0000 7fdc19c34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39822/0 (socket says 172.21.15.16:39822) 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 -- 172.21.15.16:0/2160402123 learned_addr learned my addr 172.21.15.16:0/2160402123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc1a435640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bd6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bc78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 -- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 msgr2=0x559457bd6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bd6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 -- 172.21.15.16:0/2160402123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 msgr2=0x559457bc78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bc78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 -- 172.21.15.16:0/2160402123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559457bc8080 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc1a435640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bd6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc19c34640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fdc04008010 tx=0x7fdc0400aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ba1f8d43f5a424 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdbfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0401b040 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8565+0+0 (secure 0 0 0) 0x7fdc040166c0 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc10003ea0 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc0403f050 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc1ea72740 10 client.19011 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.909+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559457c487f0 con 0x559457bc35d0 2024-09-12T01:07:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10003ea0 con 0x559457bc35d0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdc04015040 con 0x559457bc35d0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 10 client.19011 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 20 client.19011 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457c48be0 0x559457c4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 4 client.19011 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559457b30d70 con 0x559457c48be0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.910+0000 7fdc1a435640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457c48be0 0x559457c4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1a435640 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457c48be0 0x559457c4afd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277796e49ac93755 server_cookie=4f299eff2e65ca82 in_seq=0 out_seq=0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdbfdffb640 10 client.19011 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdbfdffb640 1 -- 172.21.15.16:0/2160402123 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559457b30d70 con 0x559457c48be0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdbfdffb640 10 client.19011 handle_command_reply: tid=0 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 client.19011 shutdown 2024-09-12T01:07:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 msgr2=0x559457bd6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fdc04008010 tx=0x7fdc0400aba0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 shutdown_connections 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559457c48be0 0x559457c4afd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559457bce370 0x559457bc78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559457bc4b40 0x559457bd6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdbfdffb640 0 client.19011 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 --2- 172.21.15.16:0/2160402123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559457bc35d0 0x559457bd6660 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 >> 172.21.15.16:0/2160402123 conn(0x559457b8c310 msgr2=0x559457bcd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 shutdown_connections 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 1 -- 172.21.15.16:0/2160402123 wait complete. 2024-09-12T01:07:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.911+0000 7fdc1ea72740 20 client.19011 trim_cache size 0 max 16384 2024-09-12T01:07:09.922 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 msgr2=0x7fdc000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 0x7fdc000a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0edfb0 tx=0x7fdc0c0b1d20 comp rx=0 tx=0).stop 2024-09-12T01:07:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 msgr2=0x7fdc1419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c02ae70 tx=0x7fdc0c02f5b0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 shutdown_connections 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fdc000a2cf0 0x7fdc000a51b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc1410c180 0x7fdc1419cd50 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc141036c0 0x7fdc141989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 --2- 172.21.15.16:0/2460137828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc14102da0 0x7fdc14198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 >> 172.21.15.16:0/2460137828 conn(0x7fdc140fc780 msgr2=0x7fdc140fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 shutdown_connections 2024-09-12T01:07:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:09.912+0000 7fdc1c699640 1 -- 172.21.15.16:0/2460137828 wait complete. 2024-09-12T01:07:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:10 smithi099 ceph-mon[35654]: pgmap v343: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:10 smithi159 ceph-mon[35566]: pgmap v343: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:11.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:10 smithi016 ceph-mon[30972]: pgmap v343: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:11.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:07:11.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:07:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 -- 172.21.15.16:0/2366411335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1730102d40 msgr2=0x7f1730103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 --2- 172.21.15.16:0/2366411335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1730102d40 0x7f1730103120 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f172c009900 tx=0x7f172c02a480 comp rx=0 tx=0).stop 2024-09-12T01:07:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 -- 172.21.15.16:0/2366411335 shutdown_connections 2024-09-12T01:07:12.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 --2- 172.21.15.16:0/2366411335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f173010c170 0x7f173010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 --2- 172.21.15.16:0/2366411335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730103660 0x7f173010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 --2- 172.21.15.16:0/2366411335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1730102d40 0x7f1730103120 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 -- 172.21.15.16:0/2366411335 >> 172.21.15.16:0/2366411335 conn(0x7f17300fc7a0 msgr2=0x7f17300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 -- 172.21.15.16:0/2366411335 shutdown_connections 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.543+0000 7f17381cb640 1 -- 172.21.15.16:0/2366411335 wait complete. 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 Processor -- start 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 -- start start 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730102d40 0x7f1730198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1730103660 0x7f17301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1730111f70 con 0x7f173010c170 2024-09-12T01:07:12.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1730111df0 con 0x7f1730102d40 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f17381cb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17301120f0 con 0x7f1730103660 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f1736768640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f1736768640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39842/0 (socket says 172.21.15.16:39842) 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f1736768640 1 -- 172.21.15.16:0/1540697256 learned_addr learned my addr 172.21.15.16:0/1540697256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f1735766640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1730103660 0x7f17301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.544+0000 7f1735f67640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730102d40 0x7f1730198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 -- 172.21.15.16:0/1540697256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1730103660 msgr2=0x7f17301989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1730103660 0x7f17301989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 -- 172.21.15.16:0/1540697256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730102d40 msgr2=0x7f1730198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730102d40 0x7f1730198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 -- 172.21.15.16:0/1540697256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173019d440 con 0x7f173010c170 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1736768640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f172800eaa0 tx=0x7f172800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1726ffd640 1 -- 172.21.15.16:0/1540697256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172800cdb0 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1726ffd640 1 -- 172.21.15.16:0/1540697256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1728005020 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173019d730 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f1726ffd640 1 -- 172.21.15.16:0/1540697256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280116b0 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.545+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a5010 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.547+0000 7f1726ffd640 1 -- 172.21.15.16:0/1540697256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1728005560 con 0x7f173010c170 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 Processor -- start 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 -- start start 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d4234f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4242130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587d419a280 con 0x5587d4235480 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587d4141b10 con 0x5587d423fd40 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f173a5a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5587d4196240 con 0x5587d4234b60 2024-09-12T01:07:12.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39858/0 (socket says 172.21.15.16:39858) 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f1735f67640 1 -- 172.21.15.16:0/769899703 learned_addr learned my addr 172.21.15.16:0/769899703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f1726ffd640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 0x7f171c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.548+0000 7f1735766640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d4234f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735766640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4242130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1736768640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 0x7f171c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 -- 172.21.15.16:0/769899703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 msgr2=0x5587d4234f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d4234f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 -- 172.21.15.16:0/769899703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 msgr2=0x5587d4242130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4242130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 -- 172.21.15.16:0/769899703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d414e1f0 con 0x5587d4235480 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735766640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d4234f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1726ffd640 1 -- 172.21.15.16:0/1540697256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f17280cf590 con 0x7f173010c170 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1735f67640 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f172c009900 tx=0x7f172c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cda816f47913f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f1736768640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 0x7f171c0a5230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f172809ba40 tx=0x7f172806da70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f171bfff640 1 -- 172.21.15.16:0/769899703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c03f070 con 0x5587d4235480 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f171bfff640 1 -- 172.21.15.16:0/769899703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c004030 con 0x5587d4235480 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.549+0000 7f171bfff640 1 -- 172.21.15.16:0/769899703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c035070 con 0x5587d4235480 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- 172.21.15.16:0/769899703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 msgr2=0x5587d423f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f172c009900 tx=0x7f172c005e80 comp rx=0 tx=0).stop 2024-09-12T01:07:12.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- 172.21.15.16:0/769899703 shutdown_connections 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4242130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423f800 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 --2- 172.21.15.16:0/769899703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d4234f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- 172.21.15.16:0/769899703 >> 172.21.15.16:0/769899703 conn(0x5587d42310f0 msgr2=0x5587d42314d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- 172.21.15.16:0/769899703 shutdown_connections 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- 172.21.15.16:0/769899703 wait complete. 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.550+0000 7f173a5a4740 1 -- start start 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 Processor -- start 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d423cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d423d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587d4242910 con 0x5587d4235480 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587d4154f10 con 0x5587d4234b60 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5587d4242a90 con 0x5587d423fd40 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f173a5a4740 10 client.0 _ll_register_callbacks cb 0x5587d411da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39860/0 (socket says 172.21.15.16:39860) 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 -- 172.21.15.16:0/2883485334 learned_addr learned my addr 172.21.15.16:0/2883485334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735766640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d423cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735766640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d423d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 -- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 msgr2=0x5587d423d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d423d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 -- 172.21.15.16:0/2883485334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 msgr2=0x5587d423cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d423cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735f67640 1 -- 172.21.15.16:0/2883485334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d3f0b6a0 con 0x5587d4235480 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.551+0000 7f1735766640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d423cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f1735f67640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f172c006010 tx=0x7f172c0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafa3e951a606b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c035070 con 0x5587d4235480 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f172c0317b0 con 0x5587d4235480 2024-09-12T01:07:12.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c004030 con 0x5587d4235480 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f172c03f040 con 0x5587d4235480 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 10 client.19023 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d424e4f0 con 0x5587d4235480 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c004030 con 0x5587d4235480 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f172c003820 con 0x5587d4235480 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 10 client.19023 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 20 client.19023 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.552+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d424f840 0x5587d4251c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.553+0000 7f173a5a4740 4 client.19023 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:12.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.553+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d41a77f0 con 0x5587d424f840 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.553+0000 7f1735766640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d424f840 0x5587d4251c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.553+0000 7f1735766640 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d424f840 0x5587d4251c30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d984b4af48d578d7 server_cookie=c60f1a3d7ace69ed in_seq=0 out_seq=0 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.553+0000 7f17197fa640 10 client.19023 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f17197fa640 1 -- 172.21.15.16:0/2883485334 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587d41a77f0 con 0x5587d424f840 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f17197fa640 10 client.19023 handle_command_reply: tid=0 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 client.19023 shutdown 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 msgr2=0x5587d423d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f172c006010 tx=0x7f172c0095a0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 shutdown_connections 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d424f840 0x5587d4251c30 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d423d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f17197fa640 0 client.19023 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d423d140 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 --2- 172.21.15.16:0/2883485334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d423cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 >> 172.21.15.16:0/2883485334 conn(0x5587d42310f0 msgr2=0x5587d4230270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 shutdown_connections 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 1 -- 172.21.15.16:0/2883485334 wait complete. 2024-09-12T01:07:12.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.554+0000 7f173a5a4740 20 client.19023 trim_cache size 0 max 16384 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 Processor -- start 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 -- start start 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d42375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4239e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17301ad8d0 con 0x5587d4235480 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17301af410 con 0x5587d423fd40 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f173a5a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17301adfa0 con 0x5587d4234b60 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39870/0 (socket says 172.21.15.16:39870) 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 -- 172.21.15.16:0/1186454099 learned_addr learned my addr 172.21.15.16:0/1186454099 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735766640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d42375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735766640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4239e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 -- 172.21.15.16:0/1186454099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 msgr2=0x5587d42375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d42375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 -- 172.21.15.16:0/1186454099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 msgr2=0x5587d4239e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4239e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735f67640 1 -- 172.21.15.16:0/1186454099 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d423f4d0 con 0x5587d4235480 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.572+0000 7f1735766640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d42375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f1735f67640 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f172c003c20 tx=0x7f172c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67157f33cc8529 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f171b7fe640 1 -- 172.21.15.16:0/1186454099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c035070 con 0x5587d4235480 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f171b7fe640 1 -- 172.21.15.16:0/1186454099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c0037f0 con 0x5587d4235480 2024-09-12T01:07:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f171b7fe640 1 -- 172.21.15.16:0/1186454099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c004b20 con 0x5587d4235480 2024-09-12T01:07:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f173a5a4740 1 -- 172.21.15.16:0/1186454099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 msgr2=0x5587d424b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.573+0000 7f173a5a4740 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f172c003c20 tx=0x7f172c005e80 comp rx=0 tx=0).stop 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- 172.21.15.16:0/1186454099 shutdown_connections 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d423fd40 0x5587d4239e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d424b3f0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- 172.21.15.16:0/1186454099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d4234b60 0x5587d42375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- 172.21.15.16:0/1186454099 >> 172.21.15.16:0/1186454099 conn(0x5587d42310f0 msgr2=0x5587d424ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- 172.21.15.16:0/1186454099 shutdown_connections 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- 172.21.15.16:0/1186454099 wait complete. 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- start start 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 Processor -- start 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d4254fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d42534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17301aed40 con 0x5587d4235480 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17301ae670 con 0x5587d4234b60 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17301ab8d0 con 0x5587d423fd40 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.574+0000 7f173a5a4740 10 client.0 _ll_register_callbacks cb 0x7f172c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39880/0 (socket says 172.21.15.16:39880) 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 -- 172.21.15.16:0/4271146407 learned_addr learned my addr 172.21.15.16:0/4271146407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735766640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d4254fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735766640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d42534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 -- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 msgr2=0x5587d42534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d42534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 -- 172.21.15.16:0/4271146407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 msgr2=0x5587d4254fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d4254fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 -- 172.21.15.16:0/4271146407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d4253ca0 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735766640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d4254fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f1735f67640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x5587d422c9d0 tx=0x7f172c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733be780f6cbfba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f17197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c063060 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f172c004860 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c0037f0 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f172c035040 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f173a5a4740 10 client.19029 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.575+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d42bae10 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c0037f0 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f172c056640 con 0x5587d4235480 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 10 client.19029 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 20 client.19029 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d42bb470 0x5587d42bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 4 client.19029 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587d41a1d00 con 0x5587d42bb470 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f1735766640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d42bb470 0x5587d42bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f17197fa640 10 client.19029 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.576+0000 7f1735766640 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d42bb470 0x5587d42bd890 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c499bfe381a5fad server_cookie=53a2f91a9295ed8c in_seq=0 out_seq=0 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.577+0000 7f17197fa640 1 -- 172.21.15.16:0/4271146407 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587d41a1d00 con 0x5587d42bb470 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f17197fa640 10 client.19029 handle_command_reply: tid=0 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 client.19029 shutdown 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 msgr2=0x5587d42554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x5587d422c9d0 tx=0x7f172c031a50 comp rx=0 tx=0).stop 2024-09-12T01:07:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 shutdown_connections 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5587d42bb470 0x5587d42bd890 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587d423fd40 0x5587d42534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587d4235480 0x5587d42554f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f17197fa640 0 client.19029 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 --2- 172.21.15.16:0/4271146407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587d4234b60 0x5587d4254fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 >> 172.21.15.16:0/4271146407 conn(0x5587d42310f0 msgr2=0x5587d424b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 shutdown_connections 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 1 -- 172.21.15.16:0/4271146407 wait complete. 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.578+0000 7f173a5a4740 20 client.19029 trim_cache size 0 max 16384 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:07:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "85a00ce9-eb91-4532-86ba-6b3803d0ab7a", 2024-09-12T01:07:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:07:12.583 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 msgr2=0x7f171c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 0x7f171c0a5230 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f172809ba40 tx=0x7f172806da70 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 msgr2=0x7f173019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f172800eaa0 tx=0x7f172800ef60 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 shutdown_connections 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f171c0a2d70 0x7f171c0a5230 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173010c170 0x7f173019cd60 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1730103660 0x7f17301989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 --2- 172.21.15.16:0/1540697256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1730102d40 0x7f1730198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 >> 172.21.15.16:0/1540697256 conn(0x7f17300fc7a0 msgr2=0x7f17300fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 shutdown_connections 2024-09-12T01:07:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:12.579+0000 7f17381cb640 1 -- 172.21.15.16:0/1540697256 wait complete. 2024-09-12T01:07:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:12 smithi016 ceph-mon[30972]: pgmap v344: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:12.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:12 smithi016 ceph-mon[30972]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:07:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:12 smithi099 ceph-mon[35654]: pgmap v344: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:12 smithi099 ceph-mon[35654]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:07:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:12 smithi159 ceph-mon[35566]: pgmap v344: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-09-12T01:07:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:12 smithi159 ceph-mon[35566]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T01:07:13.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:07:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:13 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:07:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:13 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:13 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:07:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:13 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:13 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:07:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:13 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:13 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:07:14.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:13 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- 172.21.15.16:0/1567657246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c108300 msgr2=0x7fac6c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- 172.21.15.16:0/1567657246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c108300 0x7fac6c10ec70 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009cd0 tx=0x7fac5c02a8d0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- 172.21.15.16:0/1567657246 shutdown_connections 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- 172.21.15.16:0/1567657246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c108300 0x7fac6c10ec70 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- 172.21.15.16:0/1567657246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c103780 0x7fac6c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- 172.21.15.16:0/1567657246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 0x7fac6c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- 172.21.15.16:0/1567657246 >> 172.21.15.16:0/1567657246 conn(0x7fac6c0fc7d0 msgr2=0x7fac6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- 172.21.15.16:0/1567657246 shutdown_connections 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- 172.21.15.16:0/1567657246 wait complete. 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 Processor -- start 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- start start 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 0x7fac6c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c108300 0x7fac6c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1120d0 con 0x7fac6c103780 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c111f50 con 0x7fac6c102dd0 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.051+0000 7fac72ba2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c112250 con 0x7fac6c108300 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39896/0 (socket says 172.21.15.16:39896) 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 -- 172.21.15.16:0/204128840 learned_addr learned my addr 172.21.15.16:0/204128840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac7093e640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 0x7fac6c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac7113f640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c108300 0x7fac6c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 -- 172.21.15.16:0/204128840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c108300 msgr2=0x7fac6c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c108300 0x7fac6c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 -- 172.21.15.16:0/204128840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 msgr2=0x7fac6c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 0x7fac6c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 -- 172.21.15.16:0/204128840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c19d540 con 0x7fac6c103780 2024-09-12T01:07:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac6bfff640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9c0 tx=0x7fac5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac69ffb640 1 -- 172.21.15.16:0/204128840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5400ccc0 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c19d830 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac69ffb640 1 -- 172.21.15.16:0/204128840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac54005500 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.052+0000 7fac69ffb640 1 -- 172.21.15.16:0/204128840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54010690 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.053+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1a5110 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.055+0000 7fac69ffb640 1 -- 172.21.15.16:0/204128840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fac54004e80 con 0x7fac6c103780 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.055+0000 7fac74f7b740 1 Processor -- start 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.055+0000 7fac74f7b740 1 -- start start 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 0x564c98c12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c12dc0 0x564c98c1d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c98b782e0 con 0x564c98c1d680 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c98b1fce0 con 0x564c98c124a0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac74f7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c98b742a0 con 0x564c98c12dc0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac69ffb640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 0x7fac580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39910/0 (socket says 172.21.15.16:39910) 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 -- 172.21.15.16:0/2959722829 learned_addr learned my addr 172.21.15.16:0/2959722829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac7113f640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 0x564c98c12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac7113f640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 0x7fac580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac7093e640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c12dc0 0x564c98c1d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac69ffb640 1 -- 172.21.15.16:0/204128840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fac540cf8c0 con 0x7fac6c103780 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 -- 172.21.15.16:0/2959722829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c12dc0 msgr2=0x564c98c1d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c12dc0 0x564c98c1d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 -- 172.21.15.16:0/2959722829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 msgr2=0x564c98c12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 0x564c98c12880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac6bfff640 1 -- 172.21.15.16:0/2959722829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c98b04440 con 0x564c98c1d680 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.056+0000 7fac7113f640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 0x564c98c12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fac540ccf20 tx=0x7fac54004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a518c0e363fff78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac7113f640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 0x7fac580a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac5c004af0 tx=0x7fac5c0095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac62ffd640 1 -- 172.21.15.16:0/2959722829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5409c480 con 0x564c98c1d680 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac62ffd640 1 -- 172.21.15.16:0/2959722829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5409bd80 con 0x564c98c1d680 2024-09-12T01:07:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac62ffd640 1 -- 172.21.15.16:0/2959722829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540cd230 con 0x564c98c1d680 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 -- 172.21.15.16:0/2959722829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 msgr2=0x564c98c1fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fac540ccf20 tx=0x7fac54004b00 comp rx=0 tx=0).stop 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 -- 172.21.15.16:0/2959722829 shutdown_connections 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c1d680 0x564c98c1fa70 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c12dc0 0x564c98c1d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 --2- 172.21.15.16:0/2959722829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c124a0 0x564c98c12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 -- 172.21.15.16:0/2959722829 >> 172.21.15.16:0/2959722829 conn(0x564c98c0ea30 msgr2=0x564c98c0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.057+0000 7fac74f7b740 1 -- 172.21.15.16:0/2959722829 shutdown_connections 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 -- 172.21.15.16:0/2959722829 wait complete. 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 -- start start 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 Processor -- start 2024-09-12T01:07:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c1a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c1a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c98c20250 con 0x564c98c124a0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c98b33050 con 0x564c98c12dc0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c98c203d0 con 0x564c98c1d680 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac74f7b740 10 client.0 _ll_register_callbacks cb 0x564c98b73160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39920/0 (socket says 172.21.15.16:39920) 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.058+0000 7fac7093e640 1 -- 172.21.15.16:0/3275563204 learned_addr learned my addr 172.21.15.16:0/3275563204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7113f640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c1a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac6bfff640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c1a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 -- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 msgr2=0x564c98c1a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c1a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 -- 172.21.15.16:0/3275563204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 msgr2=0x564c98c1a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c1a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 -- 172.21.15.16:0/3275563204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c988e93c0 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac7093e640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac6400bad0 tx=0x7fac6400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2151797fd2ba02d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64005020 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fac640051c0 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac5409bd80 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac64005400 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac74f7b740 10 client.19041 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.059+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c98c2bc80 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5409bd80 con 0x564c98c124a0 2024-09-12T01:07:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac640246e0 con 0x564c98c124a0 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 10 client.19041 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 20 client.19041 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c2d090 0x564c98c2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 4 client.19041 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c98b85850 con 0x564c98c2d090 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.060+0000 7fac6bfff640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c2d090 0x564c98c2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.061+0000 7fac6bfff640 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c2d090 0x564c98c2f4e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ae938cf1f8218c server_cookie=65d97098b57a464c in_seq=0 out_seq=0 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.061+0000 7fac4bfff640 10 client.19041 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.061+0000 7fac4bfff640 1 -- 172.21.15.16:0/3275563204 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c98b85850 con 0x564c98c2d090 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.061+0000 7fac4bfff640 10 client.19041 handle_command_reply: tid=0 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 client.19041 shutdown 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 msgr2=0x564c98c1cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fac6400bad0 tx=0x7fac6400bed0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 shutdown_connections 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c2d090 0x564c98c2f4e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c1a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c1a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 --2- 172.21.15.16:0/3275563204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c1cfd0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac4bfff640 0 client.19041 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 >> 172.21.15.16:0/3275563204 conn(0x564c98c0ea30 msgr2=0x564c98c14710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 shutdown_connections 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 1 -- 172.21.15.16:0/3275563204 wait complete. 2024-09-12T01:07:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.062+0000 7fac74f7b740 20 client.19041 trim_cache size 0 max 16384 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 Processor -- start 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 -- start start 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ad8f0 con 0x564c98c124a0 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1af430 con 0x564c98c12dc0 2024-09-12T01:07:14.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.079+0000 7fac74f7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1adfc0 con 0x564c98c1d680 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39936/0 (socket says 172.21.15.16:39936) 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 -- 172.21.15.16:0/1850700087 learned_addr learned my addr 172.21.15.16:0/1850700087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac6bfff640 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 -- 172.21.15.16:0/1850700087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 msgr2=0x564c98c352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 -- 172.21.15.16:0/1850700087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 msgr2=0x564c98c34db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c34db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 -- 172.21.15.16:0/1850700087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c98c19230 con 0x564c98c124a0 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac7093e640 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac64024b00 tx=0x7fac640044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976cf4b806bb4c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac627fc640 1 -- 172.21.15.16:0/1850700087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400b400 con 0x564c98c124a0 2024-09-12T01:07:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac627fc640 1 -- 172.21.15.16:0/1850700087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6401b4a0 con 0x564c98c124a0 2024-09-12T01:07:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.080+0000 7fac627fc640 1 -- 172.21.15.16:0/1850700087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac640116a0 con 0x564c98c124a0 2024-09-12T01:07:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- 172.21.15.16:0/1850700087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 msgr2=0x564c98c294b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fac64024b00 tx=0x7fac640044e0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- 172.21.15.16:0/1850700087 shutdown_connections 2024-09-12T01:07:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c34db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 --2- 172.21.15.16:0/1850700087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c294b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- 172.21.15.16:0/1850700087 >> 172.21.15.16:0/1850700087 conn(0x564c98c0ea30 msgr2=0x564c98c15900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- 172.21.15.16:0/1850700087 shutdown_connections 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- 172.21.15.16:0/1850700087 wait complete. 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 -- start start 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.081+0000 7fac74f7b740 1 Processor -- start 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c21ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1aed60 con 0x564c98c124a0 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ae690 con 0x564c98c12dc0 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ab8f0 con 0x564c98c1d680 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac74f7b740 10 client.0 _ll_register_callbacks cb 0x7fac640040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39942/0 (socket says 172.21.15.16:39942) 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 -- 172.21.15.16:0/386326386 learned_addr learned my addr 172.21.15.16:0/386326386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7113f640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c21ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac6bfff640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 -- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 msgr2=0x564c98c21ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c21ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 -- 172.21.15.16:0/386326386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 msgr2=0x564c98c219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 -- 172.21.15.16:0/386326386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c98c96e80 con 0x564c98c124a0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac6bfff640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac7093e640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fac6400bad0 tx=0x7fac6400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23b12b09fa772b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.082+0000 7fac4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac640247d0 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fac6401b650 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac6401b4a0 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac6403e050 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 10 client.19047 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c98c98150 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6401b4a0 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac64035070 con 0x564c98c124a0 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 10 client.19047 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 20 client.19047 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c98700 0x564c98c9aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 4 client.19047 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:07:14.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.083+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c98b7fd60 con 0x564c98c98700 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac6bfff640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c98700 0x564c98c9aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac6bfff640 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c98700 0x564c98c9aaf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad39e835b4933ed7 server_cookie=18bad23ebf975e3b in_seq=0 out_seq=0 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac4bfff640 10 client.19047 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac4bfff640 1 -- 172.21.15.16:0/386326386 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564c98b7fd60 con 0x564c98c98700 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac4bfff640 10 client.19047 handle_command_reply: tid=0 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 client.19047 shutdown 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 msgr2=0x564c98c21460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fac6400bad0 tx=0x7fac6400b5c0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 shutdown_connections 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c98c98700 0x564c98c9aaf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c98c1d680 0x564c98c21ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c98c12dc0 0x564c98c219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 --2- 172.21.15.16:0/386326386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c98c124a0 0x564c98c21460 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac4bfff640 0 client.19047 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.084+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 >> 172.21.15.16:0/386326386 conn(0x564c98c0ea30 msgr2=0x564c98c15000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 shutdown_connections 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac74f7b740 1 -- 172.21.15.16:0/386326386 wait complete. 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac74f7b740 20 client.19047 trim_cache size 0 max 16384 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stdout: "85a00ce9-eb91-4532-86ba-6b3803d0ab7a": { 2024-09-12T01:07:14.091 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "85a00ce9-eb91-4532-86ba-6b3803d0ab7a", 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 msgr2=0x7fac580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 0x7fac580a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac5c004af0 tx=0x7fac5c0095c0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 msgr2=0x7fac6c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e9c0 tx=0x7fac5400ee80 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 shutdown_connections 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fac580a2cf0 0x7fac580a51b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac6c108300 0x7fac6c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c103780 0x7fac6c198ad0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 --2- 172.21.15.16:0/204128840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac6c102dd0 0x7fac6c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 >> 172.21.15.16:0/204128840 conn(0x7fac6c0fc7d0 msgr2=0x7fac6c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.085+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 shutdown_connections 2024-09-12T01:07:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:14.086+0000 7fac72ba2640 1 -- 172.21.15.16:0/204128840 wait complete. 2024-09-12T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:13 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:13 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:13 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:13 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:14.827 INFO:tasks.cephfs.filesystem:scrub status for tag:85a00ce9-eb91-4532-86ba-6b3803d0ab7a - {'path': '/', 'tag': '85a00ce9-eb91-4532-86ba-6b3803d0ab7a', 'options': 'recursive,force'} 2024-09-12T01:07:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:14 smithi099 ceph-mon[35654]: pgmap v346: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 8 op/s 2024-09-12T01:07:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:14 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:07:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:14 smithi159 ceph-mon[35566]: pgmap v346: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 8 op/s 2024-09-12T01:07:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:14 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:07:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:14 smithi016 ceph-mon[30972]: pgmap v346: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 8 op/s 2024-09-12T01:07:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:14 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:07:17.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:16 smithi099 ceph-mon[35654]: pgmap v347: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:16 smithi159 ceph-mon[35566]: pgmap v347: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:16 smithi016 ceph-mon[30972]: pgmap v347: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:18 smithi099 ceph-mon[35654]: pgmap v348: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:18 smithi159 ceph-mon[35566]: pgmap v348: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:19.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:18 smithi016 ceph-mon[30972]: pgmap v348: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-09-12T01:07:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:20 smithi099 ceph-mon[35654]: pgmap v349: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:20 smithi159 ceph-mon[35566]: pgmap v349: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:20 smithi016 ceph-mon[30972]: pgmap v349: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:22 smithi099 ceph-mon[35654]: pgmap v350: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:07:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:22 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:22 smithi159 ceph-mon[35566]: pgmap v350: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:07:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:22 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:22 smithi016 ceph-mon[30972]: pgmap v350: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-09-12T01:07:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:07:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:22 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:24.017 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:07:24.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:07:24.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:24.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:07:24.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:07:24.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:07:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:07:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:07:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:24 smithi099 ceph-mon[35654]: pgmap v351: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-12T01:07:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:24 smithi159 ceph-mon[35566]: pgmap v351: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-12T01:07:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:24 smithi016 ceph-mon[30972]: pgmap v351: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-09-12T01:07:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:26 smithi099 ceph-mon[35654]: pgmap v352: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 20 op/s 2024-09-12T01:07:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:26 smithi159 ceph-mon[35566]: pgmap v352: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 20 op/s 2024-09-12T01:07:27.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:26 smithi016 ceph-mon[30972]: pgmap v352: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 79 KiB/s wr, 20 op/s 2024-09-12T01:07:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:28 smithi099 ceph-mon[35654]: pgmap v353: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-12T01:07:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:28 smithi159 ceph-mon[35566]: pgmap v353: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-12T01:07:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:28 smithi016 ceph-mon[30972]: pgmap v353: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-12T01:07:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:30 smithi099 ceph-mon[35654]: pgmap v354: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:30 smithi159 ceph-mon[35566]: pgmap v354: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:30 smithi016 ceph-mon[30972]: pgmap v354: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:32 smithi099 ceph-mon[35654]: pgmap v355: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:32 smithi159 ceph-mon[35566]: pgmap v355: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:33.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:32 smithi016 ceph-mon[30972]: pgmap v355: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:34 smithi099 ceph-mon[35654]: pgmap v356: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:34 smithi159 ceph-mon[35566]: pgmap v356: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:35.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:34 smithi016 ceph-mon[30972]: pgmap v356: 97 pgs: 97 active+clean; 16 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:36 smithi099 ceph-mon[35654]: pgmap v357: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:37.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:36 smithi159 ceph-mon[35566]: pgmap v357: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:37.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:36 smithi016 ceph-mon[30972]: pgmap v357: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:38 smithi099 ceph-mon[35654]: pgmap v358: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 12 op/s 2024-09-12T01:07:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:38 smithi159 ceph-mon[35566]: pgmap v358: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 12 op/s 2024-09-12T01:07:39.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:38 smithi016 ceph-mon[30972]: pgmap v358: 97 pgs: 97 active+clean; 16 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 12 op/s 2024-09-12T01:07:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:40 smithi099 ceph-mon[35654]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:40 smithi159 ceph-mon[35566]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:41.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:40 smithi016 ceph-mon[30972]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 77 KiB/s wr, 19 op/s 2024-09-12T01:07:43.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:42 smithi099 ceph-mon[35654]: pgmap v360: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:42 smithi159 ceph-mon[35566]: pgmap v360: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:42 smithi016 ceph-mon[30972]: pgmap v360: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:44.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:07:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:44 smithi099 ceph-mon[35654]: pgmap v361: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:45.097 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:44 smithi159 ceph-mon[35566]: pgmap v361: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:45.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:44 smithi016 ceph-mon[30972]: pgmap v361: 97 pgs: 97 active+clean; 17 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-09-12T01:07:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 -- 172.21.15.16:0/2876003606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b810e500 msgr2=0x7ff4b8110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2876003606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b810e500 0x7ff4b8110910 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009ca0 tx=0x7ff4b002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 -- 172.21.15.16:0/2876003606 shutdown_connections 2024-09-12T01:07:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2876003606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b810e500 0x7ff4b8110910 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2876003606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b8101440 0x7ff4b810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2876003606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 -- 172.21.15.16:0/2876003606 >> 172.21.15.16:0/2876003606 conn(0x7ff4b80fc7d0 msgr2=0x7ff4b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 -- 172.21.15.16:0/2876003606 shutdown_connections 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.584+0000 7ff4bf969640 1 -- 172.21.15.16:0/2876003606 wait complete. 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 Processor -- start 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 -- start start 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 0x7ff4b819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81142a0 con 0x7ff4b8101440 2024-09-12T01:07:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114120 con 0x7ff4b8100b20 2024-09-12T01:07:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bf969640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b8114420 con 0x7ff4b810e500 2024-09-12T01:07:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45062/0 (socket says 172.21.15.16:45062) 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.585+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2793585928 learned_addr learned my addr 172.21.15.16:0/2793585928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 0x7ff4b819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bd705640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2793585928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 msgr2=0x7ff4b819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 0x7ff4b819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2793585928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 msgr2=0x7ff4b819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2793585928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b819f670 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 0x7ff4b819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bd705640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d950 tx=0x7ff4a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4ae7fc640 1 -- 172.21.15.16:0/2793585928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8017070 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4ae7fc640 1 -- 172.21.15.16:0/2793585928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a8009c30 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b819f960 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4ae7fc640 1 -- 172.21.15.16:0/2793585928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800f600 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.586+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b81a7240 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.588+0000 7ff4ae7fc640 1 -- 172.21.15.16:0/2793585928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff4a801a070 con 0x7ff4b8101440 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 Processor -- start 2024-09-12T01:07:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 -- start start 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b744a0 0x55fef0b74880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fef0ada2e0 con 0x55fef0b7f680 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fef0a81ce0 con 0x55fef0b744a0 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fef0ad62a0 con 0x55fef0b74dc0 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4ae7fc640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 0x7ff4a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45076/0 (socket says 172.21.15.16:45076) 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4bcf04640 1 -- 172.21.15.16:0/1846671155 learned_addr learned my addr 172.21.15.16:0/1846671155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4bd705640 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.589+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b744a0 0x55fef0b74880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 0x7ff4a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4ae7fc640 1 -- 172.21.15.16:0/2793585928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff4a80cef70 con 0x7ff4b8101440 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 -- 172.21.15.16:0/1846671155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 msgr2=0x55fef0b7f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b7f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 -- 172.21.15.16:0/1846671155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b744a0 msgr2=0x55fef0b74880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b744a0 0x55fef0b74880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 -- 172.21.15.16:0/1846671155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef0a66440 con 0x55fef0b7f680 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005d80 tx=0x7ff4a8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e0df54f34a6721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 0x7ff4a40a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002f540 tx=0x7ff4b00095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4a37fe640 1 -- 172.21.15.16:0/1846671155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8021070 con 0x55fef0b7f680 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4a37fe640 1 -- 172.21.15.16:0/1846671155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a809f7e0 con 0x55fef0b7f680 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.590+0000 7ff4a37fe640 1 -- 172.21.15.16:0/1846671155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a809a880 con 0x55fef0b7f680 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1846671155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 msgr2=0x55fef0b81a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8005d80 tx=0x7ff4a8010ce0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1846671155 shutdown_connections 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b7f680 0x55fef0b81a70 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b7f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1846671155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b744a0 0x55fef0b74880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1846671155 >> 172.21.15.16:0/1846671155 conn(0x55fef0b70a30 msgr2=0x55fef0b70e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1846671155 shutdown_connections 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1846671155 wait complete. 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.591+0000 7ff4c1d42740 1 -- start start 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 Processor -- start 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 0x55fef0b7c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b7f680 0x55fef0b7c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fef0b821f0 con 0x55fef0b744a0 2024-09-12T01:07:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fef0a95050 con 0x55fef0b74dc0 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fef0b82370 con 0x55fef0b7f680 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4c1d42740 10 client.0 _ll_register_callbacks cb 0x55fef0ad5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45084/0 (socket says 172.21.15.16:45084) 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 -- 172.21.15.16:0/3855233897 learned_addr learned my addr 172.21.15.16:0/3855233897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 0x55fef0b7c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b7f680 0x55fef0b7c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 -- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b7f680 msgr2=0x55fef0b7c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b7f680 0x55fef0b7c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 -- 172.21.15.16:0/3855233897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 msgr2=0x55fef0b7c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 0x55fef0b7c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bd705640 1 -- 172.21.15.16:0/3855233897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef084b3c0 con 0x55fef0b744a0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.592+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 0x55fef0b7c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4bd705640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010840 tx=0x7ff4b4010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e777ddb4bf1d333d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400eda0 con 0x55fef0b744a0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff4b4005020 con 0x55fef0b744a0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a809f7e0 con 0x55fef0b744a0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4b4005260 con 0x55fef0b744a0 2024-09-12T01:07:45.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 10 client.19059 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fef0b8dbb0 con 0x55fef0b744a0 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a809f7e0 con 0x55fef0b744a0 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4b4018550 con 0x55fef0b744a0 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 10 client.19059 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 20 client.19059 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0b8f070 0x55fef0b914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 4 client.19059 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.593+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef0ae7850 con 0x55fef0b8f070 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.594+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0b8f070 0x55fef0b914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.594+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0b8f070 0x55fef0b914f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f97af6020a1a70 server_cookie=69b2fbfda4b14471 in_seq=0 out_seq=0 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.594+0000 7ff4a0ff9640 10 client.19059 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/3855233897 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fef0ae7850 con 0x55fef0b8f070 2024-09-12T01:07:45.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4a0ff9640 10 client.19059 handle_command_reply: tid=0 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 client.19059 shutdown 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 msgr2=0x55fef0b7ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010840 tx=0x7ff4b4010d00 comp rx=0 tx=0).stop 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 shutdown_connections 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0b8f070 0x55fef0b914f0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b7f680 0x55fef0b7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b74dc0 0x55fef0b7c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4a0ff9640 0 client.19059 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/3855233897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b7ef00 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 >> 172.21.15.16:0/3855233897 conn(0x55fef0b70a30 msgr2=0x55fef0b76710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 shutdown_connections 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 1 -- 172.21.15.16:0/3855233897 wait complete. 2024-09-12T01:07:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.595+0000 7ff4c1d42740 20 client.19059 trim_cache size 0 max 16384 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 Processor -- start 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 -- start start 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81afa30 con 0x55fef0b744a0 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81b1570 con 0x55fef0b7f680 2024-09-12T01:07:45.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81b0100 con 0x55fef0b74dc0 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45100/0 (socket says 172.21.15.16:45100) 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 -- 172.21.15.16:0/1613559933 learned_addr learned my addr 172.21.15.16:0/1613559933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 -- 172.21.15.16:0/1613559933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 msgr2=0x55fef0b96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b96e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 -- 172.21.15.16:0/1613559933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 msgr2=0x55fef0b97350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b97350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bd705640 1 -- 172.21.15.16:0/1613559933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef0b91b70 con 0x55fef0b744a0 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:45.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.613+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4bd705640 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b310 tx=0x7ff4b4018bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65803b0df54aec68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4a2ffd640 1 -- 172.21.15.16:0/1613559933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4005600 con 0x55fef0b744a0 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4a2ffd640 1 -- 172.21.15.16:0/1613559933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b402f070 con 0x55fef0b744a0 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4a2ffd640 1 -- 172.21.15.16:0/1613559933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b402abb0 con 0x55fef0b744a0 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1613559933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 msgr2=0x55fef0b8b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400b310 tx=0x7ff4b4018bb0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1613559933 shutdown_connections 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b97350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b74dc0 0x55fef0b96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/1613559933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b744a0 0x55fef0b8b450 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1613559933 >> 172.21.15.16:0/1613559933 conn(0x55fef0b70a30 msgr2=0x55fef0b7aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.614+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1613559933 shutdown_connections 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 -- 172.21.15.16:0/1613559933 wait complete. 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 -- start start 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 Processor -- start 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b744a0 0x55fef0b87890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b82c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81b0ea0 con 0x55fef0b74dc0 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81b07d0 con 0x55fef0b7f680 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b81ada30 con 0x55fef0b744a0 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.615+0000 7ff4c1d42740 10 client.0 _ll_register_callbacks cb 0x7ff4b4010450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45112/0 (socket says 172.21.15.16:45112) 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2098952182 learned_addr learned my addr 172.21.15.16:0/2098952182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bd705640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b744a0 0x55fef0b87890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b82c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b744a0 msgr2=0x55fef0b87890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b744a0 0x55fef0b87890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2098952182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 msgr2=0x55fef0b82c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b82c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 -- 172.21.15.16:0/2098952182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef0b83450 con 0x55fef0b74dc0 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bdf06640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b82c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4bcf04640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80a3b70 tx=0x7ff4a8097cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4612b93bd65c8f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:45.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8021070 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff4a809a880 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4b402f070 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4a80ccaf0 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4c1d42740 10 client.19065 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.616+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fef0bf98e0 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b402f070 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff4a80b42b0 con 0x55fef0b74dc0 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 10 client.19065 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 10 client.19065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:45.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 20 client.19065 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0bf9c10 0x55fef0bfc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 4 client.19065 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fef0ae1d60 con 0x55fef0bf9c10 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.617+0000 7ff4bd705640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0bf9c10 0x55fef0bfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4bd705640 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0bf9c10 0x55fef0bfc000 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8516ae4b13c116bd server_cookie=de5859ff7b9d028 in_seq=0 out_seq=0 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4a0ff9640 10 client.19065 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4a0ff9640 1 -- 172.21.15.16:0/2098952182 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fef0ae1d60 con 0x55fef0bf9c10 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4a0ff9640 10 client.19065 handle_command_reply: tid=0 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 client.19065 shutdown 2024-09-12T01:07:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 msgr2=0x55fef0b87dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80a3b70 tx=0x7ff4a8097cd0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 shutdown_connections 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55fef0bf9c10 0x55fef0bfc000 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fef0b7f680 0x55fef0b82c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fef0b74dc0 0x55fef0b87dd0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 --2- 172.21.15.16:0/2098952182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fef0b744a0 0x55fef0b87890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4a0ff9640 0 client.19065 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 >> 172.21.15.16:0/2098952182 conn(0x55fef0b70a30 msgr2=0x55fef0b6fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 shutdown_connections 2024-09-12T01:07:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 1 -- 172.21.15.16:0/2098952182 wait complete. 2024-09-12T01:07:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.618+0000 7ff4c1d42740 20 client.19065 trim_cache size 0 max 16384 2024-09-12T01:07:45.631 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:07:45.632 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:07:45.632 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 msgr2=0x7ff4a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 0x7ff4a40a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002f540 tx=0x7ff4b00095c0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 msgr2=0x7ff4b819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800d950 tx=0x7ff4a800de10 comp rx=0 tx=0).stop 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 shutdown_connections 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff4a40a2cf0 0x7ff4a40a51b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4b810e500 0x7ff4b819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b8101440 0x7ff4b819ac00 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 --2- 172.21.15.16:0/2793585928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4b8100b20 0x7ff4b819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 >> 172.21.15.16:0/2793585928 conn(0x7ff4b80fc7d0 msgr2=0x7ff4b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 shutdown_connections 2024-09-12T01:07:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:45.619+0000 7ff4bf969640 1 -- 172.21.15.16:0/2793585928 wait complete. 2024-09-12T01:07:46.435 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:07:46.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:07:46.693 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:46 smithi099 ceph-mon[35654]: pgmap v362: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:47.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:46 smithi016 ceph-mon[30972]: pgmap v362: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:46 smithi159 ceph-mon[35566]: pgmap v362: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 19 op/s 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3218616588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4102790 msgr2=0x7f58c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3218616588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4102790 0x7f58c41067e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac02a3f0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3218616588 shutdown_connections 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3218616588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4106f10 0x7f58c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3218616588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4102790 0x7f58c41067e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3218616588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4101de0 0x7f58c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3218616588 >> 172.21.15.16:0/3218616588 conn(0x7f58c40fd8c0 msgr2=0x7f58c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3218616588 shutdown_connections 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.176+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3218616588 wait complete. 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 Processor -- start 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 -- start start 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4101de0 0x7f58c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102790 0x7f58c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4111f30 con 0x7f58c4106f10 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4111db0 con 0x7f58c4102790 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c9f3a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41120b0 con 0x7f58c4101de0 2024-09-12T01:07:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45134/0 (socket says 172.21.15.16:45134) 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c3fff640 1 -- 172.21.15.16:0/3573281127 learned_addr learned my addr 172.21.15.16:0/3573281127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58bbfff640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102790 0x7f58c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.177+0000 7f58c37fe640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4101de0 0x7f58c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 -- 172.21.15.16:0/3573281127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4101de0 msgr2=0x7f58c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4101de0 0x7f58c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 -- 172.21.15.16:0/3573281127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102790 msgr2=0x7f58c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102790 0x7f58c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 -- 172.21.15.16:0/3573281127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c41a6880 con 0x7f58c4106f10 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c3fff640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cad0 tx=0x7f58bc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00ad60 con 0x7f58c4106f10 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc005020 con 0x7f58c4106f10 2024-09-12T01:07:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c41a6b70 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00f6b0 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.178+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a7080 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.179+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c41021c0 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.180+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f58bc005560 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.181+0000 7f58c17fa640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 0x7f58b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.181+0000 7f58c37fe640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 0x7f58b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.181+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f58bc0cf330 con 0x7f58c4106f10 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.182+0000 7f58c37fe640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 0x7f58b00a5230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58b400abe0 tx=0x7f58b400a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:47.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.185+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58bc014030 con 0x7f58c4106f10 2024-09-12T01:07:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.398+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c402d7c0 con 0x7f58c4106f10 2024-09-12T01:07:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.398+0000 7f58c17fa640 1 -- 172.21.15.16:0/3573281127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f58bc09bf50 con 0x7f58c4106f10 2024-09-12T01:07:47.397 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:07:47.397 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:07:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.400+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 msgr2=0x7f58b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.400+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 0x7f58b00a5230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f58b400abe0 tx=0x7f58b400a2d0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.400+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 msgr2=0x7f58c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.400+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cad0 tx=0x7f58bc00ced0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 shutdown_connections 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f58b00a2d70 0x7f58b00a5230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58c4106f10 0x7f58c4074f70 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c4102790 0x7f58c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 --2- 172.21.15.16:0/3573281127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58c4101de0 0x7f58c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 >> 172.21.15.16:0/3573281127 conn(0x7f58c40fd8c0 msgr2=0x7f58c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 shutdown_connections 2024-09-12T01:07:47.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:47.401+0000 7f58c9f3a640 1 -- 172.21.15.16:0/3573281127 wait complete. 2024-09-12T01:07:48.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3573281127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:48.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3573281127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3573281127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:07:48.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:07:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:49.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:48 smithi016 ceph-mon[30972]: pgmap v363: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:49.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:48 smithi159 ceph-mon[35566]: pgmap v363: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 -- 172.21.15.16:0/2583431787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 msgr2=0x7fee6810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/2583431787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee6810d950 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fee60009cd0 tx=0x7fee6002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 -- 172.21.15.16:0/2583431787 shutdown_connections 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/2583431787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee6810d950 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/2583431787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/2583431787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 -- 172.21.15.16:0/2583431787 >> 172.21.15.16:0/2583431787 conn(0x7fee680fc7a0 msgr2=0x7fee680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 -- 172.21.15.16:0/2583431787 shutdown_connections 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.086+0000 7fee6ef7d640 1 -- 172.21.15.16:0/2583431787 wait complete. 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 Processor -- start 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 -- start start 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee681984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee6819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee68111fc0 con 0x7fee68101420 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee68111e40 con 0x7fee68100a80 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6ef7d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee68112140 con 0x7fee68106a80 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43834/0 (socket says 172.21.15.16:43834) 2024-09-12T01:07:49.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee5ffff640 1 -- 172.21.15.16:0/3604546559 learned_addr learned my addr 172.21.15.16:0/3604546559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.087+0000 7fee6d51a640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee6819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee6cd19640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee681984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 -- 172.21.15.16:0/3604546559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 msgr2=0x7fee6819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee6819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 -- 172.21.15.16:0/3604546559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 msgr2=0x7fee681984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee681984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 -- 172.21.15.16:0/3604546559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6819d450 con 0x7fee68101420 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee6cd19640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee681984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5ffff640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b4a0 tx=0x7fee5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5dffb640 1 -- 172.21.15.16:0/3604546559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58014070 con 0x7fee68101420 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5dffb640 1 -- 172.21.15.16:0/3604546559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58010af0 con 0x7fee68101420 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6819d740 con 0x7fee68101420 2024-09-12T01:07:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee5dffb640 1 -- 172.21.15.16:0/3604546559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58017d10 con 0x7fee68101420 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.088+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee681a5020 con 0x7fee68101420 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.090+0000 7fee5dffb640 1 -- 172.21.15.16:0/3604546559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee58017070 con 0x7fee68101420 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 Processor -- start 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 -- start start 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee5dffb640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 0x7fee500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf9eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bedbef32f0 con 0x55bedbf8eb40 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bedbe9acf0 con 0x55bedbf8d5d0 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee71356740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bedbeef2b0 con 0x55bedbf9c780 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee6d51a640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 0x7fee500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43848/0 (socket says 172.21.15.16:43848) 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.091+0000 7fee6cd19640 1 -- 172.21.15.16:0/2778081722 learned_addr learned my addr 172.21.15.16:0/2778081722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee5dffb640 1 -- 172.21.15.16:0/3604546559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fee580ce770 con 0x7fee68101420 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee5ffff640 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6d51a640 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf9eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 -- 172.21.15.16:0/2778081722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 msgr2=0x55bedbf9eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf9eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 -- 172.21.15.16:0/2778081722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 msgr2=0x55bedbf8da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf8da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 -- 172.21.15.16:0/2778081722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedbe7f3a0 con 0x55bedbf8eb40 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6d51a640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 0x7fee500a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fee60004ab0 tx=0x7fee600317a0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee6cd19640 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d40 tx=0x7fee640117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad71cfd8e044ded4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee4effd640 1 -- 172.21.15.16:0/2778081722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64012650 con 0x55bedbf8eb40 2024-09-12T01:07:49.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee4effd640 1 -- 172.21.15.16:0/2778081722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee64009e30 con 0x55bedbf8eb40 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.092+0000 7fee4effd640 1 -- 172.21.15.16:0/2778081722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6400fbd0 con 0x55bedbf8eb40 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- 172.21.15.16:0/2778081722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 msgr2=0x55bedbf8d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d40 tx=0x7fee640117b0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- 172.21.15.16:0/2778081722 shutdown_connections 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf9eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf8da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 --2- 172.21.15.16:0/2778081722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8eb40 0x55bedbf8d090 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- 172.21.15.16:0/2778081722 >> 172.21.15.16:0/2778081722 conn(0x55bedbf56310 msgr2=0x55bedbf89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- 172.21.15.16:0/2778081722 shutdown_connections 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- 172.21.15.16:0/2778081722 wait complete. 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 -- start start 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.093+0000 7fee71356740 1 Processor -- start 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8eb40 0x55bedbf99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bedbf9f480 con 0x55bedbf8d5d0 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bedbeae060 con 0x55bedbf8eb40 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bedbf9f600 con 0x55bedbf9c780 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee71356740 10 client.0 _ll_register_callbacks cb 0x55bedbc018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43860/0 (socket says 172.21.15.16:43860) 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 -- 172.21.15.16:0/696394986 learned_addr learned my addr 172.21.15.16:0/696394986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6d51a640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee5ffff640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8eb40 0x55bedbf99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 -- 172.21.15.16:0/696394986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 msgr2=0x55bedbf998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 -- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8eb40 msgr2=0x55bedbf99380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8eb40 0x55bedbf99380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6cd19640 1 -- 172.21.15.16:0/696394986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedbc643c0 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.094+0000 7fee6d51a640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee6cd19640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d10 tx=0x7fee6400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392d085e924d84c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64028020 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee64009e30 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fee64005240 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee64033050 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 10 client.19083 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bedbfaae90 con 0x55bedbf8d5d0 2024-09-12T01:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee64009e30 con 0x55bedbf8d5d0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee640124e0 con 0x55bedbf8d5d0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 10 client.19083 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 10 client.19083 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 20 client.19083 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedbfac2f0 0x55bedbfae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 4 client.19083 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.095+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bedbf00860 con 0x55bedbfac2f0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.096+0000 7fee5ffff640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedbfac2f0 0x55bedbfae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.096+0000 7fee5ffff640 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedbfac2f0 0x55bedbfae6e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef43447a95804278 server_cookie=cba34d476dc51a2 in_seq=0 out_seq=0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.096+0000 7fee47fff640 10 client.19083 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee47fff640 1 -- 172.21.15.16:0/696394986 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bedbf00860 con 0x55bedbfac2f0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee47fff640 10 client.19083 handle_command_reply: tid=0 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 client.19083 shutdown 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 msgr2=0x55bedbf9c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fee64009d10 tx=0x7fee6400fba0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 shutdown_connections 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedbfac2f0 0x55bedbfae6e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf9c780 0x55bedbf998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8eb40 0x55bedbf99380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee47fff640 0 client.19083 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 --2- 172.21.15.16:0/696394986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbf9c0d0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 >> 172.21.15.16:0/696394986 conn(0x55bedbf56310 msgr2=0x55bedbf8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 shutdown_connections 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.097+0000 7fee71356740 1 -- 172.21.15.16:0/696394986 wait complete. 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.098+0000 7fee71356740 20 client.19083 trim_cache size 0 max 16384 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.110+0000 7fee71356740 1 Processor -- start 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.110+0000 7fee71356740 1 -- start start 2024-09-12T01:07:49.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbf88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ab750 con 0x55bedbf9c780 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ad290 con 0x55bedbf8d5d0 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee71356740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee681abe20 con 0x55bedbf8eb40 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43862/0 (socket says 172.21.15.16:43862) 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 -- 172.21.15.16:0/4281028073 learned_addr learned my addr 172.21.15.16:0/4281028073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee5ffff640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbf88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6cd19640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 -- 172.21.15.16:0/4281028073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 msgr2=0x55bedbf88c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbf88c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 -- 172.21.15.16:0/4281028073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 msgr2=0x55bedbf98c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf98c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 -- 172.21.15.16:0/4281028073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bedbf8dd50 con 0x55bedbf9c780 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee5ffff640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbf88c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6cd19640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee6d51a640 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fee60009cd0 tx=0x7fee60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394fabe2d5fd96b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.111+0000 7fee4e7fc640 1 -- 172.21.15.16:0/4281028073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6004e070 con 0x55bedbf9c780 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee4e7fc640 1 -- 172.21.15.16:0/4281028073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee60036070 con 0x55bedbf9c780 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee4e7fc640 1 -- 172.21.15.16:0/4281028073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6004a430 con 0x55bedbf9c780 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- 172.21.15.16:0/4281028073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 msgr2=0x55bedbfb3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fee60009cd0 tx=0x7fee60004a40 comp rx=0 tx=0).stop 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- 172.21.15.16:0/4281028073 shutdown_connections 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf9c780 0x55bedbfb3d40 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbf88c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 --2- 172.21.15.16:0/4281028073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf8d5d0 0x55bedbf98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- 172.21.15.16:0/4281028073 >> 172.21.15.16:0/4281028073 conn(0x55bedbf56310 msgr2=0x55bedbf88500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- 172.21.15.16:0/4281028073 shutdown_connections 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- 172.21.15.16:0/4281028073 wait complete. 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.112+0000 7fee71356740 1 -- start start 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 Processor -- start 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbfa0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf9c780 0x55bedc015d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee681acbc0 con 0x55bedbf8d5d0 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ac4f0 con 0x55bedbf9c780 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee681a9750 con 0x55bedbf8eb40 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee71356740 10 client.0 _ll_register_callbacks cb 0x7fee58096fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43864/0 (socket says 172.21.15.16:43864) 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 -- 172.21.15.16:0/864847372 learned_addr learned my addr 172.21.15.16:0/864847372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6d51a640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf9c780 0x55bedc015d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee5ffff640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbfa0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 -- 172.21.15.16:0/864847372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 msgr2=0x55bedbfa0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbfa0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 -- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf9c780 msgr2=0x55bedc015d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf9c780 0x55bedc015d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee6cd19640 1 -- 172.21.15.16:0/864847372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bedbfa1750 con 0x55bedbf8d5d0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.113+0000 7fee5ffff640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbfa0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee6cd19640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fee640117f0 tx=0x7fee6400fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511c743f01d25fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64028040 con 0x55bedbf8d5d0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fee64012650 con 0x55bedbf8d5d0 2024-09-12T01:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee60036070 con 0x55bedbf8d5d0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee6400f5a0 con 0x55bedbf8d5d0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 10 client.19089 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bedc017350 con 0x55bedbf8d5d0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee60036070 con 0x55bedbf8d5d0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee64004c00 con 0x55bedbf8d5d0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 10 client.19089 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 10 client.19089 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 20 client.19089 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedc017a60 0x55bedc019e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 4 client.19089 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.114+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bedbefad70 con 0x55bedc017a60 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee5ffff640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedc017a60 0x55bedc019e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee47fff640 10 client.19089 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee5ffff640 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedc017a60 0x55bedc019e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae54d91fcfa4e343 server_cookie=d336cf8d9a10d08b in_seq=0 out_seq=0 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee47fff640 1 -- 172.21.15.16:0/864847372 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bedbefad70 con 0x55bedc017a60 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee47fff640 10 client.19089 handle_command_reply: tid=0 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee71356740 1 client.19089 shutdown 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 msgr2=0x55bedbfa0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fee640117f0 tx=0x7fee6400fc10 comp rx=0 tx=0).stop 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 shutdown_connections 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.115+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55bedc017a60 0x55bedc019e50 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bedbf9c780 0x55bedc015d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bedbf8eb40 0x55bedbfa0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 --2- 172.21.15.16:0/864847372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bedbf8d5d0 0x55bedbfa0950 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 >> 172.21.15.16:0/864847372 conn(0x55bedbf56310 msgr2=0x55bedbf8f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee47fff640 0 client.19089 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 shutdown_connections 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 1 -- 172.21.15.16:0/864847372 wait complete. 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee71356740 20 client.19089 trim_cache size 0 max 16384 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 msgr2=0x7fee500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 0x7fee500a51b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fee60004ab0 tx=0x7fee600317a0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 msgr2=0x7fee681989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b4a0 tx=0x7fee5800b960 comp rx=0 tx=0).stop 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 shutdown_connections 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee500a2cf0 0x7fee500a51b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee6819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee68101420 0x7fee681989e0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 --2- 172.21.15.16:0/3604546559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee68100a80 0x7fee681984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 >> 172.21.15.16:0/3604546559 conn(0x7fee680fc7a0 msgr2=0x7fee680fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 shutdown_connections 2024-09-12T01:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:49.116+0000 7fee6ef7d640 1 -- 172.21.15.16:0/3604546559 wait complete. 2024-09-12T01:07:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:48 smithi099 ceph-mon[35654]: pgmap v363: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:49.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:07:50.138 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 -- 172.21.15.16:0/2720491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8102ef0 msgr2=0x7fd2f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2720491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8102ef0 0x7fd2f810da40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009ca0 tx=0x7fd2f002a950 comp rx=0 tx=0).stop 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 -- 172.21.15.16:0/2720491634 shutdown_connections 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2720491634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8102ef0 0x7fd2f810da40 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2720491634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2720491634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8100e20 0x7fd2f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 -- 172.21.15.16:0/2720491634 >> 172.21.15.16:0/2720491634 conn(0x7fd2f80fc7d0 msgr2=0x7fd2f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 -- 172.21.15.16:0/2720491634 shutdown_connections 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.653+0000 7fd2fd511640 1 -- 172.21.15.16:0/2720491634 wait complete. 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 Processor -- start 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 -- start start 2024-09-12T01:07:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8102ef0 0x7fd2f819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8111020 con 0x7fd2f8100e20 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8110ea0 con 0x7fd2f8102570 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2fd511640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81111a0 con 0x7fd2f8102ef0 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43878/0 (socket says 172.21.15.16:43878) 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2f6ffd640 1 -- 172.21.15.16:0/2322308563 learned_addr learned my addr 172.21.15.16:0/2322308563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.654+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8102ef0 0x7fd2f819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 -- 172.21.15.16:0/2322308563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8102ef0 msgr2=0x7fd2f819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8102ef0 0x7fd2f819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 -- 172.21.15.16:0/2322308563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 msgr2=0x7fd2f8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 -- 172.21.15.16:0/2322308563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f819d5b0 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800b4a0 tx=0x7fd2e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2effff640 1 -- 172.21.15.16:0/2322308563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8014070 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f819d8a0 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2effff640 1 -- 172.21.15.16:0/2322308563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e8010af0 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2effff640 1 -- 172.21.15.16:0/2322308563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8017d10 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.655+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f81a5180 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2effff640 1 -- 172.21.15.16:0/2322308563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd2e8017070 con 0x7fd2f8100e20 2024-09-12T01:07:50.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 Processor -- start 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 -- start start 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e21e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641a7d832f0 con 0x5641a7e2c780 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641a7d2acf0 con 0x5641a7e219e0 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641a7d7f2b0 con 0x5641a7e22f50 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.658+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43884/0 (socket says 172.21.15.16:43884) 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 -- 172.21.15.16:0/4068020161 learned_addr learned my addr 172.21.15.16:0/4068020161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e21e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2effff640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 0x7fd2e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 -- 172.21.15.16:0/4068020161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 msgr2=0x5641a7e214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 -- 172.21.15.16:0/4068020161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 msgr2=0x5641a7e21e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e21e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 -- 172.21.15.16:0/4068020161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a7d0f3a0 con 0x5641a7e2c780 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 0x7fd2e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2effff640 1 -- 172.21.15.16:0/2322308563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fd2e80ce770 con 0x7fd2f8100e20 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a9d0 tx=0x7fd2e000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e131d6c61953205d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2ed7fa640 1 -- 172.21.15.16:0/4068020161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0015070 con 0x5641a7e2c780 2024-09-12T01:07:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2ed7fa640 1 -- 172.21.15.16:0/4068020161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e000cc60 con 0x5641a7e2c780 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.659+0000 7fd2ed7fa640 1 -- 172.21.15.16:0/4068020161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00105b0 con 0x5641a7e2c780 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/4068020161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 msgr2=0x5641a7e2eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 0x7fd2e40a51b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009a60 tx=0x7fd2f00095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a9d0 tx=0x7fd2e000ae90 comp rx=0 tx=0).stop 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/4068020161 shutdown_connections 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e2eb70 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e21e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/4068020161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/4068020161 >> 172.21.15.16:0/4068020161 conn(0x5641a7de6310 msgr2=0x5641a7e1e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/4068020161 shutdown_connections 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.660+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/4068020161 wait complete. 2024-09-12T01:07:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 -- start start 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 Processor -- start 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 0x5641a7e298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5641a7e2f480 con 0x5641a7e22f50 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641a7d3e060 con 0x5641a7e219e0 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641a7e2f600 con 0x5641a7e2c780 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2ff8ea740 10 client.0 _ll_register_callbacks cb 0x5641a7a918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43900/0 (socket says 172.21.15.16:43900) 2024-09-12T01:07:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 -- 172.21.15.16:0/675761875 learned_addr learned my addr 172.21.15.16:0/675761875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 -- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 msgr2=0x5641a7e298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 0x5641a7e298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 0x5641a7e298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 -- 172.21.15.16:0/675761875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 msgr2=0x5641a7e2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f67fc640 1 -- 172.21.15.16:0/675761875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a7af43c0 con 0x5641a7e22f50 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.661+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 0x5641a7e298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0008010 tx=0x7fd2e0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bed1032e1d3333a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0020070 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fd2e001a450 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e000cc60 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2e0015040 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2ff8ea740 10 client.19101 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641a7e3ae90 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2e000cc60 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.662+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2e002c770 con 0x5641a7e22f50 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 10 client.19101 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 10 client.19101 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 20 client.19101 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7e3c290 0x5641a7e3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 4 client.19101 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a7d90860 con 0x5641a7e3c290 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7e3c290 0x5641a7e3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2de7fc640 10 client.19101 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.663+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7e3c290 0x5641a7e3e680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ee1b02bea64d8a4 server_cookie=2913012799163959 in_seq=0 out_seq=0 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2de7fc640 1 -- 172.21.15.16:0/675761875 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641a7d90860 con 0x5641a7e3c290 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2de7fc640 10 client.19101 handle_command_reply: tid=0 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 client.19101 shutdown 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 msgr2=0x5641a7e29390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0008010 tx=0x7fd2e0011ba0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 shutdown_connections 2024-09-12T01:07:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e2c780 0x5641a7e298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7e3c290 0x5641a7e3e680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e29390 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/675761875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 >> 172.21.15.16:0/675761875 conn(0x5641a7de6310 msgr2=0x5641a7e23d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2de7fc640 0 client.19101 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 shutdown_connections 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/675761875 wait complete. 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.664+0000 7fd2ff8ea740 20 client.19101 trim_cache size 0 max 16384 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.677+0000 7fd2ff8ea740 1 Processor -- start 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.677+0000 7fd2ff8ea740 1 -- start start 2024-09-12T01:07:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ab830 con 0x5641a7e2c780 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ad370 con 0x5641a7e219e0 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81abf00 con 0x5641a7e22f50 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43912/0 (socket says 172.21.15.16:43912) 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 -- 172.21.15.16:0/3812393637 learned_addr learned my addr 172.21.15.16:0/3812393637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 -- 172.21.15.16:0/3812393637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 msgr2=0x5641a7e28c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e28c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 -- 172.21.15.16:0/3812393637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 msgr2=0x5641a7e1e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e1e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 -- 172.21.15.16:0/3812393637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a7e28380 con 0x5641a7e2c780 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.678+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f003c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a85be549b10ed9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ecff9640 1 -- 172.21.15.16:0/3812393637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0054070 con 0x5641a7e2c780 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ecff9640 1 -- 172.21.15.16:0/3812393637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f0003c30 con 0x5641a7e2c780 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ecff9640 1 -- 172.21.15.16:0/3812393637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0034040 con 0x5641a7e2c780 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/3812393637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 msgr2=0x5641a7e38720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f003c030 comp rx=0 tx=0).stop 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/3812393637 shutdown_connections 2024-09-12T01:07:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e2c780 0x5641a7e38720 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e22f50 0x5641a7e28c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/3812393637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e219e0 0x5641a7e1e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/3812393637 >> 172.21.15.16:0/3812393637 conn(0x5641a7de6310 msgr2=0x5641a7e1db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/3812393637 shutdown_connections 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.679+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/3812393637 wait complete. 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 -- start start 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 Processor -- start 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e219e0 0x5641a7e34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e2c780 0x5641a7e25be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81acca0 con 0x5641a7e22f50 2024-09-12T01:07:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ac5d0 con 0x5641a7e2c780 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81a9830 con 0x5641a7e219e0 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2ff8ea740 10 client.0 _ll_register_callbacks cb 0x7fd2e000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43922/0 (socket says 172.21.15.16:43922) 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2f67fc640 1 -- 172.21.15.16:0/87548128 learned_addr learned my addr 172.21.15.16:0/87548128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e219e0 0x5641a7e34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.680+0000 7fd2f77fe640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e2c780 0x5641a7e25be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 -- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e219e0 msgr2=0x5641a7e34a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e219e0 0x5641a7e34a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 -- 172.21.15.16:0/87548128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e2c780 msgr2=0x5641a7e25be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e2c780 0x5641a7e25be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 -- 172.21.15.16:0/87548128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a7e26410 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2f67fc640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a760 tx=0x7fd2e0006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6e48bf331b7308 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0020040 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fd2e002c5f0 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f0003c30 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd2e002c830 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2ff8ea740 10 client.19107 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641a7ea6780 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f0003c30 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.681+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd2e0033060 con 0x5641a7e22f50 2024-09-12T01:07:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 10 client.19107 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 10 client.19107 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 20 client.19107 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7ea6e40 0x5641a7ea9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 4 client.19107 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641a7d8ad70 con 0x5641a7ea6e40 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7ea6e40 0x5641a7ea9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2de7fc640 10 client.19107 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2f6ffd640 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7ea6e40 0x5641a7ea9230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce3343345638aead server_cookie=c21d16d17f1924a9 in_seq=0 out_seq=0 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2de7fc640 1 -- 172.21.15.16:0/87548128 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641a7d8ad70 con 0x5641a7ea6e40 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2de7fc640 10 client.19107 handle_command_reply: tid=0 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.682+0000 7fd2ff8ea740 1 client.19107 shutdown 2024-09-12T01:07:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 msgr2=0x5641a7e34fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000a760 tx=0x7fd2e0006cf0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 shutdown_connections 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641a7e2c780 0x5641a7e25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5641a7ea6e40 0x5641a7ea9230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5641a7e22f50 0x5641a7e34fb0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 --2- 172.21.15.16:0/87548128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641a7e219e0 0x5641a7e34a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2de7fc640 0 client.19107 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:07:50.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 >> 172.21.15.16:0/87548128 conn(0x5641a7de6310 msgr2=0x5641a7e2b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 shutdown_connections 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 1 -- 172.21.15.16:0/87548128 wait complete. 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2ff8ea740 20 client.19107 trim_cache size 0 max 16384 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 msgr2=0x7fd2e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 0x7fd2e40a51b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009a60 tx=0x7fd2f00095c0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 msgr2=0x7fd2f8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800b4a0 tx=0x7fd2e800b960 comp rx=0 tx=0).stop 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 shutdown_connections 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd2e40a2cf0 0x7fd2e40a51b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.683+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2f8102ef0 0x7fd2f819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.684+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd2f8102570 0x7fd2f8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.684+0000 7fd2fd511640 1 --2- 172.21.15.16:0/2322308563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2f8100e20 0x7fd2f8198600 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.684+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 >> 172.21.15.16:0/2322308563 conn(0x7fd2f80fc7d0 msgr2=0x7fd2f8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.684+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 shutdown_connections 2024-09-12T01:07:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:50.684+0000 7fd2fd511640 1 -- 172.21.15.16:0/2322308563 wait complete. 2024-09-12T01:07:50.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:50 smithi016 ceph-mon[30972]: pgmap v364: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 19 op/s 2024-09-12T01:07:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:50 smithi159 ceph-mon[35566]: pgmap v364: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 19 op/s 2024-09-12T01:07:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:50 smithi099 ceph-mon[35654]: pgmap v364: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 19 op/s 2024-09-12T01:07:51.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:07:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 -- 172.21.15.16:0/2133809062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7941036c0 msgr2=0x7fa79410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/2133809062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7941036c0 0x7fa79410bbb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fa784009900 tx=0x7fa78402a480 comp rx=0 tx=0).stop 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 -- 172.21.15.16:0/2133809062 shutdown_connections 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/2133809062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79410c180 0x7fa79410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/2133809062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7941036c0 0x7fa79410bbb0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/2133809062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa794103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 -- 172.21.15.16:0/2133809062 >> 172.21.15.16:0/2133809062 conn(0x7fa7940fc7a0 msgr2=0x7fa7940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 -- 172.21.15.16:0/2133809062 shutdown_connections 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.137+0000 7fa79ad4d640 1 -- 172.21.15.16:0/2133809062 wait complete. 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 Processor -- start 2024-09-12T01:07:52.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 -- start start 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa79419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 0x7fa79419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa794111fb0 con 0x7fa79410c180 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa794111e30 con 0x7fa7941036c0 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa79ad4d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa794112130 con 0x7fa794102da0 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43936/0 (socket says 172.21.15.16:43936) 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 -- 172.21.15.16:0/465396475 learned_addr learned my addr 172.21.15.16:0/465396475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa78bfff640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 0x7fa79419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa798ae9640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa79419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 -- 172.21.15.16:0/465396475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 msgr2=0x7fa79419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa79419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 -- 172.21.15.16:0/465396475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 msgr2=0x7fa79419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 0x7fa79419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa7992ea640 1 -- 172.21.15.16:0/465396475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79419f590 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa798ae9640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa79419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.138+0000 7fa78bfff640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 0x7fa79419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa7992ea640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00cad0 tx=0x7fa78c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa789ffb640 1 -- 172.21.15.16:0/465396475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00ae20 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79419f880 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa789ffb640 1 -- 172.21.15.16:0/465396475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c005020 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa789ffb640 1 -- 172.21.15.16:0/465396475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f6f0 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.139+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7941a7160 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.141+0000 7fa789ffb640 1 -- 172.21.15.16:0/465396475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa78c0051d0 con 0x7fa79410c180 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 Processor -- start 2024-09-12T01:07:52.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 -- start start 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c7307090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c7314760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5618c726d2f0 con 0x5618c73075d0 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618c7214cf0 con 0x5618c7308b40 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa79d126740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618c72692b0 con 0x5618c7312370 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43938/0 (socket says 172.21.15.16:43938) 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 -- 172.21.15.16:0/2475431426 learned_addr learned my addr 172.21.15.16:0/2475431426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa789ffb640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 0x7fa77c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c7307090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c7314760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa7992ea640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 0x7fa77c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 -- 172.21.15.16:0/2475431426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 msgr2=0x5618c7314760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c7314760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 -- 172.21.15.16:0/2475431426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 msgr2=0x5618c7307090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c7307090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa798ae9640 1 -- 172.21.15.16:0/2475431426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c71f93a0 con 0x5618c73075d0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.142+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c7314760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa789ffb640 1 -- 172.21.15.16:0/465396475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fa78c0cf530 con 0x7fa79410c180 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa79000bc20 tx=0x7fa79000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27b4e15309c1556 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa77affd640 1 -- 172.21.15.16:0/2475431426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790017620 con 0x5618c73075d0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa77affd640 1 -- 172.21.15.16:0/2475431426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790017ca0 con 0x5618c73075d0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa77affd640 1 -- 172.21.15.16:0/2475431426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79000fcf0 con 0x5618c73075d0 2024-09-12T01:07:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa7992ea640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 0x7fa77c0a51b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0ccd30 tx=0x7fa78c06da10 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa79d126740 1 -- 172.21.15.16:0/2475431426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 msgr2=0x5618c7307a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.143+0000 7fa79d126740 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa79000bc20 tx=0x7fa79000d710 comp rx=0 tx=0).stop 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- 172.21.15.16:0/2475431426 shutdown_connections 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c7314760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7307a10 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- 172.21.15.16:0/2475431426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c7307090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- 172.21.15.16:0/2475431426 >> 172.21.15.16:0/2475431426 conn(0x5618c72d0310 msgr2=0x5618c7303c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- 172.21.15.16:0/2475431426 shutdown_connections 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- 172.21.15.16:0/2475431426 wait complete. 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- start start 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 Processor -- start 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c730f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5618c7315070 con 0x5618c73075d0 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618c7228060 con 0x5618c7308b40 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.144+0000 7fa79d126740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618c73151f0 con 0x5618c7312370 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa79d126740 10 client.0 _ll_register_callbacks cb 0x5618c6f7b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43942/0 (socket says 172.21.15.16:43942) 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 -- 172.21.15.16:0/2035008307 learned_addr learned my addr 172.21.15.16:0/2035008307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 -- 172.21.15.16:0/2035008307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 msgr2=0x5618c730f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c730f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 -- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 msgr2=0x5618c730efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 -- 172.21.15.16:0/2035008307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c6fde3c0 con 0x5618c73075d0 2024-09-12T01:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa784031550 tx=0x7fa7840038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e362d7f752dc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa773fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784035040 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fa784005020 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.145+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790017ca0 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa784005220 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 10 client.19119 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618c7320a40 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790017ca0 con 0x5618c73075d0 2024-09-12T01:07:52.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa78403a040 con 0x5618c73075d0 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 10 client.19119 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 10 client.19119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 20 client.19119 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c7321e50 0x5618c7324240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 4 client.19119 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c727a860 con 0x5618c7321e50 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.146+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c7321e50 0x5618c7324240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.147+0000 7fa773fff640 10 client.19119 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.147+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c7321e50 0x5618c7324240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9db9970d8ff44161 server_cookie=2f1310d15d1ec4a5 in_seq=0 out_seq=0 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.147+0000 7fa773fff640 1 -- 172.21.15.16:0/2035008307 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618c727a860 con 0x5618c7321e50 2024-09-12T01:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.147+0000 7fa773fff640 10 client.19119 handle_command_reply: tid=0 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.147+0000 7fa79d126740 1 client.19119 shutdown 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 msgr2=0x5618c7311d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa784031550 tx=0x7fa7840038b0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 shutdown_connections 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c7321e50 0x5618c7324240 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c7312370 0x5618c730f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa773fff640 0 client.19119 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 --2- 172.21.15.16:0/2035008307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c73075d0 0x5618c7311d30 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 >> 172.21.15.16:0/2035008307 conn(0x5618c72d0310 msgr2=0x5618c7309960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 shutdown_connections 2024-09-12T01:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 1 -- 172.21.15.16:0/2035008307 wait complete. 2024-09-12T01:07:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.148+0000 7fa79d126740 20 client.19119 trim_cache size 0 max 16384 2024-09-12T01:07:52.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 Processor -- start 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 -- start start 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c730a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941af9d0 con 0x5618c7312370 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941b1510 con 0x5618c7308b40 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa79d126740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941b00a0 con 0x5618c73075d0 2024-09-12T01:07:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43956/0 (socket says 172.21.15.16:43956) 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa78bfff640 1 -- 172.21.15.16:0/2366172404 learned_addr learned my addr 172.21.15.16:0/2366172404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c730a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.161+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 -- 172.21.15.16:0/2366172404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 msgr2=0x5618c730a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c730a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 -- 172.21.15.16:0/2366172404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 msgr2=0x5618c730a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 -- 172.21.15.16:0/2366172404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c73248a0 con 0x5618c7312370 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa798ae9640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa78bfff640 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa784003650 tx=0x7fa78403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d20175d18b074 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa77a7fc640 1 -- 172.21.15.16:0/2366172404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784035030 con 0x5618c7312370 2024-09-12T01:07:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa77a7fc640 1 -- 172.21.15.16:0/2366172404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784031e40 con 0x5618c7312370 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.162+0000 7fa77a7fc640 1 -- 172.21.15.16:0/2366172404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784004a00 con 0x5618c7312370 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- 172.21.15.16:0/2366172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 msgr2=0x5618c7329af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa784003650 tx=0x7fa78403f040 comp rx=0 tx=0).stop 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- 172.21.15.16:0/2366172404 shutdown_connections 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7329af0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c730a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 --2- 172.21.15.16:0/2366172404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c730a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- 172.21.15.16:0/2366172404 >> 172.21.15.16:0/2366172404 conn(0x5618c72d0310 msgr2=0x5618c7302d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- 172.21.15.16:0/2366172404 shutdown_connections 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- 172.21.15.16:0/2366172404 wait complete. 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 -- start start 2024-09-12T01:07:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 Processor -- start 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.163+0000 7fa79d126740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c731a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c731a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941b0e40 con 0x5618c7312370 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941b0770 con 0x5618c7308b40 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7941ad9d0 con 0x5618c73075d0 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 10 client.0 _ll_register_callbacks cb 0x7fa784009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43968/0 (socket says 172.21.15.16:43968) 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 -- 172.21.15.16:0/3015451607 learned_addr learned my addr 172.21.15.16:0/3015451607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa798ae9640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c731a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c731a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 -- 172.21.15.16:0/3015451607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 msgr2=0x5618c731a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c731a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 -- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 msgr2=0x5618c731a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c731a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 -- 172.21.15.16:0/3015451607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c731b100 con 0x5618c7312370 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa798ae9640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c731a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa78bfff640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa784031550 tx=0x7fa784032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab71ff06e4770a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa773fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784064060 con 0x5618c7312370 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fa7840055c0 con 0x5618c7312370 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.164+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618c7274d70 con 0x5618c7312370 2024-09-12T01:07:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa784035070 con 0x5618c7312370 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 10 client.19125 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5618c738d0d0 con 0x5618c7312370 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5618c7274d70 con 0x5618c7312370 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa784004e10 con 0x5618c7312370 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 10 client.19125 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 10 client.19125 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 20 client.19125 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c738d6d0 0x5618c738faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 4 client.19125 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618c71f1170 con 0x5618c738d6d0 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.165+0000 7fa798ae9640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c738d6d0 0x5618c738faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa773fff640 10 client.19125 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa798ae9640 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c738d6d0 0x5618c738faf0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f32decae4b678185 server_cookie=908209db50a48bb9 in_seq=0 out_seq=0 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa773fff640 1 -- 172.21.15.16:0/3015451607 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618c71f1170 con 0x5618c738d6d0 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa773fff640 10 client.19125 handle_command_reply: tid=0 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 client.19125 shutdown 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 msgr2=0x5618c7317450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa784031550 tx=0x7fa784032f80 comp rx=0 tx=0).stop 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 shutdown_connections 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5618c738d6d0 0x5618c738faf0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5618c7312370 0x5618c7317450 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa773fff640 0 client.19125 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618c7308b40 0x5618c731a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 --2- 172.21.15.16:0/3015451607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c73075d0 0x5618c731a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 >> 172.21.15.16:0/3015451607 conn(0x5618c72d0310 msgr2=0x5618c7310840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 shutdown_connections 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.166+0000 7fa79d126740 1 -- 172.21.15.16:0/3015451607 wait complete. 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79d126740 20 client.19125 trim_cache size 0 max 16384 2024-09-12T01:07:52.175 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 msgr2=0x7fa77c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 0x7fa77c0a51b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0ccd30 tx=0x7fa78c06da10 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 msgr2=0x7fa79419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00cad0 tx=0x7fa78c00cf90 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 shutdown_connections 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa77c0a2cf0 0x7fa77c0a51b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79410c180 0x7fa79419eeb0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7941036c0 0x7fa79419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 --2- 172.21.15.16:0/465396475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa794102da0 0x7fa79419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:52.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 >> 172.21.15.16:0/465396475 conn(0x7fa7940fc7a0 msgr2=0x7fa794100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:52.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 shutdown_connections 2024-09-12T01:07:52.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:52.167+0000 7fa79ad4d640 1 -- 172.21.15.16:0/465396475 wait complete. 2024-09-12T01:07:52.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:07:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:53.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:52 smithi159 ceph-mon[35566]: pgmap v365: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:53.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:52 smithi016 ceph-mon[30972]: pgmap v365: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:53.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:52 smithi099 ceph-mon[35654]: pgmap v365: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 -- 172.21.15.16:0/2939335970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28102790 msgr2=0x7f1e281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2939335970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28102790 0x7f1e281067e0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e1802a410 comp rx=0 tx=0).stop 2024-09-12T01:07:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 -- 172.21.15.16:0/2939335970 shutdown_connections 2024-09-12T01:07:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2939335970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28106f10 0x7f1e2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2939335970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28102790 0x7f1e281067e0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2939335970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28101de0 0x7f1e281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 -- 172.21.15.16:0/2939335970 >> 172.21.15.16:0/2939335970 conn(0x7f1e280fd8c0 msgr2=0x7f1e280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 -- 172.21.15.16:0/2939335970 shutdown_connections 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.597+0000 7f1e2f292640 1 -- 172.21.15.16:0/2939335970 wait complete. 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 Processor -- start 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 -- start start 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28101de0 0x7f1e280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28102790 0x7f1e28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28111f70 con 0x7f1e28106f10 2024-09-12T01:07:53.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28111df0 con 0x7f1e28102790 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2f292640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281120f0 con 0x7f1e28101de0 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43988/0 (socket says 172.21.15.16:43988) 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 -- 172.21.15.16:0/2925008602 learned_addr learned my addr 172.21.15.16:0/2925008602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28101de0 0x7f1e280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28102790 0x7f1e28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 -- 172.21.15.16:0/2925008602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28101de0 msgr2=0x7f1e280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28101de0 0x7f1e280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 -- 172.21.15.16:0/2925008602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28102790 msgr2=0x7f1e28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28102790 0x7f1e28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.598+0000 7f1e2d82f640 1 -- 172.21.15.16:0/2925008602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e281a6870 con 0x7f1e28106f10 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e2d82f640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000cce0 tx=0x7f1e2000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e1e7fc640 1 -- 172.21.15.16:0/2925008602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000f450 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e281a6b60 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e1e7fc640 1 -- 172.21.15.16:0/2925008602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e2000fad0 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e1e7fc640 1 -- 172.21.15.16:0/2925008602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20018c10 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.599+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281a7070 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e1e7fc640 1 -- 172.21.15.16:0/2925008602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1e2000f5f0 con 0x7f1e28106f10 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 Processor -- start 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 -- start start 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8eab2f0 con 0x55b1d8f50370 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8e52cf0 con 0x55b1d8f455d0 2024-09-12T01:07:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e3166b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8ea72b0 con 0x55b1d8f46b40 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44004/0 (socket says 172.21.15.16:44004) 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e2c82d640 1 -- 172.21.15.16:0/3810606923 learned_addr learned my addr 172.21.15.16:0/3810606923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.602+0000 7f1e1e7fc640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 0x7f1e140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 -- 172.21.15.16:0/3810606923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 msgr2=0x55b1d8f45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f45090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 -- 172.21.15.16:0/3810606923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 msgr2=0x55b1d8f45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 -- 172.21.15.16:0/3810606923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d8e373a0 con 0x55b1d8f50370 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2d82f640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 0x7f1e140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e1e7fc640 1 -- 172.21.15.16:0/2925008602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f1e200cfa20 con 0x7f1e28106f10 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e180314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1099b139f21e3b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e137fe640 1 -- 172.21.15.16:0/3810606923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035070 con 0x55b1d8f50370 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e137fe640 1 -- 172.21.15.16:0/3810606923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18003ad0 con 0x55b1d8f50370 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.603+0000 7f1e137fe640 1 -- 172.21.15.16:0/3810606923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1803d7b0 con 0x55b1d8f50370 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- 172.21.15.16:0/3810606923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 msgr2=0x55b1d8f52760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e180314d0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e2d82f640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 0x7f1e140a51b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e200cd220 tx=0x7f1e2006ae70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- 172.21.15.16:0/3810606923 shutdown_connections 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f50370 0x55b1d8f52760 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 --2- 172.21.15.16:0/3810606923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- 172.21.15.16:0/3810606923 >> 172.21.15.16:0/3810606923 conn(0x55b1d8f0e310 msgr2=0x55b1d8f41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- 172.21.15.16:0/3810606923 shutdown_connections 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- 172.21.15.16:0/3810606923 wait complete. 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 -- start start 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.604+0000 7f1e3166b740 1 Processor -- start 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8f53010 con 0x55b1d8f46b40 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8e66060 con 0x55b1d8f455d0 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1d8f53190 con 0x55b1d8f50370 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e3166b740 10 client.0 _ll_register_callbacks cb 0x55b1d8bb98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44018/0 (socket says 172.21.15.16:44018) 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 -- 172.21.15.16:0/1122708580 learned_addr learned my addr 172.21.15.16:0/1122708580 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 -- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 msgr2=0x55b1d8f4d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 -- 172.21.15.16:0/1122708580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 msgr2=0x55b1d8f4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 -- 172.21.15.16:0/1122708580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d8c1c3c0 con 0x55b1d8f46b40 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1e24010820 tx=0x7f1e24010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6ccc2cb9ee7a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.605+0000 7f1e10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400fb90 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f1e2400fd30 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e18003ad0 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e24005020 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 10 client.19137 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1d8f5eb60 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18003ad0 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e24019410 con 0x55b1d8f46b40 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 10 client.19137 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 10 client.19137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 20 client.19137 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8f600c0 0x55b1d8f624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 4 client.19137 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.606+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1d8eb8860 con 0x55b1d8f600c0 2024-09-12T01:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.607+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8f600c0 0x55b1d8f624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.607+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8f600c0 0x55b1d8f624e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29ffc90b87d1ae34 server_cookie=fbaf5f237e647f10 in_seq=0 out_seq=0 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.607+0000 7f1e10ff9640 10 client.19137 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e10ff9640 1 -- 172.21.15.16:0/1122708580 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1d8eb8860 con 0x55b1d8f600c0 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e10ff9640 10 client.19137 handle_command_reply: tid=0 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 client.19137 shutdown 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 msgr2=0x55b1d8f4cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1e24010820 tx=0x7f1e24010ce0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 shutdown_connections 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4cff0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8f600c0 0x55b1d8f624e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1122708580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 >> 172.21.15.16:0/1122708580 conn(0x55b1d8f0e310 msgr2=0x55b1d8f47960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e10ff9640 0 client.19137 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 shutdown_connections 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 1 -- 172.21.15.16:0/1122708580 wait complete. 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.608+0000 7f1e3166b740 20 client.19137 trim_cache size 0 max 16384 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 Processor -- start 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 -- start start 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f50370 0x55b1d8f67db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281af8c0 con 0x55b1d8f455d0 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281b1400 con 0x55b1d8f50370 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e3166b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281aff90 con 0x55b1d8f46b40 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44030/0 (socket says 172.21.15.16:44030) 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 -- 172.21.15.16:0/1596769479 learned_addr learned my addr 172.21.15.16:0/1596769479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 -- 172.21.15.16:0/1596769479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 msgr2=0x55b1d8f5c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f5c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 -- 172.21.15.16:0/1596769479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f50370 msgr2=0x55b1d8f67db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f50370 0x55b1d8f67db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 -- 172.21.15.16:0/1596769479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d8f4bfc0 con 0x55b1d8f455d0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.622+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e18031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e86ea819d02199c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e12ffd640 1 -- 172.21.15.16:0/1596769479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035040 con 0x55b1d8f455d0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e12ffd640 1 -- 172.21.15.16:0/1596769479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18009520 con 0x55b1d8f455d0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e12ffd640 1 -- 172.21.15.16:0/1596769479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1803db20 con 0x55b1d8f455d0 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 -- 172.21.15.16:0/1596769479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 msgr2=0x55b1d8f4cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009900 tx=0x7f1e18031a10 comp rx=0 tx=0).stop 2024-09-12T01:07:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 -- 172.21.15.16:0/1596769479 shutdown_connections 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f50370 0x55b1d8f67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f46b40 0x55b1d8f5c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 --2- 172.21.15.16:0/1596769479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4cfc0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 -- 172.21.15.16:0/1596769479 >> 172.21.15.16:0/1596769479 conn(0x55b1d8f0e310 msgr2=0x55b1d8f4c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 -- 172.21.15.16:0/1596769479 shutdown_connections 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.623+0000 7f1e3166b740 1 -- 172.21.15.16:0/1596769479 wait complete. 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 -- start start 2024-09-12T01:07:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 Processor -- start 2024-09-12T01:07:53.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281b0d30 con 0x55b1d8f455d0 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281b0660 con 0x55b1d8f46b40 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e281ad8c0 con 0x55b1d8f50370 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e3166b740 10 client.0 _ll_register_callbacks cb 0x7f1e2400a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e2c82d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44032/0 (socket says 172.21.15.16:44032) 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.624+0000 7f1e2c82d640 1 -- 172.21.15.16:0/203875925 learned_addr learned my addr 172.21.15.16:0/203875925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 -- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 msgr2=0x55b1d8f4aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 -- 172.21.15.16:0/203875925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f46b40 msgr2=0x55b1d8f4a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 -- 172.21.15.16:0/203875925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d8fc9d80 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e2c82d640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1e180314d0 tx=0x7f1e1803d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15eb5666c33e06e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035050 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7f1e18005220 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e18009520 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e18065050 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e3166b740 10 client.19143 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.625+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1d8fcb060 con 0x55b1d8f455d0 2024-09-12T01:07:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e18009520 con 0x55b1d8f455d0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e1803d810 con 0x55b1d8f455d0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 10 client.19143 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 10 client.19143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 20 client.19143 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8fcb2e0 0x55b1d8fcd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 4 client.19143 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1d8eb2d70 con 0x55b1d8fcb2e0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.626+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8fcb2e0 0x55b1d8fcd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e2d02e640 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8fcb2e0 0x55b1d8fcd6d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f7162859c32753c server_cookie=d012e5b2d10464aa in_seq=0 out_seq=0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e10ff9640 10 client.19143 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e10ff9640 1 -- 172.21.15.16:0/203875925 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1d8eb2d70 con 0x55b1d8fcb2e0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e10ff9640 10 client.19143 handle_command_reply: tid=0 2024-09-12T01:07:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 client.19143 shutdown 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 msgr2=0x55b1d8f4a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1e180314d0 tx=0x7f1e1803d700 comp rx=0 tx=0).stop 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 shutdown_connections 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1d8f50370 0x55b1d8f4aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55b1d8fcb2e0 0x55b1d8fcd6d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1d8f46b40 0x55b1d8f4a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 --2- 172.21.15.16:0/203875925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1d8f455d0 0x55b1d8f4a030 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 >> 172.21.15.16:0/203875925 conn(0x55b1d8f0e310 msgr2=0x55b1d8f4e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e10ff9640 0 client.19143 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 shutdown_connections 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 1 -- 172.21.15.16:0/203875925 wait complete. 2024-09-12T01:07:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.627+0000 7f1e3166b740 20 client.19143 trim_cache size 0 max 16384 2024-09-12T01:07:53.642 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 msgr2=0x7f1e140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 0x7f1e140a51b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1e200cd220 tx=0x7f1e2006ae70 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 msgr2=0x7f1e28074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000cce0 tx=0x7f1e2000a620 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 shutdown_connections 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1e140a2cf0 0x7f1e140a51b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e28106f10 0x7f1e28074f70 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e28102790 0x7f1e28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 --2- 172.21.15.16:0/2925008602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e28101de0 0x7f1e280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 >> 172.21.15.16:0/2925008602 conn(0x7f1e280fd8c0 msgr2=0x7f1e280fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 shutdown_connections 2024-09-12T01:07:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:53.628+0000 7f1e2f292640 1 -- 172.21.15.16:0/2925008602 wait complete. 2024-09-12T01:07:54.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:07:54.681 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:55.079 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:54 smithi016 ceph-mon[30972]: pgmap v366: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:54 smithi159 ceph-mon[35566]: pgmap v366: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 -- 172.21.15.16:0/3997728610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c102790 msgr2=0x7fc35c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 --2- 172.21.15.16:0/3997728610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c102790 0x7fc35c1067e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009900 tx=0x7fc34c02a410 comp rx=0 tx=0).stop 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 -- 172.21.15.16:0/3997728610 shutdown_connections 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 --2- 172.21.15.16:0/3997728610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c106f10 0x7fc35c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 --2- 172.21.15.16:0/3997728610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c102790 0x7fc35c1067e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 --2- 172.21.15.16:0/3997728610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c101de0 0x7fc35c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 -- 172.21.15.16:0/3997728610 >> 172.21.15.16:0/3997728610 conn(0x7fc35c0fd8c0 msgr2=0x7fc35c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 -- 172.21.15.16:0/3997728610 shutdown_connections 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.165+0000 7fc363841640 1 -- 172.21.15.16:0/3997728610 wait complete. 2024-09-12T01:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 Processor -- start 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 -- start start 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c102790 0x7fc35c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 0x7fc35c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111f70 con 0x7fc35c101de0 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c111df0 con 0x7fc35c106f10 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc363841640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1120f0 con 0x7fc35c102790 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44038/0 (socket says 172.21.15.16:44038) 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 -- 172.21.15.16:0/826633126 learned_addr learned my addr 172.21.15.16:0/826633126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc361dde640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 0x7fc35c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 -- 172.21.15.16:0/826633126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c102790 msgr2=0x7fc35c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc360ddc640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c102790 0x7fc35c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c102790 0x7fc35c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 -- 172.21.15.16:0/826633126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 msgr2=0x7fc35c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 0x7fc35c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc3615dd640 1 -- 172.21.15.16:0/826633126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c1a6820 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.166+0000 7fc361dde640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 0x7fc35c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc3615dd640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c920 tx=0x7fc35800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc3527fc640 1 -- 172.21.15.16:0/826633126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358007c70 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c1a6b10 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc3527fc640 1 -- 172.21.15.16:0/826633126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc358007e10 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc3527fc640 1 -- 172.21.15.16:0/826633126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358018890 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.167+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a7050 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.169+0000 7fc3527fc640 1 -- 172.21.15.16:0/826633126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc358018a70 con 0x7fc35c101de0 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 Processor -- start 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 -- start start 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc611090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc611a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc5772f0 con 0x55a6fc61c370 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc51ecf0 con 0x55a6fc612b40 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc365c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc5732b0 con 0x55a6fc6115d0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc3527fc640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 0x7fc3480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc360ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc361dde640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 0x7fc3480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc360ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44046/0 (socket says 172.21.15.16:44046) 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc3615dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc611a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.170+0000 7fc360ddc640 1 -- 172.21.15.16:0/1956481329 learned_addr learned my addr 172.21.15.16:0/1956481329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc3527fc640 1 -- 172.21.15.16:0/826633126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fc3580cf650 con 0x7fc35c101de0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc361dde640 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc611090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 -- 172.21.15.16:0/1956481329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 msgr2=0x55a6fc611a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc611a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 -- 172.21.15.16:0/1956481329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 msgr2=0x55a6fc611090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc611090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 -- 172.21.15.16:0/1956481329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fc5033a0 con 0x55a6fc61c370 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc3615dd640 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc361dde640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 0x7fc3480a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc354005c40 tx=0x7fc354009d50 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc360ddc640 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006010 tx=0x7fc34c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1648ae35a2291b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc3477fe640 1 -- 172.21.15.16:0/1956481329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c03f070 con 0x55a6fc61c370 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc3477fe640 1 -- 172.21.15.16:0/1956481329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c004030 con 0x55a6fc61c370 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.171+0000 7fc3477fe640 1 -- 172.21.15.16:0/1956481329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c035070 con 0x55a6fc61c370 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- 172.21.15.16:0/1956481329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 msgr2=0x55a6fc61e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc34c006010 tx=0x7fc34c005e80 comp rx=0 tx=0).stop 2024-09-12T01:07:55.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- 172.21.15.16:0/1956481329 shutdown_connections 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61e760 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1956481329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc611090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- 172.21.15.16:0/1956481329 >> 172.21.15.16:0/1956481329 conn(0x55a6fc5da310 msgr2=0x55a6fc60dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- 172.21.15.16:0/1956481329 shutdown_connections 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- 172.21.15.16:0/1956481329 wait complete. 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 -- start start 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.172+0000 7fc365c1a740 1 Processor -- start 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc612b40 0x55a6fc618f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc61c370 0x55a6fc619490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc61f010 con 0x55a6fc6115d0 2024-09-12T01:07:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc532060 con 0x55a6fc61c370 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a6fc61f190 con 0x55a6fc612b40 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc365c1a740 10 client.0 _ll_register_callbacks cb 0x55a6fc2858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44062/0 (socket says 172.21.15.16:44062) 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 -- 172.21.15.16:0/883815330 learned_addr learned my addr 172.21.15.16:0/883815330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc361dde640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc61c370 0x55a6fc619490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc360ddc640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc612b40 0x55a6fc618f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 -- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc612b40 msgr2=0x55a6fc618f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc612b40 0x55a6fc618f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 -- 172.21.15.16:0/883815330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc61c370 msgr2=0x55a6fc619490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc61c370 0x55a6fc619490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 -- 172.21.15.16:0/883815330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fc2e83c0 con 0x55a6fc6115d0 2024-09-12T01:07:55.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc3615dd640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc3580cd3a0 tx=0x7fc358015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b087455cbf9b6626 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.173+0000 7fc344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35809bb70 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fc35801dd40 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34c004030 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc35809ff20 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 10 client.19155 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6fc62aab0 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc34c004030 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3580662e0 con 0x55a6fc6115d0 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 10 client.19155 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 10 client.19155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 20 client.19155 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc62bf70 0x55a6fc62e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 4 client.19155 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.174+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6fc584860 con 0x55a6fc62bf70 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.175+0000 7fc360ddc640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc62bf70 0x55a6fc62e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.175+0000 7fc360ddc640 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc62bf70 0x55a6fc62e3c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5387c9b8a1b02a6 server_cookie=7d91b02a594474d5 in_seq=0 out_seq=0 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.175+0000 7fc344ff9640 10 client.19155 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc344ff9640 1 -- 172.21.15.16:0/883815330 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6fc584860 con 0x55a6fc62bf70 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc344ff9640 10 client.19155 handle_command_reply: tid=0 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 client.19155 shutdown 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 msgr2=0x55a6fc61bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fc3580cd3a0 tx=0x7fc358015aa0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 shutdown_connections 2024-09-12T01:07:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc62bf70 0x55a6fc62e3c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc61c370 0x55a6fc619490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc612b40 0x55a6fc618f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc344ff9640 0 client.19155 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 --2- 172.21.15.16:0/883815330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc6115d0 0x55a6fc61bca0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 >> 172.21.15.16:0/883815330 conn(0x55a6fc5da310 msgr2=0x55a6fc613960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 shutdown_connections 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 1 -- 172.21.15.16:0/883815330 wait complete. 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.176+0000 7fc365c1a740 20 client.19155 trim_cache size 0 max 16384 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 Processor -- start 2024-09-12T01:07:55.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 -- start start 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc6282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc60d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1af820 con 0x55a6fc61c370 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b1360 con 0x55a6fc612b40 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.189+0000 7fc365c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1afef0 con 0x55a6fc6115d0 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44074/0 (socket says 172.21.15.16:44074) 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 -- 172.21.15.16:0/1448153396 learned_addr learned my addr 172.21.15.16:0/1448153396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc3615dd640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc6282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc360ddc640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc60d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 -- 172.21.15.16:0/1448153396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 msgr2=0x55a6fc6282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc6282b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 -- 172.21.15.16:0/1448153396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 msgr2=0x55a6fc60d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc60d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 -- 172.21.15.16:0/1448153396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fc60ded0 con 0x55a6fc61c370 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc3615dd640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc6282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc361dde640 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc3540097c0 tx=0x7fc354004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd1bc95f223b2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc346ffd640 1 -- 172.21.15.16:0/1448153396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354004b20 con 0x55a6fc61c370 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc346ffd640 1 -- 172.21.15.16:0/1448153396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc354004cc0 con 0x55a6fc61c370 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.190+0000 7fc346ffd640 1 -- 172.21.15.16:0/1448153396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354005790 con 0x55a6fc61c370 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- 172.21.15.16:0/1448153396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 msgr2=0x55a6fc61d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc3540097c0 tx=0x7fc354004ab0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- 172.21.15.16:0/1448153396 shutdown_connections 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc61c370 0x55a6fc61d760 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc612b40 0x55a6fc60d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 --2- 172.21.15.16:0/1448153396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc6115d0 0x55a6fc6282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- 172.21.15.16:0/1448153396 >> 172.21.15.16:0/1448153396 conn(0x55a6fc5da310 msgr2=0x55a6fc617820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- 172.21.15.16:0/1448153396 shutdown_connections 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- 172.21.15.16:0/1448153396 wait complete. 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 -- start start 2024-09-12T01:07:55.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.191+0000 7fc365c1a740 1 Processor -- start 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc6115d0 0x55a6fc624420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc61c370 0x55a6fc615870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b0c90 con 0x55a6fc612b40 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1b05c0 con 0x55a6fc6115d0 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c1ad820 con 0x55a6fc61c370 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc365c1a740 10 client.0 _ll_register_callbacks cb 0x7fc35809b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44080/0 (socket says 172.21.15.16:44080) 2024-09-12T01:07:55.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 -- 172.21.15.16:0/178615423 learned_addr learned my addr 172.21.15.16:0/178615423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc3615dd640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc6115d0 0x55a6fc624420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc361dde640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc61c370 0x55a6fc615870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 -- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc61c370 msgr2=0x55a6fc615870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc61c370 0x55a6fc615870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 -- 172.21.15.16:0/178615423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc6115d0 msgr2=0x55a6fc624420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc6115d0 0x55a6fc624420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.192+0000 7fc360ddc640 1 -- 172.21.15.16:0/178615423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fc616e00 con 0x55a6fc612b40 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc360ddc640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02a450 tx=0x7fc34c02a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34daa13427b0ee8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c035040 con 0x55a6fc612b40 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fc34c0317b0 con 0x55a6fc612b40 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc354004cc0 con 0x55a6fc612b40 2024-09-12T01:07:55.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc34c03f040 con 0x55a6fc612b40 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 10 client.19161 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a6fc697030 con 0x55a6fc612b40 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc354004cc0 con 0x55a6fc612b40 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc34c04d040 con 0x55a6fc612b40 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 10 client.19161 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 10 client.19161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 20 client.19161 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc697300 0x55a6fc6996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 4 client.19161 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.193+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6fc57ed70 con 0x55a6fc697300 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc3615dd640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc697300 0x55a6fc6996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc3615dd640 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc697300 0x55a6fc6996f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854301a1cf36716b server_cookie=81e9061dcb7f2dd3 in_seq=0 out_seq=0 2024-09-12T01:07:55.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc344ff9640 10 client.19161 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc344ff9640 1 -- 172.21.15.16:0/178615423 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6fc57ed70 con 0x55a6fc697300 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc344ff9640 10 client.19161 handle_command_reply: tid=0 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 client.19161 shutdown 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 msgr2=0x55a6fc624960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc34c02a450 tx=0x7fc34c02a9b0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 shutdown_connections 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55a6fc697300 0x55a6fc6996f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.194+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a6fc61c370 0x55a6fc615870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6fc612b40 0x55a6fc624960 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 1 --2- 172.21.15.16:0/178615423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a6fc6115d0 0x55a6fc624420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc344ff9640 0 client.19161 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 >> 172.21.15.16:0/178615423 conn(0x55a6fc5da310 msgr2=0x55a6fc60c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 shutdown_connections 2024-09-12T01:07:55.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 1 -- 172.21.15.16:0/178615423 wait complete. 2024-09-12T01:07:55.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc365c1a740 20 client.19161 trim_cache size 0 max 16384 2024-09-12T01:07:55.210 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 msgr2=0x7fc3480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 0x7fc3480a51b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc354005c40 tx=0x7fc354009d50 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 msgr2=0x7fc35c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c920 tx=0x7fc35800cde0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 shutdown_connections 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3480a2cf0 0x7fc3480a51b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc35c106f10 0x7fc35c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc35c102790 0x7fc35c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 --2- 172.21.15.16:0/826633126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c101de0 0x7fc35c0744f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 >> 172.21.15.16:0/826633126 conn(0x7fc35c0fd8c0 msgr2=0x7fc35c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:55.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 shutdown_connections 2024-09-12T01:07:55.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:55.195+0000 7fc363841640 1 -- 172.21.15.16:0/826633126 wait complete. 2024-09-12T01:07:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:54 smithi099 ceph-mon[35654]: pgmap v366: 97 pgs: 97 active+clean; 17 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-12T01:07:56.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:07:56.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:07:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:56 smithi159 ceph-mon[35566]: pgmap v367: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-12T01:07:57.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:56 smithi016 ceph-mon[30972]: pgmap v367: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-12T01:07:57.151 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:56 smithi099 ceph-mon[35654]: pgmap v367: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 73 KiB/s wr, 19 op/s 2024-09-12T01:07:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.634+0000 7ff255fe0640 1 -- 172.21.15.16:0/2933927467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250101de0 msgr2=0x7ff2501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.634+0000 7ff255fe0640 1 --2- 172.21.15.16:0/2933927467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250101de0 0x7ff2501021c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff240009900 tx=0x7ff24002a410 comp rx=0 tx=0).stop 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 -- 172.21.15.16:0/2933927467 shutdown_connections 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 --2- 172.21.15.16:0/2933927467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250106f10 0x7ff25010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 --2- 172.21.15.16:0/2933927467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250102790 0x7ff2501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 --2- 172.21.15.16:0/2933927467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250101de0 0x7ff2501021c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 -- 172.21.15.16:0/2933927467 >> 172.21.15.16:0/2933927467 conn(0x7ff2500fd8c0 msgr2=0x7ff2500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 -- 172.21.15.16:0/2933927467 shutdown_connections 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 -- 172.21.15.16:0/2933927467 wait complete. 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 Processor -- start 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.635+0000 7ff255fe0640 1 -- start start 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 0x7ff250198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250106f10 0x7ff25019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff250111f70 con 0x7ff250102790 2024-09-12T01:07:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff250111df0 con 0x7ff250101de0 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501120f0 con 0x7ff250106f10 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44108/0 (socket says 172.21.15.16:44108) 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 -- 172.21.15.16:0/1428748547 learned_addr learned my addr 172.21.15.16:0/1428748547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24ffff640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250106f10 0x7ff25019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 0x7ff250198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 -- 172.21.15.16:0/1428748547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250106f10 msgr2=0x7ff25019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250106f10 0x7ff25019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 -- 172.21.15.16:0/1428748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 msgr2=0x7ff250198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 0x7ff250198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 -- 172.21.15.16:0/1428748547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25019d3e0 con 0x7ff250102790 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 0x7ff250198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff246ffd640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff23800d770 tx=0x7ff23800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24d7fa640 1 -- 172.21.15.16:0/1428748547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238017070 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24d7fa640 1 -- 172.21.15.16:0/1428748547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff238005500 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25019d6d0 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.636+0000 7ff24d7fa640 1 -- 172.21.15.16:0/1428748547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238004e00 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.637+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a4fb0 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff24d7fa640 1 -- 172.21.15.16:0/1428748547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff238004fa0 con 0x7ff250102790 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 Processor -- start 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 -- start start 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14645e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d14657540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559d145ab280 con 0x559d14646360 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d14552b10 con 0x559d14655150 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff2583b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559d145a7240 con 0x559d14645a40 2024-09-12T01:07:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff24f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff24f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44116/0 (socket says 172.21.15.16:44116) 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.639+0000 7ff24f7fe640 1 -- 172.21.15.16:0/1609846435 learned_addr learned my addr 172.21.15.16:0/1609846435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff246ffd640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d14657540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24ffff640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14645e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24d7fa640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 0x7ff23c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 -- 172.21.15.16:0/1609846435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 msgr2=0x559d14645e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14645e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 -- 172.21.15.16:0/1609846435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 msgr2=0x559d14657540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d14657540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 -- 172.21.15.16:0/1609846435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d1455f1f0 con 0x559d14646360 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff246ffd640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d14657540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24ffff640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 0x7ff23c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24d7fa640 1 -- 172.21.15.16:0/1428748547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff2380ceed0 con 0x7ff250102790 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff240031550 tx=0x7ff240032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eea46b4b1e92bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff245ffb640 1 -- 172.21.15.16:0/1609846435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24003f070 con 0x559d14646360 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff245ffb640 1 -- 172.21.15.16:0/1609846435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2400317b0 con 0x559d14646360 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.640+0000 7ff245ffb640 1 -- 172.21.15.16:0/1609846435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240035070 con 0x559d14646360 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- 172.21.15.16:0/1609846435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 msgr2=0x559d14654c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff240031550 tx=0x7ff240032cd0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff24ffff640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 0x7ff23c0a5230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff248008d40 tx=0x7ff248004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- 172.21.15.16:0/1609846435 shutdown_connections 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d14657540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14654c10 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 --2- 172.21.15.16:0/1609846435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14645e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- 172.21.15.16:0/1609846435 >> 172.21.15.16:0/1609846435 conn(0x559d146420f0 msgr2=0x559d146424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- 172.21.15.16:0/1609846435 shutdown_connections 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- 172.21.15.16:0/1609846435 wait complete. 2024-09-12T01:07:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.641+0000 7ff2583b9740 1 -- start start 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 Processor -- start 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14652020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 0x559d14652aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559d14657d20 con 0x559d14646360 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d14565f10 con 0x559d14645a40 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559d14657ea0 con 0x559d14655150 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff2583b9740 10 client.0 _ll_register_callbacks cb 0x559d1452ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44132/0 (socket says 172.21.15.16:44132) 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 -- 172.21.15.16:0/3795129252 learned_addr learned my addr 172.21.15.16:0/3795129252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14652020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff24ffff640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 0x559d14652aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 -- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 msgr2=0x559d14652aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 0x559d14652aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 -- 172.21.15.16:0/3795129252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 msgr2=0x559d14652020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14652020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff246ffd640 1 -- 172.21.15.16:0/3795129252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1431c6a0 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.642+0000 7ff24ffff640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 0x559d14652aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff246ffd640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff23809b0f0 tx=0x7ff23809ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe4b592c109250c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238020070 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff2380cc8a0 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2400317b0 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2380046c0 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 10 client.19173 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d14663960 con 0x559d14646360 2024-09-12T01:07:57.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2400317b0 con 0x559d14646360 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff238097a60 con 0x559d14646360 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 10 client.19173 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 20 client.19173 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d14667210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 4 client.19173 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.643+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d145b87f0 con 0x559d14664e20 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.644+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d14667210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.644+0000 7ff22effd640 10 client.19173 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.644+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d14667210 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edad67947027c83 server_cookie=52648d23c54b9385 in_seq=0 out_seq=0 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff22effd640 1 -- 172.21.15.16:0/3795129252 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d145b87f0 con 0x559d14664e20 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff22effd640 10 client.19173 handle_command_reply: tid=0 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 client.19173 shutdown 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 msgr2=0x559d14652560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff23809b0f0 tx=0x7ff23809ad50 comp rx=0 tx=0).stop 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 shutdown_connections 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d14667210 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14655150 0x559d14652aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14652560 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff22effd640 0 client.19173 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 --2- 172.21.15.16:0/3795129252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14652020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 >> 172.21.15.16:0/3795129252 conn(0x559d146420f0 msgr2=0x559d14647e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 shutdown_connections 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 1 -- 172.21.15.16:0/3795129252 wait complete. 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.645+0000 7ff2583b9740 20 client.19173 trim_cache size 0 max 16384 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 Processor -- start 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 -- start start 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14653e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 0x559d1466ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ab650 con 0x559d14655150 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ad190 con 0x559d14645a40 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.662+0000 7ff2583b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501abd20 con 0x559d14646360 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44144/0 (socket says 172.21.15.16:44144) 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 -- 172.21.15.16:0/4166157362 learned_addr learned my addr 172.21.15.16:0/4166157362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14653e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff246ffd640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 0x559d1466ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 -- 172.21.15.16:0/4166157362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 msgr2=0x559d1466ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 0x559d1466ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 -- 172.21.15.16:0/4166157362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 msgr2=0x559d14653e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14653e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 -- 172.21.15.16:0/4166157362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d14650570 con 0x559d14655150 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14653e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff246ffd640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 0x559d1466ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff24ffff640 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff248012770 tx=0x7ff248013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e43ab5e60424c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff2457fa640 1 -- 172.21.15.16:0/4166157362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24801d800 con 0x559d14655150 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff2457fa640 1 -- 172.21.15.16:0/4166157362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24801b440 con 0x559d14655150 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.663+0000 7ff2457fa640 1 -- 172.21.15.16:0/4166157362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248026620 con 0x559d14655150 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- 172.21.15.16:0/4166157362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 msgr2=0x559d1466cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff248012770 tx=0x7ff248013000 comp rx=0 tx=0).stop 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- 172.21.15.16:0/4166157362 shutdown_connections 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14655150 0x559d1466cfd0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14646360 0x559d1466ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 --2- 172.21.15.16:0/4166157362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14645a40 0x559d14653e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- 172.21.15.16:0/4166157362 >> 172.21.15.16:0/4166157362 conn(0x559d146420f0 msgr2=0x559d14653770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- 172.21.15.16:0/4166157362 shutdown_connections 2024-09-12T01:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- 172.21.15.16:0/4166157362 wait complete. 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.664+0000 7ff2583b9740 1 -- start start 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 Processor -- start 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14659230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d146cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501acac0 con 0x559d14646360 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501ac3f0 con 0x559d14655150 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff2501a9650 con 0x559d14645a40 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff2583b9740 10 client.0 _ll_register_callbacks cb 0x7ff23800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44158/0 (socket says 172.21.15.16:44158) 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 -- 172.21.15.16:0/828074135 learned_addr learned my addr 172.21.15.16:0/828074135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14659230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff24ffff640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d146cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 -- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 msgr2=0x559d14659230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14659230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 -- 172.21.15.16:0/828074135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 msgr2=0x559d146cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d146cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff246ffd640 1 -- 172.21.15.16:0/828074135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1465a070 con 0x559d14646360 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14659230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.665+0000 7ff24ffff640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d146cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff246ffd640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff23809b380 tx=0x7ff23809ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d99e50a0eb3e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238020040 con 0x559d14646360 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7ff2380d22b0 con 0x559d14646360 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24801b440 con 0x559d14646360 2024-09-12T01:07:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff23809af30 con 0x559d14646360 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 10 client.19179 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d146d0110 con 0x559d14646360 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24801b440 con 0x559d14646360 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff23809a960 con 0x559d14646360 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 10 client.19179 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 20 client.19179 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d146d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 4 client.19179 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.666+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d145b2d00 con 0x559d14664e20 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.667+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d146d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.667+0000 7ff22effd640 10 client.19179 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:57.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.667+0000 7ff24f7fe640 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d146d28c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b626f550979fe9 server_cookie=a7cd216e796cc0f in_seq=0 out_seq=0 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.668+0000 7ff22effd640 1 -- 172.21.15.16:0/828074135 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d145b2d00 con 0x559d14664e20 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.668+0000 7ff22effd640 10 client.19179 handle_command_reply: tid=0 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.668+0000 7ff2583b9740 1 client.19179 shutdown 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 msgr2=0x559d14659770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff23809b380 tx=0x7ff23809ae80 comp rx=0 tx=0).stop 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 shutdown_connections 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x559d14664e20 0x559d146d28c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d14655150 0x559d146cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d14646360 0x559d14659770 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 --2- 172.21.15.16:0/828074135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d14645a40 0x559d14659230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 >> 172.21.15.16:0/828074135 conn(0x559d146420f0 msgr2=0x559d14647d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff22effd640 0 client.19179 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 shutdown_connections 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 1 -- 172.21.15.16:0/828074135 wait complete. 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.669+0000 7ff2583b9740 20 client.19179 trim_cache size 0 max 16384 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3cda534c-403a-4228-8470-0d6a6bb6f34a", 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:07:57.682 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 msgr2=0x7ff23c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 0x7ff23c0a5230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff248008d40 tx=0x7ff248004a40 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 msgr2=0x7ff250198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff23800d770 tx=0x7ff23800dc30 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 shutdown_connections 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff23c0a2d70 0x7ff23c0a5230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff250106f10 0x7ff25019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff250102790 0x7ff250198970 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 --2- 172.21.15.16:0/1428748547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff250101de0 0x7ff250198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 >> 172.21.15.16:0/1428748547 conn(0x7ff2500fd8c0 msgr2=0x7ff2500fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 shutdown_connections 2024-09-12T01:07:57.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:57.670+0000 7ff255fe0640 1 -- 172.21.15.16:0/1428748547 wait complete. 2024-09-12T01:07:58.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:07:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:07:59.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:58 smithi016 ceph-mon[30972]: pgmap v368: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:07:59.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:58 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:07:59.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:58 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:07:59.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:58 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:58 smithi099 ceph-mon[35654]: pgmap v368: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:07:59.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:58 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:07:59.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:58 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:07:59.076 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:58 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:58 smithi159 ceph-mon[35566]: pgmap v368: 97 pgs: 97 active+clean; 17 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:07:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:58 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:07:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:58 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:07:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:58 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 -- 172.21.15.16:0/2524351397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c103660 msgr2=0x7fe51c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2524351397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c103660 0x7fe51c10bb50 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fe510009900 tx=0x7fe51002a480 comp rx=0 tx=0).stop 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 -- 172.21.15.16:0/2524351397 shutdown_connections 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2524351397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 0x7fe51c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2524351397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c103660 0x7fe51c10bb50 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2524351397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c102d40 0x7fe51c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 -- 172.21.15.16:0/2524351397 >> 172.21.15.16:0/2524351397 conn(0x7fe51c0fc7a0 msgr2=0x7fe51c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.156+0000 7fe521f4c640 1 -- 172.21.15.16:0/2524351397 shutdown_connections 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.157+0000 7fe521f4c640 1 -- 172.21.15.16:0/2524351397 wait complete. 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.157+0000 7fe521f4c640 1 Processor -- start 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.157+0000 7fe521f4c640 1 -- start start 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.157+0000 7fe521f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c103660 0x7fe51c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 0x7fe51c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c111fd0 con 0x7fe51c102d40 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c111e50 con 0x7fe51c10c170 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c112150 con 0x7fe51c103660 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46822/0 (socket says 172.21.15.16:46822) 2024-09-12T01:07:59.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 -- 172.21.15.16:0/2330433641 learned_addr learned my addr 172.21.15.16:0/2330433641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51affd640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c103660 0x7fe51c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51bfff640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 0x7fe51c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 -- 172.21.15.16:0/2330433641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c103660 msgr2=0x7fe51c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c103660 0x7fe51c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 -- 172.21.15.16:0/2330433641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 msgr2=0x7fe51c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 0x7fe51c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 -- 172.21.15.16:0/2330433641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19d460 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00d770 tx=0x7fe50c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe518ff9640 1 -- 172.21.15.16:0/2330433641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c017070 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19d750 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.158+0000 7fe518ff9640 1 -- 172.21.15.16:0/2330433641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe50c005500 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.159+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a5030 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.159+0000 7fe518ff9640 1 -- 172.21.15.16:0/2330433641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c020060 con 0x7fe51c102d40 2024-09-12T01:07:59.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe518ff9640 1 -- 172.21.15.16:0/2330433641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe50c004da0 con 0x7fe51c102d40 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 Processor -- start 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 -- start start 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e95260550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95260a90 0x564e95262e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e951b72e0 con 0x564e952513a0 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e9515ece0 con 0x564e95251cc0 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.161+0000 7fe524325740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e951b32a0 con 0x564e95260a90 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe518ff9640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 0x7fe5040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46824/0 (socket says 172.21.15.16:46824) 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 -- 172.21.15.16:0/950459404 learned_addr learned my addr 172.21.15.16:0/950459404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:59.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 0x7fe5040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51affd640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95260a90 0x564e95262e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe518ff9640 1 -- 172.21.15.16:0/2330433641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fe50c0cf280 con 0x7fe51c102d40 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 -- 172.21.15.16:0/950459404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95260a90 msgr2=0x564e95262e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95260a90 0x564e95262e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 -- 172.21.15.16:0/950459404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 msgr2=0x564e95260550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e95260550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e95260550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 -- 172.21.15.16:0/950459404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e95143440 con 0x564e952513a0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e95260550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe51bfff640 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe51400cb20 tx=0x7fe5140072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c407d51aac9a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe501ffb640 1 -- 172.21.15.16:0/950459404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514015070 con 0x564e952513a0 2024-09-12T01:07:59.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe501ffb640 1 -- 172.21.15.16:0/950459404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514016040 con 0x564e952513a0 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.162+0000 7fe501ffb640 1 -- 172.21.15.16:0/950459404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e020 con 0x564e952513a0 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- 172.21.15.16:0/950459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 msgr2=0x564e95251780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe51bfff640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 0x7fe5040a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe51401f000 tx=0x7fe51401b000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe51400cb20 tx=0x7fe5140072b0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- 172.21.15.16:0/950459404 shutdown_connections 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95260a90 0x564e95262e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e95260550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 --2- 172.21.15.16:0/950459404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e952513a0 0x564e95251780 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- 172.21.15.16:0/950459404 >> 172.21.15.16:0/950459404 conn(0x564e9524da30 msgr2=0x564e9524de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- 172.21.15.16:0/950459404 shutdown_connections 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- 172.21.15.16:0/950459404 wait complete. 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.163+0000 7fe524325740 1 -- start start 2024-09-12T01:07:59.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 Processor -- start 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95260460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e9525d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564e952636c0 con 0x564e95260a90 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e95172050 con 0x564e95251cc0 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e95263840 con 0x564e952513a0 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe524325740 10 client.0 _ll_register_callbacks cb 0x564e951b2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46828/0 (socket says 172.21.15.16:46828) 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 -- 172.21.15.16:0/890545673 learned_addr learned my addr 172.21.15.16:0/890545673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95260460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51affd640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e9525d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 -- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 msgr2=0x564e95260460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95260460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 -- 172.21.15.16:0/890545673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 msgr2=0x564e9525d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e9525d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51bfff640 1 -- 172.21.15.16:0/890545673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e94f283c0 con 0x564e95260a90 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.164+0000 7fe51affd640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e9525d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe51bfff640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe51401ae10 tx=0x7fe51401af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd96f9df0f856f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e050 con 0x564e95260a90 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514016040 con 0x564e95260a90 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fe51401a440 con 0x564e95260a90 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe514015040 con 0x564e95260a90 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 10 client.19191 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:59.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e9526f0f0 con 0x564e95260a90 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514016040 con 0x564e95260a90 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe51402a070 con 0x564e95260a90 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 10 client.19191 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.165+0000 7fe524325740 20 client.19191 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952704a0 0x564e95272890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe524325740 4 client.19191 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e951c4850 con 0x564e952704a0 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952704a0 0x564e95272890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe4faffd640 10 client.19191 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.166+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952704a0 0x564e95272890 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0e18535c899858 server_cookie=7af0a1acac4c4b1 in_seq=0 out_seq=0 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe4faffd640 1 -- 172.21.15.16:0/890545673 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e951c4850 con 0x564e952704a0 2024-09-12T01:07:59.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe4faffd640 10 client.19191 handle_command_reply: tid=0 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 client.19191 shutdown 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 msgr2=0x564e9525dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe51401ae10 tx=0x7fe51401af90 comp rx=0 tx=0).stop 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 shutdown_connections 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952704a0 0x564e95272890 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e9525dd00 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95251cc0 0x564e9525d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 --2- 172.21.15.16:0/890545673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95260460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 >> 172.21.15.16:0/890545673 conn(0x564e9524da30 msgr2=0x564e95253760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe4faffd640 0 client.19191 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 shutdown_connections 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 1 -- 172.21.15.16:0/890545673 wait complete. 2024-09-12T01:07:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.167+0000 7fe524325740 20 client.19191 trim_cache size 0 max 16384 2024-09-12T01:07:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 Processor -- start 2024-09-12T01:07:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 -- start start 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e952513a0 0x564e9526c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 0x564e9525cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ab5e0 con 0x564e95260a90 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ad120 con 0x564e952513a0 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe524325740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1abcb0 con 0x564e95251cc0 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 0x564e9525cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 0x564e9525cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:55748/0 (socket says 172.21.15.16:55748) 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51affd640 1 -- 172.21.15.16:0/3528156234 learned_addr learned my addr 172.21.15.16:0/3528156234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46832/0 (socket says 172.21.15.16:46832) 2024-09-12T01:07:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e952513a0 0x564e9526c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 -- 172.21.15.16:0/3528156234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 msgr2=0x564e9525cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 0x564e9525cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 -- 172.21.15.16:0/3528156234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e952513a0 msgr2=0x564e9526c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e952513a0 0x564e9526c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.185+0000 7fe51bfff640 1 -- 172.21.15.16:0/3528156234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e95252230 con 0x564e95260a90 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe51bfff640 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c700 tx=0x7fe5140072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da4d812cc3a8b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe5017fa640 1 -- 172.21.15.16:0/3528156234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e070 con 0x564e95260a90 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe5017fa640 1 -- 172.21.15.16:0/3528156234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51402c440 con 0x564e95260a90 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe5017fa640 1 -- 172.21.15.16:0/3528156234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514023420 con 0x564e95260a90 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 -- 172.21.15.16:0/3528156234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 msgr2=0x564e95278020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe51402c700 tx=0x7fe5140072b0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 -- 172.21.15.16:0/3528156234 shutdown_connections 2024-09-12T01:07:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95260a90 0x564e95278020 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e95251cc0 0x564e9525cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 --2- 172.21.15.16:0/3528156234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e952513a0 0x564e9526c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 -- 172.21.15.16:0/3528156234 >> 172.21.15.16:0/3528156234 conn(0x564e9524da30 msgr2=0x564e9525f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 -- 172.21.15.16:0/3528156234 shutdown_connections 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.186+0000 7fe524325740 1 -- 172.21.15.16:0/3528156234 wait complete. 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 -- start start 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 Processor -- start 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95264c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95260a90 0x564e952d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1aca50 con 0x564e95251cc0 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ac380 con 0x564e95260a90 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1a95e0 con 0x564e952513a0 2024-09-12T01:07:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe524325740 10 client.0 _ll_register_callbacks cb 0x7fe51400c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe51affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe51affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46834/0 (socket says 172.21.15.16:46834) 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe51affd640 1 -- 172.21.15.16:0/3509689437 learned_addr learned my addr 172.21.15.16:0/3509689437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.187+0000 7fe51bfff640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95260a90 0x564e952d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95264c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 -- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 msgr2=0x564e95264c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95264c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 -- 172.21.15.16:0/3509689437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95260a90 msgr2=0x564e952d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95260a90 0x564e952d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 -- 172.21.15.16:0/3509689437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e952da530 con 0x564e95251cc0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe51affd640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe510005940 tx=0x7fe510031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0636bd5bd511b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe4faffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510035040 con 0x564e95251cc0 2024-09-12T01:07:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8521+0+0 (secure 0 0 0) 0x7fe510004450 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51402c440 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe510005030 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe524325740 10 client.19197 fetch_fsmap learned FSMap version 28 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564e952db720 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.188+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51402c440 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe510045020 con 0x564e95251cc0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 10 client.19197 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 10 client.19197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 20 client.19197 populate_metadata read hostname 'smithi016' 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952dbed0 0x564e952de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 4 client.19197 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e951bed60 con 0x564e952dbed0 2024-09-12T01:07:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952dbed0 0x564e952de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe4faffd640 10 client.19197 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.189+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952dbed0 0x564e952de2f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66083fbcb70192e4 server_cookie=3d45031686882a26 in_seq=0 out_seq=0 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe4faffd640 1 -- 172.21.15.16:0/3509689437 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564e951bed60 con 0x564e952dbed0 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe4faffd640 10 client.19197 handle_command_reply: tid=0 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 client.19197 shutdown 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 msgr2=0x564e952651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe510005940 tx=0x7fe510031990 comp rx=0 tx=0).stop 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 shutdown_connections 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564e952dbed0 0x564e952de2f0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e95260a90 0x564e952d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e95251cc0 0x564e952651b0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 --2- 172.21.15.16:0/3509689437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e952513a0 0x564e95264c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 >> 172.21.15.16:0/3509689437 conn(0x564e9524da30 msgr2=0x564e95252ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe4faffd640 0 client.19197 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 shutdown_connections 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 1 -- 172.21.15.16:0/3509689437 wait complete. 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.190+0000 7fe524325740 20 client.19197 trim_cache size 0 max 16384 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stdout: "3cda534c-403a-4228-8470-0d6a6bb6f34a": { 2024-09-12T01:07:59.201 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:07:59.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "3cda534c-403a-4228-8470-0d6a6bb6f34a", 2024-09-12T01:07:59.204 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:07:59.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:07:59.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:07:59.205 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:07:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 msgr2=0x7fe5040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 0x7fe5040a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe51401f000 tx=0x7fe51401b000 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 msgr2=0x7fe51c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00d770 tx=0x7fe50c00dc30 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 shutdown_connections 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5040a2cf0 0x7fe5040a51b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe51c10c170 0x7fe51c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe51c103660 0x7fe51c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 --2- 172.21.15.16:0/2330433641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51c102d40 0x7fe51c1984b0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 >> 172.21.15.16:0/2330433641 conn(0x7fe51c0fc7a0 msgr2=0x7fe51c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 shutdown_connections 2024-09-12T01:07:59.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:07:59.191+0000 7fe521f4c640 1 -- 172.21.15.16:0/2330433641 wait complete. 2024-09-12T01:07:59.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:07:59 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:07:59.954 INFO:tasks.cephfs.filesystem:scrub status for tag:3cda534c-403a-4228-8470-0d6a6bb6f34a - {'path': '/', 'tag': '3cda534c-403a-4228-8470-0d6a6bb6f34a', 'options': 'recursive,force'} 2024-09-12T01:08:00.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:07:59 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:08:00.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:07:59 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:08:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:00 smithi159 ceph-mon[35566]: pgmap v369: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 73 KiB/s wr, 20 op/s 2024-09-12T01:08:01.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:00 smithi016 ceph-mon[30972]: pgmap v369: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 73 KiB/s wr, 20 op/s 2024-09-12T01:08:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:00 smithi099 ceph-mon[35654]: pgmap v369: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 73 KiB/s wr, 20 op/s 2024-09-12T01:08:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:02 smithi159 ceph-mon[35566]: pgmap v370: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:02 smithi016 ceph-mon[30972]: pgmap v370: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:02 smithi099 ceph-mon[35654]: pgmap v370: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:04 smithi159 ceph-mon[35566]: pgmap v371: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:05.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:04 smithi016 ceph-mon[30972]: pgmap v371: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:05.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:04 smithi099 ceph-mon[35654]: pgmap v371: 97 pgs: 97 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-09-12T01:08:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:05 smithi159 ceph-mon[35566]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:08:06.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:05 smithi016 ceph-mon[30972]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:08:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:05 smithi099 ceph-mon[35654]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T01:08:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:06 smithi159 ceph-mon[35566]: pgmap v373: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:07.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:06 smithi016 ceph-mon[30972]: pgmap v373: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:07.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:06 smithi099 ceph-mon[35654]: pgmap v373: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:08.578 INFO:tasks.workunit.client.0.smithi016.stdout:375/600: open fd = 3 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:376/600: open fd = 3 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:377/600: open fd = 3 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.579 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:378/600: open fd = 3 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:379/600: open fd = 3 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:380/600: open fd = 3 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.580 INFO:tasks.workunit.client.0.smithi016.stdout:381/600: open fd = 3 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:382/600: open fd = 3 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:383/600: open fd = 3 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:384/600: open fd = 3 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.581 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:385/600: open fd = 3 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:386/600: open fd = 3 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:387/600: open fd = 3 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.582 INFO:tasks.workunit.client.0.smithi016.stdout:388/600: open fd = 3 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:389/600: open fd = 3 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:390/600: open fd = 3 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:391/600: open fd = 3 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.583 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:392/600: open fd = 3 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:393/600: open fd = 3 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:394/600: open fd = 3 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.584 INFO:tasks.workunit.client.0.smithi016.stdout:395/600: open fd = 3 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:396/600: open fd = 3 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:397/600: open fd = 3 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:398/600: open fd = 3 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.585 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:399/600: open fd = 3 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:400/600: open fd = 3 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:401/600: open fd = 3 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.586 INFO:tasks.workunit.client.0.smithi016.stdout:402/600: open fd = 3 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:403/600: open fd = 3 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:404/600: open fd = 3 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:405/600: open fd = 3 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.587 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:406/600: open fd = 3 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:407/600: open fd = 3 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:408/600: open fd = 3 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.588 INFO:tasks.workunit.client.0.smithi016.stdout:409/600: open fd = 3 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:410/600: open fd = 3 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:411/600: open fd = 3 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:412/600: open fd = 3 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.589 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:413/600: open fd = 3 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:414/600: open fd = 3 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:415/600: open fd = 3 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.590 INFO:tasks.workunit.client.0.smithi016.stdout:416/600: open fd = 3 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:417/600: open fd = 3 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:418/600: open fd = 3 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:419/600: open fd = 3 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.591 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:420/600: open fd = 3 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:421/600: open fd = 3 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:422/600: open fd = 3 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.592 INFO:tasks.workunit.client.0.smithi016.stdout:423/600: open fd = 3 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:424/600: open fd = 3 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:425/600: open fd = 3 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:426/600: open fd = 3 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.593 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:427/600: open fd = 3 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:428/600: open fd = 3 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:429/600: open fd = 3 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.594 INFO:tasks.workunit.client.0.smithi016.stdout:430/600: open fd = 3 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:431/600: open fd = 3 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:432/600: open fd = 3 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:433/600: open fd = 3 2024-09-12T01:08:08.595 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:434/600: open fd = 3 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:435/600: open fd = 3 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:436/600: open fd = 3 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:08:08.596 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:08:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:08 smithi159 ceph-mon[35566]: pgmap v374: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:09.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:08 smithi016 ceph-mon[30972]: pgmap v374: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:08 smithi099 ceph-mon[35654]: pgmap v374: 97 pgs: 97 active+clean; 18 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 118 KiB/s wr, 36 op/s 2024-09-12T01:08:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:10 smithi159 ceph-mon[35566]: pgmap v375: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:11.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:10 smithi016 ceph-mon[30972]: pgmap v375: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:10 smithi099 ceph-mon[35654]: pgmap v375: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:12 smithi159 ceph-mon[35566]: pgmap v376: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:12 smithi159 ceph-mon[35566]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:08:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:12 smithi016 ceph-mon[30972]: pgmap v376: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:13.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:12 smithi016 ceph-mon[30972]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:08:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:12 smithi099 ceph-mon[35654]: pgmap v376: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-09-12T01:08:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:12 smithi099 ceph-mon[35654]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T01:08:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:14 smithi159 ceph-mon[35566]: pgmap v378: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 8 op/s 2024-09-12T01:08:15.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:14 smithi016 ceph-mon[30972]: pgmap v378: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 8 op/s 2024-09-12T01:08:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:14 smithi099 ceph-mon[35654]: pgmap v378: 97 pgs: 97 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 8 op/s 2024-09-12T01:08:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:16 smithi159 ceph-mon[35566]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:17.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:16 smithi016 ceph-mon[30972]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:16 smithi099 ceph-mon[35654]: pgmap v379: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:18.373 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T01:08:18.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:08:18.645 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:19.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:18 smithi016 ceph-mon[30972]: pgmap v380: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:18 smithi159 ceph-mon[35566]: pgmap v380: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1625329379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 msgr2=0x7f3ff410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1625329379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff410ea90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009ca0 tx=0x7f3fec02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1625329379 shutdown_connections 2024-09-12T01:08:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1625329379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff410ea90 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1625329379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4102790 0x7f3ff41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1625329379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4101de0 0x7f3ff41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1625329379 >> 172.21.15.16:0/1625329379 conn(0x7f3ff40fd8c0 msgr2=0x7f3ff40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1625329379 shutdown_connections 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.139+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1625329379 wait complete. 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 Processor -- start 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 -- start start 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4101de0 0x7f3ff4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4102790 0x7f3ff41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4111f90 con 0x7f3ff4106f10 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4111e10 con 0x7f3ff4101de0 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffbeec640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4112110 con 0x7f3ff4102790 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffa489640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffa489640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41356/0 (socket says 172.21.15.16:41356) 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ffa489640 1 -- 172.21.15.16:0/1724667786 learned_addr learned my addr 172.21.15.16:0/1724667786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ff9487640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4102790 0x7f3ff41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.140+0000 7f3ff9c88640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4101de0 0x7f3ff4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 -- 172.21.15.16:0/1724667786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4102790 msgr2=0x7f3ff41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4102790 0x7f3ff41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 -- 172.21.15.16:0/1724667786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4101de0 msgr2=0x7f3ff4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4101de0 0x7f3ff4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 -- 172.21.15.16:0/1724667786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff419d420 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffa489640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3fec031a60 tx=0x7f3fec031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec003c30 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fec003dd0 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff419d6b0 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0047e0 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.141+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41a4ff0 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.142+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff41021c0 con 0x7f3ff4106f10 2024-09-12T01:08:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.143+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3fec004b40 con 0x7f3ff4106f10 2024-09-12T01:08:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.144+0000 7f3feaffd640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 0x7f3fe00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.144+0000 7f3ff9c88640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 0x7f3fe00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.144+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f3fec0f0910 con 0x7f3ff4106f10 2024-09-12T01:08:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.144+0000 7f3ff9c88640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 0x7f3fe00a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009890 tx=0x7f3ff0008040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:19.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.147+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fec0bd5e0 con 0x7f3ff4106f10 2024-09-12T01:08:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:18 smithi099 ceph-mon[35654]: pgmap v380: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:19.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.358+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff41997d0 con 0x7f3ff4106f10 2024-09-12T01:08:19.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.359+0000 7f3feaffd640 1 -- 172.21.15.16:0/1724667786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f3ff41997d0 con 0x7f3ff4106f10 2024-09-12T01:08:19.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:08:19.359 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:08:19.359 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 msgr2=0x7f3fe00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 0x7f3fe00a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009890 tx=0x7f3ff0008040 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 msgr2=0x7f3ff419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f3fec031a60 tx=0x7f3fec031a90 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 shutdown_connections 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3fe00a2cf0 0x7f3fe00a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ff4106f10 0x7f3ff419cd40 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ff4102790 0x7f3ff41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 --2- 172.21.15.16:0/1724667786 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff4101de0 0x7f3ff4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 >> 172.21.15.16:0/1724667786 conn(0x7f3ff40fd8c0 msgr2=0x7f3ff40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 shutdown_connections 2024-09-12T01:08:19.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:19.361+0000 7f3ffbeec640 1 -- 172.21.15.16:0/1724667786 wait complete. 2024-09-12T01:08:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:19 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1724667786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:20.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:19 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1724667786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:20.180 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T01:08:20.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T01:08:20.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:19 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1724667786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:20.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 -- 172.21.15.16:0/1043338102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c102d40 msgr2=0x7f4d4c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 --2- 172.21.15.16:0/1043338102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c102d40 0x7f4d4c103120 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009900 tx=0x7f4d3c02a480 comp rx=0 tx=0).stop 2024-09-12T01:08:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 -- 172.21.15.16:0/1043338102 shutdown_connections 2024-09-12T01:08:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 --2- 172.21.15.16:0/1043338102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c10c170 0x7f4d4c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 --2- 172.21.15.16:0/1043338102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c103660 0x7f4d4c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 --2- 172.21.15.16:0/1043338102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c102d40 0x7f4d4c103120 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 -- 172.21.15.16:0/1043338102 >> 172.21.15.16:0/1043338102 conn(0x7f4d4c0fc7a0 msgr2=0x7f4d4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 -- 172.21.15.16:0/1043338102 shutdown_connections 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.932+0000 7f4d5230d640 1 -- 172.21.15.16:0/1043338102 wait complete. 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.933+0000 7f4d5230d640 1 Processor -- start 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.933+0000 7f4d5230d640 1 -- start start 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c102d40 0x7f4d4c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c10c170 0x7f4d4c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c112030 con 0x7f4d4c103660 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c111eb0 con 0x7f4d4c10c170 2024-09-12T01:08:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d5230d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1121b0 con 0x7f4d4c102d40 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41378/0 (socket says 172.21.15.16:41378) 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 -- 172.21.15.16:0/2023010846 learned_addr learned my addr 172.21.15.16:0/2023010846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c102d40 0x7f4d4c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d508aa640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c10c170 0x7f4d4c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 -- 172.21.15.16:0/2023010846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c102d40 msgr2=0x7f4d4c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c102d40 0x7f4d4c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 -- 172.21.15.16:0/2023010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c10c170 msgr2=0x7f4d4c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c10c170 0x7f4d4c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 -- 172.21.15.16:0/2023010846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19d5c0 con 0x7f4d4c103660 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b3f0 tx=0x7f4d4000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.934+0000 7f4d497fa640 1 -- 172.21.15.16:0/2023010846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40015020 con 0x7f4d4c103660 2024-09-12T01:08:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.935+0000 7f4d497fa640 1 -- 172.21.15.16:0/2023010846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40011d10 con 0x7f4d4c103660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.935+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19d8b0 con 0x7f4d4c103660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.935+0000 7f4d497fa640 1 -- 172.21.15.16:0/2023010846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000f2e0 con 0x7f4d4c103660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.935+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c1a5190 con 0x7f4d4c103660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d497fa640 1 -- 172.21.15.16:0/2023010846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4d40018020 con 0x7f4d4c103660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 Processor -- start 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 -- start start 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991660 0x5636e8991a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 0x5636e89a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636e88f72a0 con 0x5636e89a0d50 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636e889eb10 con 0x5636e8991f80 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d546e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636e88f3260 con 0x5636e8991660 2024-09-12T01:08:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.937+0000 7f4d508aa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991660 0x5636e8991a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 0x5636e89a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41392/0 (socket says 172.21.15.16:41392) 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4b7fe640 1 -- 172.21.15.16:0/1762686892 learned_addr learned my addr 172.21.15.16:0/1762686892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d497fa640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 0x7f4d340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d508aa640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 0x7f4d340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 -- 172.21.15.16:0/1762686892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991660 msgr2=0x5636e8991a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991660 0x5636e8991a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 -- 172.21.15.16:0/1762686892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 msgr2=0x5636e89a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 -- 172.21.15.16:0/1762686892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e8668b90 con 0x5636e8991f80 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d497fa640 1 -- 172.21.15.16:0/2023010846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4d400cec50 con 0x7f4d4c103660 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 0x5636e89a0810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009230 tx=0x7f4d3c031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb14b4a1ac6de9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d508aa640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 0x7f4d340a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fd0 tx=0x7f4d4400e040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.938+0000 7f4d327fc640 1 -- 172.21.15.16:0/1762686892 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03f070 con 0x5636e8991f80 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d327fc640 1 -- 172.21.15.16:0/1762686892 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c031a20 con 0x5636e8991f80 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d327fc640 1 -- 172.21.15.16:0/1762686892 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035070 con 0x5636e8991f80 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 -- 172.21.15.16:0/1762686892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 msgr2=0x5636e89a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 0x5636e89a0810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009230 tx=0x7f4d3c031630 comp rx=0 tx=0).stop 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 -- 172.21.15.16:0/1762686892 shutdown_connections 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991f80 0x5636e89a0810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 --2- 172.21.15.16:0/1762686892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991660 0x5636e8991a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 -- 172.21.15.16:0/1762686892 >> 172.21.15.16:0/1762686892 conn(0x5636e898dcf0 msgr2=0x5636e898e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 -- 172.21.15.16:0/1762686892 shutdown_connections 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.939+0000 7f4d546e6740 1 -- 172.21.15.16:0/1762686892 wait complete. 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 -- start start 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 Processor -- start 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e899dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e899e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636e89a3920 con 0x5636e89a0d50 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636e887f6e0 con 0x5636e8991660 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636e89a3aa0 con 0x5636e8991f80 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d546e6740 10 client.0 _ll_register_callbacks cb 0x5636e88a3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d508aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d508aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41396/0 (socket says 172.21.15.16:41396) 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d508aa640 1 -- 172.21.15.16:0/3633125242 learned_addr learned my addr 172.21.15.16:0/3633125242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e899e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.940+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e899dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 -- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 msgr2=0x5636e899e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e899e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 -- 172.21.15.16:0/3633125242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 msgr2=0x5636e899dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e899dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 -- 172.21.15.16:0/3633125242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e89acea0 con 0x5636e89a0d50 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d508aa640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4d44007d30 tx=0x7f4d44004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69374a95b527c3e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44015050 con 0x5636e89a0d50 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c031a20 con 0x5636e89a0d50 2024-09-12T01:08:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4d4401d410 con 0x5636e89a0d50 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d4401d610 con 0x5636e89a0d50 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d546e6740 10 client.19215 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636e89af890 con 0x5636e89a0d50 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c031a20 con 0x5636e89a0d50 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.941+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d44019070 con 0x5636e89a0d50 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 10 client.19215 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 10 client.19215 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 20 client.19215 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e89b0ca0 0x5636e89b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 4 client.19215 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e89049d0 con 0x5636e89b0ca0 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e89b0ca0 0x5636e89b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d2b7fe640 10 client.19215 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.942+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e89b0ca0 0x5636e89b3110 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df2a46d72484295 server_cookie=81ab96a049afbacd in_seq=0 out_seq=0 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/3633125242 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636e89049d0 con 0x5636e89b0ca0 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d2b7fe640 10 client.19215 handle_command_reply: tid=0 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 client.19215 shutdown 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 msgr2=0x5636e899e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f4d44007d30 tx=0x7f4d44004b50 comp rx=0 tx=0).stop 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 shutdown_connections 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e89b0ca0 0x5636e89b3110 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e899e710 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e899e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 --2- 172.21.15.16:0/3633125242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e899dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 >> 172.21.15.16:0/3633125242 conn(0x5636e898dcf0 msgr2=0x5636e8993a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d2b7fe640 0 client.19215 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:20.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.943+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 shutdown_connections 2024-09-12T01:08:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.944+0000 7f4d546e6740 1 -- 172.21.15.16:0/3633125242 wait complete. 2024-09-12T01:08:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.944+0000 7f4d546e6740 20 client.19215 trim_cache size 0 max 16384 2024-09-12T01:08:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 Processor -- start 2024-09-12T01:08:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 -- start start 2024-09-12T01:08:20.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 0x5636e899aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ab730 con 0x5636e8991660 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ad270 con 0x5636e89a0d50 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d546e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1abe00 con 0x5636e8991f80 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41406/0 (socket says 172.21.15.16:41406) 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 -- 172.21.15.16:0/4223312020 learned_addr learned my addr 172.21.15.16:0/4223312020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d508aa640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 0x5636e899aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 -- 172.21.15.16:0/4223312020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 msgr2=0x5636e89b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 -- 172.21.15.16:0/4223312020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 msgr2=0x5636e899aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 0x5636e899aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 -- 172.21.15.16:0/4223312020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e8989260 con 0x5636e8991660 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d508aa640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 0x5636e899aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.961+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c031550 tx=0x7f4d3c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390c6467d532fda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d31ffb640 1 -- 172.21.15.16:0/4223312020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03f040 con 0x5636e8991660 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d31ffb640 1 -- 172.21.15.16:0/4223312020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c051070 con 0x5636e8991660 2024-09-12T01:08:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d31ffb640 1 -- 172.21.15.16:0/4223312020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035040 con 0x5636e8991660 2024-09-12T01:08:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 -- 172.21.15.16:0/4223312020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 msgr2=0x5636e89b8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c031550 tx=0x7f4d3c005a20 comp rx=0 tx=0).stop 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 -- 172.21.15.16:0/4223312020 shutdown_connections 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e89a0d50 0x5636e899aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 --2- 172.21.15.16:0/4223312020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e8991660 0x5636e89b8a30 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 -- 172.21.15.16:0/4223312020 >> 172.21.15.16:0/4223312020 conn(0x5636e898dcf0 msgr2=0x5636e898ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 -- 172.21.15.16:0/4223312020 shutdown_connections 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.962+0000 7f4d546e6740 1 -- 172.21.15.16:0/4223312020 wait complete. 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 -- start start 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 Processor -- start 2024-09-12T01:08:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e89a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1acba0 con 0x5636e89a0d50 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1ac4d0 con 0x5636e8991660 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c1a9730 con 0x5636e8991f80 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d546e6740 10 client.0 _ll_register_callbacks cb 0x7f4d44007980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d508aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d508aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41410/0 (socket says 172.21.15.16:41410) 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d508aa640 1 -- 172.21.15.16:0/672203957 learned_addr learned my addr 172.21.15.16:0/672203957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e89a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.963+0000 7f4d4b7fe640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 -- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 msgr2=0x5636e89a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 -- 172.21.15.16:0/672203957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 msgr2=0x5636e89a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e89a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 -- 172.21.15.16:0/672203957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e8a1aad0 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d508aa640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4d4402ea20 tx=0x7f4d4402e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6317e77fe1b358 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44015070 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c051070 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4d4401d9c0 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d4400c560 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d546e6740 10 client.19221 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.964+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636e8a1bd00 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c051070 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d44036070 con 0x5636e89a0d50 2024-09-12T01:08:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 10 client.19221 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 10 client.19221 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 20 client.19221 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e8a1c4e0 0x5636e8a1e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 4 client.19221 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x5636e88fed20 con 0x5636e8a1c4e0 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e8a1c4e0 0x5636e8a1e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d4bfff640 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e8a1c4e0 0x5636e8a1e900 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff284aa35fda926 server_cookie=d483cdf7e082dc2a in_seq=0 out_seq=0 2024-09-12T01:08:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.965+0000 7f4d2b7fe640 10 client.19221 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.977+0000 7f4d2b7fe640 1 -- 172.21.15.16:0/672203957 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5636e88fed20 con 0x5636e8a1c4e0 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.977+0000 7f4d2b7fe640 10 client.19221 handle_command_reply: tid=0 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.977+0000 7f4d546e6740 1 client.19221 shutdown 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 msgr2=0x5636e89a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4d4402ea20 tx=0x7f4d4402e710 comp rx=0 tx=0).stop 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 shutdown_connections 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5636e8a1c4e0 0x5636e8a1e900 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636e89a0d50 0x5636e89a5680 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636e8991f80 0x5636e89a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 --2- 172.21.15.16:0/672203957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636e8991660 0x5636e89a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d2b7fe640 0 client.19221 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:20.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 >> 172.21.15.16:0/672203957 conn(0x5636e898dcf0 msgr2=0x5636e899f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 shutdown_connections 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 1 -- 172.21.15.16:0/672203957 wait complete. 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d546e6740 20 client.19221 trim_cache size 0 max 16384 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 46, 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "22fc04e0": { 2024-09-12T01:08:20.979 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 46, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.2, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:08:20.980 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.978+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 msgr2=0x7f4d340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 0x7f4d340a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009fd0 tx=0x7f4d4400e040 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 msgr2=0x7f4d4c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b3f0 tx=0x7f4d4000b8b0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 shutdown_connections 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4d340a2cf0 0x7f4d340a51b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d4c10c170 0x7f4d4c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d4c103660 0x7f4d4c198b50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 --2- 172.21.15.16:0/2023010846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d4c102d40 0x7f4d4c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 >> 172.21.15.16:0/2023010846 conn(0x7f4d4c0fc7a0 msgr2=0x7f4d4c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 shutdown_connections 2024-09-12T01:08:20.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:20.979+0000 7f4d5230d640 1 -- 172.21.15.16:0/2023010846 wait complete. 2024-09-12T01:08:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:20 smithi159 ceph-mon[35566]: pgmap v381: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:20 smithi016 ceph-mon[30972]: pgmap v381: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:20 smithi099 ceph-mon[35654]: pgmap v381: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:21.804 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '22fc04e0', quiesce took 3.4 seconds. Will release after: 23.8 2024-09-12T01:08:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:22 smithi159 ceph-mon[35566]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:22 smithi016 ceph-mon[30972]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:22 smithi099 ceph-mon[35654]: pgmap v382: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-12T01:08:24.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:08:24.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:08:24.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:08:24.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:08:24.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:08:24.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:08:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:08:24.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:08:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:24 smithi159 ceph-mon[35566]: pgmap v383: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-12T01:08:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:24 smithi016 ceph-mon[30972]: pgmap v383: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-12T01:08:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:24 smithi099 ceph-mon[35654]: pgmap v383: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-12T01:08:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:26 smithi159 ceph-mon[35566]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-12T01:08:27.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:26 smithi016 ceph-mon[30972]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-12T01:08:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:26 smithi099 ceph-mon[35654]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-12T01:08:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:28 smithi159 ceph-mon[35566]: pgmap v385: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-12T01:08:29.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:28 smithi016 ceph-mon[30972]: pgmap v385: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-12T01:08:29.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:28 smithi099 ceph-mon[35654]: pgmap v385: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 11 op/s 2024-09-12T01:08:29.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:08:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 -- 172.21.15.16:0/2682735076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e04102d40 msgr2=0x7f7e04103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 --2- 172.21.15.16:0/2682735076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e04102d40 0x7f7e04103120 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df402a480 comp rx=0 tx=0).stop 2024-09-12T01:08:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 -- 172.21.15.16:0/2682735076 shutdown_connections 2024-09-12T01:08:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 --2- 172.21.15.16:0/2682735076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e0410c170 0x7f7e0410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 --2- 172.21.15.16:0/2682735076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e0410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 --2- 172.21.15.16:0/2682735076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e04102d40 0x7f7e04103120 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 -- 172.21.15.16:0/2682735076 >> 172.21.15.16:0/2682735076 conn(0x7f7e040fc7a0 msgr2=0x7f7e040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 -- 172.21.15.16:0/2682735076 shutdown_connections 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.741+0000 7f7e09de3640 1 -- 172.21.15.16:0/2682735076 wait complete. 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 Processor -- start 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 -- start start 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e04102d40 0x7f7e041009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e04100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e04112030 con 0x7f7e0410c170 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e04111eb0 con 0x7f7e04103660 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.742+0000 7f7e09de3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041121b0 con 0x7f7e04102d40 2024-09-12T01:08:30.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57558/0 (socket says 172.21.15.16:57558) 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 -- 172.21.15.16:0/620206661 learned_addr learned my addr 172.21.15.16:0/620206661 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e037fe640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e04102d40 0x7f7e041009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e04100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 -- 172.21.15.16:0/620206661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e04102d40 msgr2=0x7f7e041009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e04102d40 0x7f7e041009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 -- 172.21.15.16:0/620206661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 msgr2=0x7f7e04100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e04100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 -- 172.21.15.16:0/620206661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e041a6ad0 con 0x7f7e0410c170 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e03fff640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00b970 tx=0x7f7dfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e017fa640 1 -- 172.21.15.16:0/620206661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc019070 con 0x7f7e0410c170 2024-09-12T01:08:30.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e017fa640 1 -- 172.21.15.16:0/620206661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dfc00cb40 con 0x7f7e0410c170 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e041a6dc0 con 0x7f7e0410c170 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e017fa640 1 -- 172.21.15.16:0/620206661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc007650 con 0x7f7e0410c170 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.743+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e041a72d0 con 0x7f7e0410c170 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e017fa640 1 -- 172.21.15.16:0/620206661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7dfc01c070 con 0x7f7e0410c170 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 Processor -- start 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 -- start start 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc61c4b2e0 con 0x55bc61ce54a0 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc61bf2ce0 con 0x55bc61cf0680 2024-09-12T01:08:30.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc61c472a0 con 0x55bc61ce5dc0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57562/0 (socket says 172.21.15.16:57562) 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7dfaffd640 1 -- 172.21.15.16:0/487262506 learned_addr learned my addr 172.21.15.16:0/487262506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.746+0000 7f7e037fe640 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 -- 172.21.15.16:0/487262506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 msgr2=0x55bc61cf0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 -- 172.21.15.16:0/487262506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 msgr2=0x55bc61cf2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 -- 172.21.15.16:0/487262506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61bd7440 con 0x55bc61ce54a0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7e017fa640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 0x7f7df00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7e037fe640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 0x7f7df00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00aa20 tx=0x7f7dec00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d153b1e5d63d85b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7e017fa640 1 -- 172.21.15.16:0/620206661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7dfc0ce8c0 con 0x7f7e0410c170 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7df9ffb640 1 -- 172.21.15.16:0/487262506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00f040 con 0x55bc61ce54a0 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7df9ffb640 1 -- 172.21.15.16:0/487262506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec013460 con 0x55bc61ce54a0 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.747+0000 7f7df9ffb640 1 -- 172.21.15.16:0/487262506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0125c0 con 0x55bc61ce54a0 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/487262506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 msgr2=0x55bc61ce5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e037fe640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 0x7f7df00a5230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000c00 tx=0x7f7df40058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00aa20 tx=0x7f7dec00aee0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/487262506 shutdown_connections 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/487262506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ce5880 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/487262506 >> 172.21.15.16:0/487262506 conn(0x55bc61ce1a30 msgr2=0x55bc61ce1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/487262506 shutdown_connections 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/487262506 wait complete. 2024-09-12T01:08:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.748+0000 7f7e0c1bc740 1 -- start start 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 Processor -- start 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61ced220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61ced760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc61cf3250 con 0x55bc61ce54a0 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc61c06050 con 0x55bc61cf0680 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc61cf33d0 con 0x55bc61ce5dc0 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e0c1bc740 10 client.0 _ll_register_callbacks cb 0x55bc61c46160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57570/0 (socket says 172.21.15.16:57570) 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 -- 172.21.15.16:0/3329570446 learned_addr learned my addr 172.21.15.16:0/3329570446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e03fff640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61ced760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61ced220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 -- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 msgr2=0x55bc61ced220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61ced220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 -- 172.21.15.16:0/3329570446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 msgr2=0x55bc61ced760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61ced760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e037fe640 1 -- 172.21.15.16:0/3329570446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc619bc3c0 con 0x55bc61ce54a0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.749+0000 7f7e03fff640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61ced760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7e037fe640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc4f3b18fa40d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df404e070 con 0x55bc61ce54a0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7df404a450 con 0x55bc61ce54a0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dec013460 con 0x55bc61ce54a0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7df405a050 con 0x55bc61ce54a0 2024-09-12T01:08:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7e0c1bc740 10 client.19233 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc61cfebc0 con 0x55bc61ce54a0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec013460 con 0x55bc61ce54a0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.750+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7df403d6b0 con 0x55bc61ce54a0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 10 client.19233 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 10 client.19233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 20 client.19233 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61cfffd0 0x55bc61d02440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 4 client.19233 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc61c58850 con 0x55bc61cfffd0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61cfffd0 0x55bc61d02440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7de2ffd640 10 client.19233 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.751+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61cfffd0 0x55bc61d02440 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a3d9f777e702f8 server_cookie=328e655c7b085965 in_seq=0 out_seq=0 2024-09-12T01:08:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7de2ffd640 1 -- 172.21.15.16:0/3329570446 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc61c58850 con 0x55bc61cfffd0 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7de2ffd640 10 client.19233 handle_command_reply: tid=0 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 client.19233 shutdown 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 msgr2=0x55bc61ceff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009900 tx=0x7f7df4009260 comp rx=0 tx=0).stop 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 shutdown_connections 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61cfffd0 0x55bc61d02440 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61ced760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61ced220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7de2ffd640 0 client.19233 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/3329570446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61ceff70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 >> 172.21.15.16:0/3329570446 conn(0x55bc61ce1a30 msgr2=0x55bc61ce7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 shutdown_connections 2024-09-12T01:08:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/3329570446 wait complete. 2024-09-12T01:08:30.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.752+0000 7f7e0c1bc740 20 client.19233 trim_cache size 0 max 16384 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 Processor -- start 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 -- start start 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce54a0 0x55bc61d07d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041afb30 con 0x55bc61ce5dc0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041b1670 con 0x55bc61cf0680 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041b0200 con 0x55bc61ce54a0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57574/0 (socket says 172.21.15.16:57574) 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce54a0 0x55bc61d07d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 -- 172.21.15.16:0/2811554191 learned_addr learned my addr 172.21.15.16:0/2811554191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7e03fff640 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 -- 172.21.15.16:0/2811554191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce54a0 msgr2=0x55bc61d07d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce54a0 0x55bc61d07d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 -- 172.21.15.16:0/2811554191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 msgr2=0x55bc61cea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cea480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.770+0000 7f7dfaffd640 1 -- 172.21.15.16:0/2811554191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc61cf02e0 con 0x55bc61ce5dc0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00a9f0 tx=0x7f7dec00afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4a3395cfde5d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7df97fa640 1 -- 172.21.15.16:0/2811554191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00eb60 con 0x55bc61ce5dc0 2024-09-12T01:08:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7df97fa640 1 -- 172.21.15.16:0/2811554191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec00ed00 con 0x55bc61ce5dc0 2024-09-12T01:08:30.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7df97fa640 1 -- 172.21.15.16:0/2811554191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec027650 con 0x55bc61ce5dc0 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/2811554191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 msgr2=0x55bc61d08250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00a9f0 tx=0x7f7dec00afb0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/2811554191 shutdown_connections 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce5dc0 0x55bc61d08250 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/2811554191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce54a0 0x55bc61d07d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.771+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/2811554191 >> 172.21.15.16:0/2811554191 conn(0x55bc61ce1a30 msgr2=0x55bc61ce0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/2811554191 shutdown_connections 2024-09-12T01:08:30.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/2811554191 wait complete. 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- start start 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 Processor -- start 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041b0fa0 con 0x55bc61ce54a0 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041b08d0 con 0x55bc61cf0680 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e041adb30 con 0x55bc61ce5dc0 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e0c1bc740 10 client.0 _ll_register_callbacks cb 0x7f7df40315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.772+0000 7f7e037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57586/0 (socket says 172.21.15.16:57586) 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 -- 172.21.15.16:0/579102974 learned_addr learned my addr 172.21.15.16:0/579102974 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e03fff640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 -- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 msgr2=0x55bc61cf5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 -- 172.21.15.16:0/579102974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 msgr2=0x55bc61cf6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 -- 172.21.15.16:0/579102974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc61cf6840 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e037fe640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7df403dab0 tx=0x7f7df4004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d394d71e5326aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4062070 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7df403d650 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dec00ed00 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7df406e050 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e0c1bc740 10 client.19239 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.773+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc61d6a9f0 con 0x55bc61ce54a0 2024-09-12T01:08:30.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec00ed00 con 0x55bc61ce54a0 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7df4067440 con 0x55bc61ce54a0 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 10 client.19239 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 10 client.19239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 20 client.19239 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61d6b110 0x55bc61d6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 4 client.19239 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc61c52d60 con 0x55bc61d6b110 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.774+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61d6b110 0x55bc61d6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7de2ffd640 10 client.19239 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7dfaffd640 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61d6b110 0x55bc61d6d530 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e669e60d4b58e53d server_cookie=ce9a18cc133ef361 in_seq=0 out_seq=0 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7de2ffd640 1 -- 172.21.15.16:0/579102974 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc61c52d60 con 0x55bc61d6b110 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7de2ffd640 10 client.19239 handle_command_reply: tid=0 2024-09-12T01:08:30.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 client.19239 shutdown 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 msgr2=0x55bc61cf8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f7df403dab0 tx=0x7f7df4004000 comp rx=0 tx=0).stop 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 shutdown_connections 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bc61d6b110 0x55bc61d6d530 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc61cf0680 0x55bc61cf6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc61ce5dc0 0x55bc61cf5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 --2- 172.21.15.16:0/579102974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc61ce54a0 0x55bc61cf8a70 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 >> 172.21.15.16:0/579102974 conn(0x55bc61ce1a30 msgr2=0x55bc61ceea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7de2ffd640 0 client.19239 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 shutdown_connections 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 1 -- 172.21.15.16:0/579102974 wait complete. 2024-09-12T01:08:30.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.775+0000 7f7e0c1bc740 20 client.19239 trim_cache size 0 max 16384 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 msgr2=0x7f7df00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 0x7f7df00a5230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7df4000c00 tx=0x7f7df40058d0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 msgr2=0x7f7e04101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00b970 tx=0x7f7dfc00be30 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 shutdown_connections 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7df00a2d70 0x7f7df00a5230 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0410c170 0x7f7e04101420 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e04103660 0x7f7e04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 --2- 172.21.15.16:0/620206661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e04102d40 0x7f7e041009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 >> 172.21.15.16:0/620206661 conn(0x7f7e040fc7a0 msgr2=0x7f7e040fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 shutdown_connections 2024-09-12T01:08:30.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:30.776+0000 7f7e09de3640 1 -- 172.21.15.16:0/620206661 wait complete. 2024-09-12T01:08:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:30 smithi159 ceph-mon[35566]: pgmap v386: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-12T01:08:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:30 smithi016 ceph-mon[30972]: pgmap v386: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-12T01:08:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:30 smithi099 ceph-mon[35654]: pgmap v386: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-09-12T01:08:31.698 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:08:31.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:08:31.961 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 -- 172.21.15.16:0/4049604788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc100930 msgr2=0x7f42dc100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 --2- 172.21.15.16:0/4049604788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc100930 0x7f42dc100d10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009900 tx=0x7f42cc02a410 comp rx=0 tx=0).stop 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 -- 172.21.15.16:0/4049604788 shutdown_connections 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 --2- 172.21.15.16:0/4049604788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc10e4e0 0x7f42dc1108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 --2- 172.21.15.16:0/4049604788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 0x7f42dc10dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 --2- 172.21.15.16:0/4049604788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc100930 0x7f42dc100d10 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 -- 172.21.15.16:0/4049604788 >> 172.21.15.16:0/4049604788 conn(0x7f42dc0fc780 msgr2=0x7f42dc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:32.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 -- 172.21.15.16:0/4049604788 shutdown_connections 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.442+0000 7f42e0854640 1 -- 172.21.15.16:0/4049604788 wait complete. 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 Processor -- start 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 -- start start 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc100930 0x7f42dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 0x7f42dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc114170 con 0x7f42dc10e4e0 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc113ff0 con 0x7f42dc1012e0 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42e0854640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1142f0 con 0x7f42dc100930 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57602/0 (socket says 172.21.15.16:57602) 2024-09-12T01:08:32.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42dad9d640 1 -- 172.21.15.16:0/1142515772 learned_addr learned my addr 172.21.15.16:0/1142515772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.443+0000 7f42d1d9b640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 0x7f42dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42da59c640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc100930 0x7f42dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 -- 172.21.15.16:0/1142515772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc100930 msgr2=0x7f42dc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc100930 0x7f42dc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 -- 172.21.15.16:0/1142515772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 msgr2=0x7f42dc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 0x7f42dc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 -- 172.21.15.16:0/1142515772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc19f620 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42dad9d640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f42d400dac0 tx=0x7f42d400dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4017030 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc19f910 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4003e50 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4010460 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.444+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc1a71f0 con 0x7f42dc10e4e0 2024-09-12T01:08:32.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.445+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc100d10 con 0x7f42dc10e4e0 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.446+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f42d401a070 con 0x7f42dc10e4e0 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.447+0000 7f42d3fff640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 0x7f42c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.447+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f42d40cf0e0 con 0x7f42dc10e4e0 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.447+0000 7f42da59c640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 0x7f42c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.448+0000 7f42da59c640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 0x7f42c80a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005ab0 tx=0x7f42cc005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:32.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.451+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d409bda0 con 0x7f42dc10e4e0 2024-09-12T01:08:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.663+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42dc02d7c0 con 0x7f42dc10e4e0 2024-09-12T01:08:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.664+0000 7f42d3fff640 1 -- 172.21.15.16:0/1142515772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f42d409b6a0 con 0x7f42dc10e4e0 2024-09-12T01:08:32.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:08:32.664 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:08:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:08:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 msgr2=0x7f42c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 0x7f42c80a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005ab0 tx=0x7f42cc005a20 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 msgr2=0x7f42dc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f42d400dac0 tx=0x7f42d400dec0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 shutdown_connections 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f42c80a2cf0 0x7f42c80a51b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc10e4e0 0x7f42dc19ef40 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42dc1012e0 0x7f42dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 --2- 172.21.15.16:0/1142515772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42dc100930 0x7f42dc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 >> 172.21.15.16:0/1142515772 conn(0x7f42dc0fc780 msgr2=0x7f42dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 shutdown_connections 2024-09-12T01:08:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:32.666+0000 7f42e0854640 1 -- 172.21.15.16:0/1142515772 wait complete. 2024-09-12T01:08:33.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:32 smithi016 ceph-mon[30972]: pgmap v387: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:33.038 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1142515772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:32 smithi159 ceph-mon[35566]: pgmap v387: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:32 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1142515772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:32 smithi099 ceph-mon[35654]: pgmap v387: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:32 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1142515772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:33.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:08:33.671 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 -- 172.21.15.16:0/1355426216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 msgr2=0x7f5ce0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 --2- 172.21.15.16:0/1355426216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce0106e40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009930 tx=0x7f5cdc02a420 comp rx=0 tx=0).stop 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 -- 172.21.15.16:0/1355426216 shutdown_connections 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 --2- 172.21.15.16:0/1355426216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0101400 0x7f5ce010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 --2- 172.21.15.16:0/1355426216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0100a60 0x7f5ce0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 --2- 172.21.15.16:0/1355426216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce0106e40 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 -- 172.21.15.16:0/1355426216 >> 172.21.15.16:0/1355426216 conn(0x7f5ce00fc780 msgr2=0x7f5ce00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 -- 172.21.15.16:0/1355426216 shutdown_connections 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.160+0000 7f5ce7374640 1 -- 172.21.15.16:0/1355426216 wait complete. 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 Processor -- start 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 -- start start 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 0x7f5ce0102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0101400 0x7f5ce0102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0110e60 con 0x7f5ce0106a60 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0110ce0 con 0x7f5ce0100a60 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce7374640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce0110fe0 con 0x7f5ce0101400 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57620/0 (socket says 172.21.15.16:57620) 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 -- 172.21.15.16:0/2305968363 learned_addr learned my addr 172.21.15.16:0/2305968363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce490f640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0101400 0x7f5ce0102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5110640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 0x7f5ce0102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 -- 172.21.15.16:0/2305968363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0101400 msgr2=0x7f5ce0102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0101400 0x7f5ce0102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 -- 172.21.15.16:0/2305968363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 msgr2=0x7f5ce0102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 0x7f5ce0102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5911640 1 -- 172.21.15.16:0/2305968363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce01a6960 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.161+0000 7f5ce5110640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 0x7f5ce0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5ce5911640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800cad0 tx=0x7f5cd800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5cd67fc640 1 -- 172.21.15.16:0/2305968363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800ae20 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01a6c50 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5cd67fc640 1 -- 172.21.15.16:0/2305968363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd8005020 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5cd67fc640 1 -- 172.21.15.16:0/2305968363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800f6f0 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.162+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01a7190 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.164+0000 7f5cd67fc640 1 -- 172.21.15.16:0/2305968363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5cd80051d0 con 0x7f5ce0106a60 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 Processor -- start 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 -- start start 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002db40 0x55c4a002c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a002c5d0 0x55c4a002ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c49ff922f0 con 0x55c4a0037370 2024-09-12T01:08:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c49ff39cf0 con 0x55c4a002c5d0 2024-09-12T01:08:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce974d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c49ff8e2b0 con 0x55c4a002db40 2024-09-12T01:08:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57624/0 (socket says 172.21.15.16:57624) 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 -- 172.21.15.16:0/67329038 learned_addr learned my addr 172.21.15.16:0/67329038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5cd67fc640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 0x7f5ccc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002db40 0x55c4a002c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce5110640 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a002c5d0 0x55c4a002ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce5911640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 0x7f5ccc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 -- 172.21.15.16:0/67329038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002db40 msgr2=0x55c4a002c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002db40 0x55c4a002c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 -- 172.21.15.16:0/67329038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a002c5d0 msgr2=0x55c4a002ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a002c5d0 0x55c4a002ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5cd67fc640 1 -- 172.21.15.16:0/2305968363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f5cd80cf4a0 con 0x7f5ce0106a60 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.165+0000 7f5ce490f640 1 -- 172.21.15.16:0/67329038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c49ff1e3a0 con 0x55c4a0037370 2024-09-12T01:08:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.166+0000 7f5ce490f640 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00127c0 tx=0x7f5cd000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c124c58fa5bfb2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.166+0000 7f5ce5911640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 0x7f5ccc0a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5cd809b8f0 tx=0x7f5cd806d920 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.166+0000 7f5ccb7fe640 1 -- 172.21.15.16:0/67329038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0012e00 con 0x55c4a0037370 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.166+0000 7f5ccb7fe640 1 -- 172.21.15.16:0/67329038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0010c50 con 0x55c4a0037370 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.166+0000 7f5ccb7fe640 1 -- 172.21.15.16:0/67329038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd001eb50 con 0x55c4a0037370 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- 172.21.15.16:0/67329038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 msgr2=0x55c4a0039760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00127c0 tx=0x7f5cd000b7c0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- 172.21.15.16:0/67329038 shutdown_connections 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a0037370 0x55c4a0039760 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a002c5d0 0x55c4a002ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 --2- 172.21.15.16:0/67329038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002db40 0x55c4a002c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- 172.21.15.16:0/67329038 >> 172.21.15.16:0/67329038 conn(0x55c49fff5310 msgr2=0x55c4a0028c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- 172.21.15.16:0/67329038 shutdown_connections 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- 172.21.15.16:0/67329038 wait complete. 2024-09-12T01:08:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 -- start start 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.167+0000 7f5ce974d740 1 Processor -- start 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0036d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a00345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4a003a070 con 0x55c4a002db40 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c49ff4d060 con 0x55c4a0037370 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c4a003a1f0 con 0x55c4a002c5d0 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 10 client.0 _ll_register_callbacks cb 0x55c49fca08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57628/0 (socket says 172.21.15.16:57628) 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 -- 172.21.15.16:0/1922037410 learned_addr learned my addr 172.21.15.16:0/1922037410 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce490f640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a00345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce490f640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0036d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 -- 172.21.15.16:0/1922037410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 msgr2=0x55c4a0036d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0036d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 -- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 msgr2=0x55c4a00345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a00345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 -- 172.21.15.16:0/1922037410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c49fd033c0 con 0x55c4a002db40 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce490f640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0036d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce5110640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009930 tx=0x7f5cdc031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85775702a608c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5cc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc035040 con 0x55c4a002db40 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f5cdc004240 con 0x55c4a002db40 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd0010c50 con 0x55c4a002db40 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc004440 con 0x55c4a002db40 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.168+0000 7f5ce974d740 10 client.19257 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4a0045af0 con 0x55c4a002db40 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd0010c50 con 0x55c4a002db40 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cdc004c80 con 0x55c4a002db40 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 10 client.19257 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 10 client.19257 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 20 client.19257 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a0046fb0 0x55c4a00493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 4 client.19257 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c49ff9f860 con 0x55c4a0046fb0 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.169+0000 7f5ce490f640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a0046fb0 0x55c4a00493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.170+0000 7f5ce490f640 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a0046fb0 0x55c4a00493f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fff5b314b7b02df server_cookie=e3e12716161b992 in_seq=0 out_seq=0 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.170+0000 7f5cc8ff9640 10 client.19257 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.170+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/1922037410 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c49ff9f860 con 0x55c4a0046fb0 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.170+0000 7f5cc8ff9640 10 client.19257 handle_command_reply: tid=0 2024-09-12T01:08:34.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.170+0000 7f5ce974d740 1 client.19257 shutdown 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 msgr2=0x55c4a0034000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009930 tx=0x7f5cdc031920 comp rx=0 tx=0).stop 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 shutdown_connections 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a0046fb0 0x55c4a00493f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a00345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0034000 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 --2- 172.21.15.16:0/1922037410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0036d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5cc8ff9640 0 client.19257 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 >> 172.21.15.16:0/1922037410 conn(0x55c49fff5310 msgr2=0x55c4a002e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 shutdown_connections 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 1 -- 172.21.15.16:0/1922037410 wait complete. 2024-09-12T01:08:34.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.171+0000 7f5ce974d740 20 client.19257 trim_cache size 0 max 16384 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 Processor -- start 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 -- start start 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0043310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a0038760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01afb20 con 0x55c4a002db40 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01b1660 con 0x55c4a0037370 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce974d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01b01f0 con 0x55c4a002c5d0 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57636/0 (socket says 172.21.15.16:57636) 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 -- 172.21.15.16:0/401882695 learned_addr learned my addr 172.21.15.16:0/401882695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce490f640 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0043310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 -- 172.21.15.16:0/401882695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 msgr2=0x55c4a0043310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0043310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 -- 172.21.15.16:0/401882695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 msgr2=0x55c4a0038760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a0038760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.184+0000 7f5ce5110640 1 -- 172.21.15.16:0/401882695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a0028ed0 con 0x55c4a002db40 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce5110640 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc003640 tx=0x7f5cdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecc20ebbbdc6497 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ccaffd640 1 -- 172.21.15.16:0/401882695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc052070 con 0x55c4a002db40 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ccaffd640 1 -- 172.21.15.16:0/401882695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc005070 con 0x55c4a002db40 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ccaffd640 1 -- 172.21.15.16:0/401882695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc035070 con 0x55c4a002db40 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 -- 172.21.15.16:0/401882695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 msgr2=0x55c4a0032fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc003640 tx=0x7f5cdc000bc0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 -- 172.21.15.16:0/401882695 shutdown_connections 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a0038760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a0032fe0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 --2- 172.21.15.16:0/401882695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a0043310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 -- 172.21.15.16:0/401882695 >> 172.21.15.16:0/401882695 conn(0x55c49fff5310 msgr2=0x55c4a00328d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 -- 172.21.15.16:0/401882695 shutdown_connections 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.185+0000 7f5ce974d740 1 -- 172.21.15.16:0/401882695 wait complete. 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 -- start start 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 Processor -- start 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a004e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a003e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01b0f90 con 0x55c4a002db40 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01b08c0 con 0x55c4a0037370 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce01adb20 con 0x55c4a002c5d0 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce974d740 10 client.0 _ll_register_callbacks cb 0x7f5cdc0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce5110640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57646/0 (socket says 172.21.15.16:57646) 2024-09-12T01:08:34.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce5110640 1 -- 172.21.15.16:0/3712725705 learned_addr learned my addr 172.21.15.16:0/3712725705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.186+0000 7f5ce490f640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a004e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce490f640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a003e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 -- 172.21.15.16:0/3712725705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 msgr2=0x55c4a004e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a004e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 -- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 msgr2=0x55c4a003e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a003e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 -- 172.21.15.16:0/3712725705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a003ea40 con 0x55c4a002db40 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce490f640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a003e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce5110640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc003830 tx=0x7f5cdc003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7e64d7a0240c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5cc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc035050 con 0x55c4a002db40 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f5cdc0048c0 con 0x55c4a002db40 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cdc005070 con 0x55c4a002db40 2024-09-12T01:08:34.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cdc066050 con 0x55c4a002db40 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce974d740 10 client.19263 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4a003fd20 con 0x55c4a002db40 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.187+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cdc005070 con 0x55c4a002db40 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cdc05f060 con 0x55c4a002db40 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 10 client.19263 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 10 client.19263 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 20 client.19263 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a00b1d80 0x55c4a00b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 4 client.19263 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c49ff99d70 con 0x55c4a00b1d80 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce490f640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a00b1d80 0x55c4a00b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5ce490f640 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a00b1d80 0x55c4a00b41a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1d48a4c6a388756 server_cookie=14338714350669c in_seq=0 out_seq=0 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5cc8ff9640 10 client.19263 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:34.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.188+0000 7f5cc8ff9640 1 -- 172.21.15.16:0/3712725705 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5cd002f020 con 0x55c4a00b1d80 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5cc8ff9640 10 client.19263 handle_command_reply: tid=0 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 client.19263 shutdown 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 msgr2=0x55c4a004e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc003830 tx=0x7f5cdc003ec0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 shutdown_connections 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55c4a00b1d80 0x55c4a00b41a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c4a0037370 0x55c4a003e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5cc8ff9640 0 client.19263 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4a002db40 0x55c4a004e7e0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 --2- 172.21.15.16:0/3712725705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a002c5d0 0x55c4a004e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 >> 172.21.15.16:0/3712725705 conn(0x55c49fff5310 msgr2=0x55c4a002e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 shutdown_connections 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 1 -- 172.21.15.16:0/3712725705 wait complete. 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce974d740 20 client.19263 trim_cache size 0 max 16384 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 msgr2=0x7f5ccc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 0x7f5ccc0a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5cd809b8f0 tx=0x7f5cd806d920 comp rx=0 tx=0).stop 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 msgr2=0x7f5ce01030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800cad0 tx=0x7f5cd800cf90 comp rx=0 tx=0).stop 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.189+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 shutdown_connections 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5ccc0a2cf0 0x7f5ccc0a51b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ce0106a60 0x7f5ce01030b0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ce0101400 0x7f5ce0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 --2- 172.21.15.16:0/2305968363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ce0100a60 0x7f5ce0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 >> 172.21.15.16:0/2305968363 conn(0x7f5ce00fc780 msgr2=0x7f5ce006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 shutdown_connections 2024-09-12T01:08:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:34.190+0000 7f5ce7374640 1 -- 172.21.15.16:0/2305968363 wait complete. 2024-09-12T01:08:35.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:34 smithi016 ceph-mon[30972]: pgmap v388: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:35.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:08:35.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:34 smithi099 ceph-mon[35654]: pgmap v388: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:35.347 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:34 smithi159 ceph-mon[35566]: pgmap v388: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-09-12T01:08:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 -- 172.21.15.16:0/2886451232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 msgr2=0x7f96a410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/2886451232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a410e610 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f969c009ca0 tx=0x7f969c02a950 comp rx=0 tx=0).stop 2024-09-12T01:08:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 -- 172.21.15.16:0/2886451232 shutdown_connections 2024-09-12T01:08:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/2886451232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a410e610 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/2886451232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 0x7f96a410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/2886451232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 0x7f96a41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 -- 172.21.15.16:0/2886451232 >> 172.21.15.16:0/2886451232 conn(0x7f96a40fc7a0 msgr2=0x7f96a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 -- 172.21.15.16:0/2886451232 shutdown_connections 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 -- 172.21.15.16:0/2886451232 wait complete. 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.813+0000 7f96ab1f1640 1 Processor -- start 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 -- start start 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 0x7f96a4100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 0x7f96a4100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4111f30 con 0x7f96a410c180 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4111db0 con 0x7f96a4102de0 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96ab1f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41120b0 con 0x7f96a4103700 2024-09-12T01:08:35.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57658/0 (socket says 172.21.15.16:57658) 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 -- 172.21.15.16:0/4008006250 learned_addr learned my addr 172.21.15.16:0/4008006250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a3fff640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 0x7f96a4100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 0x7f96a4100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 -- 172.21.15.16:0/4008006250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 msgr2=0x7f96a4100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 0x7f96a4100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 -- 172.21.15.16:0/4008006250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 msgr2=0x7f96a4100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 0x7f96a4100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.814+0000 7f96a978e640 1 -- 172.21.15.16:0/4008006250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41a68b0 con 0x7f96a410c180 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96a978e640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f969c02ae70 tx=0x7f969c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96a1ffb640 1 -- 172.21.15.16:0/4008006250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c034040 con 0x7f96a410c180 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41a6b40 con 0x7f96a410c180 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96a1ffb640 1 -- 172.21.15.16:0/4008006250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c030920 con 0x7f96a410c180 2024-09-12T01:08:35.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96a1ffb640 1 -- 172.21.15.16:0/4008006250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c045480 con 0x7f96a410c180 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.815+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41a7020 con 0x7f96a410c180 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96a1ffb640 1 -- 172.21.15.16:0/4008006250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f969c037020 con 0x7f96a410c180 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 Processor -- start 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 -- start start 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc921090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc92e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc8872f0 con 0x55d1bc9215d0 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc82ecf0 con 0x55d1bc92c370 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.818+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc8832b0 con 0x55d1bc922b40 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57666/0 (socket says 172.21.15.16:57666) 2024-09-12T01:08:35.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 -- 172.21.15.16:0/470592150 learned_addr learned my addr 172.21.15.16:0/470592150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a1ffb640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a3fff640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc921090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a3fff640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc92e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 -- 172.21.15.16:0/470592150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 msgr2=0x55d1bc921090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc921090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 -- 172.21.15.16:0/470592150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 msgr2=0x55d1bc92e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc92e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a978e640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 -- 172.21.15.16:0/470592150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bc8133a0 con 0x55d1bc9215d0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a3fff640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc92e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a1ffb640 1 -- 172.21.15.16:0/4008006250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f969c0f01c0 con 0x7f96a410c180 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f969400bd10 tx=0x7f96940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9668d6efa04d158e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f969affd640 1 -- 172.21.15.16:0/470592150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694004030 con 0x55d1bc9215d0 2024-09-12T01:08:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f969affd640 1 -- 172.21.15.16:0/470592150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x55d1bc9215d0 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f969affd640 1 -- 172.21.15.16:0/470592150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694011820 con 0x55d1bc9215d0 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.819+0000 7f96a978e640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 0x7f96900a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f969c0edad0 tx=0x7f969c03c040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 -- 172.21.15.16:0/470592150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 msgr2=0x55d1bc921a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f969400bd10 tx=0x7f96940107a0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 -- 172.21.15.16:0/470592150 shutdown_connections 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc92e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc921a10 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/470592150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc921090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 -- 172.21.15.16:0/470592150 >> 172.21.15.16:0/470592150 conn(0x55d1bc8ea310 msgr2=0x55d1bc91dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 -- 172.21.15.16:0/470592150 shutdown_connections 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.820+0000 7f96ad5ca740 1 -- 172.21.15.16:0/470592150 wait complete. 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 -- start start 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 Processor -- start 2024-09-12T01:08:35.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc9215d0 0x55d1bc92bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc928ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 0x55d1bc9295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc92f070 con 0x55d1bc92c370 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc842060 con 0x55d1bc9215d0 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc92f1f0 con 0x55d1bc922b40 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96ad5ca740 10 client.0 _ll_register_callbacks cb 0x55d1bc5958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 0x55d1bc9295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 0x55d1bc9295e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57678/0 (socket says 172.21.15.16:57678) 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96a3fff640 1 -- 172.21.15.16:0/3317280045 learned_addr learned my addr 172.21.15.16:0/3317280045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc928ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.821+0000 7f96a3fff640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc9215d0 0x55d1bc92bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 -- 172.21.15.16:0/3317280045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc9215d0 msgr2=0x55d1bc92bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc9215d0 0x55d1bc92bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 -- 172.21.15.16:0/3317280045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 msgr2=0x55d1bc9295e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 0x55d1bc9295e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 -- 172.21.15.16:0/3317280045 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bc5f83c0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc928ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f969400bd10 tx=0x7f96940058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4793d8dad8b5cead server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f9683fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694029060 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96940041d0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f96940035f0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f969400b8e0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96ad5ca740 10 client.28198 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.822+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1bc93aab0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9694004860 con 0x55d1bc922b40 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 10 client.28198 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 10 client.28198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 20 client.28198 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc93be60 0x55d1bc93e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 4 client.28198 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1bc894860 con 0x55d1bc93be60 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.823+0000 7f96a3fff640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc93be60 0x55d1bc93e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.824+0000 7f96a3fff640 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc93be60 0x55d1bc93e250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a583c7b3c74b4b5a server_cookie=ec0256a3eb06bba1 in_seq=0 out_seq=0 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.824+0000 7f9683fff640 10 client.28198 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.824+0000 7f9683fff640 1 -- 172.21.15.16:0/3317280045 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1bc894860 con 0x55d1bc93be60 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.824+0000 7f9683fff640 10 client.28198 handle_command_reply: tid=0 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.824+0000 7f96ad5ca740 1 client.28198 shutdown 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 msgr2=0x55d1bc928ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc928ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f969400bd10 tx=0x7f96940058e0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 shutdown_connections 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc93be60 0x55d1bc93e250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc92c370 0x55d1bc9295e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc922b40 0x55d1bc928ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/3317280045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc9215d0 0x55d1bc92bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 >> 172.21.15.16:0/3317280045 conn(0x55d1bc8ea310 msgr2=0x55d1bc923930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f9683fff640 0 client.28198 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 shutdown_connections 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 1 -- 172.21.15.16:0/3317280045 wait complete. 2024-09-12T01:08:35.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.825+0000 7f96ad5ca740 20 client.28198 trim_cache size 0 max 16384 2024-09-12T01:08:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 Processor -- start 2024-09-12T01:08:35.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 -- start start 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 0x55d1bc927fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc92c370 0x55d1bc943870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41afa00 con 0x55d1bc9215d0 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b1540 con 0x55d1bc922b40 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.838+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b00d0 con 0x55d1bc92c370 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57688/0 (socket says 172.21.15.16:57688) 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 -- 172.21.15.16:0/410079261 learned_addr learned my addr 172.21.15.16:0/410079261 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc92c370 0x55d1bc943870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 0x55d1bc927fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 -- 172.21.15.16:0/410079261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc92c370 msgr2=0x55d1bc943870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc92c370 0x55d1bc943870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 -- 172.21.15.16:0/410079261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 msgr2=0x55d1bc927fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 0x55d1bc927fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 -- 172.21.15.16:0/410079261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bc924120 con 0x55d1bc9215d0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 0x55d1bc927fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f96a3fff640 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f968c0043c0 tx=0x7f968c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b90fb1dc86a659b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f969a7fc640 1 -- 172.21.15.16:0/410079261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c012580 con 0x55d1bc9215d0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f969a7fc640 1 -- 172.21.15.16:0/410079261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c012c00 con 0x55d1bc9215d0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.839+0000 7f969a7fc640 1 -- 172.21.15.16:0/410079261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c022af0 con 0x55d1bc9215d0 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- 172.21.15.16:0/410079261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 msgr2=0x55d1bc938370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f968c0043c0 tx=0x7f968c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- 172.21.15.16:0/410079261 shutdown_connections 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc92c370 0x55d1bc943870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc922b40 0x55d1bc927fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/410079261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc9215d0 0x55d1bc938370 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- 172.21.15.16:0/410079261 >> 172.21.15.16:0/410079261 conn(0x55d1bc8ea310 msgr2=0x55d1bc9278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- 172.21.15.16:0/410079261 shutdown_connections 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- 172.21.15.16:0/410079261 wait complete. 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.840+0000 7f96ad5ca740 1 -- start start 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 Processor -- start 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc9215d0 0x55d1bc934ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc9260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b0e70 con 0x55d1bc922b40 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b07a0 con 0x55d1bc92c370 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41ada00 con 0x55d1bc9215d0 2024-09-12T01:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96ad5ca740 10 client.0 _ll_register_callbacks cb 0x7f969400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57692/0 (socket says 172.21.15.16:57692) 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 -- 172.21.15.16:0/1288005280 learned_addr learned my addr 172.21.15.16:0/1288005280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a3fff640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc9260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 -- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc9215d0 msgr2=0x55d1bc934ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc9215d0 0x55d1bc934ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 -- 172.21.15.16:0/1288005280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 msgr2=0x55d1bc9260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc9260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a8f8d640 1 -- 172.21.15.16:0/1288005280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bc926790 con 0x55d1bc922b40 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.841+0000 7f96a3fff640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc9260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96a8f8d640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9694011010 tx=0x7f9694004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7826d5109a42cbc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694029070 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f9694011860 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968c012c00 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96940041d0 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 10 client.19281 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1bc9a6640 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c012c00 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9694005410 con 0x55d1bc922b40 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 10 client.19281 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 10 client.19281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 20 client.19281 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc9a6db0 0x55d1bc9a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 4 client.19281 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.842+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1bc88ed70 con 0x55d1bc9a6db0 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f96a3fff640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc9a6db0 0x55d1bc9a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f96a3fff640 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc9a6db0 0x55d1bc9a9210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bc574fdb1f34091 server_cookie=95b442b484400ac7 in_seq=0 out_seq=0 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f9683fff640 10 client.19281 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f9683fff640 1 -- 172.21.15.16:0/1288005280 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1bc88ed70 con 0x55d1bc9a6db0 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f9683fff640 10 client.19281 handle_command_reply: tid=0 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.843+0000 7f96ad5ca740 1 client.19281 shutdown 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 msgr2=0x55d1bc925bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9694011010 tx=0x7f9694004a70 comp rx=0 tx=0).stop 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 shutdown_connections 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc92c370 0x55d1bc9260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55d1bc9a6db0 0x55d1bc9a9210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1bc922b40 0x55d1bc925bb0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 --2- 172.21.15.16:0/1288005280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1bc9215d0 0x55d1bc934ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 >> 172.21.15.16:0/1288005280 conn(0x55d1bc8ea310 msgr2=0x55d1bc9280e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f9683fff640 0 client.19281 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 shutdown_connections 2024-09-12T01:08:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 1 -- 172.21.15.16:0/1288005280 wait complete. 2024-09-12T01:08:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ad5ca740 20 client.19281 trim_cache size 0 max 16384 2024-09-12T01:08:35.858 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 msgr2=0x7f96900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 0x7f96900a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f969c0edad0 tx=0x7f969c03c040 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 msgr2=0x7f96a41014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.844+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f969c02ae70 tx=0x7f969c02f5b0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 shutdown_connections 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f96900a2cf0 0x7f96900a51b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a410c180 0x7f96a41014c0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96a4103700 0x7f96a4100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 --2- 172.21.15.16:0/4008006250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96a4102de0 0x7f96a4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 >> 172.21.15.16:0/4008006250 conn(0x7f96a40fc7a0 msgr2=0x7f96a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 shutdown_connections 2024-09-12T01:08:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:35.845+0000 7f96ab1f1640 1 -- 172.21.15.16:0/4008006250 wait complete. 2024-09-12T01:08:36.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:08:36.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:36 smithi016 ceph-mon[30972]: pgmap v389: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T01:08:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:36 smithi099 ceph-mon[35654]: pgmap v389: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T01:08:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:36 smithi159 ceph-mon[35566]: pgmap v389: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/1156960217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88102d40 msgr2=0x7f0a88103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/1156960217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88102d40 0x7f0a88103120 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009900 tx=0x7f0a7802a480 comp rx=0 tx=0).stop 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/1156960217 shutdown_connections 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/1156960217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a8810c170 0x7f0a8810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/1156960217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a88103660 0x7f0a8810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/1156960217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88102d40 0x7f0a88103120 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/1156960217 >> 172.21.15.16:0/1156960217 conn(0x7f0a880fc7a0 msgr2=0x7f0a880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/1156960217 shutdown_connections 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.448+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/1156960217 wait complete. 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.449+0000 7f0a8e9ae640 1 Processor -- start 2024-09-12T01:08:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.449+0000 7f0a8e9ae640 1 -- start start 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 0x7f0a881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a8810c170 0x7f0a88101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88111fc0 con 0x7f0a88103660 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88111e40 con 0x7f0a8810c170 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88112140 con 0x7f0a88102d40 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57708/0 (socket says 172.21.15.16:57708) 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 -- 172.21.15.16:0/2671219060 learned_addr learned my addr 172.21.15.16:0/2671219060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a8810c170 0x7f0a88101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a87fff640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 0x7f0a881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 -- 172.21.15.16:0/2671219060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 msgr2=0x7f0a881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 0x7f0a881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 -- 172.21.15.16:0/2671219060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a8810c170 msgr2=0x7f0a88101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a8810c170 0x7f0a88101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 -- 172.21.15.16:0/2671219060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a881a68c0 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a87fff640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 0x7f0a881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a877fe640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00d740 tx=0x7f0a7c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a857fa640 1 -- 172.21.15.16:0/2671219060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c017070 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a881a6bb0 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a857fa640 1 -- 172.21.15.16:0/2671219060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c00de50 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a857fa640 1 -- 172.21.15.16:0/2671219060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00f620 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.450+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a881a7140 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.453+0000 7f0a857fa640 1 -- 172.21.15.16:0/2671219060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0a7c009c30 con 0x7f0a88103660 2024-09-12T01:08:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.453+0000 7f0a90d87740 1 Processor -- start 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.453+0000 7f0a90d87740 1 -- start start 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb59838090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb598385d0 0x55bb59838a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb5979e2f0 con 0x55bb59843370 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb59745cf0 con 0x55bb59839b40 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a90d87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb5979a2b0 con 0x55bb598385d0 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb59838090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57722/0 (socket says 172.21.15.16:57722) 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a877fe640 1 -- 172.21.15.16:0/318227530 learned_addr learned my addr 172.21.15.16:0/318227530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a857fa640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 0x7f0a700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a87fff640 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb598385d0 0x55bb59838a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 0x7f0a700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/318227530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb598385d0 msgr2=0x55bb59838a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a857fa640 1 -- 172.21.15.16:0/2671219060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a7c0cf010 con 0x7f0a88103660 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb598385d0 0x55bb59838a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/318227530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 msgr2=0x55bb59845760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/318227530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5972a3a0 con 0x55bb59839b40 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.454+0000 7f0a877fe640 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb59838090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0a800138f0 tx=0x7f0a80013db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb127784669dc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 0x7f0a700a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000a000 tx=0x7f0a8001d000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a6e7fc640 1 -- 172.21.15.16:0/318227530 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80022070 con 0x55bb59839b40 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a6e7fc640 1 -- 172.21.15.16:0/318227530 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a80013250 con 0x55bb59839b40 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a6e7fc640 1 -- 172.21.15.16:0/318227530 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800126b0 con 0x55bb59839b40 2024-09-12T01:08:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 -- 172.21.15.16:0/318227530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 msgr2=0x55bb59838090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb59838090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0a800138f0 tx=0x7f0a80013db0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 -- 172.21.15.16:0/318227530 shutdown_connections 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59845760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb598385d0 0x55bb59838a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 --2- 172.21.15.16:0/318227530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb59838090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.455+0000 7f0a90d87740 1 -- 172.21.15.16:0/318227530 >> 172.21.15.16:0/318227530 conn(0x55bb59801310 msgr2=0x55bb59834c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- 172.21.15.16:0/318227530 shutdown_connections 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- 172.21.15.16:0/318227530 wait complete. 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- start start 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 Processor -- start 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb598385d0 0x55bb59842d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59839b40 0x55bb59840010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb59846010 con 0x55bb59843370 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bb59759060 con 0x55bb598385d0 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb59846190 con 0x55bb59839b40 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a90d87740 10 client.0 _ll_register_callbacks cb 0x55bb594ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.456+0000 7f0a8cf4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57732/0 (socket says 172.21.15.16:57732) 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/3395975421 learned_addr learned my addr 172.21.15.16:0/3395975421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb598385d0 0x55bb59842d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a877fe640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59839b40 0x55bb59840010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/3395975421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59839b40 msgr2=0x55bb59840010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59839b40 0x55bb59840010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb598385d0 msgr2=0x55bb59842d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb598385d0 0x55bb59842d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 -- 172.21.15.16:0/3395975421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5950f3c0 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0a80007970 tx=0x7f0a80012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731386284166611 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a677fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80022040 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a80013250 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a80004bf0 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a80038050 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a90d87740 10 client.19293 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.457+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb59851b00 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a80013250 con 0x55bb59843370 2024-09-12T01:08:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a80005050 con 0x55bb59843370 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 10 client.19293 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 10 client.19293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 20 client.19293 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb59852eb0 0x55bb598552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 4 client.19293 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb597ab860 con 0x55bb59852eb0 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.458+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb59852eb0 0x55bb598552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb59852eb0 0x55bb598552c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcc90f1df4de8e1a server_cookie=3876167b3fa46f6a in_seq=0 out_seq=0 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a677fe640 10 client.19293 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a677fe640 1 -- 172.21.15.16:0/3395975421 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb597ab860 con 0x55bb59852eb0 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a677fe640 10 client.19293 handle_command_reply: tid=0 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 client.19293 shutdown 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 msgr2=0x55bb59840600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f0a80007970 tx=0x7f0a80012680 comp rx=0 tx=0).stop 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 shutdown_connections 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb59852eb0 0x55bb598552c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.459+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb59843370 0x55bb59840600 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59839b40 0x55bb59840010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3395975421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb598385d0 0x55bb59842d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 >> 172.21.15.16:0/3395975421 conn(0x55bb59801310 msgr2=0x55bb5983a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a677fe640 0 client.19293 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 shutdown_connections 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 1 -- 172.21.15.16:0/3395975421 wait complete. 2024-09-12T01:08:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.460+0000 7f0a90d87740 20 client.19293 trim_cache size 0 max 16384 2024-09-12T01:08:37.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 Processor -- start 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 -- start start 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5983f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5983b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881afa20 con 0x55bb598385d0 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881b1560 con 0x55bb59839b40 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a90d87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881b00f0 con 0x55bb59843370 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57734/0 (socket says 172.21.15.16:57734) 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 -- 172.21.15.16:0/1754577421 learned_addr learned my addr 172.21.15.16:0/1754577421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a877fe640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5983f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5983b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 -- 172.21.15.16:0/1754577421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 msgr2=0x55bb5983b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5983b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 -- 172.21.15.16:0/1754577421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 msgr2=0x55bb5983f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5983f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a87fff640 1 -- 172.21.15.16:0/1754577421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb598430d0 con 0x55bb598385d0 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a877fe640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5983f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.473+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5983b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a87fff640 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802aa40 tx=0x7f0a78031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e826938358a66d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a6dffb640 1 -- 172.21.15.16:0/1754577421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78035040 con 0x55bb598385d0 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a6dffb640 1 -- 172.21.15.16:0/1754577421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7804a070 con 0x55bb598385d0 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a6dffb640 1 -- 172.21.15.16:0/1754577421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78031810 con 0x55bb598385d0 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a90d87740 1 -- 172.21.15.16:0/1754577421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 msgr2=0x55bb5984f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.474+0000 7f0a90d87740 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802aa40 tx=0x7f0a78031400 comp rx=0 tx=0).stop 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 -- 172.21.15.16:0/1754577421 shutdown_connections 2024-09-12T01:08:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5983b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5983f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 --2- 172.21.15.16:0/1754577421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5984f2b0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 -- 172.21.15.16:0/1754577421 >> 172.21.15.16:0/1754577421 conn(0x55bb59801310 msgr2=0x55bb5983e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 -- 172.21.15.16:0/1754577421 shutdown_connections 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 -- 172.21.15.16:0/1754577421 wait complete. 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 -- start start 2024-09-12T01:08:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 Processor -- start 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.475+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5984b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5984b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881b0e90 con 0x55bb598385d0 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881b07c0 con 0x55bb59839b40 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a881ada20 con 0x55bb59843370 2024-09-12T01:08:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57740/0 (socket says 172.21.15.16:57740) 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 10 client.0 _ll_register_callbacks cb 0x7f0a80020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 -- 172.21.15.16:0/3624879978 learned_addr learned my addr 172.21.15.16:0/3624879978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a877fe640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5984b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a8cf4b640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5984b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 -- 172.21.15.16:0/3624879978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 msgr2=0x55bb5984b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5984b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 -- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 msgr2=0x55bb5984b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5984b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 -- 172.21.15.16:0/3624879978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5983c280 con 0x55bb598385d0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a877fe640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5984b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a87fff640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802a4c0 tx=0x7f0a78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcd5e7f14bbb14d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a677fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a78052070 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a78005240 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.476+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a7804a070 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a78035040 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 10 client.19299 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bb598bcd60 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7804a070 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a780035f0 con 0x55bb598385d0 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 10 client.19299 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 10 client.19299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 20 client.19299 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb598bd410 0x55bb598bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 4 client.19299 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb597a5d70 con 0x55bb598bd410 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.477+0000 7f0a877fe640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb598bd410 0x55bb598bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a877fe640 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb598bd410 0x55bb598bf830 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7355cb68bde545e7 server_cookie=20ddcb777f3b9c0e in_seq=0 out_seq=0 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a677fe640 10 client.19299 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a677fe640 1 -- 172.21.15.16:0/3624879978 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb597a5d70 con 0x55bb598bd410 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a677fe640 10 client.19299 handle_command_reply: tid=0 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 client.19299 shutdown 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 msgr2=0x55bb5983dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f0a7802a4c0 tx=0x7f0a78000bc0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 shutdown_connections 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55bb598bd410 0x55bb598bf830 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb59843370 0x55bb5984b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bb59839b40 0x55bb5984b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a677fe640 0 client.19299 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 --2- 172.21.15.16:0/3624879978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb598385d0 0x55bb5983dc20 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 >> 172.21.15.16:0/3624879978 conn(0x55bb59801310 msgr2=0x55bb5983e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 shutdown_connections 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 1 -- 172.21.15.16:0/3624879978 wait complete. 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.478+0000 7f0a90d87740 20 client.19299 trim_cache size 0 max 16384 2024-09-12T01:08:37.484 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 msgr2=0x7f0a700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 0x7f0a700a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000a000 tx=0x7f0a8001d000 comp rx=0 tx=0).stop 2024-09-12T01:08:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 msgr2=0x7f0a88100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00d740 tx=0x7f0a7c00dc00 comp rx=0 tx=0).stop 2024-09-12T01:08:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 shutdown_connections 2024-09-12T01:08:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a700a2cf0 0x7f0a700a51b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a8810c170 0x7f0a88101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a88103660 0x7f0a88100ee0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 --2- 172.21.15.16:0/2671219060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a88102d40 0x7f0a881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 >> 172.21.15.16:0/2671219060 conn(0x7f0a880fc7a0 msgr2=0x7f0a880fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 shutdown_connections 2024-09-12T01:08:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:37.479+0000 7f0a8e9ae640 1 -- 172.21.15.16:0/2671219060 wait complete. 2024-09-12T01:08:38.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:08:38.603 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:39.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:38 smithi016 ceph-mon[30972]: pgmap v390: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- 172.21.15.16:0/4248333355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e410c170 msgr2=0x7f22e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/4248333355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e410c170 0x7f22e410e610 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ca0 tx=0x7f22dc02a950 comp rx=0 tx=0).stop 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- 172.21.15.16:0/4248333355 shutdown_connections 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/4248333355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e410c170 0x7f22e410e610 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/4248333355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4103660 0x7f22e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/4248333355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e4102d40 0x7f22e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- 172.21.15.16:0/4248333355 >> 172.21.15.16:0/4248333355 conn(0x7f22e40fc7a0 msgr2=0x7f22e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- 172.21.15.16:0/4248333355 shutdown_connections 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- 172.21.15.16:0/4248333355 wait complete. 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 Processor -- start 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.091+0000 7f22ea9b4640 1 -- start start 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4102d40 0x7f22e4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e410c170 0x7f22e419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4111f70 con 0x7f22e4103660 2024-09-12T01:08:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4111df0 con 0x7f22e4102d40 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22ea9b4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41120f0 con 0x7f22e410c170 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 -- 172.21.15.16:0/3117800300 learned_addr learned my addr 172.21.15.16:0/3117800300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e8f51640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e410c170 0x7f22e419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e3fff640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4102d40 0x7f22e4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 -- 172.21.15.16:0/3117800300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e410c170 msgr2=0x7f22e419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e410c170 0x7f22e419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 -- 172.21.15.16:0/3117800300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4102d40 msgr2=0x7f22e4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4102d40 0x7f22e4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 -- 172.21.15.16:0/3117800300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e419d320 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.092+0000 7f22e37fe640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c970 tx=0x7f22d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.093+0000 7f22e17fa640 1 -- 172.21.15.16:0/3117800300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8007bb0 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.093+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e419d610 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.093+0000 7f22e17fa640 1 -- 172.21.15.16:0/3117800300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d8007d50 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.093+0000 7f22e17fa640 1 -- 172.21.15.16:0/3117800300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8018820 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.093+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41a4ef0 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e17fa640 1 -- 172.21.15.16:0/3117800300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f22d80189c0 con 0x7f22e4103660 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 Processor -- start 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 -- start start 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a29370 0x55ff75a2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff759842f0 con 0x55ff75a1fb40 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ff7592bcf0 con 0x55ff75a29370 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff759802b0 con 0x55ff75a1e5d0 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e8f51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e8f51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50768/0 (socket says 172.21.15.16:50768) 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e8f51640 1 -- 172.21.15.16:0/688232041 learned_addr learned my addr 172.21.15.16:0/688232041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e17fa640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 0x7f22cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.096+0000 7f22e3fff640 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e37fe640 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a29370 0x55ff75a2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 -- 172.21.15.16:0/688232041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1e5d0 msgr2=0x55ff75a1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 -- 172.21.15.16:0/688232041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a29370 msgr2=0x55ff75a2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a29370 0x55ff75a2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 -- 172.21.15.16:0/688232041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff759103a0 con 0x55ff75a1fb40 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e17fa640 1 -- 172.21.15.16:0/3117800300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f22d80cf510 con 0x7f22e4103660 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 0x7f22cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22e8f51640 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009a20 tx=0x7f22dc02f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35342fc515b0590 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22ca7fc640 1 -- 172.21.15.16:0/688232041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc034040 con 0x55ff75a1fb40 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22ca7fc640 1 -- 172.21.15.16:0/688232041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc003e50 con 0x55ff75a1fb40 2024-09-12T01:08:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.097+0000 7f22ca7fc640 1 -- 172.21.15.16:0/688232041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc004860 con 0x55ff75a1fb40 2024-09-12T01:08:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 -- 172.21.15.16:0/688232041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 msgr2=0x55ff75a1e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22e8f51640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 0x7f22cc0a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f22dc049000 tx=0x7f22dc0307e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009a20 tx=0x7f22dc02f940 comp rx=0 tx=0).stop 2024-09-12T01:08:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 -- 172.21.15.16:0/688232041 shutdown_connections 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a29370 0x55ff75a2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/688232041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a1e090 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 -- 172.21.15.16:0/688232041 >> 172.21.15.16:0/688232041 conn(0x55ff759e7310 msgr2=0x55ff75a1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 -- 172.21.15.16:0/688232041 shutdown_connections 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.098+0000 7f22ecd8d740 1 -- 172.21.15.16:0/688232041 wait complete. 2024-09-12T01:08:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 -- start start 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 Processor -- start 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1fb40 0x55ff75a26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ff75a2c070 con 0x55ff75a1e5d0 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ff7593f060 con 0x55ff75a1fb40 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff75a2c1f0 con 0x55ff75a29370 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22ecd8d740 10 client.0 _ll_register_callbacks cb 0x55ff756928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50772/0 (socket says 172.21.15.16:50772) 2024-09-12T01:08:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22e3fff640 1 -- 172.21.15.16:0/706833498 learned_addr learned my addr 172.21.15.16:0/706833498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22e37fe640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1fb40 0x55ff75a26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.099+0000 7f22e8f51640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 -- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 msgr2=0x55ff75a265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 -- 172.21.15.16:0/706833498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1fb40 msgr2=0x55ff75a26000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1fb40 0x55ff75a26000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 -- 172.21.15.16:0/706833498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff756f53c0 con 0x55ff75a1e5d0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22e3fff640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f22d400be60 tx=0x7f22d4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93992142064ac605 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4004230 con 0x55ff75a1e5d0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f22d40043d0 con 0x55ff75a1e5d0 2024-09-12T01:08:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22dc003e50 con 0x55ff75a1e5d0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22d40106a0 con 0x55ff75a1e5d0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22ecd8d740 10 client.19311 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff75a37b50 con 0x55ff75a1e5d0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.100+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22dc003e50 con 0x55ff75a1e5d0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22d4019070 con 0x55ff75a1e5d0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 10 client.19311 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 10 client.19311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 20 client.19311 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75a3b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 4 client.19311 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff75991860 con 0x55ff75a39010 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22e37fe640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75a3b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22c37fe640 10 client.19311 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.101+0000 7f22e37fe640 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75a3b490 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e379c9011324874 server_cookie=661d193e6855f88e in_seq=0 out_seq=0 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22c37fe640 1 -- 172.21.15.16:0/706833498 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff75991860 con 0x55ff75a39010 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22c37fe640 10 client.19311 handle_command_reply: tid=0 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 client.19311 shutdown 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 msgr2=0x55ff75a28d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f22d400be60 tx=0x7f22d4012820 comp rx=0 tx=0).stop 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 shutdown_connections 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75a3b490 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22c37fe640 0 client.19311 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1fb40 0x55ff75a26000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/706833498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1e5d0 0x55ff75a28d50 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 >> 172.21.15.16:0/706833498 conn(0x55ff759e7310 msgr2=0x55ff75a20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 shutdown_connections 2024-09-12T01:08:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.102+0000 7f22ecd8d740 1 -- 172.21.15.16:0/706833498 wait complete. 2024-09-12T01:08:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.103+0000 7f22ecd8d740 20 client.19311 trim_cache size 0 max 16384 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.115+0000 7f22ecd8d740 1 Processor -- start 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 -- start start 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a40db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41ad810 con 0x55ff75a1fb40 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41af350 con 0x55ff75a1e5d0 2024-09-12T01:08:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41adee0 con 0x55ff75a29370 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50780/0 (socket says 172.21.15.16:50780) 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 -- 172.21.15.16:0/1037925516 learned_addr learned my addr 172.21.15.16:0/1037925516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e3fff640 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 -- 172.21.15.16:0/1037925516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 msgr2=0x55ff75a40db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a40db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 -- 172.21.15.16:0/1037925516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 msgr2=0x55ff75a1ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 -- 172.21.15.16:0/1037925516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff75a24550 con 0x55ff75a1fb40 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22e37fe640 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f22d8007590 tx=0x7f22d806eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab530096ff4e2d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.116+0000 7f22c9ffb640 1 -- 172.21.15.16:0/1037925516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d809c130 con 0x55ff75a1fb40 2024-09-12T01:08:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22c9ffb640 1 -- 172.21.15.16:0/1037925516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d809ba30 con 0x55ff75a1fb40 2024-09-12T01:08:39.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22c9ffb640 1 -- 172.21.15.16:0/1037925516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d80ccee0 con 0x55ff75a1fb40 2024-09-12T01:08:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- 172.21.15.16:0/1037925516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 msgr2=0x55ff75a35310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f22d8007590 tx=0x7f22d806eaa0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- 172.21.15.16:0/1037925516 shutdown_connections 2024-09-12T01:08:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a29370 0x55ff75a40db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a1fb40 0x55ff75a35310 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/1037925516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a1ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- 172.21.15.16:0/1037925516 >> 172.21.15.16:0/1037925516 conn(0x55ff759e7310 msgr2=0x55ff75a1a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- 172.21.15.16:0/1037925516 shutdown_connections 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- 172.21.15.16:0/1037925516 wait complete. 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.117+0000 7f22ecd8d740 1 -- start start 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 Processor -- start 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a2d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1fb40 0x55ff75a2d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41aec80 con 0x55ff75a29370 2024-09-12T01:08:39.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41ae5b0 con 0x55ff75a1e5d0 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22e41ab810 con 0x55ff75a1fb40 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22ecd8d740 10 client.0 _ll_register_callbacks cb 0x7f22d400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50796/0 (socket says 172.21.15.16:50796) 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 -- 172.21.15.16:0/2477336084 learned_addr learned my addr 172.21.15.16:0/2477336084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e3fff640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a2d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e37fe640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1fb40 0x55ff75a2d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 -- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1fb40 msgr2=0x55ff75a2d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1fb40 0x55ff75a2d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 -- 172.21.15.16:0/2477336084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 msgr2=0x55ff75a2d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a2d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e8f51640 1 -- 172.21.15.16:0/2477336084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff75aa2e50 con 0x55ff75a29370 2024-09-12T01:08:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.118+0000 7f22e3fff640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a2d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22e8f51640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f22dc030ce0 tx=0x7f22dc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d7aab7c0b4e1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc034040 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d809ba30 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f22dc003b80 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22dc009540 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 10 client.19317 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff75aa4130 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22d809ba30 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22dc0045a0 con 0x55ff75a29370 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 10 client.19317 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 10 client.19317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:08:39.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.119+0000 7f22ecd8d740 20 client.19317 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75aa6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22ecd8d740 4 client.19317 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff7598bd70 con 0x55ff75a39010 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22e3fff640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75aa6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22e3fff640 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75aa6470 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=875d0b17bd9491af server_cookie=d6416e81ea79e3d7 in_seq=0 out_seq=0 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22c37fe640 10 client.19317 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22c37fe640 1 -- 172.21.15.16:0/2477336084 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff7598bd70 con 0x55ff75a39010 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22c37fe640 10 client.19317 handle_command_reply: tid=0 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.120+0000 7f22ecd8d740 1 client.19317 shutdown 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 msgr2=0x55ff75a2dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f22dc030ce0 tx=0x7f22dc040000 comp rx=0 tx=0).stop 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 shutdown_connections 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff75a29370 0x55ff75a2dd20 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff75a1fb40 0x55ff75a2d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55ff75a39010 0x55ff75aa6470 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 --2- 172.21.15.16:0/2477336084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ff75a1e5d0 0x55ff75a2d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 >> 172.21.15.16:0/2477336084 conn(0x55ff759e7310 msgr2=0x55ff75a20110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22c37fe640 0 client.19317 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 shutdown_connections 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 1 -- 172.21.15.16:0/2477336084 wait complete. 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ecd8d740 20 client.19317 trim_cache size 0 max 16384 2024-09-12T01:08:39.126 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 msgr2=0x7f22cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.121+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 0x7f22cc0a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f22dc049000 tx=0x7f22dc0307e0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 msgr2=0x7f22e41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c970 tx=0x7f22d800ce30 comp rx=0 tx=0).stop 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 shutdown_connections 2024-09-12T01:08:39.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22cc0a2cf0 0x7f22cc0a51b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22e410c170 0x7f22e419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e4103660 0x7f22e41988b0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 --2- 172.21.15.16:0/3117800300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22e4102d40 0x7f22e4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 >> 172.21.15.16:0/3117800300 conn(0x7f22e40fc7a0 msgr2=0x7f22e40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 shutdown_connections 2024-09-12T01:08:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:39.122+0000 7f22ea9b4640 1 -- 172.21.15.16:0/3117800300 wait complete. 2024-09-12T01:08:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:38 smithi099 ceph-mon[35654]: pgmap v390: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:38 smithi159 ceph-mon[35566]: pgmap v390: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:40.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:08:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.757+0000 7fcbf1574640 1 -- 172.21.15.16:0/2577531369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 msgr2=0x7fcbec106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.757+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2577531369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec106e60 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009930 tx=0x7fcbdc02a420 comp rx=0 tx=0).stop 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- 172.21.15.16:0/2577531369 shutdown_connections 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2577531369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec101420 0x7fcbec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2577531369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec100a80 0x7fcbec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2577531369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec106e60 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- 172.21.15.16:0/2577531369 >> 172.21.15.16:0/2577531369 conn(0x7fcbec0fc7a0 msgr2=0x7fcbec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- 172.21.15.16:0/2577531369 shutdown_connections 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- 172.21.15.16:0/2577531369 wait complete. 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 Processor -- start 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- start start 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec100a80 0x7fcbec19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 0x7fcbec19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec110e80 con 0x7fcbec106a80 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec110d00 con 0x7fcbec100a80 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.758+0000 7fcbf1574640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec111000 con 0x7fcbec101420 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50820/0 (socket says 172.21.15.16:50820) 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 -- 172.21.15.16:0/2688247583 learned_addr learned my addr 172.21.15.16:0/2688247583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 0x7fcbec19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec100a80 0x7fcbec19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 -- 172.21.15.16:0/2688247583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 msgr2=0x7fcbec19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 0x7fcbec19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 -- 172.21.15.16:0/2688247583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec100a80 msgr2=0x7fcbec19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec100a80 0x7fcbec19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 -- 172.21.15.16:0/2688247583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbec19f5f0 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 0x7fcbec19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbeb7fe640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400cce0 tx=0x7fcbe400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbe8ff9640 1 -- 172.21.15.16:0/2688247583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400f450 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbe8ff9640 1 -- 172.21.15.16:0/2688247583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe400fad0 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbec19f8e0 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbe8ff9640 1 -- 172.21.15.16:0/2688247583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4018c10 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.759+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbec1a71c0 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.762+0000 7fcbe8ff9640 1 -- 172.21.15.16:0/2688247583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcbe400f5f0 con 0x7fcbec106a80 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.762+0000 7fcbf394d740 1 Processor -- start 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.762+0000 7fcbf394d740 1 -- start start 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.762+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6372090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f6372a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbf394d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614f62d82f0 con 0x5614f637d370 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbf394d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5614f627fcf0 con 0x5614f6373b40 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbf394d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5614f62d42b0 con 0x5614f63725d0 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe8ff9640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 0x7fcbd80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50822/0 (socket says 172.21.15.16:50822) 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbeb7fe640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 0x7fcbd80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 -- 172.21.15.16:0/2659792683 learned_addr learned my addr 172.21.15.16:0/2659792683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6372090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f6372a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe8ff9640 1 -- 172.21.15.16:0/2688247583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fcbe40cfa80 con 0x7fcbec106a80 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 -- 172.21.15.16:0/2659792683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 msgr2=0x5614f6372a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f6372a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 -- 172.21.15.16:0/2659792683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 msgr2=0x5614f6372090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6372090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 -- 172.21.15.16:0/2659792683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f62643a0 con 0x5614f637d370 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4014a30 tx=0x7fcbd400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ede147db0e63c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbeb7fe640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 0x7fcbd80a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40cd280 tx=0x7fcbe40186b0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.763+0000 7fcbe17fa640 1 -- 172.21.15.16:0/2659792683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40142d0 con 0x5614f637d370 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbe17fa640 1 -- 172.21.15.16:0/2659792683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd400f820 con 0x5614f637d370 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbe17fa640 1 -- 172.21.15.16:0/2659792683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4003850 con 0x5614f637d370 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 -- 172.21.15.16:0/2659792683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 msgr2=0x5614f637f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4014a30 tx=0x7fcbd400c570 comp rx=0 tx=0).stop 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 -- 172.21.15.16:0/2659792683 shutdown_connections 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f637f760 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f6372a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2659792683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6372090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 -- 172.21.15.16:0/2659792683 >> 172.21.15.16:0/2659792683 conn(0x5614f633b310 msgr2=0x5614f636ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 -- 172.21.15.16:0/2659792683 shutdown_connections 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.764+0000 7fcbf394d740 1 -- 172.21.15.16:0/2659792683 wait complete. 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 -- start start 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 Processor -- start 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f637cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 0x5614f637a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614f6380010 con 0x5614f6373b40 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5614f6293060 con 0x5614f637d370 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5614f6380190 con 0x5614f63725d0 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbf394d740 10 client.0 _ll_register_callbacks cb 0x5614f5fe68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50836/0 (socket says 172.21.15.16:50836) 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 -- 172.21.15.16:0/2025916554 learned_addr learned my addr 172.21.15.16:0/2025916554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f637cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 0x5614f637a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 -- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 msgr2=0x5614f637cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f637cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 -- 172.21.15.16:0/2025916554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 msgr2=0x5614f637a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 0x5614f637a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbeaffd640 1 -- 172.21.15.16:0/2025916554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f60493c0 con 0x5614f6373b40 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.765+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 0x5614f637a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc031550 tx=0x7fcbdc031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f8015d0465b03d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc035070 con 0x5614f6373b40 2024-09-12T01:08:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fcbdc004240 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd400f820 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbdc004440 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 10 client.19329 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614f638bae0 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd400f820 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcbdc004bc0 con 0x5614f6373b40 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 10 client.19329 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 20 client.19329 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f638cef0 0x5614f638f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 4 client.19329 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.766+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f62e5860 con 0x5614f638cef0 2024-09-12T01:08:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.767+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f638cef0 0x5614f638f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.767+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f638cef0 0x5614f638f350 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16baa9229681e470 server_cookie=62ce7f10bab7d9dd in_seq=0 out_seq=0 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.767+0000 7fcbd27fc640 10 client.19329 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbd27fc640 1 -- 172.21.15.16:0/2025916554 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614f62e5860 con 0x5614f638cef0 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbd27fc640 10 client.19329 handle_command_reply: tid=0 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 client.19329 shutdown 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 msgr2=0x5614f637a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc031550 tx=0x7fcbdc031860 comp rx=0 tx=0).stop 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 shutdown_connections 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f638cef0 0x5614f638f350 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f637d370 0x5614f637a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f6373b40 0x5614f637a030 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbd27fc640 0 client.19329 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:40.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 --2- 172.21.15.16:0/2025916554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f63725d0 0x5614f637cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 >> 172.21.15.16:0/2025916554 conn(0x5614f633b310 msgr2=0x5614f6374960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 shutdown_connections 2024-09-12T01:08:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 1 -- 172.21.15.16:0/2025916554 wait complete. 2024-09-12T01:08:40.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.768+0000 7fcbf394d740 20 client.19329 trim_cache size 0 max 16384 2024-09-12T01:08:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.781+0000 7fcbf394d740 1 Processor -- start 2024-09-12T01:08:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.781+0000 7fcbf394d740 1 -- start start 2024-09-12T01:08:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f63725d0 0x5614f637bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f6373b40 0x5614f6375120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1af990 con 0x5614f637d370 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b14d0 con 0x5614f63725d0 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbf394d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b0060 con 0x5614f6373b40 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 -- 172.21.15.16:0/30517442 learned_addr learned my addr 172.21.15.16:0/30517442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f63725d0 0x5614f637bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f6373b40 0x5614f6375120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 -- 172.21.15.16:0/30517442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f6373b40 msgr2=0x5614f6375120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f6373b40 0x5614f6375120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 -- 172.21.15.16:0/30517442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f63725d0 msgr2=0x5614f637bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f63725d0 0x5614f637bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 -- 172.21.15.16:0/30517442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f638f9d0 con 0x5614f637d370 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4014fb0 tx=0x7fcbd400cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a7fdea9981c085 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.782+0000 7fcbe0ff9640 1 -- 172.21.15.16:0/30517442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4014150 con 0x5614f637d370 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbe0ff9640 1 -- 172.21.15.16:0/30517442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd4005020 con 0x5614f637d370 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbe0ff9640 1 -- 172.21.15.16:0/30517442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400fa80 con 0x5614f637d370 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 -- 172.21.15.16:0/30517442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 msgr2=0x5614f6375660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4014fb0 tx=0x7fcbd400cfd0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 -- 172.21.15.16:0/30517442 shutdown_connections 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f637d370 0x5614f6375660 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f6373b40 0x5614f6375120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 --2- 172.21.15.16:0/30517442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f63725d0 0x5614f637bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 -- 172.21.15.16:0/30517442 >> 172.21.15.16:0/30517442 conn(0x5614f633b310 msgr2=0x5614f637b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 -- 172.21.15.16:0/30517442 shutdown_connections 2024-09-12T01:08:40.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.783+0000 7fcbf394d740 1 -- 172.21.15.16:0/30517442 wait complete. 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 -- start start 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 Processor -- start 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6376ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f637d370 0x5614f6377550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b0e00 con 0x5614f63725d0 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1b0730 con 0x5614f6373b40 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbec1ad990 con 0x5614f637d370 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbf394d740 10 client.0 _ll_register_callbacks cb 0x7fcbdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50856/0 (socket says 172.21.15.16:50856) 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 -- 172.21.15.16:0/3161785212 learned_addr learned my addr 172.21.15.16:0/3161785212 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 -- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f637d370 msgr2=0x5614f6377550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f637d370 0x5614f6377550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 -- 172.21.15.16:0/3161785212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 msgr2=0x5614f6376ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6376ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.784+0000 7fcbe3fff640 1 -- 172.21.15.16:0/3161785212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f63f6bf0 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbe3fff640 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c900 tx=0x7fcbd40141e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2c92fe681c0fda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4004030 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fcbd40041d0 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd4005020 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcbd40043d0 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 10 client.19335 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614f63f7d40 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbd4005020 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcbd4004bb0 con 0x5614f63725d0 2024-09-12T01:08:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 10 client.19335 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 20 client.19335 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f63f8240 0x5614f63fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 4 client.19335 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.785+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614f62dfd70 con 0x5614f63f8240 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f63f8240 0x5614f63fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbeaffd640 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f63f8240 0x5614f63fa660 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca132df43a485e8 server_cookie=3fa8230ef08b93ee in_seq=0 out_seq=0 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbd27fc640 10 client.19335 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbd27fc640 1 -- 172.21.15.16:0/3161785212 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614f62dfd70 con 0x5614f63f8240 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbd27fc640 10 client.19335 handle_command_reply: tid=0 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 client.19335 shutdown 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 msgr2=0x5614f6385c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400c900 tx=0x7fcbd40141e0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 shutdown_connections 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5614f63f8240 0x5614f63fa660 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.786+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614f637d370 0x5614f6377550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbd27fc640 0 client.19335 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5614f6373b40 0x5614f6376ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 1 --2- 172.21.15.16:0/3161785212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614f63725d0 0x5614f6385c90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 >> 172.21.15.16:0/3161785212 conn(0x5614f633b310 msgr2=0x5614f637c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 shutdown_connections 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 1 -- 172.21.15.16:0/3161785212 wait complete. 2024-09-12T01:08:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf394d740 20 client.19335 trim_cache size 0 max 16384 2024-09-12T01:08:40.794 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 msgr2=0x7fcbd80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 0x7fcbd80a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40cd280 tx=0x7fcbe40186b0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 msgr2=0x7fcbec19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400cce0 tx=0x7fcbe400a620 comp rx=0 tx=0).stop 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 shutdown_connections 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcbd80a2d70 0x7fcbd80a5230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec106a80 0x7fcbec19ef10 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbec101420 0x7fcbec19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 --2- 172.21.15.16:0/2688247583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbec100a80 0x7fcbec19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 >> 172.21.15.16:0/2688247583 conn(0x7fcbec0fc7a0 msgr2=0x7fcbec06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 shutdown_connections 2024-09-12T01:08:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:40.787+0000 7fcbf1574640 1 -- 172.21.15.16:0/2688247583 wait complete. 2024-09-12T01:08:41.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:40 smithi016 ceph-mon[30972]: pgmap v391: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T01:08:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:40 smithi099 ceph-mon[35654]: pgmap v391: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T01:08:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:40 smithi159 ceph-mon[35566]: pgmap v391: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T01:08:42.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:08:42.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:08:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:43.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:42 smithi016 ceph-mon[30972]: pgmap v392: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:42 smithi099 ceph-mon[35654]: pgmap v392: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:42 smithi159 ceph-mon[35566]: pgmap v392: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/1240645826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58100e20 msgr2=0x7f7b58101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/1240645826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58100e20 0x7f7b58101200 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009900 tx=0x7f7b5402a480 comp rx=0 tx=0).stop 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/1240645826 shutdown_connections 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/1240645826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 0x7f7b5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/1240645826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58102570 0x7f7b581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/1240645826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58100e20 0x7f7b58101200 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/1240645826 >> 172.21.15.16:0/1240645826 conn(0x7f7b580fc7d0 msgr2=0x7f7b580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/1240645826 shutdown_connections 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/1240645826 wait complete. 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.437+0000 7f7b5f7f8640 1 Processor -- start 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 -- start start 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 0x7f7b58198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 0x7f7b5819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58110fa0 con 0x7f7b58102570 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58110e20 con 0x7f7b58100e20 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5f7f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58111120 con 0x7f7b58102ef0 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50878/0 (socket says 172.21.15.16:50878) 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3034312411 learned_addr learned my addr 172.21.15.16:0/3034312411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 0x7f7b58198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 0x7f7b5819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3034312411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 msgr2=0x7f7b5819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 0x7f7b5819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3034312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 msgr2=0x7f7b58198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 0x7f7b58198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3034312411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5819d510 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 0x7f7b58198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.438+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c890 tx=0x7f7b4800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.439+0000 7f7b4e7fc640 1 -- 172.21.15.16:0/3034312411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48004220 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.439+0000 7f7b4e7fc640 1 -- 172.21.15.16:0/3034312411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b480043c0 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.439+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5819d800 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.439+0000 7f7b4e7fc640 1 -- 172.21.15.16:0/3034312411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b480049e0 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.439+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581a50e0 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.441+0000 7f7b4e7fc640 1 -- 172.21.15.16:0/3034312411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b480079b0 con 0x7f7b58102570 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.441+0000 7f7b61bd1740 1 Processor -- start 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.441+0000 7f7b61bd1740 1 -- start start 2024-09-12T01:08:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f720547f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720555130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f7204ad280 con 0x55f720548480 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f720454b10 con 0x55f720552d40 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f7204a9240 con 0x55f720547b60 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b4e7fc640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50886/0 (socket says 172.21.15.16:50886) 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 -- 172.21.15.16:0/1300840659 learned_addr learned my addr 172.21.15.16:0/1300840659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b4e7fc640 1 -- 172.21.15.16:0/3034312411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7b480cf560 con 0x7f7b58102570 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720555130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f720547f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 -- 172.21.15.16:0/1300840659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 msgr2=0x55f720547f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f720547f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 -- 172.21.15.16:0/1300840659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 msgr2=0x55f720555130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720555130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 -- 172.21.15.16:0/1300840659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7204611f0 con 0x55f720548480 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f720547f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5d594640 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7b540314e0 tx=0x7f7b54004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d154cca967414cb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.442+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 0x7f7b440a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c650 tx=0x7f7b50009810 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b437fe640 1 -- 172.21.15.16:0/1300840659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54035030 con 0x55f720548480 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b437fe640 1 -- 172.21.15.16:0/1300840659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54003ad0 con 0x55f720548480 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b437fe640 1 -- 172.21.15.16:0/1300840659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54031c70 con 0x55f720548480 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 -- 172.21.15.16:0/1300840659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 msgr2=0x55f720552800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7b540314e0 tx=0x7f7b54004200 comp rx=0 tx=0).stop 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 -- 172.21.15.16:0/1300840659 shutdown_connections 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720555130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f720552800 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/1300840659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f720547f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 -- 172.21.15.16:0/1300840659 >> 172.21.15.16:0/1300840659 conn(0x55f7205440f0 msgr2=0x55f7205444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 -- 172.21.15.16:0/1300840659 shutdown_connections 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.443+0000 7f7b61bd1740 1 -- 172.21.15.16:0/1300840659 wait complete. 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 -- start start 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 Processor -- start 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720548480 0x55f7205500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720550620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f720555910 con 0x55f720547b60 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f720467f10 con 0x55f720552d40 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f720555a90 con 0x55f720548480 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b61bd1740 10 client.0 _ll_register_callbacks cb 0x55f720430a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b5d594640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b5d594640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50898/0 (socket says 172.21.15.16:50898) 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.444+0000 7f7b5d594640 1 -- 172.21.15.16:0/492937487 learned_addr learned my addr 172.21.15.16:0/492937487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720548480 0x55f7205500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720550620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 -- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720548480 msgr2=0x55f7205500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720548480 0x55f7205500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 -- 172.21.15.16:0/492937487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 msgr2=0x55f720550620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720550620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 -- 172.21.15.16:0/492937487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72021e6a0 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720550620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b5d594640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402a4c0 tx=0x7f7b540316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907943dad0c90e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54035030 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b54003ad0 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7b540035f0 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b540037f0 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b61bd1740 10 client.19347 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.445+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f720561410 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54003ad0 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b5403cb20 con 0x55f720547b60 2024-09-12T01:08:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 10 client.19347 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 10 client.19347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 20 client.19347 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f720562760 0x55f720564b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 4 client.19347 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7204ba7f0 con 0x55f720562760 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.446+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f720562760 0x55f720564b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.447+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f720562760 0x55f720564b50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2065cfb676b8d123 server_cookie=ad9deb2eff01c7dd in_seq=0 out_seq=0 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.447+0000 7f7b40ff9640 10 client.19347 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b40ff9640 1 -- 172.21.15.16:0/492937487 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7204ba7f0 con 0x55f720562760 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b40ff9640 10 client.19347 handle_command_reply: tid=0 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 client.19347 shutdown 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 msgr2=0x55f72054fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7b5402a4c0 tx=0x7f7b540316c0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 shutdown_connections 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f720562760 0x55f720564b50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720552d40 0x55f720550620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720548480 0x55f7205500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b40ff9640 0 client.19347 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/492937487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720547b60 0x55f72054fba0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 >> 172.21.15.16:0/492937487 conn(0x55f7205440f0 msgr2=0x55f720543270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 shutdown_connections 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 1 -- 172.21.15.16:0/492937487 wait complete. 2024-09-12T01:08:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.448+0000 7f7b61bd1740 20 client.19347 trim_cache size 0 max 16384 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 Processor -- start 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 -- start start 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f72054a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 0x55f72054ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581ad920 con 0x55f720552d40 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581af460 con 0x55f720548480 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581adff0 con 0x55f720547b60 2024-09-12T01:08:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50904/0 (socket says 172.21.15.16:50904) 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 -- 172.21.15.16:0/2524226069 learned_addr learned my addr 172.21.15.16:0/2524226069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 0x55f72054ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5d594640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f72054a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 -- 172.21.15.16:0/2524226069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 msgr2=0x55f72054a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f72054a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 -- 172.21.15.16:0/2524226069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 msgr2=0x55f72054ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 0x55f72054ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5dd95640 1 -- 172.21.15.16:0/2524226069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7205482b0 con 0x55f720552d40 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.466+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 0x55f72054ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f7b500070b0 tx=0x7f7b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f693d4d0ecbd9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b42ffd640 1 -- 172.21.15.16:0/2524226069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50016070 con 0x55f720552d40 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b42ffd640 1 -- 172.21.15.16:0/2524226069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5001ae30 con 0x55f720552d40 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b42ffd640 1 -- 172.21.15.16:0/2524226069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50019390 con 0x55f720552d40 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b61bd1740 1 -- 172.21.15.16:0/2524226069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 msgr2=0x55f72054d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f7b500070b0 tx=0x7f7b50004a40 comp rx=0 tx=0).stop 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.467+0000 7f7b61bd1740 1 -- 172.21.15.16:0/2524226069 shutdown_connections 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720552d40 0x55f72054d340 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720548480 0x55f72054ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/2524226069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720547b60 0x55f72054a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 -- 172.21.15.16:0/2524226069 >> 172.21.15.16:0/2524226069 conn(0x55f7205440f0 msgr2=0x55f72055e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 -- 172.21.15.16:0/2524226069 shutdown_connections 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 -- 172.21.15.16:0/2524226069 wait complete. 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 -- start start 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.468+0000 7f7b61bd1740 1 Processor -- start 2024-09-12T01:08:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 0x55f720568160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720552d40 0x55f7205659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581aed90 con 0x55f720548480 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581ae6c0 con 0x55f720547b60 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581ab920 con 0x55f720552d40 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 10 client.0 _ll_register_callbacks cb 0x7f7b48004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50920/0 (socket says 172.21.15.16:50920) 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3492270652 learned_addr learned my addr 172.21.15.16:0/3492270652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5dd95640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720552d40 0x55f7205659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 0x55f720568160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720552d40 msgr2=0x55f7205659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720552d40 0x55f7205659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3492270652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 msgr2=0x55f720568160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 0x55f720568160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 -- 172.21.15.16:0/3492270652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f720566090 con 0x55f720548480 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 0x55f720568160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b5cd93640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f7b48016000 tx=0x7f7b480054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebce1e86a3376a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4801d040 con 0x55f720548480 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f7b480980f0 con 0x55f720548480 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.469+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5001ae30 con 0x55f720548480 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b480a46f0 con 0x55f720548480 2024-09-12T01:08:43.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 10 client.19353 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7205ccfd0 con 0x55f720548480 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5001ae30 con 0x55f720548480 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b480a42f0 con 0x55f720548480 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 10 client.19353 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 10 client.19353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 20 client.19353 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f7205cd6a0 0x55f7205cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 4 client.19353 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f7204b4d00 con 0x55f7205cd6a0 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.470+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f7205cd6a0 0x55f7205cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.471+0000 7f7b5d594640 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f7205cd6a0 0x55f7205cfa90 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19bc8f24edf9518 server_cookie=898734d67121621e in_seq=0 out_seq=0 2024-09-12T01:08:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.471+0000 7f7b40ff9640 10 client.19353 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b40ff9640 1 -- 172.21.15.16:0/3492270652 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f7204b4d00 con 0x55f7205cd6a0 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b40ff9640 10 client.19353 handle_command_reply: tid=0 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 client.19353 shutdown 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 msgr2=0x55f7205654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f7b48016000 tx=0x7f7b480054a0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 shutdown_connections 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f7205cd6a0 0x55f7205cfa90 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f720552d40 0x55f7205659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f720548480 0x55f7205654b0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 --2- 172.21.15.16:0/3492270652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f720547b60 0x55f720568160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 >> 172.21.15.16:0/3492270652 conn(0x55f7205440f0 msgr2=0x55f720542a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.472+0000 7f7b40ff9640 0 client.19353 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 shutdown_connections 2024-09-12T01:08:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b61bd1740 1 -- 172.21.15.16:0/3492270652 wait complete. 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b61bd1740 20 client.19353 trim_cache size 0 max 16384 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b00510a1-6d94-4f97-a196-aaf5d02258ca", 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:08:43.480 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 msgr2=0x7f7b440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 0x7f7b440a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c650 tx=0x7f7b50009810 comp rx=0 tx=0).stop 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 msgr2=0x7f7b58198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c890 tx=0x7f7b4800cd50 comp rx=0 tx=0).stop 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 shutdown_connections 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b440a2cf0 0x7f7b440a51b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b58102ef0 0x7f7b5819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b58102570 0x7f7b58198aa0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 --2- 172.21.15.16:0/3034312411 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b58100e20 0x7f7b58198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.473+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 >> 172.21.15.16:0/3034312411 conn(0x7f7b580fc7d0 msgr2=0x7f7b58069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.474+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 shutdown_connections 2024-09-12T01:08:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:43.474+0000 7f7b5f7f8640 1 -- 172.21.15.16:0/3034312411 wait complete. 2024-09-12T01:08:44.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:43 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:08:44.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:43 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:08:44.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:43 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:43 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:08:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:43 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:08:44.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:43 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:08:44.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:43 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:08:44.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:43 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:08:44.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:43 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:08:44.558 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:44.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:44 smithi016 ceph-mon[30972]: pgmap v393: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:44.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:44 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/4220481479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0102d40 msgr2=0x7fd7e0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/4220481479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0102d40 0x7fd7e0103120 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009900 tx=0x7fd7d002a480 comp rx=0 tx=0).stop 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/4220481479 shutdown_connections 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/4220481479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 0x7fd7e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/4220481479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0103660 0x7fd7e010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/4220481479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0102d40 0x7fd7e0103120 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/4220481479 >> 172.21.15.16:0/4220481479 conn(0x7fd7e00fc7a0 msgr2=0x7fd7e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/4220481479 shutdown_connections 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.052+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/4220481479 wait complete. 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.053+0000 7fd7e4cb4640 1 Processor -- start 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.053+0000 7fd7e4cb4640 1 -- start start 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0102d40 0x7fd7e01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 0x7fd7e019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0111f70 con 0x7fd7e0103660 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0111df0 con 0x7fd7e010c170 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7e4cb4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01120f0 con 0x7fd7e0102d40 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50938/0 (socket says 172.21.15.16:50938) 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/1483270174 learned_addr learned my addr 172.21.15.16:0/1483270174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 0x7fd7e019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0102d40 0x7fd7e01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/1483270174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0102d40 msgr2=0x7fd7e01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0102d40 0x7fd7e01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/1483270174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 msgr2=0x7fd7e019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 0x7fd7e019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/1483270174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e019d490 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400d750 tx=0x7fd7d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.054+0000 7fd7cf7fe640 1 -- 172.21.15.16:0/1483270174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4017070 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.055+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e019d780 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.055+0000 7fd7cf7fe640 1 -- 172.21.15.16:0/1483270174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d4005020 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.055+0000 7fd7cf7fe640 1 -- 172.21.15.16:0/1483270174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4004d00 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.055+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e01a5060 con 0x7fd7e0103660 2024-09-12T01:08:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.057+0000 7fd7cf7fe640 1 -- 172.21.15.16:0/1483270174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd7d4009a30 con 0x7fd7e0103660 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.057+0000 7fd7cf7fe640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 0x7fd7c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.057+0000 7fd7e708d740 1 Processor -- start 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7cf7fe640 1 -- 172.21.15.16:0/1483270174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd7d40ceec0 con 0x7fd7e0103660 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 0x7fd7c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 -- start start 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c41780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c50550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba23ba72e0 con 0x55ba23c50a90 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ba23b4ece0 con 0x55ba23c413a0 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7e708d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ba23ba32a0 con 0x55ba23c41cc0 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50952/0 (socket says 172.21.15.16:50952) 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 -- 172.21.15.16:0/169946407 learned_addr learned my addr 172.21.15.16:0/169946407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 0x7fd7c80a51b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0005ab0 tx=0x7fd7d0005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c50550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7de59c640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c41780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 -- 172.21.15.16:0/169946407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 msgr2=0x55ba23c50550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c50550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 -- 172.21.15.16:0/169946407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 msgr2=0x55ba23c41780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c41780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ded9d640 1 -- 172.21.15.16:0/169946407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba23b33440 con 0x55ba23c50a90 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.058+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c50550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.059+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009f70 tx=0x7fd7d8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf24ba584c2f214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.059+0000 7fd7ccff9640 1 -- 172.21.15.16:0/169946407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8016070 con 0x55ba23c50a90 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.059+0000 7fd7ccff9640 1 -- 172.21.15.16:0/169946407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8004d20 con 0x55ba23c50a90 2024-09-12T01:08:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.059+0000 7fd7ccff9640 1 -- 172.21.15.16:0/169946407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8012850 con 0x55ba23c50a90 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- 172.21.15.16:0/169946407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 msgr2=0x55ba23c52e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009f70 tx=0x7fd7d8010800 comp rx=0 tx=0).stop 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- 172.21.15.16:0/169946407 shutdown_connections 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c52e80 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c50550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 --2- 172.21.15.16:0/169946407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c41780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- 172.21.15.16:0/169946407 >> 172.21.15.16:0/169946407 conn(0x55ba23c3da30 msgr2=0x55ba23c3de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- 172.21.15.16:0/169946407 shutdown_connections 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- 172.21.15.16:0/169946407 wait complete. 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 -- start start 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.060+0000 7fd7e708d740 1 Processor -- start 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c50410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ba23c536c0 con 0x55ba23c50a90 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ba23b62050 con 0x55ba23c413a0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ba23c53840 con 0x55ba23c41cc0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 10 client.0 _ll_register_callbacks cb 0x55ba23ba2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7de59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c50410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50964/0 (socket says 172.21.15.16:50964) 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 -- 172.21.15.16:0/1221726459 learned_addr learned my addr 172.21.15.16:0/1221726459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 -- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 msgr2=0x55ba23c4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 -- 172.21.15.16:0/1221726459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 msgr2=0x55ba23c50410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c50410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 -- 172.21.15.16:0/1221726459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba239183c0 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80098b0 tx=0x7fd7d80125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd63e00a9a87d7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8016030 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd7d8003c40 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.061+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d8004d20 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7d8003e40 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 10 client.19365 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba23c5f060 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d8004d20 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d800f040 con 0x55ba23c50a90 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 10 client.19365 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 10 client.19365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 20 client.19365 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23c604c0 0x55ba23c628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 4 client.19365 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba23bb4850 con 0x55ba23c604c0 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.062+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23c604c0 0x55ba23c628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.063+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23c604c0 0x55ba23c628b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbc67b3f788de39 server_cookie=cc6febaeddf80fc7 in_seq=0 out_seq=0 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.063+0000 7fd7c5ffb640 10 client.19365 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.063+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1221726459 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba23bb4850 con 0x55ba23c604c0 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7c5ffb640 10 client.19365 handle_command_reply: tid=0 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 client.19365 shutdown 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 msgr2=0x55ba23c4dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd7d80098b0 tx=0x7fd7d80125d0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 shutdown_connections 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23c604c0 0x55ba23c628b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c50a90 0x55ba23c4dc40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c41cc0 0x55ba23c4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7c5ffb640 0 client.19365 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1221726459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c50410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 >> 172.21.15.16:0/1221726459 conn(0x55ba23c3da30 msgr2=0x55ba23c43790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 shutdown_connections 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 1 -- 172.21.15.16:0/1221726459 wait complete. 2024-09-12T01:08:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.064+0000 7fd7e708d740 20 client.19365 trim_cache size 0 max 16384 2024-09-12T01:08:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 Processor -- start 2024-09-12T01:08:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 -- start start 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c5c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c67e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ab6a0 con 0x55ba23c41cc0 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ad1e0 con 0x55ba23c413a0 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7e708d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01abd70 con 0x55ba23c50a90 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7ddd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50978/0 (socket says 172.21.15.16:50978) 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.081+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/853957187 learned_addr learned my addr 172.21.15.16:0/853957187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7de59c640 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c5c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/853957187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 msgr2=0x55ba23c67e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c67e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/853957187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 msgr2=0x55ba23c5c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c5c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 -- 172.21.15.16:0/853957187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba23c50780 con 0x55ba23c41cc0 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7de59c640 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c5c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400b620 tx=0x7fd7d409bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f6c5788e640e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7c7fff640 1 -- 172.21.15.16:0/853957187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40b4070 con 0x55ba23c41cc0 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7c7fff640 1 -- 172.21.15.16:0/853957187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d40a4050 con 0x55ba23c41cc0 2024-09-12T01:08:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.082+0000 7fd7c7fff640 1 -- 172.21.15.16:0/853957187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4020040 con 0x55ba23c41cc0 2024-09-12T01:08:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- 172.21.15.16:0/853957187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 msgr2=0x55ba23c4f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400b620 tx=0x7fd7d409bae0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- 172.21.15.16:0/853957187 shutdown_connections 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c67e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c41cc0 0x55ba23c4f630 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- 172.21.15.16:0/853957187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c413a0 0x55ba23c5c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- 172.21.15.16:0/853957187 >> 172.21.15.16:0/853957187 conn(0x55ba23c3da30 msgr2=0x55ba23c4ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- 172.21.15.16:0/853957187 shutdown_connections 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- 172.21.15.16:0/853957187 wait complete. 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- start start 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 Processor -- start 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 0x55ba23c54e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01acb10 con 0x55ba23c413a0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ac440 con 0x55ba23c41cc0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.083+0000 7fd7e708d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01a96a0 con 0x55ba23c50a90 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7e708d740 10 client.0 _ll_register_callbacks cb 0x7fd7d8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50980/0 (socket says 172.21.15.16:50980) 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 -- 172.21.15.16:0/1815535792 learned_addr learned my addr 172.21.15.16:0/1815535792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 0x55ba23c54e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 -- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 msgr2=0x55ba23c55340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c55340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 -- 172.21.15.16:0/1815535792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 msgr2=0x55ba23c54e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 0x55ba23c54e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 -- 172.21.15.16:0/1815535792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba23cc9ed0 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7ded9d640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c55340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 0x55ba23c54e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7de59c640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd7d002a9a0 tx=0x7fd7d002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31954479b75a7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0055020 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd7d0005040 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.084+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d40a4050 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7d0060050 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 10 client.19371 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba23ccb110 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d40a4050 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d00035f0 con 0x55ba23c413a0 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 10 client.19371 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 10 client.19371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 20 client.19371 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23ccb860 0x55ba23ccdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 4 client.19371 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba23baed60 con 0x55ba23ccb860 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.085+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23ccb860 0x55ba23ccdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7ddd9b640 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23ccb860 0x55ba23ccdc80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ed1d36cc81133c server_cookie=6cb23a8d1f7ccb7f in_seq=0 out_seq=0 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7c5ffb640 10 client.19371 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/1815535792 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba23baed60 con 0x55ba23ccb860 2024-09-12T01:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7c5ffb640 10 client.19371 handle_command_reply: tid=0 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 client.19371 shutdown 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 msgr2=0x55ba23c548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd7d002a9a0 tx=0x7fd7d002a480 comp rx=0 tx=0).stop 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 shutdown_connections 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55ba23ccb860 0x55ba23ccdc80 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba23c50a90 0x55ba23c55340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ba23c41cc0 0x55ba23c54e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7c5ffb640 0 client.19371 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 --2- 172.21.15.16:0/1815535792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ba23c413a0 0x55ba23c548c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 >> 172.21.15.16:0/1815535792 conn(0x55ba23c3da30 msgr2=0x55ba23c4f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 shutdown_connections 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 1 -- 172.21.15.16:0/1815535792 wait complete. 2024-09-12T01:08:45.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.086+0000 7fd7e708d740 20 client.19371 trim_cache size 0 max 16384 2024-09-12T01:08:45.091 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:08:45.091 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:08:45.091 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 msgr2=0x7fd7c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 0x7fd7c80a51b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0005ab0 tx=0x7fd7d0005a20 comp rx=0 tx=0).stop 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 msgr2=0x7fd7e0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd7d400d750 tx=0x7fd7d400dc10 comp rx=0 tx=0).stop 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 shutdown_connections 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd7c80a2cf0 0x7fd7c80a51b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7e010c170 0x7fd7e019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7e0103660 0x7fd7e0198a20 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 --2- 172.21.15.16:0/1483270174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7e0102d40 0x7fd7e01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:45.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 >> 172.21.15.16:0/1483270174 conn(0x7fd7e00fc7a0 msgr2=0x7fd7e00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:45.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 shutdown_connections 2024-09-12T01:08:45.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:45.087+0000 7fd7e4cb4640 1 -- 172.21.15.16:0/1483270174 wait complete. 2024-09-12T01:08:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:44 smithi099 ceph-mon[35654]: pgmap v393: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:44 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:08:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:44 smithi159 ceph-mon[35566]: pgmap v393: 97 pgs: 97 active+clean; 19 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-12T01:08:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:44 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:08:45.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:08:46.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:08:46.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:08:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.586+0000 7fa140e45640 1 -- 172.21.15.16:0/4160313283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c101420 msgr2=0x7fa13c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.586+0000 7fa140e45640 1 --2- 172.21.15.16:0/4160313283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c101420 0x7fa13c10d950 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fa134009cd0 tx=0x7fa13402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 -- 172.21.15.16:0/4160313283 shutdown_connections 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 --2- 172.21.15.16:0/4160313283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c101420 0x7fa13c10d950 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 --2- 172.21.15.16:0/4160313283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c100a80 0x7fa13c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 --2- 172.21.15.16:0/4160313283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 0x7fa13c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 -- 172.21.15.16:0/4160313283 >> 172.21.15.16:0/4160313283 conn(0x7fa13c0fc7a0 msgr2=0x7fa13c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 -- 172.21.15.16:0/4160313283 shutdown_connections 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 -- 172.21.15.16:0/4160313283 wait complete. 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 Processor -- start 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.587+0000 7fa140e45640 1 -- start start 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c101420 0x7fa13c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 0x7fa13c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c110ec0 con 0x7fa13c100a80 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c110d40 con 0x7fa13c101420 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa140e45640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c111040 con 0x7fa13c106a80 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51000/0 (socket says 172.21.15.16:51000) 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 -- 172.21.15.16:0/691130916 learned_addr learned my addr 172.21.15.16:0/691130916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:46.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa139d9b640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c101420 0x7fa13c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13ad9d640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 0x7fa13c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 -- 172.21.15.16:0/691130916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 msgr2=0x7fa13c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 0x7fa13c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 -- 172.21.15.16:0/691130916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c101420 msgr2=0x7fa13c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c101420 0x7fa13c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 -- 172.21.15.16:0/691130916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c19d4f0 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa13a59c640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d770 tx=0x7fa12c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.588+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c017070 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.589+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c009820 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.589+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c19d7e0 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.589+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c004cd0 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.589+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1a50c0 con 0x7fa13c100a80 2024-09-12T01:08:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.590+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c106e60 con 0x7fa13c100a80 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.591+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa12c0099d0 con 0x7fa13c100a80 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.592+0000 7fa12b7fe640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.592+0000 7fa139d9b640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.592+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fa12c0cf2b0 con 0x7fa13c100a80 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.592+0000 7fa139d9b640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 0x7fa1240a51b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0fec70 tx=0x7fa130009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.595+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa12c09bed0 con 0x7fa13c100a80 2024-09-12T01:08:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.822+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13c1a5290 con 0x7fa13c100a80 2024-09-12T01:08:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.822+0000 7fa12b7fe640 1 -- 172.21.15.16:0/691130916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fa12c09b7d0 con 0x7fa13c100a80 2024-09-12T01:08:46.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:08:46.821 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:08:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.824+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 msgr2=0x7fa1240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.824+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 0x7fa1240a51b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0fec70 tx=0x7fa130009110 comp rx=0 tx=0).stop 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.824+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 msgr2=0x7fa13c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.824+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00d770 tx=0x7fa12c00db70 comp rx=0 tx=0).stop 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 shutdown_connections 2024-09-12T01:08:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa1240a2cf0 0x7fa1240a51b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa13c106a80 0x7fa13c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa13c101420 0x7fa13c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 --2- 172.21.15.16:0/691130916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c100a80 0x7fa13c198770 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 >> 172.21.15.16:0/691130916 conn(0x7fa13c0fc7a0 msgr2=0x7fa13c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 shutdown_connections 2024-09-12T01:08:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:46.825+0000 7fa140e45640 1 -- 172.21.15.16:0/691130916 wait complete. 2024-09-12T01:08:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:46 smithi016 ceph-mon[30972]: pgmap v394: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-12T01:08:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:46 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/691130916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:46 smithi099 ceph-mon[35654]: pgmap v394: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-12T01:08:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:46 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/691130916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:46 smithi159 ceph-mon[35566]: pgmap v394: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-09-12T01:08:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:46 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/691130916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 -- 172.21.15.16:0/953305812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c102da0 msgr2=0x7f199c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:47.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 --2- 172.21.15.16:0/953305812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c102da0 0x7f199c103180 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1998009900 tx=0x7f199802a480 comp rx=0 tx=0).stop 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 -- 172.21.15.16:0/953305812 shutdown_connections 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 --2- 172.21.15.16:0/953305812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c10c180 0x7f199c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 --2- 172.21.15.16:0/953305812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c1036c0 0x7f199c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 --2- 172.21.15.16:0/953305812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c102da0 0x7f199c103180 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.367+0000 7f19a346d640 1 -- 172.21.15.16:0/953305812 >> 172.21.15.16:0/953305812 conn(0x7f199c0fc7a0 msgr2=0x7f199c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.368+0000 7f19a346d640 1 -- 172.21.15.16:0/953305812 shutdown_connections 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.368+0000 7f19a346d640 1 -- 172.21.15.16:0/953305812 wait complete. 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.368+0000 7f19a346d640 1 Processor -- start 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.368+0000 7f19a346d640 1 -- start start 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c102da0 0x7f199c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:47.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 0x7f199c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f199c111f60 con 0x7f199c10c180 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f199c111de0 con 0x7f199c102da0 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a346d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1120e0 con 0x7f199c1036c0 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51012/0 (socket says 172.21.15.16:51012) 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 -- 172.21.15.16:0/3014613032 learned_addr learned my addr 172.21.15.16:0/3014613032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1209640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c102da0 0x7f199c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a0a08640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 0x7f199c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 -- 172.21.15.16:0/3014613032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 msgr2=0x7f199c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 0x7f199c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 -- 172.21.15.16:0/3014613032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c102da0 msgr2=0x7f199c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c102da0 0x7f199c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 -- 172.21.15.16:0/3014613032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c19d3d0 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a0a08640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 0x7f199c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19a1a0a640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f199400cad0 tx=0x7f199400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.369+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400ae20 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.370+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1994005020 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.370+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c19d6c0 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.370+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400f6f0 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.370+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c1a4fa0 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.371+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c103180 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.372+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19940051d0 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.372+0000 7f19927fc640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 0x7f19880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.373+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f19940cf740 con 0x7f199c10c180 2024-09-12T01:08:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.373+0000 7f19a1209640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 0x7f19880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.373+0000 7f19a1209640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 0x7f19880a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1998009230 tx=0x7f19980058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:47.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.376+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1994014030 con 0x7f199c10c180 2024-09-12T01:08:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.589+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f199c106c90 con 0x7f199c10c180 2024-09-12T01:08:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.590+0000 7f19927fc640 1 -- 172.21.15.16:0/3014613032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f199409c360 con 0x7f199c10c180 2024-09-12T01:08:47.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:08:47.589 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:08:47.589 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 msgr2=0x7f19880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 0x7f19880a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1998009230 tx=0x7f19980058d0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 msgr2=0x7f199c19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f199400cad0 tx=0x7f199400cf90 comp rx=0 tx=0).stop 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 shutdown_connections 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f19880a2cf0 0x7f19880a51b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c10c180 0x7f199c19ccf0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1036c0 0x7f199c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 --2- 172.21.15.16:0/3014613032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f199c102da0 0x7f199c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.592+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 >> 172.21.15.16:0/3014613032 conn(0x7f199c0fc7a0 msgr2=0x7f199c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.593+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 shutdown_connections 2024-09-12T01:08:47.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:47.593+0000 7f19a346d640 1 -- 172.21.15.16:0/3014613032 wait complete. 2024-09-12T01:08:48.003 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 22fc04e0 --release --await' 2024-09-12T01:08:48.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 22fc04e0 --release --await 2024-09-12T01:08:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:47 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3014613032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:48.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:47 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3014613032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:47 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3014613032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:08:48.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:08:48.849 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:49.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:48 smithi016 ceph-mon[30972]: pgmap v395: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T01:08:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:48 smithi099 ceph-mon[35654]: pgmap v395: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T01:08:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 -- 172.21.15.16:0/2038756901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0101de0 msgr2=0x7f84a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 --2- 172.21.15.16:0/2038756901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0101de0 0x7f84a01021c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8490009900 tx=0x7f849002a410 comp rx=0 tx=0).stop 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 -- 172.21.15.16:0/2038756901 shutdown_connections 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 --2- 172.21.15.16:0/2038756901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0106f10 0x7f84a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 --2- 172.21.15.16:0/2038756901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0102790 0x7f84a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 --2- 172.21.15.16:0/2038756901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0101de0 0x7f84a01021c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 -- 172.21.15.16:0/2038756901 >> 172.21.15.16:0/2038756901 conn(0x7f84a00fd8c0 msgr2=0x7f84a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 -- 172.21.15.16:0/2038756901 shutdown_connections 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.313+0000 7f84a4e19640 1 -- 172.21.15.16:0/2038756901 wait complete. 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.314+0000 7f84a4e19640 1 Processor -- start 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.314+0000 7f84a4e19640 1 -- start start 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0101de0 0x7f84a019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0102790 0x7f84a019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111f70 con 0x7f84a0106f10 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0111df0 con 0x7f84a0102790 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01120f0 con 0x7f84a0101de0 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46088/0 (socket says 172.21.15.16:46088) 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 -- 172.21.15.16:0/1076746678 learned_addr learned my addr 172.21.15.16:0/1076746678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f8495d9b640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0102790 0x7f84a019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849e59c640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0101de0 0x7f84a019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 -- 172.21.15.16:0/1076746678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0101de0 msgr2=0x7f84a019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0101de0 0x7f84a019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 -- 172.21.15.16:0/1076746678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0102790 msgr2=0x7f84a019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0102790 0x7f84a019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 -- 172.21.15.16:0/1076746678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a019f5f0 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f849ed9d640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f849800eaa0 tx=0x7f849800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f8497fff640 1 -- 172.21.15.16:0/1076746678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849800cdb0 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a019f8e0 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f8497fff640 1 -- 172.21.15.16:0/1076746678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8498005020 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.315+0000 7f8497fff640 1 -- 172.21.15.16:0/1076746678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980116b0 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.316+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a01a71c0 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.318+0000 7f8497fff640 1 -- 172.21.15.16:0/1076746678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8498005560 con 0x7f84a0106f10 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.318+0000 7f84a71f2740 1 Processor -- start 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.318+0000 7f84a71f2740 1 -- start start 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.318+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b8ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b8b240 0x55be31b8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f84a71f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be31ae5620 con 0x55be31b80060 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f84a71f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be31a8ce50 con 0x55be31b80980 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f84a71f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be31ae15e0 con 0x55be31b8b240 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46098/0 (socket says 172.21.15.16:46098) 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 -- 172.21.15.16:0/2181702405 learned_addr learned my addr 172.21.15.16:0/2181702405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8497fff640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f849e59c640 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b8ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f849ed9d640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 -- 172.21.15.16:0/2181702405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b8b240 msgr2=0x55be31b8d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b8b240 0x55be31b8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8497fff640 1 -- 172.21.15.16:0/1076746678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f84980cf640 con 0x7f84a0106f10 2024-09-12T01:08:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 -- 172.21.15.16:0/2181702405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 msgr2=0x55be31b8ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b8ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 -- 172.21.15.16:0/2181702405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be31a7dd40 con 0x55be31b80060 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f8495d9b640 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f8488005d00 tx=0x7f84880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b420926cbc6084c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.319+0000 7f849ed9d640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 0x7f848c0a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f849806eba0 tx=0x7f849806db20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f8494d99640 1 -- 172.21.15.16:0/2181702405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488004230 con 0x55be31b80060 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f8494d99640 1 -- 172.21.15.16:0/2181702405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848800be70 con 0x55be31b80060 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f8494d99640 1 -- 172.21.15.16:0/2181702405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488011860 con 0x55be31b80060 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 -- 172.21.15.16:0/2181702405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 msgr2=0x55be31b80440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f8488005d00 tx=0x7f84880107a0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 -- 172.21.15.16:0/2181702405 shutdown_connections 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b8b240 0x55be31b8d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b8ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 --2- 172.21.15.16:0/2181702405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b80440 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 -- 172.21.15.16:0/2181702405 >> 172.21.15.16:0/2181702405 conn(0x55be31b7c5f0 msgr2=0x55be31b7c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 -- 172.21.15.16:0/2181702405 shutdown_connections 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.320+0000 7f84a71f2740 1 -- 172.21.15.16:0/2181702405 wait complete. 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 -- start start 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 Processor -- start 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b8b240 0x55be31b88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be31b8de10 con 0x55be31b80060 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be31a6df10 con 0x55be31b8b240 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be31b8df90 con 0x55be31b80980 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f84a71f2740 10 client.0 _ll_register_callbacks cb 0x55be31a91db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46114/0 (socket says 172.21.15.16:46114) 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f8495d9b640 1 -- 172.21.15.16:0/852310289 learned_addr learned my addr 172.21.15.16:0/852310289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f849e59c640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.321+0000 7f8495d9b640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b8b240 0x55be31b88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 -- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 msgr2=0x55be31b886e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b886e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 -- 172.21.15.16:0/852310289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b8b240 msgr2=0x55be31b88c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b8b240 0x55be31b88c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 -- 172.21.15.16:0/852310289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be317f4580 con 0x55be31b80060 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8495d9b640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8488003f40 tx=0x7f8488005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c88ce4941203a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880037f0 con 0x55be31b80060 2024-09-12T01:08:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f8488003990 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848800be70 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8488003b90 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f84a71f2740 10 client.19395 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be318be960 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848800be70 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.322+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8488004be0 con 0x55be31b80060 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 10 client.19395 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 10 client.19395 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 20 client.19395 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b9ad20 0x55be31b9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 4 client.19395 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be31af2b90 con 0x55be31b9ad20 2024-09-12T01:08:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f849e59c640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b9ad20 0x55be31b9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f8485ffb640 10 client.19395 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.323+0000 7f849e59c640 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b9ad20 0x55be31b9d110 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82bfda9cf855da0b server_cookie=e1223320140fe051 in_seq=0 out_seq=0 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f8485ffb640 1 -- 172.21.15.16:0/852310289 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be31af2b90 con 0x55be31b9ad20 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f8485ffb640 10 client.19395 handle_command_reply: tid=0 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 client.19395 shutdown 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 msgr2=0x55be31b881a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8488003f40 tx=0x7f8488005c90 comp rx=0 tx=0).stop 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 shutdown_connections 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b9ad20 0x55be31b9d110 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b8b240 0x55be31b88c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b886e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f8485ffb640 0 client.19395 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 --2- 172.21.15.16:0/852310289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b80060 0x55be31b881a0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 >> 172.21.15.16:0/852310289 conn(0x55be31b7c5f0 msgr2=0x55be31b82320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 shutdown_connections 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 1 -- 172.21.15.16:0/852310289 wait complete. 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.324+0000 7f84a71f2740 20 client.19395 trim_cache size 0 max 16384 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.341+0000 7f84a71f2740 1 Processor -- start 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.341+0000 7f84a71f2740 1 -- start start 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.341+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80060 0x55be31b968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f84a71f2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01afaf0 con 0x55be31b8b240 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f84a71f2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b1630 con 0x55be31b80060 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f84a71f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b01c0 con 0x55be31b80980 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46118/0 (socket says 172.21.15.16:46118) 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 -- 172.21.15.16:0/911953361 learned_addr learned my addr 172.21.15.16:0/911953361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80060 0x55be31b968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 -- 172.21.15.16:0/911953361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 msgr2=0x55be31b85400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 -- 172.21.15.16:0/911953361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80060 msgr2=0x55be31b968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80060 0x55be31b968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 -- 172.21.15.16:0/911953361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be31b8df20 con 0x55be31b8b240 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.342+0000 7f8495d9b640 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8488011730 tx=0x7f84880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3da5d329b04c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f8487fff640 1 -- 172.21.15.16:0/911953361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880042f0 con 0x55be31b8b240 2024-09-12T01:08:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f8487fff640 1 -- 172.21.15.16:0/911953361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848801f030 con 0x55be31b8b240 2024-09-12T01:08:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f8487fff640 1 -- 172.21.15.16:0/911953361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84880035f0 con 0x55be31b8b240 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 -- 172.21.15.16:0/911953361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 msgr2=0x55be31b85940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8488011730 tx=0x7f84880107a0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 -- 172.21.15.16:0/911953361 shutdown_connections 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b85940 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80980 0x55be31b85400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 --2- 172.21.15.16:0/911953361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80060 0x55be31b968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 -- 172.21.15.16:0/911953361 >> 172.21.15.16:0/911953361 conn(0x55be31b7c5f0 msgr2=0x55be31b961e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.343+0000 7f84a71f2740 1 -- 172.21.15.16:0/911953361 shutdown_connections 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 -- 172.21.15.16:0/911953361 wait complete. 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 -- start start 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 Processor -- start 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80060 0x55be31b9e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b9e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.344+0000 7f84a71f2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46122/0 (socket says 172.21.15.16:46122) 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 -- 172.21.15.16:0/1356386141 learned_addr learned my addr 172.21.15.16:0/1356386141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b0f60 con 0x55be31b8b240 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01b0890 con 0x55be31b80980 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84a01adaf0 con 0x55be31b80060 2024-09-12T01:08:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f84a71f2740 10 client.0 _ll_register_callbacks cb 0x7f8488003fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80060 0x55be31b9e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f849e59c640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b9e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 -- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80060 msgr2=0x55be31b9e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80060 0x55be31b9e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 -- 172.21.15.16:0/1356386141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 msgr2=0x55be31b9e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b9e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be31c04930 con 0x55be31b8b240 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8495d9b640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f848800b540 tx=0x7f848800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c086bdb6b0b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.345+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488039040 con 0x55be31b8b240 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f8488032b40 con 0x55be31b8b240 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848801f030 con 0x55be31b8b240 2024-09-12T01:08:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84880042f0 con 0x55be31b8b240 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 10 client.19401 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be31c05c70 con 0x55be31b8b240 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848801f030 con 0x55be31b8b240 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8488005020 con 0x55be31b8b240 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 10 client.19401 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 10 client.19401 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 20 client.19401 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31c06270 0x55be31c08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 4 client.19401 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "22fc04e0", "release": true, "await": true}] 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.346+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "22fc04e0", "release": true, "await": true}) -- 0x55be31aed0a0 con 0x55be31c06270 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.347+0000 7f849e59c640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31c06270 0x55be31c08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.347+0000 7f8485ffb640 10 client.19401 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.347+0000 7f849e59c640 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31c06270 0x55be31c08660 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70b2ad8e02adacc server_cookie=da45c208ca7821fc in_seq=0 out_seq=0 2024-09-12T01:08:49.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f8485ffb640 1 -- 172.21.15.16:0/1356386141 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55be31aed0a0 con 0x55be31c06270 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f8485ffb640 10 client.19401 handle_command_reply: tid=0 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 client.19401 shutdown 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 msgr2=0x55be31b9ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f848800b540 tx=0x7f848800bd00 comp rx=0 tx=0).stop 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 shutdown_connections 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31c06270 0x55be31c08660 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31b8b240 0x55be31b9ec40 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31b80980 0x55be31b9e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 --2- 172.21.15.16:0/1356386141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31b80060 0x55be31b9e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f8485ffb640 0 client.19401 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.352+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 >> 172.21.15.16:0/1356386141 conn(0x55be31b7c5f0 msgr2=0x55be31b96a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 shutdown_connections 2024-09-12T01:08:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a71f2740 1 -- 172.21.15.16:0/1356386141 wait complete. 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a71f2740 20 client.19401 trim_cache size 0 max 16384 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 48, 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "22fc04e0": { 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 48, 2024-09-12T01:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 147.2, 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:49.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:49.363 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:08:49.363 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 msgr2=0x7f848c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 0x7f848c0a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f849806eba0 tx=0x7f849806db20 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 msgr2=0x7f84a019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.353+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f849800eaa0 tx=0x7f849800ef60 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 shutdown_connections 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f848c0a2cf0 0x7f848c0a51b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a0106f10 0x7f84a019ef10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84a0102790 0x7f84a019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 --2- 172.21.15.16:0/1076746678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84a0101de0 0x7f84a019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 >> 172.21.15.16:0/1076746678 conn(0x7f84a00fd8c0 msgr2=0x7f84a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 shutdown_connections 2024-09-12T01:08:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:49.354+0000 7f84a4e19640 1 -- 172.21.15.16:0/1076746678 wait complete. 2024-09-12T01:08:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:48 smithi159 ceph-mon[35566]: pgmap v395: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-09-12T01:08:50.144 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '22fc04e0', total seconds elapsed: 31.8 2024-09-12T01:08:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:50.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.720+0000 7f2db9bb0640 1 -- 172.21.15.16:0/999951158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4100a80 msgr2=0x7f2db4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.720+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/999951158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4100a80 0x7f2db4100ee0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009930 tx=0x7f2da802a420 comp rx=0 tx=0).stop 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.720+0000 7f2db9bb0640 1 -- 172.21.15.16:0/999951158 shutdown_connections 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/999951158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4101420 0x7f2db410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/999951158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4100a80 0x7f2db4100ee0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/999951158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4106a80 0x7f2db4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 -- 172.21.15.16:0/999951158 >> 172.21.15.16:0/999951158 conn(0x7f2db40fc7a0 msgr2=0x7f2db40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 -- 172.21.15.16:0/999951158 shutdown_connections 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 -- 172.21.15.16:0/999951158 wait complete. 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 Processor -- start 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.721+0000 7f2db9bb0640 1 -- start start 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4100a80 0x7f2db419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4106a80 0x7f2db419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4110ea0 con 0x7f2db4101420 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4110d20 con 0x7f2db4100a80 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db9bb0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4111020 con 0x7f2db4106a80 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46136/0 (socket says 172.21.15.16:46136) 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 -- 172.21.15.16:0/733176836 learned_addr learned my addr 172.21.15.16:0/733176836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db37fe640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4100a80 0x7f2db419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db3fff640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4106a80 0x7f2db419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 -- 172.21.15.16:0/733176836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4106a80 msgr2=0x7f2db419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4106a80 0x7f2db419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 -- 172.21.15.16:0/733176836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4100a80 msgr2=0x7f2db419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4100a80 0x7f2db419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 -- 172.21.15.16:0/733176836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db419f660 con 0x7f2db4101420 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009230 tx=0x7f2da8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.722+0000 7f2db0ff9640 1 -- 172.21.15.16:0/733176836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da803f030 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.723+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db419f8f0 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.723+0000 7f2db0ff9640 1 -- 172.21.15.16:0/733176836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da8003ad0 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.723+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db41a7230 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.723+0000 7f2db0ff9640 1 -- 172.21.15.16:0/733176836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8049060 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2db0ff9640 1 -- 172.21.15.16:0/733176836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2da8042030 con 0x7f2db4101420 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 Processor -- start 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 -- start start 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac4b40 0x562e56ac3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2db0ff9640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 0x7f2d9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ad0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.725+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e56a292f0 con 0x562e56ac35d0 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e569d0cf0 con 0x562e56ace370 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e56a252b0 con 0x562e56ac4b40 2024-09-12T01:08:50.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db37fe640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 0x7f2d9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46148/0 (socket says 172.21.15.16:46148) 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3458997029 learned_addr learned my addr 172.21.15.16:0/3458997029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db3fff640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac4b40 0x562e56ac3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db0ff9640 1 -- 172.21.15.16:0/733176836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f2da80ef8d0 con 0x7f2db4101420 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db3fff640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ad0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3458997029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac4b40 msgr2=0x562e56ac3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac4b40 0x562e56ac3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3458997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 msgr2=0x562e56ad0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ad0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3458997029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e569b53a0 con 0x562e56ac35d0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db3fff640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ad0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db37fe640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 0x7f2d9c0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009990 tx=0x7f2da4009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2da80b8250 tx=0x7f2da80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2df850555d5521 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.726+0000 7f2d99ffb640 1 -- 172.21.15.16:0/3458997029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8035070 con 0x562e56ac35d0 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2d99ffb640 1 -- 172.21.15.16:0/3458997029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da80c5920 con 0x562e56ac35d0 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2d99ffb640 1 -- 172.21.15.16:0/3458997029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80c01a0 con 0x562e56ac35d0 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3458997029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 msgr2=0x562e56ac3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2da80b8250 tx=0x7f2da80045f0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3458997029 shutdown_connections 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ad0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac35d0 0x562e56ac3a10 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3458997029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac4b40 0x562e56ac3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3458997029 >> 172.21.15.16:0/3458997029 conn(0x562e56a8c310 msgr2=0x562e56abfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3458997029 shutdown_connections 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.727+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3458997029 wait complete. 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 -- start start 2024-09-12T01:08:50.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 Processor -- start 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56acddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56acb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562e56ad0e30 con 0x562e56ac4b40 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e569e4060 con 0x562e56ace370 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e56ad0fb0 con 0x562e56ac35d0 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2dbbf89740 10 client.0 _ll_register_callbacks cb 0x562e567378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46156/0 (socket says 172.21.15.16:46156) 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 -- 172.21.15.16:0/831162989 learned_addr learned my addr 172.21.15.16:0/831162989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db37fe640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56acddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db3fff640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56acb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 -- 172.21.15.16:0/831162989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 msgr2=0x562e56acddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56acddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 -- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 msgr2=0x562e56acb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.728+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56acb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2db2ffd640 1 -- 172.21.15.16:0/831162989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5679a3c0 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f2da80bbd80 tx=0x7f2da802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207bdf02c6ed62d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8035040 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f2da8003c80 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da80c5920 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2da80bc780 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 10 client.19413 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e56ad0e30 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da80c5920 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2da80bc0e0 con 0x562e56ac4b40 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 10 client.19413 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 10 client.19413 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:08:50.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 20 client.19413 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56addb60 0x562e56adff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 4 client.19413 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.729+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e56a36860 con 0x562e56addb60 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.730+0000 7f2db37fe640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56addb60 0x562e56adff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.730+0000 7f2db37fe640 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56addb60 0x562e56adff80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5358d7f58e32698e server_cookie=e5fdd452df095565 in_seq=0 out_seq=0 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.730+0000 7f2d92ffd640 10 client.19413 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2d92ffd640 1 -- 172.21.15.16:0/831162989 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e56a36860 con 0x562e56addb60 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2d92ffd640 10 client.19413 handle_command_reply: tid=0 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 client.19413 shutdown 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 msgr2=0x562e56acb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f2da80bbd80 tx=0x7f2da802a420 comp rx=0 tx=0).stop 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 shutdown_connections 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56addb60 0x562e56adff80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56acb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56acb070 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/831162989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56acddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 >> 172.21.15.16:0/831162989 conn(0x562e56a8c310 msgr2=0x562e56ac5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2d92ffd640 0 client.19413 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 shutdown_connections 2024-09-12T01:08:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 1 -- 172.21.15.16:0/831162989 wait complete. 2024-09-12T01:08:50.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.731+0000 7f2dbbf89740 20 client.19413 trim_cache size 0 max 16384 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.744+0000 7f2dbbf89740 1 Processor -- start 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.744+0000 7f2dbbf89740 1 -- start start 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ac35d0 0x562e56acafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ace370 0x562e56ae57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41af980 con 0x562e56ac4b40 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41b14c0 con 0x562e56ac35d0 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41b0050 con 0x562e56ace370 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46172/0 (socket says 172.21.15.16:46172) 2024-09-12T01:08:50.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3315454131 learned_addr learned my addr 172.21.15.16:0/3315454131 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db37fe640 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ac35d0 0x562e56acafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db3fff640 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ace370 0x562e56ae57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3315454131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ace370 msgr2=0x562e56ae57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ace370 0x562e56ae57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3315454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ac35d0 msgr2=0x562e56acafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ac35d0 0x562e56acafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 -- 172.21.15.16:0/3315454131 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e56acbb70 con 0x562e56ac4b40 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2da80b8250 tx=0x7f2da8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ed3ef307cbeb01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2d997fa640 1 -- 172.21.15.16:0/3315454131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8035070 con 0x562e56ac4b40 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2d997fa640 1 -- 172.21.15.16:0/3315454131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da80d0070 con 0x562e56ac4b40 2024-09-12T01:08:50.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.745+0000 7f2d997fa640 1 -- 172.21.15.16:0/3315454131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8003c80 con 0x562e56ac4b40 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3315454131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 msgr2=0x562e56aca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2da80b8250 tx=0x7f2da8009260 comp rx=0 tx=0).stop 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3315454131 shutdown_connections 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ace370 0x562e56ae57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56aca050 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/3315454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ac35d0 0x562e56acafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3315454131 >> 172.21.15.16:0/3315454131 conn(0x562e56a8c310 msgr2=0x562e56ac9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3315454131 shutdown_connections 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- 172.21.15.16:0/3315454131 wait complete. 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.746+0000 7f2dbbf89740 1 -- start start 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 Processor -- start 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56ad65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ac80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41b0df0 con 0x562e56ac4b40 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41b0720 con 0x562e56ace370 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2db41ad980 con 0x562e56ac35d0 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2dbbf89740 10 client.0 _ll_register_callbacks cb 0x7f2dac009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46188/0 (socket says 172.21.15.16:46188) 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 -- 172.21.15.16:0/1262444747 learned_addr learned my addr 172.21.15.16:0/1262444747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db37fe640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56ad65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 -- 172.21.15.16:0/1262444747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 msgr2=0x562e56ad65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56ad65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 -- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 msgr2=0x562e56ac80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ac80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.747+0000 7f2db2ffd640 1 -- 172.21.15.16:0/1262444747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e56ac8620 con 0x562e56ac4b40 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2db2ffd640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2da80bbd80 tx=0x7f2da802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc5ee035005863 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8035070 con 0x562e56ac4b40 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f2da80bc050 con 0x562e56ac4b40 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da80d0070 con 0x562e56ac4b40 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2da80bc290 con 0x562e56ac4b40 2024-09-12T01:08:50.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 10 client.19419 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e56ac8620 con 0x562e56ac4b40 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da80d0070 con 0x562e56ac4b40 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2da8082070 con 0x562e56ac4b40 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 10 client.19419 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 10 client.19419 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 20 client.19419 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56b48740 0x562e56b4ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 4 client.19419 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.748+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e56a30d70 con 0x562e56b48740 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2db37fe640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56b48740 0x562e56b4ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2d92ffd640 10 client.19419 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2db37fe640 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56b48740 0x562e56b4ab30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88501652629502b7 server_cookie=b784854e5c887da0 in_seq=0 out_seq=0 2024-09-12T01:08:50.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2d92ffd640 1 -- 172.21.15.16:0/1262444747 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e56a30d70 con 0x562e56b48740 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2d92ffd640 10 client.19419 handle_command_reply: tid=0 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2dbbf89740 1 client.19419 shutdown 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 msgr2=0x562e56ac7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.749+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2da80bbd80 tx=0x7f2da802a420 comp rx=0 tx=0).stop 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 shutdown_connections 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x562e56b48740 0x562e56b4ab30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e56ace370 0x562e56ac80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e56ac4b40 0x562e56ac7ba0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2d92ffd640 0 client.19419 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 --2- 172.21.15.16:0/1262444747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e56ac35d0 0x562e56ad65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 >> 172.21.15.16:0/1262444747 conn(0x562e56a8c310 msgr2=0x562e56ad8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 shutdown_connections 2024-09-12T01:08:50.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 1 -- 172.21.15.16:0/1262444747 wait complete. 2024-09-12T01:08:50.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2dbbf89740 20 client.19419 trim_cache size 0 max 16384 2024-09-12T01:08:50.754 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 msgr2=0x7f2d9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 0x7f2d9c0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009990 tx=0x7f2da4009110 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 msgr2=0x7f2db419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009230 tx=0x7f2da8004270 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 shutdown_connections 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d9c0a2cf0 0x7f2d9c0a51b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2db4106a80 0x7f2db419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2db4101420 0x7f2db419abf0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 --2- 172.21.15.16:0/733176836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4100a80 0x7f2db419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.750+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 >> 172.21.15.16:0/733176836 conn(0x7f2db40fc7a0 msgr2=0x7f2db40fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.751+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 shutdown_connections 2024-09-12T01:08:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:50.751+0000 7f2db9bb0640 1 -- 172.21.15.16:0/733176836 wait complete. 2024-09-12T01:08:51.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:50 smithi016 ceph-mon[30972]: pgmap v396: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:08:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:50 smithi099 ceph-mon[35654]: pgmap v396: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:08:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:50 smithi159 ceph-mon[35566]: pgmap v396: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.7 KiB/s wr, 13 op/s 2024-09-12T01:08:51.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:08:51.923 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 -- 172.21.15.16:0/190448476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 msgr2=0x7f34bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 --2- 172.21.15.16:0/190448476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc1067e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009900 tx=0x7f34b002a410 comp rx=0 tx=0).stop 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 -- 172.21.15.16:0/190448476 shutdown_connections 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 --2- 172.21.15.16:0/190448476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 0x7f34bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 --2- 172.21.15.16:0/190448476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc1067e0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 --2- 172.21.15.16:0/190448476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 0x7f34bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 -- 172.21.15.16:0/190448476 >> 172.21.15.16:0/190448476 conn(0x7f34bc0fd8c0 msgr2=0x7f34bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 -- 172.21.15.16:0/190448476 shutdown_connections 2024-09-12T01:08:52.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.395+0000 7f34c22b4640 1 -- 172.21.15.16:0/190448476 wait complete. 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 Processor -- start 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 -- start start 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 0x7f34bc19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 0x7f34bc19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc111f70 con 0x7f34bc102790 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc111df0 con 0x7f34bc106f10 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34c22b4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1120f0 con 0x7f34bc101de0 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46198/0 (socket says 172.21.15.16:46198) 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.396+0000 7f34bb7fe640 1 -- 172.21.15.16:0/2677190266 learned_addr learned my addr 172.21.15.16:0/2677190266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bbfff640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 0x7f34bc19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34c0851640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 0x7f34bc19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 -- 172.21.15.16:0/2677190266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 msgr2=0x7f34bc19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 0x7f34bc19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 -- 172.21.15.16:0/2677190266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 msgr2=0x7f34bc19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 0x7f34bc19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 -- 172.21.15.16:0/2677190266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc19f670 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34b97fa640 1 -- 172.21.15.16:0/2677190266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b003f030 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34b97fa640 1 -- 172.21.15.16:0/2677190266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b0003ce0 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc19f900 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34b97fa640 1 -- 172.21.15.16:0/2677190266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0035070 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.397+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a7240 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34b97fa640 1 -- 172.21.15.16:0/2677190266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f34b0042030 con 0x7f34bc102790 2024-09-12T01:08:52.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 Processor -- start 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 -- start start 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 0x55f248cb2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f248c182f0 con 0x55f248cb3b40 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f248bbfcf0 con 0x55f248cb25d0 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c468d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f248c142b0 con 0x55f248cbd370 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.400+0000 7f34c0851640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46206/0 (socket says 172.21.15.16:46206) 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 -- 172.21.15.16:0/719968692 learned_addr learned my addr 172.21.15.16:0/719968692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34bbfff640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 0x55f248cb2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34b97fa640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 0x7f34a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 -- 172.21.15.16:0/719968692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 msgr2=0x55f248cbf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cbf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 -- 172.21.15.16:0/719968692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 msgr2=0x55f248cb2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 0x55f248cb2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 -- 172.21.15.16:0/719968692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f248ba43a0 con 0x55f248cb3b40 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34bbfff640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 0x55f248cb2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34b97fa640 1 -- 172.21.15.16:0/2677190266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f34b00ef4f0 con 0x7f34bc102790 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 0x7f34a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34c0851640 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f34b4011590 tx=0x7f34b4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc641638ec7804a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34a27fc640 1 -- 172.21.15.16:0/719968692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4017050 con 0x55f248cb3b40 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.401+0000 7f34a27fc640 1 -- 172.21.15.16:0/719968692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b401bb90 con 0x55f248cb3b40 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34a27fc640 1 -- 172.21.15.16:0/719968692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b400e3b0 con 0x55f248cb3b40 2024-09-12T01:08:52.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 -- 172.21.15.16:0/719968692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 msgr2=0x55f248cb2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f34b4011590 tx=0x7f34b4011a50 comp rx=0 tx=0).stop 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c0851640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 0x7f34a40a5230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34b4022000 tx=0x7f34b4023000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 -- 172.21.15.16:0/719968692 shutdown_connections 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cbf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb25d0 0x55f248cb2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 --2- 172.21.15.16:0/719968692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cb2090 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 -- 172.21.15.16:0/719968692 >> 172.21.15.16:0/719968692 conn(0x55f248c7b310 msgr2=0x55f248caec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 -- 172.21.15.16:0/719968692 shutdown_connections 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.402+0000 7f34c468d740 1 -- 172.21.15.16:0/719968692 wait complete. 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 -- start start 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 Processor -- start 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cbcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 0x55f248cba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f248cc0070 con 0x55f248cb3b40 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f248bd3060 con 0x55f248cbd370 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f248cc01f0 con 0x55f248cb25d0 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c468d740 10 client.0 _ll_register_callbacks cb 0x55f2489268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46222/0 (socket says 172.21.15.16:46222) 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34bb7fe640 1 -- 172.21.15.16:0/3577981463 learned_addr learned my addr 172.21.15.16:0/3577981463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34bbfff640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cbcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.403+0000 7f34c0851640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 0x55f248cba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 -- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 msgr2=0x55f248cbcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cbcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 -- 172.21.15.16:0/3577981463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 msgr2=0x55f248cba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 0x55f248cba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 -- 172.21.15.16:0/3577981463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2489893c0 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c0851640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 0x55f248cba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bbfff640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cbcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f34b00ecc40 tx=0x7f34b0032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801109293c9eefc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0049070 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f34b00b8190 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b401bb90 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34b00bb8f0 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c468d740 10 client.19431 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f248ccbae0 con 0x55f248cb3b40 2024-09-12T01:08:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b401bb90 con 0x55f248cb3b40 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f34b004c060 con 0x55f248cb3b40 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c468d740 10 client.19431 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.404+0000 7f34c468d740 10 client.19431 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34c468d740 20 client.19431 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248ccce30 0x55f248ccf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34c468d740 4 client.19431 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f248c25860 con 0x55f248ccce30 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34bbfff640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248ccce30 0x55f248ccf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f34bbfff640 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248ccce30 0x55f248ccf240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4394f6e0436548ab server_cookie=a54631dd31d1379a in_seq=0 out_seq=0 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.405+0000 7f349b7fe640 10 client.19431 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f349b7fe640 1 -- 172.21.15.16:0/3577981463 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f248c25860 con 0x55f248ccce30 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f349b7fe640 10 client.19431 handle_command_reply: tid=0 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 client.19431 shutdown 2024-09-12T01:08:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 msgr2=0x55f248cba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f34b00ecc40 tx=0x7f34b0032bc0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 shutdown_connections 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cbd370 0x55f248cba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248ccce30 0x55f248ccf240 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb3b40 0x55f248cba000 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 --2- 172.21.15.16:0/3577981463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cbcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 >> 172.21.15.16:0/3577981463 conn(0x55f248c7b310 msgr2=0x55f248cb4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f349b7fe640 0 client.19431 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 shutdown_connections 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 1 -- 172.21.15.16:0/3577981463 wait complete. 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.406+0000 7f34c468d740 20 client.19431 trim_cache size 0 max 16384 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.419+0000 7f34c468d740 1 Processor -- start 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.419+0000 7f34c468d740 1 -- start start 2024-09-12T01:08:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1afad0 con 0x55f248cb25d0 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b1610 con 0x55f248cb3b40 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c468d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b01a0 con 0x55f248cbd370 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 -- 172.21.15.16:0/291091565 learned_addr learned my addr 172.21.15.16:0/291091565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34c0851640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 -- 172.21.15.16:0/291091565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 msgr2=0x55f248cb8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cb8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 -- 172.21.15.16:0/291091565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 msgr2=0x55f248cb1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cb1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 -- 172.21.15.16:0/291091565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f248cad310 con 0x55f248cb25d0 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cb1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34bbfff640 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b600 tx=0x7f34ac00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c85e9e3056585ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34a1ffb640 1 -- 172.21.15.16:0/291091565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac003d50 con 0x55f248cb25d0 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34a1ffb640 1 -- 172.21.15.16:0/291091565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ac005500 con 0x55f248cb25d0 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.420+0000 7f34a1ffb640 1 -- 172.21.15.16:0/291091565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0116a0 con 0x55f248cb25d0 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- 172.21.15.16:0/291091565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 msgr2=0x55f248cc9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b600 tx=0x7f34ac00bca0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- 172.21.15.16:0/291091565 shutdown_connections 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cbd370 0x55f248cb8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cb1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 --2- 172.21.15.16:0/291091565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cb25d0 0x55f248cc9310 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- 172.21.15.16:0/291091565 >> 172.21.15.16:0/291091565 conn(0x55f248c7b310 msgr2=0x55f248cb88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- 172.21.15.16:0/291091565 shutdown_connections 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- 172.21.15.16:0/291091565 wait complete. 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.421+0000 7f34c468d740 1 -- start start 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 Processor -- start 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cc5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cc10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b0f40 con 0x55f248cbd370 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1b0870 con 0x55f248cb3b40 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1adad0 con 0x55f248cb25d0 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c468d740 10 client.0 _ll_register_callbacks cb 0x7f34b00f2c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46236/0 (socket says 172.21.15.16:46236) 2024-09-12T01:08:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 -- 172.21.15.16:0/4208281031 learned_addr learned my addr 172.21.15.16:0/4208281031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34bb7fe640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cc10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34bbfff640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cc5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 -- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 msgr2=0x55f248cc5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cc5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 -- 172.21.15.16:0/4208281031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 msgr2=0x55f248cc10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cc10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.422+0000 7f34c0851640 1 -- 172.21.15.16:0/4208281031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f248d36ae0 con 0x55f248cbd370 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c0851640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006010 tx=0x7f34b4004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38e0d1b57c845f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4033020 con 0x55f248cbd370 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f34b401bb90 con 0x55f248cbd370 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ac005500 con 0x55f248cbd370 2024-09-12T01:08:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34b4017020 con 0x55f248cbd370 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 10 client.19437 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f248d37c10 con 0x55f248cbd370 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ac005500 con 0x55f248cbd370 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f34b4025ad0 con 0x55f248cbd370 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 10 client.19437 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 10 client.19437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 20 client.19437 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248d38180 0x55f248d3a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 4 client.19437 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.423+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f248c1fd70 con 0x55f248d38180 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f34bbfff640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248d38180 0x55f248d3a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f34bbfff640 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248d38180 0x55f248d3a5a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca636afce07fc439 server_cookie=46bc42aa4889924b in_seq=0 out_seq=0 2024-09-12T01:08:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f349b7fe640 10 client.19437 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f349b7fe640 1 -- 172.21.15.16:0/4208281031 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f248c1fd70 con 0x55f248d38180 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f349b7fe640 10 client.19437 handle_command_reply: tid=0 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f34c468d740 1 client.19437 shutdown 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.424+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 msgr2=0x55f248cc1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006010 tx=0x7f34b4004dd0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 shutdown_connections 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55f248d38180 0x55f248d3a5a0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f248cbd370 0x55f248cc1850 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f248cb3b40 0x55f248cc10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 --2- 172.21.15.16:0/4208281031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f248cb25d0 0x55f248cc5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 >> 172.21.15.16:0/4208281031 conn(0x55f248c7b310 msgr2=0x55f248cb90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f349b7fe640 0 client.19437 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 shutdown_connections 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 1 -- 172.21.15.16:0/4208281031 wait complete. 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c468d740 20 client.19437 trim_cache size 0 max 16384 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 msgr2=0x7f34a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 0x7f34a40a5230 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34b4022000 tx=0x7f34b4023000 comp rx=0 tx=0).stop 2024-09-12T01:08:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 msgr2=0x7f34bc19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b00042f0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 shutdown_connections 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34a40a2d70 0x7f34a40a5230 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34bc106f10 0x7f34bc19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc102790 0x7f34bc19ac00 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 --2- 172.21.15.16:0/2677190266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34bc101de0 0x7f34bc19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.425+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 >> 172.21.15.16:0/2677190266 conn(0x7f34bc0fd8c0 msgr2=0x7f34bc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.426+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 shutdown_connections 2024-09-12T01:08:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:52.426+0000 7f34c22b4640 1 -- 172.21.15.16:0/2677190266 wait complete. 2024-09-12T01:08:53.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:52 smithi016 ceph-mon[30972]: pgmap v397: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:53.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:08:53.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:52 smithi099 ceph-mon[35654]: pgmap v397: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:53.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:52 smithi159 ceph-mon[35566]: pgmap v397: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:53.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 -- 172.21.15.16:0/734641515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 msgr2=0x7f4c8410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 --2- 172.21.15.16:0/734641515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c8410e610 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009ca0 tx=0x7f4c7c02a950 comp rx=0 tx=0).stop 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 -- 172.21.15.16:0/734641515 shutdown_connections 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 --2- 172.21.15.16:0/734641515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c8410e610 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 --2- 172.21.15.16:0/734641515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84103720 0x7f4c8410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 --2- 172.21.15.16:0/734641515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c84102e00 0x7f4c841031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 -- 172.21.15.16:0/734641515 >> 172.21.15.16:0/734641515 conn(0x7f4c840fc7a0 msgr2=0x7f4c840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 -- 172.21.15.16:0/734641515 shutdown_connections 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.930+0000 7f4c898a2640 1 -- 172.21.15.16:0/734641515 wait complete. 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 Processor -- start 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 -- start start 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84102e00 0x7f4c84100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c84103720 0x7f4c84100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84111fa0 con 0x7f4c8410c1a0 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84111e20 con 0x7f4c84103720 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c898a2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c84112120 con 0x7f4c84102e00 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46258/0 (socket says 172.21.15.16:46258) 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 -- 172.21.15.16:0/1655719544 learned_addr learned my addr 172.21.15.16:0/1655719544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84102e00 0x7f4c84100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 -- 172.21.15.16:0/1655719544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84102e00 msgr2=0x7f4c84100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84102e00 0x7f4c84100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 -- 172.21.15.16:0/1655719544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c84103720 msgr2=0x7f4c84100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c84103720 0x7f4c84100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.931+0000 7f4c837fe640 1 -- 172.21.15.16:0/1655719544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c841a6860 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c837fe640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009dc0 tx=0x7f4c7c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c888a0640 1 -- 172.21.15.16:0/1655719544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c034040 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c841a6af0 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c888a0640 1 -- 172.21.15.16:0/1655719544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c02fe20 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c888a0640 1 -- 172.21.15.16:0/1655719544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0047f0 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.932+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c841a6fd0 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.934+0000 7f4c888a0640 1 -- 172.21.15.16:0/1655719544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4c7c037020 con 0x7f4c8410c1a0 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.934+0000 7f4c8bc7b740 1 Processor -- start 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.934+0000 7f4c8bc7b740 1 -- start start 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17a5d0 0x558b7a17aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a187760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b7a0e02f0 con 0x558b7a185370 2024-09-12T01:08:53.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b7a087cf0 con 0x558b7a17bb40 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558b7a0dc2b0 con 0x558b7a17a5d0 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a187760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 -- 172.21.15.16:0/176357167 learned_addr learned my addr 172.21.15.16:0/176357167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17a5d0 0x558b7a17aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c888a0640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 0x7f4c700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 -- 172.21.15.16:0/176357167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17a5d0 msgr2=0x558b7a17aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17a5d0 0x558b7a17aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 -- 172.21.15.16:0/176357167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 msgr2=0x558b7a187760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a187760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c7bfff640 1 -- 172.21.15.16:0/176357167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7a06c3a0 con 0x558b7a17bb40 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.935+0000 7f4c837fe640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 0x7f4c700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c888a0640 1 -- 172.21.15.16:0/1655719544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4c7c0f02a0 con 0x7f4c8410c1a0 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x558b7a17b070 tx=0x7f4c6c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47d146e0d9fee4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c79ffb640 1 -- 172.21.15.16:0/176357167 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c012d60 con 0x558b7a17bb40 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c837fe640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 0x7f4c700a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c08f9d0 tx=0x7f4c7c004990 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c79ffb640 1 -- 172.21.15.16:0/176357167 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c005500 con 0x558b7a17bb40 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c79ffb640 1 -- 172.21.15.16:0/176357167 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00f760 con 0x558b7a17bb40 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/176357167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 msgr2=0x558b7a17a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.936+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x558b7a17b070 tx=0x7f4c6c005850 comp rx=0 tx=0).stop 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/176357167 shutdown_connections 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a187760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17a5d0 0x558b7a17aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/176357167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17bb40 0x558b7a17a090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/176357167 >> 172.21.15.16:0/176357167 conn(0x558b7a143310 msgr2=0x558b7a176c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/176357167 shutdown_connections 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/176357167 wait complete. 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- start start 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 Processor -- start 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a184dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a182810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558b7a188070 con 0x558b7a17bb40 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558b7a09b060 con 0x558b7a17a5d0 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.937+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558b7a1881f0 con 0x558b7a185370 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c8bc7b740 10 client.0 _ll_register_callbacks cb 0x558b79dee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46268/0 (socket says 172.21.15.16:46268) 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 -- 172.21.15.16:0/1033428486 learned_addr learned my addr 172.21.15.16:0/1033428486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a184dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a182810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 -- 172.21.15.16:0/1033428486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 msgr2=0x558b7a182810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a182810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 -- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 msgr2=0x558b7a184dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a184dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 -- 172.21.15.16:0/1033428486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b79e513c0 con 0x558b7a17bb40 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a182810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4c74012860 tx=0x7f4c74012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b941516bf9a422f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7401f060 con 0x558b7a17bb40 2024-09-12T01:08:53.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4c74010dc0 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.938+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c005500 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c7401a430 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 10 client.19449 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b7a193bf0 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c005500 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c7400e560 con 0x558b7a17bb40 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 10 client.19449 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 10 client.19449 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 20 client.19449 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1950b0 0x558b7a197500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 4 client.19449 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:53.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b7a0ed860 con 0x558b7a1950b0 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.939+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1950b0 0x558b7a197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.940+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1950b0 0x558b7a197500 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13deee1bd2497e21 server_cookie=5caf713691a74492 in_seq=0 out_seq=0 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.940+0000 7f4c62ffd640 10 client.19449 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.940+0000 7f4c62ffd640 1 -- 172.21.15.16:0/1033428486 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b7a0ed860 con 0x558b7a1950b0 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.940+0000 7f4c62ffd640 10 client.19449 handle_command_reply: tid=0 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.940+0000 7f4c8bc7b740 1 client.19449 shutdown 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 msgr2=0x558b7a182080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4c74012860 tx=0x7f4c74012d20 comp rx=0 tx=0).stop 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 shutdown_connections 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1950b0 0x558b7a197500 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a182810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a182080 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/1033428486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a184dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 >> 172.21.15.16:0/1033428486 conn(0x558b7a143310 msgr2=0x558b7a17c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c62ffd640 0 client.19449 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 shutdown_connections 2024-09-12T01:08:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/1033428486 wait complete. 2024-09-12T01:08:53.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.941+0000 7f4c8bc7b740 20 client.19449 trim_cache size 0 max 16384 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 Processor -- start 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 -- start start 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a17b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 0x558b7a176700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841afa00 con 0x558b7a185370 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841b1540 con 0x558b7a17a5d0 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841b00d0 con 0x558b7a17bb40 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46270/0 (socket says 172.21.15.16:46270) 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 -- 172.21.15.16:0/2902385956 learned_addr learned my addr 172.21.15.16:0/2902385956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 0x558b7a176700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a17b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 -- 172.21.15.16:0/2902385956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 msgr2=0x558b7a176700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 0x558b7a176700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 -- 172.21.15.16:0/2902385956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 msgr2=0x558b7a17b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a17b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c7bfff640 1 -- 172.21.15.16:0/2902385956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7a177180 con 0x558b7a185370 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.954+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 0x558b7a176700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:53.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x558b7a17b070 tx=0x7f4c6c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f31fbfad888f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c797fa640 1 -- 172.21.15.16:0/2902385956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0047a0 con 0x558b7a185370 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c797fa640 1 -- 172.21.15.16:0/2902385956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c02e030 con 0x558b7a185370 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c797fa640 1 -- 172.21.15.16:0/2902385956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c004d10 con 0x558b7a185370 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2902385956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 msgr2=0x558b7a176c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.955+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x558b7a17b070 tx=0x7f4c6c004be0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2902385956 shutdown_connections 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a185370 0x558b7a176c40 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a17bb40 0x558b7a176700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2902385956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a17b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2902385956 >> 172.21.15.16:0/2902385956 conn(0x558b7a143310 msgr2=0x558b7a171a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2902385956 shutdown_connections 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2902385956 wait complete. 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- start start 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 Processor -- start 2024-09-12T01:08:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a188480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a188f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841b0e70 con 0x558b7a17bb40 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841b07a0 con 0x558b7a17a5d0 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c841ada00 con 0x558b7a185370 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.956+0000 7f4c8bc7b740 10 client.0 _ll_register_callbacks cb 0x7f4c7400a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46272/0 (socket says 172.21.15.16:46272) 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 -- 172.21.15.16:0/2486188922 learned_addr learned my addr 172.21.15.16:0/2486188922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a188480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 -- 172.21.15.16:0/2486188922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 msgr2=0x558b7a188f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a188f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a188f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 -- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 msgr2=0x558b7a188480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a188480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 -- 172.21.15.16:0/2486188922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7a1fe110 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a188f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c82ffd640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400e980 tx=0x7f4c74010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5332c0850567f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74031060 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f4c74012240 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.957+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c02e030 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c74010e90 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 10 client.19455 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b7a1ff3b0 con 0x558b7a17bb40 2024-09-12T01:08:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c02e030 con 0x558b7a17bb40 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c7401ab40 con 0x558b7a17bb40 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 10 client.19455 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 10 client.19455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 20 client.19455 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1ffb60 0x558b7a201f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 4 client.19455 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b7a0e7d70 con 0x558b7a1ffb60 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.958+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1ffb60 0x558b7a201f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c7bfff640 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1ffb60 0x558b7a201f80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c69c957889c33a8c server_cookie=690da38858a4638 in_seq=0 out_seq=0 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c62ffd640 10 client.19455 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c62ffd640 1 -- 172.21.15.16:0/2486188922 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b7a0e7d70 con 0x558b7a1ffb60 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c62ffd640 10 client.19455 handle_command_reply: tid=0 2024-09-12T01:08:53.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 client.19455 shutdown 2024-09-12T01:08:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 msgr2=0x558b7a1889c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400e980 tx=0x7f4c74010890 comp rx=0 tx=0).stop 2024-09-12T01:08:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 shutdown_connections 2024-09-12T01:08:53.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558b7a1ffb60 0x558b7a201f80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b7a185370 0x558b7a188f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b7a17bb40 0x558b7a1889c0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c62ffd640 0 client.19455 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 --2- 172.21.15.16:0/2486188922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558b7a17a5d0 0x558b7a188480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 >> 172.21.15.16:0/2486188922 conn(0x558b7a143310 msgr2=0x558b7a171a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 shutdown_connections 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 1 -- 172.21.15.16:0/2486188922 wait complete. 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.959+0000 7f4c8bc7b740 20 client.19455 trim_cache size 0 max 16384 2024-09-12T01:08:53.964 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 msgr2=0x7f4c700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 0x7f4c700a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c08f9d0 tx=0x7f4c7c004990 comp rx=0 tx=0).stop 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 msgr2=0x7f4c841014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009dc0 tx=0x7f4c7c003f10 comp rx=0 tx=0).stop 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 shutdown_connections 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c700a2cf0 0x7f4c700a51b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8410c1a0 0x7f4c841014e0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c84103720 0x7f4c84100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 --2- 172.21.15.16:0/1655719544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c84102e00 0x7f4c84100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 >> 172.21.15.16:0/1655719544 conn(0x7f4c840fc7a0 msgr2=0x7f4c840fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 shutdown_connections 2024-09-12T01:08:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:53.960+0000 7f4c898a2640 1 -- 172.21.15.16:0/1655719544 wait complete. 2024-09-12T01:08:54.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:08:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:55.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:54 smithi016 ceph-mon[30972]: pgmap v398: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:55.168 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:54 smithi159 ceph-mon[35566]: pgmap v398: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:54 smithi099 ceph-mon[35654]: pgmap v398: 97 pgs: 97 active+clean; 19 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.7 KiB/s wr, 9 op/s 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 -- 172.21.15.16:0/1588016667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee54103660 msgr2=0x7fee5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1588016667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee54103660 0x7fee5410bb50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009900 tx=0x7fee3c02a480 comp rx=0 tx=0).stop 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 -- 172.21.15.16:0/1588016667 shutdown_connections 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1588016667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee5410c170 0x7fee5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1588016667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee54103660 0x7fee5410bb50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1588016667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 -- 172.21.15.16:0/1588016667 >> 172.21.15.16:0/1588016667 conn(0x7fee540fc7a0 msgr2=0x7fee540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 -- 172.21.15.16:0/1588016667 shutdown_connections 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.520+0000 7fee58ba2640 1 -- 172.21.15.16:0/1588016667 wait complete. 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 Processor -- start 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 -- start start 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 0x7fee5419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee54111f70 con 0x7fee5410c170 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee54111df0 con 0x7fee54103660 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee58ba2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee541120f0 con 0x7fee54102d40 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.521+0000 7fee52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee5259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46286/0 (socket says 172.21.15.16:46286) 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee5259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:59250/0 (socket says 172.21.15.16:59250) 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 -- 172.21.15.16:0/1687891779 learned_addr learned my addr 172.21.15.16:0/1687891779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee49d9b640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 0x7fee5419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 -- 172.21.15.16:0/1687891779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 msgr2=0x7fee5419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 -- 172.21.15.16:0/1687891779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 msgr2=0x7fee5419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 0x7fee5419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 -- 172.21.15.16:0/1687891779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5419f5f0 con 0x7fee5410c170 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee49d9b640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 0x7fee5419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee5259c640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee52d9d640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00eaa0 tx=0x7fee4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee4bfff640 1 -- 172.21.15.16:0/1687891779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00cdb0 con 0x7fee5410c170 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5419f8e0 con 0x7fee5410c170 2024-09-12T01:08:55.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee4bfff640 1 -- 172.21.15.16:0/1687891779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee4c005020 con 0x7fee5410c170 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee4bfff640 1 -- 172.21.15.16:0/1687891779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0116b0 con 0x7fee5410c170 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.522+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee541a71c0 con 0x7fee5410c170 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.525+0000 7fee4bfff640 1 -- 172.21.15.16:0/1687891779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee4c005560 con 0x7fee5410c170 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.525+0000 7fee5af7b740 1 Processor -- start 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.525+0000 7fee5af7b740 1 -- start start 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb9ea2f0 con 0x55a5cba8f370 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb991cf0 con 0x55a5cba845d0 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5af7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb9e62b0 con 0x55a5cba85b40 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46298/0 (socket says 172.21.15.16:46298) 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 -- 172.21.15.16:0/4054042074 learned_addr learned my addr 172.21.15.16:0/4054042074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:55.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee5259c640 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee4bfff640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 0x7fee400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee52d9d640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 0x7fee400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 -- 172.21.15.16:0/4054042074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 msgr2=0x55a5cba84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 -- 172.21.15.16:0/4054042074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 msgr2=0x55a5cba84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 -- 172.21.15.16:0/4054042074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cb9763a0 con 0x55a5cba8f370 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee4bfff640 1 -- 172.21.15.16:0/1687891779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fee4c0cf640 con 0x7fee5410c170 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.526+0000 7fee49d9b640 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fee3c031550 tx=0x7fee3c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb659359a5c6736e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee48d99640 1 -- 172.21.15.16:0/4054042074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c043030 con 0x55a5cba8f370 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee52d9d640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 0x7fee400a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06eba0 tx=0x7fee4c06ebd0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee48d99640 1 -- 172.21.15.16:0/4054042074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c009520 con 0x55a5cba8f370 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee48d99640 1 -- 172.21.15.16:0/4054042074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c035040 con 0x55a5cba8f370 2024-09-12T01:08:55.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 -- 172.21.15.16:0/4054042074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 msgr2=0x55a5cba91760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fee3c031550 tx=0x7fee3c005b70 comp rx=0 tx=0).stop 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 -- 172.21.15.16:0/4054042074 shutdown_connections 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba91760 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 --2- 172.21.15.16:0/4054042074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.527+0000 7fee5af7b740 1 -- 172.21.15.16:0/4054042074 >> 172.21.15.16:0/4054042074 conn(0x55a5cba4d310 msgr2=0x55a5cba80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- 172.21.15.16:0/4054042074 shutdown_connections 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- 172.21.15.16:0/4054042074 wait complete. 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- start start 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 Processor -- start 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba845d0 0x55a5cba8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba85b40 0x55a5cba8c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5cba92010 con 0x55a5cba8f370 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb9a5060 con 0x55a5cba85b40 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5cba92190 con 0x55a5cba845d0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee5af7b740 10 client.0 _ll_register_callbacks cb 0x55a5cb6f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46300/0 (socket says 172.21.15.16:46300) 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.528+0000 7fee49d9b640 1 -- 172.21.15.16:0/2118733031 learned_addr learned my addr 172.21.15.16:0/2118733031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba845d0 0x55a5cba8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee5259c640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba85b40 0x55a5cba8c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 -- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba845d0 msgr2=0x55a5cba8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba845d0 0x55a5cba8ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 -- 172.21.15.16:0/2118733031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba85b40 msgr2=0x55a5cba8c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba85b40 0x55a5cba8c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 -- 172.21.15.16:0/2118733031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cb75b3c0 con 0x55a5cba8f370 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005ec0 tx=0x7fee3c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72a1a9e5dce4aec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c000e60 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fee3c004d00 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c009520 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee3c05a050 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee5af7b740 10 client.19467 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.529+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5cba9dab0 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c009520 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee3c03a040 con 0x55a5cba8f370 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 10 client.19467 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 10 client.19467 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 20 client.19467 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cba9ee00 0x55a5cbaa11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 4 client.19467 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5cb9f7860 con 0x55a5cba9ee00 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.530+0000 7fee5259c640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cba9ee00 0x55a5cbaa11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.531+0000 7fee39ffb640 10 client.19467 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.531+0000 7fee5259c640 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cba9ee00 0x55a5cbaa11f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fbcfebec2ffbf32 server_cookie=49399c7ba5bbbfe8 in_seq=0 out_seq=0 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.531+0000 7fee39ffb640 1 -- 172.21.15.16:0/2118733031 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5cb9f7860 con 0x55a5cba9ee00 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.531+0000 7fee39ffb640 10 client.19467 handle_command_reply: tid=0 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.531+0000 7fee5af7b740 1 client.19467 shutdown 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 msgr2=0x55a5cba8c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005ec0 tx=0x7fee3c004610 comp rx=0 tx=0).stop 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 shutdown_connections 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8c610 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cba9ee00 0x55a5cbaa11f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba85b40 0x55a5cba8c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee39ffb640 0 client.19467 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2118733031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba845d0 0x55a5cba8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 >> 172.21.15.16:0/2118733031 conn(0x55a5cba4d310 msgr2=0x55a5cba86960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 shutdown_connections 2024-09-12T01:08:55.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 1 -- 172.21.15.16:0/2118733031 wait complete. 2024-09-12T01:08:55.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.532+0000 7fee5af7b740 20 client.19467 trim_cache size 0 max 16384 2024-09-12T01:08:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 Processor -- start 2024-09-12T01:08:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 -- start start 2024-09-12T01:08:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba9b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee541af9e0 con 0x55a5cba8f370 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee541b1520 con 0x55a5cba845d0 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee5af7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee541b00b0 con 0x55a5cba85b40 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46316/0 (socket says 172.21.15.16:46316) 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 -- 172.21.15.16:0/2196454179 learned_addr learned my addr 172.21.15.16:0/2196454179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 -- 172.21.15.16:0/2196454179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 msgr2=0x55a5cba9b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba9b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 -- 172.21.15.16:0/2196454179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 msgr2=0x55a5cba80e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.545+0000 7fee49d9b640 1 -- 172.21.15.16:0/2196454179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cba8aff0 con 0x55a5cba8f370 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee49d9b640 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fee3c03dd10 tx=0x7fee3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e59afdc8af80fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee3bfff640 1 -- 172.21.15.16:0/2196454179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c035030 con 0x55a5cba8f370 2024-09-12T01:08:55.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee3bfff640 1 -- 172.21.15.16:0/2196454179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c031920 con 0x55a5cba8f370 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee3bfff640 1 -- 172.21.15.16:0/2196454179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c043040 con 0x55a5cba8f370 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 -- 172.21.15.16:0/2196454179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 msgr2=0x55a5cba8b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fee3c03dd10 tx=0x7fee3c009780 comp rx=0 tx=0).stop 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 -- 172.21.15.16:0/2196454179 shutdown_connections 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba8b8e0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba9b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 --2- 172.21.15.16:0/2196454179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 -- 172.21.15.16:0/2196454179 >> 172.21.15.16:0/2196454179 conn(0x55a5cba4d310 msgr2=0x55a5cba80700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 -- 172.21.15.16:0/2196454179 shutdown_connections 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.546+0000 7fee5af7b740 1 -- 172.21.15.16:0/2196454179 wait complete. 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 -- start start 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 Processor -- start 2024-09-12T01:08:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba97cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee541b0e50 con 0x55a5cba8f370 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee541b0780 con 0x55a5cba845d0 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee541ad9e0 con 0x55a5cba85b40 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5af7b740 10 client.0 _ll_register_callbacks cb 0x7fee44009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee49d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46324/0 (socket says 172.21.15.16:46324) 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee49d9b640 1 -- 172.21.15.16:0/163832083 learned_addr learned my addr 172.21.15.16:0/163832083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee5259c640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba97cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.547+0000 7fee49d9b640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 -- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 msgr2=0x55a5cba97cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba97cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 -- 172.21.15.16:0/163832083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 msgr2=0x55a5cba97780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba97780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 -- 172.21.15.16:0/163832083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cba89420 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee49d9b640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x55a5cba882b0 tx=0x7fee3c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aed4c4b56a35fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c065070 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fee3c004b00 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c031920 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee3c043040 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee5af7b740 10 client.19473 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5cbb09380 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.548+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c031920 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fee3c069610 con 0x55a5cba8f370 2024-09-12T01:08:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 10 client.19473 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 10 client.19473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 20 client.19473 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cbb097a0 0x55a5cbb0bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 4 client.19473 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5cb9f1d70 con 0x55a5cbb097a0 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5259c640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cbb097a0 0x55a5cbb0bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee5259c640 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cbb097a0 0x55a5cbb0bb90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5aa0b3366e2a14d0 server_cookie=ec04c0f8282273eb in_seq=0 out_seq=0 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.549+0000 7fee39ffb640 10 client.19473 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee39ffb640 1 -- 172.21.15.16:0/163832083 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5cb9f1d70 con 0x55a5cbb097a0 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee39ffb640 10 client.19473 handle_command_reply: tid=0 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 client.19473 shutdown 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 msgr2=0x55a5cba88bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x55a5cba882b0 tx=0x7fee3c005c90 comp rx=0 tx=0).stop 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 shutdown_connections 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5cba8f370 0x55a5cba88bf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5cba85b40 0x55a5cba97cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5cbb097a0 0x55a5cbb0bb90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 --2- 172.21.15.16:0/163832083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5cba845d0 0x55a5cba97780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 >> 172.21.15.16:0/163832083 conn(0x55a5cba4d310 msgr2=0x55a5cba8d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee39ffb640 0 client.19473 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 shutdown_connections 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 1 -- 172.21.15.16:0/163832083 wait complete. 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee5af7b740 20 client.19473 trim_cache size 0 max 16384 2024-09-12T01:08:55.555 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 msgr2=0x7fee400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 0x7fee400a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee4c06eba0 tx=0x7fee4c06ebd0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 msgr2=0x7fee5419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00eaa0 tx=0x7fee4c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.550+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 shutdown_connections 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee400a2cf0 0x7fee400a51b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5410c170 0x7fee5419ef10 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee54103660 0x7fee5419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 --2- 172.21.15.16:0/1687891779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee54102d40 0x7fee5419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 >> 172.21.15.16:0/1687891779 conn(0x7fee540fc7a0 msgr2=0x7fee5410d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 shutdown_connections 2024-09-12T01:08:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:55.551+0000 7fee58ba2640 1 -- 172.21.15.16:0/1687891779 wait complete. 2024-09-12T01:08:56.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:08:56.646 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:57.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:56 smithi016 ceph-mon[30972]: pgmap v399: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1073566855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b410c170 msgr2=0x7fd9b410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1073566855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b410c170 0x7fd9b410e610 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009ca0 tx=0x7fd9ac02a950 comp rx=0 tx=0).stop 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1073566855 shutdown_connections 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1073566855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b410c170 0x7fd9b410e610 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1073566855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4103660 0x7fd9b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1073566855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4102d40 0x7fd9b4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1073566855 >> 172.21.15.16:0/1073566855 conn(0x7fd9b40fc7a0 msgr2=0x7fd9b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1073566855 shutdown_connections 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.147+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1073566855 wait complete. 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 Processor -- start 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 -- start start 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4103660 0x7fd9b4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b410c170 0x7fd9b4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112080 con 0x7fd9b4102d40 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4111f00 con 0x7fd9b410c170 2024-09-12T01:08:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b9c9e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112200 con 0x7fd9b4103660 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46346/0 (socket says 172.21.15.16:46346) 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 -- 172.21.15.16:0/1271647289 learned_addr learned my addr 172.21.15.16:0/1271647289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b410c170 0x7fd9b4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4103660 0x7fd9b4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 -- 172.21.15.16:0/1271647289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4103660 msgr2=0x7fd9b4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4103660 0x7fd9b4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 -- 172.21.15.16:0/1271647289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b410c170 msgr2=0x7fd9b4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.148+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b410c170 0x7fd9b4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b37fe640 1 -- 172.21.15.16:0/1271647289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b41a6a20 con 0x7fd9b4102d40 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b4a0 tx=0x7fd9a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b0ff9640 1 -- 172.21.15.16:0/1271647289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4014070 con 0x7fd9b4102d40 2024-09-12T01:08:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b0ff9640 1 -- 172.21.15.16:0/1271647289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a4010af0 con 0x7fd9b4102d40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b0ff9640 1 -- 172.21.15.16:0/1271647289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4017d10 con 0x7fd9b4102d40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b41a6d10 con 0x7fd9b4102d40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.149+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a71f0 con 0x7fd9b4102d40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.151+0000 7fd9b0ff9640 1 -- 172.21.15.16:0/1271647289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9a4017070 con 0x7fd9b4102d40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 Processor -- start 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 -- start start 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe747090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe754760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55babe6ad2f0 con 0x55babe7475d0 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55babe654cf0 con 0x55babe752370 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9bc077740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55babe6a92b0 con 0x55babe748b40 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b0ff9640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 0x7fd99c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46358/0 (socket says 172.21.15.16:46358) 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b37fe640 1 -- 172.21.15.16:0/545723702 learned_addr learned my addr 172.21.15.16:0/545723702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe747090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe754760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.152+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 0x7fd99c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b0ff9640 1 -- 172.21.15.16:0/1271647289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd9a40ce660 con 0x7fd9b4102d40 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 -- 172.21.15.16:0/545723702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 msgr2=0x55babe747090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe747090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 -- 172.21.15.16:0/545723702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 msgr2=0x55babe754760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe754760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 -- 172.21.15.16:0/545723702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babe6393a0 con 0x55babe7475d0 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe747090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40a3370 tx=0x7fd9a40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4128bd032e80754d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd999ffb640 1 -- 172.21.15.16:0/545723702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a401e070 con 0x55babe7475d0 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 0x7fd99c0a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009430 tx=0x7fd9ac02a950 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd999ffb640 1 -- 172.21.15.16:0/545723702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40a38a0 con 0x55babe7475d0 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.153+0000 7fd999ffb640 1 -- 172.21.15.16:0/545723702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a409a130 con 0x55babe7475d0 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- 172.21.15.16:0/545723702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 msgr2=0x55babe747a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40a3370 tx=0x7fd9a40047d0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- 172.21.15.16:0/545723702 shutdown_connections 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe754760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe747a10 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 --2- 172.21.15.16:0/545723702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe747090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- 172.21.15.16:0/545723702 >> 172.21.15.16:0/545723702 conn(0x55babe710310 msgr2=0x55babe743c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- 172.21.15.16:0/545723702 shutdown_connections 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- 172.21.15.16:0/545723702 wait complete. 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 -- start start 2024-09-12T01:08:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.154+0000 7fd9bc077740 1 Processor -- start 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe74f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe74f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55babe754fb0 con 0x55babe7475d0 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55babe668060 con 0x55babe752370 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55babe755130 con 0x55babe748b40 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9bc077740 10 client.0 _ll_register_callbacks cb 0x55babe3bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46370/0 (socket says 172.21.15.16:46370) 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 -- 172.21.15.16:0/2813153655 learned_addr learned my addr 172.21.15.16:0/2813153655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe74f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe74f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 -- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 msgr2=0x55babe74f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe74f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 -- 172.21.15.16:0/2813153655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 msgr2=0x55babe74f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe74f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 -- 172.21.15.16:0/2813153655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babe41e3c0 con 0x55babe7475d0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe74f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe74f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409ac70 tx=0x7fd9a4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12c54844612acab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a401e050 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.155+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd9a40cc140 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a40a38a0 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a409a130 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 10 client.19485 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55babe760a80 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a40a38a0 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a4079b00 con 0x55babe7475d0 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 10 client.19485 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 20 client.19485 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe761dd0 0x55babe7641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 4 client.19485 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55babe6ba860 con 0x55babe761dd0 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.156+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe761dd0 0x55babe7641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.157+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe761dd0 0x55babe7641c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c6e0c12480567a server_cookie=461c63d3cb4b6426 in_seq=0 out_seq=0 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.157+0000 7fd992ffd640 10 client.19485 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.157+0000 7fd992ffd640 1 -- 172.21.15.16:0/2813153655 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55babe6ba860 con 0x55babe761dd0 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.157+0000 7fd992ffd640 10 client.19485 handle_command_reply: tid=0 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 client.19485 shutdown 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 msgr2=0x55babe751d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fd9a409ac70 tx=0x7fd9a4009390 comp rx=0 tx=0).stop 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 shutdown_connections 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe761dd0 0x55babe7641c0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe74f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe74f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd992ffd640 0 client.19485 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2813153655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe7475d0 0x55babe751d90 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 >> 172.21.15.16:0/2813153655 conn(0x55babe710310 msgr2=0x55babe749990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 shutdown_connections 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 1 -- 172.21.15.16:0/2813153655 wait complete. 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.158+0000 7fd9bc077740 20 client.19485 trim_cache size 0 max 16384 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 Processor -- start 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 -- start start 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe7475d0 0x55babe750fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe7514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41afbc0 con 0x55babe752370 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b1700 con 0x55babe7475d0 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9bc077740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b0290 con 0x55babe748b40 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46382/0 (socket says 172.21.15.16:46382) 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.171+0000 7fd9b3fff640 1 -- 172.21.15.16:0/2870709787 learned_addr learned my addr 172.21.15.16:0/2870709787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe7514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe7475d0 0x55babe750fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 -- 172.21.15.16:0/2870709787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 msgr2=0x55babe7514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe7514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 -- 172.21.15.16:0/2870709787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe7475d0 msgr2=0x55babe750fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe7475d0 0x55babe750fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 -- 172.21.15.16:0/2870709787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babe74ffe0 con 0x55babe752370 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac035000 tx=0x7fd9ac005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4e73ea8a01f2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9997fa640 1 -- 172.21.15.16:0/2870709787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac043050 con 0x55babe752370 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9997fa640 1 -- 172.21.15.16:0/2870709787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac044040 con 0x55babe752370 2024-09-12T01:08:57.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.172+0000 7fd9997fa640 1 -- 172.21.15.16:0/2870709787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac034040 con 0x55babe752370 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- 172.21.15.16:0/2870709787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 msgr2=0x55babe74a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac035000 tx=0x7fd9ac005fa0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- 172.21.15.16:0/2870709787 shutdown_connections 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe752370 0x55babe74a120 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe748b40 0x55babe7514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2870709787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe7475d0 0x55babe750fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- 172.21.15.16:0/2870709787 >> 172.21.15.16:0/2870709787 conn(0x55babe710310 msgr2=0x55babe7508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- 172.21.15.16:0/2870709787 shutdown_connections 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- 172.21.15.16:0/2870709787 wait complete. 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 -- start start 2024-09-12T01:08:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.173+0000 7fd9bc077740 1 Processor -- start 2024-09-12T01:08:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe7475d0 0x55babe765170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe765bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b1030 con 0x55babe748b40 2024-09-12T01:08:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b0960 con 0x55babe752370 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41adbc0 con 0x55babe7475d0 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9bc077740 10 client.0 _ll_register_callbacks cb 0x7fd9ac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46390/0 (socket says 172.21.15.16:46390) 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 -- 172.21.15.16:0/2014939 learned_addr learned my addr 172.21.15.16:0/2014939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b3fff640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe765bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe7475d0 0x55babe765170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 -- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe7475d0 msgr2=0x55babe765170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe7475d0 0x55babe765170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 -- 172.21.15.16:0/2014939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 msgr2=0x55babe765bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe765bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 -- 172.21.15.16:0/2014939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babe7cb910 con 0x55babe748b40 2024-09-12T01:08:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd9b2ffd640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006e10 tx=0x7fd9a80157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f52f7c022c39157 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.174+0000 7fd992ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80147c0 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac044040 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7fd9a8014960 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a8014b60 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 10 client.19491 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55babe7ccc50 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac044040 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9a802d3b0 con 0x55babe748b40 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 10 client.19491 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 10 client.19491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 20 client.19491 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe7cd3b0 0x55babe7cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 4 client.19491 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.175+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55babe6b4d70 con 0x55babe7cd3b0 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe7cd3b0 0x55babe7cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd992ffd640 10 client.19491 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9b37fe640 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe7cd3b0 0x55babe7cf7d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e025cdac23503de server_cookie=23bc781cc5f1f5b1 in_seq=0 out_seq=0 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd992ffd640 1 -- 172.21.15.16:0/2014939 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55babe6b4d70 con 0x55babe7cd3b0 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd992ffd640 10 client.19491 handle_command_reply: tid=0 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 client.19491 shutdown 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 msgr2=0x55babe7656b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8006e10 tx=0x7fd9a80157f0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 shutdown_connections 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55babe7cd3b0 0x55babe7cf7d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55babe752370 0x55babe765bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55babe748b40 0x55babe7656b0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 --2- 172.21.15.16:0/2014939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55babe7475d0 0x55babe765170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd992ffd640 0 client.19491 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 >> 172.21.15.16:0/2014939 conn(0x55babe710310 msgr2=0x55babe75d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.176+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 shutdown_connections 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9bc077740 1 -- 172.21.15.16:0/2014939 wait complete. 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9bc077740 20 client.19491 trim_cache size 0 max 16384 2024-09-12T01:08:57.192 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 msgr2=0x7fd99c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 0x7fd99c0a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009430 tx=0x7fd9ac02a950 comp rx=0 tx=0).stop 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 msgr2=0x7fd9b41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b4a0 tx=0x7fd9a400b960 comp rx=0 tx=0).stop 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 shutdown_connections 2024-09-12T01:08:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd99c0a2cf0 0x7fd99c0a51b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b410c170 0x7fd9b4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9b4103660 0x7fd9b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 --2- 172.21.15.16:0/1271647289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b4102d40 0x7fd9b41009a0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 >> 172.21.15.16:0/1271647289 conn(0x7fd9b40fc7a0 msgr2=0x7fd9b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 shutdown_connections 2024-09-12T01:08:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:57.177+0000 7fd9b9c9e640 1 -- 172.21.15.16:0/1271647289 wait complete. 2024-09-12T01:08:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:56 smithi099 ceph-mon[35654]: pgmap v399: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-09-12T01:08:57.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:56 smithi159 ceph-mon[35566]: pgmap v399: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-09-12T01:08:58.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:08:58.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:08:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:08:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:08:58 smithi099 ceph-mon[35654]: pgmap v400: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-12T01:08:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:08:58 smithi159 ceph-mon[35566]: pgmap v400: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-12T01:08:59.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:08:58 smithi016 ceph-mon[30972]: pgmap v400: 97 pgs: 97 active+clean; 20 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-12T01:08:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 -- 172.21.15.16:0/2172925811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 msgr2=0x7f0a5c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 --2- 172.21.15.16:0/2172925811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c10de80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009900 tx=0x7f0a4c02a480 comp rx=0 tx=0).stop 2024-09-12T01:08:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 -- 172.21.15.16:0/2172925811 shutdown_connections 2024-09-12T01:08:59.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 --2- 172.21.15.16:0/2172925811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 --2- 172.21.15.16:0/2172925811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c10de80 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 --2- 172.21.15.16:0/2172925811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c074620 0x7f0a5c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 -- 172.21.15.16:0/2172925811 >> 172.21.15.16:0/2172925811 conn(0x7f0a5c0fd8c0 msgr2=0x7f0a5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 -- 172.21.15.16:0/2172925811 shutdown_connections 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.700+0000 7f0a632fb640 1 -- 172.21.15.16:0/2172925811 wait complete. 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 Processor -- start 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 -- start start 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c074620 0x7f0a5c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c114160 con 0x7f0a5c074f40 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c113fe0 con 0x7f0a5c10e3c0 2024-09-12T01:08:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a632fb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1142e0 con 0x7f0a5c074620 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33212/0 (socket says 172.21.15.16:33212) 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 -- 172.21.15.16:0/413345610 learned_addr learned my addr 172.21.15.16:0/413345610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a61898640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a61097640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c074620 0x7f0a5c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 -- 172.21.15.16:0/413345610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c074620 msgr2=0x7f0a5c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c074620 0x7f0a5c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 -- 172.21.15.16:0/413345610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 msgr2=0x7f0a5c19df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c19df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a60896640 1 -- 172.21.15.16:0/413345610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c19e6b0 con 0x7f0a5c074f40 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.701+0000 7f0a61898640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a60896640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006010 tx=0x7f0a4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a527fc640 1 -- 172.21.15.16:0/413345610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c03f030 con 0x7f0a5c074f40 2024-09-12T01:08:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c19e940 con 0x7f0a5c074f40 2024-09-12T01:08:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a527fc640 1 -- 172.21.15.16:0/413345610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c003ce0 con 0x7f0a5c074f40 2024-09-12T01:08:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a527fc640 1 -- 172.21.15.16:0/413345610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c035070 con 0x7f0a5c074f40 2024-09-12T01:08:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.702+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c1a7310 con 0x7f0a5c074f40 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.704+0000 7f0a527fc640 1 -- 172.21.15.16:0/413345610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0a4c042030 con 0x7f0a5c074f40 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 Processor -- start 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 -- start start 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa8cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561e6a9f2280 con 0x561e6aa97d40 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561e6a999b10 con 0x561e6aa8d480 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a656d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e6a9ee240 con 0x561e6aa8cb60 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33222/0 (socket says 172.21.15.16:33222) 2024-09-12T01:08:59.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 -- 172.21.15.16:0/1701391109 learned_addr learned my addr 172.21.15.16:0/1701391109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a527fc640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 0x7f0a480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a61097640 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a61898640 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa8cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 -- 172.21.15.16:0/1701391109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 msgr2=0x561e6aa8cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa8cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 -- 172.21.15.16:0/1701391109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 msgr2=0x561e6aa97800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.705+0000 7f0a60896640 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa97800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a60896640 1 -- 172.21.15.16:0/1701391109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6a9a61f0 con 0x561e6aa97d40 2024-09-12T01:08:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a61898640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 0x7f0a480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a527fc640 1 -- 172.21.15.16:0/413345610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a4c0efbc0 con 0x7f0a5c074f40 2024-09-12T01:08:59.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a60896640 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0ed2b0 tx=0x7f0a4c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d99eb5bd1f089cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a477fe640 1 -- 172.21.15.16:0/1701391109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c049030 con 0x561e6aa97d40 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a61898640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 0x7f0a480a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a54005fd0 tx=0x7f0a54004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a477fe640 1 -- 172.21.15.16:0/1701391109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c040e60 con 0x561e6aa97d40 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.706+0000 7f0a477fe640 1 -- 172.21.15.16:0/1701391109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c03b580 con 0x561e6aa97d40 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- 172.21.15.16:0/1701391109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 msgr2=0x561e6aa9a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0ed2b0 tx=0x7f0a4c02a480 comp rx=0 tx=0).stop 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- 172.21.15.16:0/1701391109 shutdown_connections 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa9a130 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa97800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1701391109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa8cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- 172.21.15.16:0/1701391109 >> 172.21.15.16:0/1701391109 conn(0x561e6aa890f0 msgr2=0x561e6aa894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- 172.21.15.16:0/1701391109 shutdown_connections 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- 172.21.15.16:0/1701391109 wait complete. 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 -- start start 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.707+0000 7f0a656d4740 1 Processor -- start 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa94b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561e6aa9a6d0 con 0x561e6aa97d40 2024-09-12T01:08:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561e6a9acf10 con 0x561e6aa8d480 2024-09-12T01:08:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e6aa9a850 con 0x561e6aa8cb60 2024-09-12T01:08:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a656d4740 10 client.0 _ll_register_callbacks cb 0x561e6a975a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:59.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33226/0 (socket says 172.21.15.16:33226) 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 -- 172.21.15.16:0/1158903885 learned_addr learned my addr 172.21.15.16:0/1158903885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a60896640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61097640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa94b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 -- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 msgr2=0x561e6aa94b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa94b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 -- 172.21.15.16:0/1158903885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 msgr2=0x561e6aa950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 -- 172.21.15.16:0/1158903885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6a7636a0 con 0x561e6aa97d40 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61097640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a61898640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f0a540138e0 tx=0x7f0a54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d14c22d2775b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.708+0000 7f0a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:59.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54013e60 con 0x561e6aa97d40 2024-09-12T01:08:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a54004d20 con 0x561e6aa97d40 2024-09-12T01:08:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e6a9ff7f0 con 0x561e6aa97d40 2024-09-12T01:08:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a5401eaf0 con 0x561e6aa97d40 2024-09-12T01:08:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 10 client.19503 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:59.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e6aaa61f0 con 0x561e6aa97d40 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561e6a9ff7f0 con 0x561e6aa97d40 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a5400f040 con 0x561e6aa97d40 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 10 client.19503 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 20 client.19503 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:59.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6aaa7600 0x561e6aaa99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 4 client.19503 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.709+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6a9f9d00 con 0x561e6aaa7600 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.710+0000 7f0a61097640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6aaa7600 0x561e6aaa99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.710+0000 7f0a61097640 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6aaa7600 0x561e6aaa99f0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be1fdf9344a74e0 server_cookie=b38cc4c7ef65ff2d in_seq=0 out_seq=0 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.710+0000 7f0a44ff9640 10 client.19503 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:59.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a44ff9640 1 -- 172.21.15.16:0/1158903885 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e6a9f9d00 con 0x561e6aaa7600 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a44ff9640 10 client.19503 handle_command_reply: tid=0 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 client.19503 shutdown 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 msgr2=0x561e6aa95610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f0a540138e0 tx=0x7f0a54004ab0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 shutdown_connections 2024-09-12T01:08:59.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6aaa7600 0x561e6aaa99f0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa97d40 0x561e6aa95610 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8d480 0x561e6aa950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1158903885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8cb60 0x561e6aa94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a44ff9640 0 client.19503 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 >> 172.21.15.16:0/1158903885 conn(0x561e6aa890f0 msgr2=0x561e6aa8ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 shutdown_connections 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 1 -- 172.21.15.16:0/1158903885 wait complete. 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.711+0000 7f0a656d4740 20 client.19503 trim_cache size 0 max 16384 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.728+0000 7f0a656d4740 1 Processor -- start 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.728+0000 7f0a656d4740 1 -- start start 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8d480 0x561e6aaaf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 0x561e6aaaf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1afb20 con 0x561e6aa8cb60 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b1660 con 0x561e6aa97d40 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a656d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b01f0 con 0x561e6aa8d480 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33242/0 (socket says 172.21.15.16:33242) 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 -- 172.21.15.16:0/1423557061 learned_addr learned my addr 172.21.15.16:0/1423557061 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 -- 172.21.15.16:0/1423557061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8d480 msgr2=0x561e6aaaf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a60896640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8d480 0x561e6aaaf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61898640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 0x561e6aaaf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8d480 0x561e6aaaf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 -- 172.21.15.16:0/1423557061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 msgr2=0x561e6aaaf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 0x561e6aaaf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 -- 172.21.15.16:0/1423557061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6aa88f20 con 0x561e6aa8cb60 2024-09-12T01:08:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61898640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 0x561e6aaaf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a61097640 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800bce0 tx=0x7f0a580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc3d1eb6509518 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a46ffd640 1 -- 172.21.15.16:0/1423557061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58004230 con 0x561e6aa8cb60 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a46ffd640 1 -- 172.21.15.16:0/1423557061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a580043d0 con 0x561e6aa8cb60 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.729+0000 7f0a46ffd640 1 -- 172.21.15.16:0/1423557061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5801d9f0 con 0x561e6aa8cb60 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- 172.21.15.16:0/1423557061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 msgr2=0x561e6aa969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800bce0 tx=0x7f0a580118d0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- 172.21.15.16:0/1423557061 shutdown_connections 2024-09-12T01:08:59.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa97d40 0x561e6aaaf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa8d480 0x561e6aaaf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 --2- 172.21.15.16:0/1423557061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8cb60 0x561e6aa969f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- 172.21.15.16:0/1423557061 >> 172.21.15.16:0/1423557061 conn(0x561e6aa890f0 msgr2=0x561e6aa962e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- 172.21.15.16:0/1423557061 shutdown_connections 2024-09-12T01:08:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- 172.21.15.16:0/1423557061 wait complete. 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 -- start start 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.730+0000 7f0a656d4740 1 Processor -- start 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 0x561e6aa9f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 0x561e6aa9c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b0f90 con 0x561e6aa8d480 2024-09-12T01:08:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1b08c0 con 0x561e6aa8cb60 2024-09-12T01:08:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c1adb20 con 0x561e6aa97d40 2024-09-12T01:08:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a656d4740 10 client.0 _ll_register_callbacks cb 0x7f0a54009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:08:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33258/0 (socket says 172.21.15.16:33258) 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 -- 172.21.15.16:0/3163465912 learned_addr learned my addr 172.21.15.16:0/3163465912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a61097640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 0x561e6aa9f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 -- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 msgr2=0x561e6aa9c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a61898640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 0x561e6aa9c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 0x561e6aa9c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 -- 172.21.15.16:0/3163465912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 msgr2=0x561e6aa9f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 0x561e6aa9f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a60896640 1 -- 172.21.15.16:0/3163465912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6aaa38a0 con 0x561e6aa8d480 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a61097640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 0x561e6aa9f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.731+0000 7f0a61898640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 0x561e6aa9c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a60896640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0b8b70 tx=0x7f0a4c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d347905eb1ac5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:08:59.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c049040 con 0x561e6aa8d480 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f0a4c0ba4a0 con 0x561e6aa8d480 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a580043d0 con 0x561e6aa8d480 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c0bbb00 con 0x561e6aa8d480 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 10 client.19509 fetch_fsmap learned FSMap version 28 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e6ab127f0 con 0x561e6aa8d480 2024-09-12T01:08:59.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a580043d0 con 0x561e6aa8d480 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0a4c0ed590 con 0x561e6aa8d480 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 10 client.19509 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 20 client.19509 populate_metadata read hostname 'smithi016' 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6ab12e20 0x561e6ab15240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:08:59.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 4 client.19509 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:08:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.732+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561e6a976110 con 0x561e6ab12e20 2024-09-12T01:08:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.733+0000 7f0a61097640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6ab12e20 0x561e6ab15240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:08:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.733+0000 7f0a61097640 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6ab12e20 0x561e6ab15240 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad052e60aa3d880f server_cookie=3c0b1a3b89a2c896 in_seq=0 out_seq=0 2024-09-12T01:08:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.733+0000 7f0a44ff9640 10 client.19509 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:59.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a44ff9640 1 -- 172.21.15.16:0/3163465912 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561e6a976110 con 0x561e6ab12e20 2024-09-12T01:08:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a44ff9640 10 client.19509 handle_command_reply: tid=0 2024-09-12T01:08:59.786 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:08:59.786 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:08:59.787 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7f07a25f-2b28-49b5-a00f-5b678ff96e3b", 2024-09-12T01:08:59.787 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:08:59.787 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:08:59.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 client.19509 shutdown 2024-09-12T01:08:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 msgr2=0x561e6aa9fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0b8b70 tx=0x7f0a4c004460 comp rx=0 tx=0).stop 2024-09-12T01:08:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 shutdown_connections 2024-09-12T01:08:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x561e6ab12e20 0x561e6ab15240 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e6aa97d40 0x561e6aa9c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e6aa8d480 0x561e6aa9fea0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a44ff9640 0 client.19509 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 --2- 172.21.15.16:0/3163465912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561e6aa8cb60 0x561e6aa9f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 >> 172.21.15.16:0/3163465912 conn(0x561e6aa890f0 msgr2=0x561e6aaa30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.734+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 shutdown_connections 2024-09-12T01:08:59.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a656d4740 1 -- 172.21.15.16:0/3163465912 wait complete. 2024-09-12T01:08:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a656d4740 20 client.19509 trim_cache size 0 max 16384 2024-09-12T01:08:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 msgr2=0x7f0a480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 0x7f0a480a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a54005fd0 tx=0x7f0a54004a40 comp rx=0 tx=0).stop 2024-09-12T01:08:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 msgr2=0x7f0a5c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:08:59.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c006010 tx=0x7f0a4c0042f0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 shutdown_connections 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0a480a2cf0 0x7f0a480a51b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a5c10e3c0 0x7f0a5c19df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a5c074f40 0x7f0a5c19aca0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 --2- 172.21.15.16:0/413345610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a5c074620 0x7f0a5c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:08:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 >> 172.21.15.16:0/413345610 conn(0x7f0a5c0fd8c0 msgr2=0x7f0a5c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:08:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 shutdown_connections 2024-09-12T01:08:59.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:08:59.735+0000 7f0a632fb640 1 -- 172.21.15.16:0/413345610 wait complete. 2024-09-12T01:09:00.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:09:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:00 smithi099 ceph-mon[35654]: pgmap v401: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T01:09:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:00 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:09:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:00 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:00 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:09:01.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:00 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:00 smithi016 ceph-mon[30972]: pgmap v401: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T01:09:01.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:00 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:09:01.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:00 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:00 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:09:01.310 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:00 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:00 smithi159 ceph-mon[35566]: pgmap v401: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 17 op/s 2024-09-12T01:09:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:00 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:09:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:00 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:00 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:09:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:00 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 -- 172.21.15.16:0/3535305203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464102790 msgr2=0x7f64641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 --2- 172.21.15.16:0/3535305203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464102790 0x7f64641067e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f644c009900 tx=0x7f644c02a410 comp rx=0 tx=0).stop 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 -- 172.21.15.16:0/3535305203 shutdown_connections 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 --2- 172.21.15.16:0/3535305203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464106f10 0x7f646410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 --2- 172.21.15.16:0/3535305203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464102790 0x7f64641067e0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 --2- 172.21.15.16:0/3535305203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464101de0 0x7f64641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 -- 172.21.15.16:0/3535305203 >> 172.21.15.16:0/3535305203 conn(0x7f64640fd8c0 msgr2=0x7f64640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 -- 172.21.15.16:0/3535305203 shutdown_connections 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 -- 172.21.15.16:0/3535305203 wait complete. 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.398+0000 7f646934d640 1 Processor -- start 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 -- start start 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6464111f30 con 0x7f6464101de0 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6464111db0 con 0x7f6464102790 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f646934d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64641120b0 con 0x7f6464106f10 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33268/0 (socket says 172.21.15.16:33268) 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 -- 172.21.15.16:0/1182701905 learned_addr learned my addr 172.21.15.16:0/1182701905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:01.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f64637fe640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f645a7fc640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 -- 172.21.15.16:0/1182701905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464106f10 msgr2=0x7f6464074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 -- 172.21.15.16:0/1182701905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464102790 msgr2=0x7f6464074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.399+0000 7f6462ffd640 1 -- 172.21.15.16:0/1182701905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64641a67d0 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f645400c920 tx=0x7f645400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f6460ff9640 1 -- 172.21.15.16:0/1182701905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454007c70 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f6460ff9640 1 -- 172.21.15.16:0/1182701905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454007e10 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641a6ac0 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f6460ff9640 1 -- 172.21.15.16:0/1182701905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454018890 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.400+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a7050 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.402+0000 7f6460ff9640 1 -- 172.21.15.16:0/1182701905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6454018a70 con 0x7f6464101de0 2024-09-12T01:09:01.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.402+0000 7f646b726740 1 Processor -- start 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.402+0000 7f646b726740 1 -- start start 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620298292e0 con 0x5620298c3dc0 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620297d0ce0 con 0x5620298ce680 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f646b726740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5620298252a0 con 0x5620298c34a0 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33276/0 (socket says 172.21.15.16:33276) 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 -- 172.21.15.16:0/2211252224 learned_addr learned my addr 172.21.15.16:0/2211252224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f645a7fc640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f64637fe640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6460ff9640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 0x7f64500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 -- 172.21.15.16:0/2211252224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 msgr2=0x5620298c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 -- 172.21.15.16:0/2211252224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 msgr2=0x5620298d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f6462ffd640 1 -- 172.21.15.16:0/2211252224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620297b5440 con 0x5620298c3dc0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f645a7fc640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.403+0000 7f64637fe640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 0x7f64500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f6460ff9640 1 -- 172.21.15.16:0/1182701905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f64540cf5b0 con 0x7f6464101de0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f64540ccc70 tx=0x7f64540c6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a477507b18aeea72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f64597fa640 1 -- 172.21.15.16:0/2211252224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645409b610 con 0x5620298c3dc0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f64597fa640 1 -- 172.21.15.16:0/2211252224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645409c1d0 con 0x5620298c3dc0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f64597fa640 1 -- 172.21.15.16:0/2211252224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540a40a0 con 0x5620298c3dc0 2024-09-12T01:09:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f646b726740 1 -- 172.21.15.16:0/2211252224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 msgr2=0x5620298ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f64637fe640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 0x7f64500a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f645c007dd0 tx=0x7f645c004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.404+0000 7f646b726740 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f64540ccc70 tx=0x7f64540c6660 comp rx=0 tx=0).stop 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- 172.21.15.16:0/2211252224 shutdown_connections 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298ce140 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- 172.21.15.16:0/2211252224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- 172.21.15.16:0/2211252224 >> 172.21.15.16:0/2211252224 conn(0x5620298bfa30 msgr2=0x5620298bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- 172.21.15.16:0/2211252224 shutdown_connections 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- 172.21.15.16:0/2211252224 wait complete. 2024-09-12T01:09:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- start start 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 Processor -- start 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 0x5620298cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5620298d1250 con 0x5620298ce680 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5620297e4050 con 0x5620298c3dc0 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.405+0000 7f646b726740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5620298d13d0 con 0x5620298c34a0 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f646b726740 10 client.0 _ll_register_callbacks cb 0x562029824160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33280/0 (socket says 172.21.15.16:33280) 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 -- 172.21.15.16:0/1795133024 learned_addr learned my addr 172.21.15.16:0/1795133024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f645a7fc640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 0x5620298cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 -- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 msgr2=0x5620298cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 -- 172.21.15.16:0/1795133024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 msgr2=0x5620298cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 0x5620298cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 -- 172.21.15.16:0/1795133024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202959a3c0 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f645a7fc640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 0x5620298cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f64637fe640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f645c005c00 tx=0x7f645c007470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dfbe7a8f271042 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f644a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c01dd00 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f645c01b450 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.406+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f645409c1d0 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f645c01b650 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 10 client.19521 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5620298dcc30 con 0x5620298ce680 2024-09-12T01:09:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645409c1d0 con 0x5620298ce680 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f645c024070 con 0x5620298ce680 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 10 client.19521 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 20 client.19521 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620298de2a0 0x5620298e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 4 client.19521 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562029836850 con 0x5620298de2a0 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.407+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620298de2a0 0x5620298e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.408+0000 7f6462ffd640 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620298de2a0 0x5620298e06b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bff18784146c24 server_cookie=39a7dfe9121bb2ca in_seq=0 out_seq=0 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.408+0000 7f644a7fc640 10 client.19521 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.408+0000 7f644a7fc640 1 -- 172.21.15.16:0/1795133024 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562029836850 con 0x5620298de2a0 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.408+0000 7f644a7fc640 10 client.19521 handle_command_reply: tid=0 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.408+0000 7f646b726740 1 client.19521 shutdown 2024-09-12T01:09:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 msgr2=0x5620298cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f645c005c00 tx=0x7f645c007470 comp rx=0 tx=0).stop 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 shutdown_connections 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620298de2a0 0x5620298e06b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298ce680 0x5620298cb730 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c3dc0 0x5620298cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f644a7fc640 0 client.19521 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 --2- 172.21.15.16:0/1795133024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 >> 172.21.15.16:0/1795133024 conn(0x5620298bfa30 msgr2=0x5620298c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 shutdown_connections 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 1 -- 172.21.15.16:0/1795133024 wait complete. 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.409+0000 7f646b726740 20 client.19521 trim_cache size 0 max 16384 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.426+0000 7f646b726740 1 Processor -- start 2024-09-12T01:09:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.426+0000 7f646b726740 1 -- start start 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c34a0 0x5620298da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 0x5620298e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64641af990 con 0x5620298c3dc0 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64641b14d0 con 0x5620298c34a0 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f646b726740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64641b0060 con 0x5620298ce680 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33284/0 (socket says 172.21.15.16:33284) 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 -- 172.21.15.16:0/3111343861 learned_addr learned my addr 172.21.15.16:0/3111343861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f64637fe640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 0x5620298e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f6462ffd640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c34a0 0x5620298da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 -- 172.21.15.16:0/3111343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 msgr2=0x5620298e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 0x5620298e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 -- 172.21.15.16:0/3111343861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c34a0 msgr2=0x5620298da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c34a0 0x5620298da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 -- 172.21.15.16:0/3111343861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620298d0db0 con 0x5620298c3dc0 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f64637fe640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 0x5620298e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f645a7fc640 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f644c031550 tx=0x7f644c032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b72bf4f0a97f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f6458ff9640 1 -- 172.21.15.16:0/3111343861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c035070 con 0x5620298c3dc0 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f6458ff9640 1 -- 172.21.15.16:0/3111343861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c0317b0 con 0x5620298c3dc0 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.427+0000 7f6458ff9640 1 -- 172.21.15.16:0/3111343861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0046c0 con 0x5620298c3dc0 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 -- 172.21.15.16:0/3111343861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 msgr2=0x5620298e5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f644c031550 tx=0x7f644c032d80 comp rx=0 tx=0).stop 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 -- 172.21.15.16:0/3111343861 shutdown_connections 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298ce680 0x5620298e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298e5f80 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 --2- 172.21.15.16:0/3111343861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298c34a0 0x5620298da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 -- 172.21.15.16:0/3111343861 >> 172.21.15.16:0/3111343861 conn(0x5620298bfa30 msgr2=0x5620298c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 -- 172.21.15.16:0/3111343861 shutdown_connections 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.428+0000 7f646b726740 1 -- 172.21.15.16:0/3111343861 wait complete. 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 -- start start 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 Processor -- start 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64641b0e00 con 0x5620298c3dc0 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64641b0730 con 0x5620298ce680 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64641ad990 con 0x5620298c34a0 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f646b726740 10 client.0 _ll_register_callbacks cb 0x7f645c007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33296/0 (socket says 172.21.15.16:33296) 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 -- 172.21.15.16:0/2087287902 learned_addr learned my addr 172.21.15.16:0/2087287902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f64637fe640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 -- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 msgr2=0x5620298d6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298d6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 -- 172.21.15.16:0/2087287902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 msgr2=0x5620298d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.429+0000 7f645a7fc640 1 -- 172.21.15.16:0/2087287902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620298d2a70 con 0x5620298c3dc0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f645a7fc640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f644c032fd0 tx=0x7f644c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2696116c724c39be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c035070 con 0x5620298c3dc0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8521+0+0 (secure 0 0 0) 0x7f644c004440 con 0x5620298c3dc0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644c0317b0 con 0x5620298c3dc0 2024-09-12T01:09:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f644c0035f0 con 0x5620298c3dc0 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 10 client.19527 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "7f07a25f-2b28-49b5-a00f-5b678ff96e3b": { 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "7f07a25f-2b28-49b5-a00f-5b678ff96e3b", 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:09:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562029948d70 con 0x5620298c3dc0 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644c0317b0 con 0x5620298c3dc0 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f644c04c020 con 0x5620298c3dc0 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 10 client.19527 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 10 client.19527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 20 client.19527 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620299494c0 0x56202994b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 4 client.19527 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.430+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562029830d60 con 0x5620299494c0 2024-09-12T01:09:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620299494c0 0x56202994b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f6462ffd640 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620299494c0 0x56202994b8e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379fe8ca6859a664 server_cookie=a594a921ef8800dd in_seq=0 out_seq=0 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f644a7fc640 10 client.19527 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f644a7fc640 1 -- 172.21.15.16:0/2087287902 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562029830d60 con 0x5620299494c0 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f644a7fc640 10 client.19527 handle_command_reply: tid=0 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f646b726740 1 client.19527 shutdown 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 msgr2=0x5620298d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.431+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f644c032fd0 tx=0x7f644c031730 comp rx=0 tx=0).stop 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 shutdown_connections 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5620299494c0 0x56202994b8e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5620298ce680 0x5620298d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5620298c3dc0 0x5620298d1ef0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f644a7fc640 0 client.19527 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 --2- 172.21.15.16:0/2087287902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620298c34a0 0x5620298d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 >> 172.21.15.16:0/2087287902 conn(0x5620298bfa30 msgr2=0x5620298c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 shutdown_connections 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 1 -- 172.21.15.16:0/2087287902 wait complete. 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646b726740 20 client.19527 trim_cache size 0 max 16384 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 msgr2=0x7f64500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 0x7f64500a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f645c007dd0 tx=0x7f645c004a40 comp rx=0 tx=0).stop 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 msgr2=0x7f64640744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f645400c920 tx=0x7f645400cde0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 shutdown_connections 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f64500a2d70 0x7f64500a5230 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6464106f10 0x7f6464074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6464102790 0x7f6464074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 --2- 172.21.15.16:0/1182701905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6464101de0 0x7f64640744f0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 >> 172.21.15.16:0/1182701905 conn(0x7f64640fd8c0 msgr2=0x7f64640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 shutdown_connections 2024-09-12T01:09:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:01.432+0000 7f646934d640 1 -- 172.21.15.16:0/1182701905 wait complete. 2024-09-12T01:09:02.328 INFO:tasks.cephfs.filesystem:scrub status for tag:7f07a25f-2b28-49b5-a00f-5b678ff96e3b - {'path': '/', 'tag': '7f07a25f-2b28-49b5-a00f-5b678ff96e3b', 'options': 'recursive,force'} 2024-09-12T01:09:03.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:02 smithi099 ceph-mon[35654]: pgmap v402: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:02 smithi159 ceph-mon[35566]: pgmap v402: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:02 smithi016 ceph-mon[30972]: pgmap v402: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:05.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:04 smithi099 ceph-mon[35654]: pgmap v403: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:05.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:04 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:09:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:04 smithi159 ceph-mon[35566]: pgmap v403: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:04 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:09:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:04 smithi016 ceph-mon[30972]: pgmap v403: 97 pgs: 97 active+clean; 20 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 12 op/s 2024-09-12T01:09:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:04 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:09:06.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:05 smithi099 ceph-mon[35654]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:09:06.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:05 smithi159 ceph-mon[35566]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:09:06.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:05 smithi016 ceph-mon[30972]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T01:09:07.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:06 smithi099 ceph-mon[35654]: pgmap v405: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:06 smithi159 ceph-mon[35566]: pgmap v405: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:06 smithi016 ceph-mon[30972]: pgmap v405: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:09.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:08 smithi099 ceph-mon[35654]: pgmap v406: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:09.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:08 smithi159 ceph-mon[35566]: pgmap v406: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:08 smithi016 ceph-mon[30972]: pgmap v406: 97 pgs: 97 active+clean; 20 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 111 KiB/s wr, 35 op/s 2024-09-12T01:09:11.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:10 smithi099 ceph-mon[35654]: pgmap v407: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:10 smithi159 ceph-mon[35566]: pgmap v407: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:11.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:10 smithi016 ceph-mon[30972]: pgmap v407: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:12 smithi099 ceph-mon[35654]: pgmap v408: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:13.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:12 smithi099 ceph-mon[35654]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:09:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:12 smithi159 ceph-mon[35566]: pgmap v408: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:12 smithi159 ceph-mon[35566]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:09:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:12 smithi016 ceph-mon[30972]: pgmap v408: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-12T01:09:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:12 smithi016 ceph-mon[30972]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T01:09:15.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:14 smithi099 ceph-mon[35654]: pgmap v410: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-09-12T01:09:15.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:14 smithi159 ceph-mon[35566]: pgmap v410: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-09-12T01:09:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:14 smithi016 ceph-mon[30972]: pgmap v410: 97 pgs: 97 active+clean; 20 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 9 op/s 2024-09-12T01:09:17.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:16 smithi099 ceph-mon[35654]: pgmap v411: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:16 smithi159 ceph-mon[35566]: pgmap v411: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:17.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:16 smithi016 ceph-mon[30972]: pgmap v411: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:19.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:18 smithi016 ceph-mon[30972]: pgmap v412: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:19.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:18 smithi099 ceph-mon[35654]: pgmap v412: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:19.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:18 smithi159 ceph-mon[35566]: pgmap v412: 97 pgs: 97 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:09:21.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:20 smithi099 ceph-mon[35654]: pgmap v413: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:21.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:20 smithi159 ceph-mon[35566]: pgmap v413: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:21.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:20 smithi016 ceph-mon[30972]: pgmap v413: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:23.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:22 smithi099 ceph-mon[35654]: pgmap v414: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:23.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:22 smithi159 ceph-mon[35566]: pgmap v414: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:23.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:22 smithi016 ceph-mon[30972]: pgmap v414: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-09-12T01:09:24.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:23 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:09:24.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:23 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:09:24.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:23 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:09:25.267 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:24 smithi159 ceph-mon[35566]: pgmap v415: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-12T01:09:25.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:09:25.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:09:25.268 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:09:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:24 smithi099 ceph-mon[35654]: pgmap v415: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-12T01:09:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:09:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:09:25.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:09:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:24 smithi016 ceph-mon[30972]: pgmap v415: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-09-12T01:09:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:09:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:09:25.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:09:27.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:26 smithi099 ceph-mon[35654]: pgmap v416: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:27.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:26 smithi159 ceph-mon[35566]: pgmap v416: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:27.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:26 smithi016 ceph-mon[30972]: pgmap v416: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:29.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:28 smithi099 ceph-mon[35654]: pgmap v417: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:29.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:28 smithi159 ceph-mon[35566]: pgmap v417: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:29.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:28 smithi016 ceph-mon[30972]: pgmap v417: 97 pgs: 97 active+clean; 21 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:31.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:30 smithi099 ceph-mon[35654]: pgmap v418: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:31.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:30 smithi159 ceph-mon[35566]: pgmap v418: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:31.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:30 smithi016 ceph-mon[30972]: pgmap v418: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:32.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:09:32.604 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:33.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:32 smithi016 ceph-mon[30972]: pgmap v419: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 -- 172.21.15.16:0/2483168803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 msgr2=0x7f9e781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 --2- 172.21.15.16:0/2483168803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e781021c0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009900 tx=0x7f9e6802a410 comp rx=0 tx=0).stop 2024-09-12T01:09:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 -- 172.21.15.16:0/2483168803 shutdown_connections 2024-09-12T01:09:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 --2- 172.21.15.16:0/2483168803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78106f10 0x7f9e7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 --2- 172.21.15.16:0/2483168803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78102790 0x7f9e781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 --2- 172.21.15.16:0/2483168803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e781021c0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 -- 172.21.15.16:0/2483168803 >> 172.21.15.16:0/2483168803 conn(0x7f9e780fd8c0 msgr2=0x7f9e780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 -- 172.21.15.16:0/2483168803 shutdown_connections 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.117+0000 7f9e7d481640 1 -- 172.21.15.16:0/2483168803 wait complete. 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 Processor -- start 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 -- start start 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78102790 0x7f9e78198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78106f10 0x7f9e7819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78111f30 con 0x7f9e78101de0 2024-09-12T01:09:33.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e78111db0 con 0x7f9e78102790 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e7d481640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781120b0 con 0x7f9e78106f10 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59284/0 (socket says 172.21.15.16:59284) 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.118+0000 7f9e76ffd640 1 -- 172.21.15.16:0/293901585 learned_addr learned my addr 172.21.15.16:0/293901585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e767fc640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78102790 0x7f9e78198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e777fe640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78106f10 0x7f9e7819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 -- 172.21.15.16:0/293901585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78106f10 msgr2=0x7f9e7819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78106f10 0x7f9e7819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 -- 172.21.15.16:0/293901585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78102790 msgr2=0x7f9e78198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78102790 0x7f9e78198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 -- 172.21.15.16:0/293901585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7819d4c0 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e5ffff640 1 -- 172.21.15.16:0/293901585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6803f030 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e5ffff640 1 -- 172.21.15.16:0/293901585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e68003ce0 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7819d750 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e5ffff640 1 -- 172.21.15.16:0/293901585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68035070 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.119+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e781a5090 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.121+0000 7f9e5ffff640 1 -- 172.21.15.16:0/293901585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9e68042030 con 0x7f9e78101de0 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 Processor -- start 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 -- start start 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f64a0 0x564c465f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c46601140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c4655c2e0 con 0x564c46601680 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c46503ce0 con 0x564c465f6dc0 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c465582a0 con 0x564c465f64a0 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e5ffff640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 0x7f9e600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59294/0 (socket says 172.21.15.16:59294) 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 -- 172.21.15.16:0/3944128486 learned_addr learned my addr 172.21.15.16:0/3944128486 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e777fe640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f64a0 0x564c465f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c46601140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e5ffff640 1 -- 172.21.15.16:0/293901585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9e680ef980 con 0x7f9e78101de0 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e777fe640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 0x7f9e600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 -- 172.21.15.16:0/3944128486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f64a0 msgr2=0x564c465f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f64a0 0x564c465f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 -- 172.21.15.16:0/3944128486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 msgr2=0x564c46601140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.122+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c46601140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e767fc640 1 -- 172.21.15.16:0/3944128486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c464e8440 con 0x564c46601680 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c46601140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bcb0 tx=0x7f9e6c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1e17b430eb9b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e5d7fa640 1 -- 172.21.15.16:0/3944128486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c004030 con 0x564c46601680 2024-09-12T01:09:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e777fe640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 0x7f9e600a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005fd0 tx=0x7f9e70009500 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e5d7fa640 1 -- 172.21.15.16:0/3944128486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0041d0 con 0x564c46601680 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.123+0000 7f9e5d7fa640 1 -- 172.21.15.16:0/3944128486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c011800 con 0x564c46601680 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3944128486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 msgr2=0x564c46603a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00bcb0 tx=0x7f9e6c0107a0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3944128486 shutdown_connections 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c46603a70 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c46601140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3944128486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f64a0 0x564c465f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3944128486 >> 172.21.15.16:0/3944128486 conn(0x564c465f2a30 msgr2=0x564c465f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3944128486 shutdown_connections 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3944128486 wait complete. 2024-09-12T01:09:33.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- start start 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 Processor -- start 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c465fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c465fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c46608660 con 0x564c465f64a0 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c46517050 con 0x564c465f6dc0 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.124+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c466087e0 con 0x564c46601680 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e7f85a740 10 client.0 _ll_register_callbacks cb 0x564c46557160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59298/0 (socket says 172.21.15.16:59298) 2024-09-12T01:09:33.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 -- 172.21.15.16:0/2428761184 learned_addr learned my addr 172.21.15.16:0/2428761184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e777fe640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c465fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 -- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 msgr2=0x564c465fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e767fc640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c465fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c465fe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 -- 172.21.15.16:0/2428761184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 msgr2=0x564c465fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c465fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 -- 172.21.15.16:0/2428761184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c462cd3c0 con 0x564c465f64a0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e777fe640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c465fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9e680bbd10 tx=0x7f9e68031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482b30267620ef41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e680b6060 con 0x564c465f64a0 2024-09-12T01:09:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9e680bc270 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c0041d0 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e68049030 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e7f85a740 10 client.19539 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c46614050 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.125+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0041d0 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e680bf070 con 0x564c465f64a0 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 10 client.19539 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 10 client.19539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 20 client.19539 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c466154b0 0x564c46617920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 4 client.19539 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c46569850 con 0x564c466154b0 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.126+0000 7f9e767fc640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c466154b0 0x564c46617920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e5aff5640 10 client.19539 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e767fc640 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c466154b0 0x564c46617920 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33057b328da37aa server_cookie=e3f0022eb48228e2 in_seq=0 out_seq=0 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2428761184 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c46569850 con 0x564c466154b0 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e5aff5640 10 client.19539 handle_command_reply: tid=0 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e7f85a740 1 client.19539 shutdown 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 msgr2=0x564c46600fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.127+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9e680bbd10 tx=0x7f9e68031a60 comp rx=0 tx=0).stop 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 shutdown_connections 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c466154b0 0x564c46617920 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c465fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f6dc0 0x564c465fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2428761184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f64a0 0x564c46600fc0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 >> 172.21.15.16:0/2428761184 conn(0x564c465f2a30 msgr2=0x564c465f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e5aff5640 0 client.19539 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 shutdown_connections 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2428761184 wait complete. 2024-09-12T01:09:33.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.128+0000 7f9e7f85a740 20 client.19539 trim_cache size 0 max 16384 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 Processor -- start 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 -- start start 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c4661d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781ab5e0 con 0x564c465f6dc0 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781ad120 con 0x564c465f64a0 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781abcb0 con 0x564c46601680 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59310/0 (socket says 172.21.15.16:59310) 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.145+0000 7f9e767fc640 1 -- 172.21.15.16:0/3563645696 learned_addr learned my addr 172.21.15.16:0/3563645696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e777fe640 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c4661d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 -- 172.21.15.16:0/3563645696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 msgr2=0x564c4661d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c4661d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 -- 172.21.15.16:0/3563645696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 msgr2=0x564c466118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 -- 172.21.15.16:0/3563645696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c465fd270 con 0x564c465f6dc0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e767fc640 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c011010 tx=0x7f9e6c00bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196236919850963f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e5cff9640 1 -- 172.21.15.16:0/3563645696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0035f0 con 0x564c465f6dc0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e5cff9640 1 -- 172.21.15.16:0/3563645696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0050c0 con 0x564c465f6dc0 2024-09-12T01:09:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.146+0000 7f9e5cff9640 1 -- 172.21.15.16:0/3563645696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0276e0 con 0x564c465f6dc0 2024-09-12T01:09:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3563645696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 msgr2=0x564c465fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c011010 tx=0x7f9e6c00bbf0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3563645696 shutdown_connections 2024-09-12T01:09:33.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c46601680 0x564c4661d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c465f6dc0 0x564c465fcb90 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/3563645696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3563645696 >> 172.21.15.16:0/3563645696 conn(0x564c465f2a30 msgr2=0x564c465f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3563645696 shutdown_connections 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- 172.21.15.16:0/3563645696 wait complete. 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 -- start start 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.147+0000 7f9e7f85a740 1 Processor -- start 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 0x564c46609ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781aca50 con 0x564c46601680 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781ac380 con 0x564c465f64a0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e781a95c0 con 0x564c465f6dc0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e7f85a740 10 client.0 _ll_register_callbacks cb 0x7f9e68003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59324/0 (socket says 172.21.15.16:59324) 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e76ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 -- 172.21.15.16:0/2631393928 learned_addr learned my addr 172.21.15.16:0/2631393928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e767fc640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 0x564c46609ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 -- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 msgr2=0x564c46609ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 0x564c46609ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 -- 172.21.15.16:0/2631393928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 msgr2=0x564c466099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:33.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 -- 172.21.15.16:0/2631393928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4667f3d0 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e767fc640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 0x564c46609ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e777fe640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9e70010de0 tx=0x7f9e70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbaa23cecfbf515 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.148+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7002d060 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f9e70028410 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c0050c0 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e70028610 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 10 client.19545 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c46680720 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c0050c0 con 0x564c46601680 2024-09-12T01:09:33.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9e7001d480 con 0x564c46601680 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 10 client.19545 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 10 client.19545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 20 client.19545 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c46680ea0 0x564c466832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 4 client.19545 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c46563d60 con 0x564c46680ea0 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.149+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c46680ea0 0x564c466832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e76ffd640 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c46680ea0 0x564c466832e0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87806c88e2abd123 server_cookie=f753e4a4ec445687 in_seq=0 out_seq=0 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e5aff5640 10 client.19545 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e5aff5640 1 -- 172.21.15.16:0/2631393928 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564c46563d60 con 0x564c46680ea0 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e5aff5640 10 client.19545 handle_command_reply: tid=0 2024-09-12T01:09:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 client.19545 shutdown 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 msgr2=0x564c4660a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9e70010de0 tx=0x7f9e70004a40 comp rx=0 tx=0).stop 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 shutdown_connections 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564c46680ea0 0x564c466832e0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c46601680 0x564c4660a420 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e5aff5640 0 client.19545 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c465f6dc0 0x564c46609ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 --2- 172.21.15.16:0/2631393928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c465f64a0 0x564c466099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.150+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 >> 172.21.15.16:0/2631393928 conn(0x564c465f2a30 msgr2=0x564c465f97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 shutdown_connections 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7f85a740 1 -- 172.21.15.16:0/2631393928 wait complete. 2024-09-12T01:09:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7f85a740 20 client.19545 trim_cache size 0 max 16384 2024-09-12T01:09:33.157 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:09:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:09:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:09:33.157 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 msgr2=0x7f9e600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 0x7f9e600a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e70005fd0 tx=0x7f9e70009500 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 msgr2=0x7f9e78198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e680042f0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 shutdown_connections 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9e600a2cf0 0x7f9e600a51b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e78106f10 0x7f9e7819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e78102790 0x7f9e78198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 --2- 172.21.15.16:0/293901585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e78101de0 0x7f9e78198510 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 >> 172.21.15.16:0/293901585 conn(0x7f9e780fd8c0 msgr2=0x7f9e780fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 shutdown_connections 2024-09-12T01:09:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:33.151+0000 7f9e7d481640 1 -- 172.21.15.16:0/293901585 wait complete. 2024-09-12T01:09:33.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:32 smithi099 ceph-mon[35654]: pgmap v419: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:33.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:32 smithi159 ceph-mon[35566]: pgmap v419: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:33.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:09:33.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:09:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.766+0000 7f18084c7640 1 -- 172.21.15.16:0/2608506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074510 msgr2=0x7f18000748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.766+0000 7f18084c7640 1 --2- 172.21.15.16:0/2608506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074510 0x7f18000748f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009900 tx=0x7f17fc02a480 comp rx=0 tx=0).stop 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- 172.21.15.16:0/2608506476 shutdown_connections 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- 172.21.15.16:0/2608506476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f180010e440 0x7f1800110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- 172.21.15.16:0/2608506476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1800074e30 0x7f180010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- 172.21.15.16:0/2608506476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074510 0x7f18000748f0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- 172.21.15.16:0/2608506476 >> 172.21.15.16:0/2608506476 conn(0x7f18000fd8c0 msgr2=0x7f18000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- 172.21.15.16:0/2608506476 shutdown_connections 2024-09-12T01:09:34.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- 172.21.15.16:0/2608506476 wait complete. 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 Processor -- start 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- start start 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1800074510 0x7f180019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f180010e440 0x7f180019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18001141e0 con 0x7f1800074e30 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1800114060 con 0x7f180010e440 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.767+0000 7f18084c7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1800114360 con 0x7f1800074510 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59344/0 (socket says 172.21.15.16:59344) 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 -- 172.21.15.16:0/2054225862 learned_addr learned my addr 172.21.15.16:0/2054225862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1806263640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1800074510 0x7f180019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1806a64640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f180010e440 0x7f180019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 -- 172.21.15.16:0/2054225862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1800074510 msgr2=0x7f180019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1800074510 0x7f180019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 -- 172.21.15.16:0/2054225862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f180010e440 msgr2=0x7f180019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f180010e440 0x7f180019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 -- 172.21.15.16:0/2054225862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180019f6b0 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f1805a62640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b4a0 tx=0x7f17f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0014070 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0010af0 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180019f9a0 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0017d10 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.768+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a7280 con 0x7f1800074e30 2024-09-12T01:09:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.769+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18000748f0 con 0x7f1800074e30 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.771+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f17f0017070 con 0x7f1800074e30 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.771+0000 7f17f77fe640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.771+0000 7f1806263640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.771+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f17f00cea50 con 0x7f1800074e30 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.772+0000 7f1806263640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 0x7f17ec0a5230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009230 tx=0x7f17fc0058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.774+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17f009b780 con 0x7f1800074e30 2024-09-12T01:09:34.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.986+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f180019ba60 con 0x7f1800074e30 2024-09-12T01:09:34.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.987+0000 7f17f77fe640 1 -- 172.21.15.16:0/2054225862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f180019ba60 con 0x7f1800074e30 2024-09-12T01:09:34.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:09:34.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:09:34.987 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 msgr2=0x7f17ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 0x7f17ec0a5230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009230 tx=0x7f17fc0058d0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 msgr2=0x7f180019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b4a0 tx=0x7f17f000b960 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 shutdown_connections 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f17ec0a2d70 0x7f17ec0a5230 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f180010e440 0x7f180019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800074e30 0x7f180019ac40 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 --2- 172.21.15.16:0/2054225862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1800074510 0x7f180019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 >> 172.21.15.16:0/2054225862 conn(0x7f18000fd8c0 msgr2=0x7f18000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:34.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 shutdown_connections 2024-09-12T01:09:34.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:34.989+0000 7f18084c7640 1 -- 172.21.15.16:0/2054225862 wait complete. 2024-09-12T01:09:35.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:34 smithi099 ceph-mon[35654]: pgmap v420: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:35.362 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:34 smithi016 ceph-mon[30972]: pgmap v420: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:35.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:34 smithi159 ceph-mon[35566]: pgmap v420: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:35.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:09:36.085 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:36.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2054225862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:36.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2054225862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:36.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2054225862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 -- 172.21.15.16:0/495934185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94106f10 msgr2=0x7f6a9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/495934185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94106f10 0x7f6a9410ea90 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009ca0 tx=0x7f6a8c02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 -- 172.21.15.16:0/495934185 shutdown_connections 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/495934185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94106f10 0x7f6a9410ea90 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/495934185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/495934185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94101de0 0x7f6a941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 -- 172.21.15.16:0/495934185 >> 172.21.15.16:0/495934185 conn(0x7f6a940fd8c0 msgr2=0x7f6a940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 -- 172.21.15.16:0/495934185 shutdown_connections 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.601+0000 7f6a9bde1640 1 -- 172.21.15.16:0/495934185 wait complete. 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 Processor -- start 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 -- start start 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 0x7f6a9419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111f90 con 0x7f6a94101de0 2024-09-12T01:09:36.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111e10 con 0x7f6a94102790 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a9bde1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94112110 con 0x7f6a94106f10 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.602+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59362/0 (socket says 172.21.15.16:59362) 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 -- 172.21.15.16:0/2465178983 learned_addr learned my addr 172.21.15.16:0/2465178983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9937c640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 0x7f6a9419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 -- 172.21.15.16:0/2465178983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 msgr2=0x7f6a9419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 0x7f6a9419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 -- 172.21.15.16:0/2465178983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 msgr2=0x7f6a941989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 -- 172.21.15.16:0/2465178983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9419d420 con 0x7f6a94101de0 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9937c640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:36.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 0x7f6a9419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d950 tx=0x7f6a9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a8affd640 1 -- 172.21.15.16:0/2465178983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017030 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9419d710 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a8affd640 1 -- 172.21.15.16:0/2465178983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a90004020 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a8affd640 1 -- 172.21.15.16:0/2465178983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a900047a0 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.603+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a4ff0 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.605+0000 7f6a8affd640 1 -- 172.21.15.16:0/2465178983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6a900041d0 con 0x7f6a94101de0 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 Processor -- start 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 -- start start 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b15d0 0x557cb76b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a8affd640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 0x7f6a800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557cb76172f0 con 0x557cb76b2b40 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557cb75becf0 con 0x557cb76b15d0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557cb76132b0 con 0x557cb76bc370 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.606+0000 7f6a9a37e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59376/0 (socket says 172.21.15.16:59376) 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 -- 172.21.15.16:0/3313180114 learned_addr learned my addr 172.21.15.16:0/3313180114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 0x7f6a800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a8affd640 1 -- 172.21.15.16:0/2465178983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f6a900cef50 con 0x7f6a94101de0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b15d0 0x557cb76b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9937c640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 -- 172.21.15.16:0/3313180114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 msgr2=0x557cb76be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 -- 172.21.15.16:0/3313180114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b15d0 msgr2=0x557cb76b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b15d0 0x557cb76b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 -- 172.21.15.16:0/3313180114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb75a33a0 con 0x557cb76b2b40 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9937c640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006010 tx=0x7f6a8c031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e26f8a6a892611 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 0x7f6a800a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009bf0 tx=0x7f6a8c047000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a7ffff640 1 -- 172.21.15.16:0/3313180114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c0305c0 con 0x557cb76b2b40 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a7ffff640 1 -- 172.21.15.16:0/3313180114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c036040 con 0x557cb76b2b40 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.607+0000 7f6a7ffff640 1 -- 172.21.15.16:0/3313180114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c049690 con 0x557cb76b2b40 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/3313180114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 msgr2=0x557cb76b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c006010 tx=0x7f6a8c031870 comp rx=0 tx=0).stop 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/3313180114 shutdown_connections 2024-09-12T01:09:36.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b15d0 0x557cb76b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/3313180114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b2b40 0x557cb76b1090 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/3313180114 >> 172.21.15.16:0/3313180114 conn(0x557cb767a310 msgr2=0x557cb76adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/3313180114 shutdown_connections 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.608+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/3313180114 wait complete. 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 -- start start 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 Processor -- start 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557cb76bf010 con 0x557cb76b15d0 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557cb75d2060 con 0x557cb76b2b40 2024-09-12T01:09:36.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557cb76bf190 con 0x557cb76bc370 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9e1ba740 10 client.0 _ll_register_callbacks cb 0x557cb73258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59390/0 (socket says 172.21.15.16:59390) 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a99b7d640 1 -- 172.21.15.16:0/467052740 learned_addr learned my addr 172.21.15.16:0/467052740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.609+0000 7f6a9937c640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 -- 172.21.15.16:0/467052740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 msgr2=0x557cb76b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 -- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 msgr2=0x557cb76b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 -- 172.21.15.16:0/467052740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb73883c0 con 0x557cb76b15d0 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6a900045f0 tx=0x7f6a9009ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1deccbfad5a625 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90014070 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f6a900cc8c0 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8c036040 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a900b9050 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a9e1ba740 10 client.19563 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb76caac0 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.610+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c036040 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a900b1630 con 0x557cb76b15d0 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 10 client.19563 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 10 client.19563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 20 client.19563 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:36.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb76cbf20 0x557cb76ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 4 client.19563 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cb7624860 con 0x557cb76cbf20 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9937c640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb76cbf20 0x557cb76ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a7d7fa640 10 client.19563 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.611+0000 7f6a9937c640 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb76cbf20 0x557cb76ce330 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9255a8b022173485 server_cookie=bd29dc0899b504d4 in_seq=0 out_seq=0 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/467052740 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cb7624860 con 0x557cb76cbf20 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a7d7fa640 10 client.19563 handle_command_reply: tid=0 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 client.19563 shutdown 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 msgr2=0x557cb76bbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f6a900045f0 tx=0x7f6a9009ade0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 shutdown_connections 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb76cbf20 0x557cb76ce330 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a7d7fa640 0 client.19563 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/467052740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76bbda0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.612+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 >> 172.21.15.16:0/467052740 conn(0x557cb767a310 msgr2=0x557cb76b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.613+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 shutdown_connections 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.613+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/467052740 wait complete. 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.613+0000 7f6a9e1ba740 20 client.19563 trim_cache size 0 max 16384 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 Processor -- start 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 -- start start 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ad910 con 0x557cb76b15d0 2024-09-12T01:09:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941af450 con 0x557cb76b2b40 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941adfe0 con 0x557cb76bc370 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59400/0 (socket says 172.21.15.16:59400) 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 -- 172.21.15.16:0/1098005325 learned_addr learned my addr 172.21.15.16:0/1098005325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 -- 172.21.15.16:0/1098005325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 msgr2=0x557cb76d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 -- 172.21.15.16:0/1098005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 msgr2=0x557cb76d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9937c640 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 -- 172.21.15.16:0/1098005325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cb76b1be0 con 0x557cb76b15d0 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a9937c640 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.626+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f6a900a3b50 tx=0x7f6a9009b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b152ceee21d65de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a7f7fe640 1 -- 172.21.15.16:0/1098005325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90064020 con 0x557cb76b15d0 2024-09-12T01:09:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a7f7fe640 1 -- 172.21.15.16:0/1098005325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9009bb10 con 0x557cb76b15d0 2024-09-12T01:09:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a7f7fe640 1 -- 172.21.15.16:0/1098005325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90014030 con 0x557cb76b15d0 2024-09-12T01:09:36.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1098005325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 msgr2=0x557cb76b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f6a900a3b50 tx=0x7f6a9009b590 comp rx=0 tx=0).stop 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1098005325 shutdown_connections 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76bc370 0x557cb76d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1098005325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76b15d0 0x557cb76b8910 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1098005325 >> 172.21.15.16:0/1098005325 conn(0x557cb767a310 msgr2=0x557cb76b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1098005325 shutdown_connections 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.627+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1098005325 wait complete. 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 -- start start 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 Processor -- start 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76b15d0 0x557cb76c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941aed80 con 0x557cb76bc370 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ae6b0 con 0x557cb76b2b40 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941ab910 con 0x557cb76b15d0 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9e1ba740 10 client.0 _ll_register_callbacks cb 0x7f6a9009afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9a37e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a99b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76b15d0 0x557cb76c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9a37e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59402/0 (socket says 172.21.15.16:59402) 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.628+0000 7f6a9a37e640 1 -- 172.21.15.16:0/1840987329 learned_addr learned my addr 172.21.15.16:0/1840987329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9937c640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 -- 172.21.15.16:0/1840987329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76b15d0 msgr2=0x557cb76c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76b15d0 0x557cb76c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 -- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 msgr2=0x557cb76bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 -- 172.21.15.16:0/1840987329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cb76c0990 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9a37e640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009430 tx=0x7f6a8c02ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18ca0140893e937 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004b40 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a9009bb10 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f6a8c004ce0 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a8c061050 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9e1ba740 10 client.19569 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cb7736a00 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.629+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9009bb10 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a8c048070 con 0x557cb76bc370 2024-09-12T01:09:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 10 client.19569 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 10 client.19569 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 20 client.19569 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb7736db0 0x557cb77391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 4 client.19569 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cb761ed70 con 0x557cb7736db0 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb7736db0 0x557cb77391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a99b7d640 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb7736db0 0x557cb77391a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56b5d4c81c764d3 server_cookie=f48e1951e80c14b1 in_seq=0 out_seq=0 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a7d7fa640 10 client.19569 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a7d7fa640 1 -- 172.21.15.16:0/1840987329 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cb761ed70 con 0x557cb7736db0 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a7d7fa640 10 client.19569 handle_command_reply: tid=0 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.630+0000 7f6a9e1ba740 1 client.19569 shutdown 2024-09-12T01:09:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 msgr2=0x557cb76c02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009430 tx=0x7f6a8c02ffe0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 shutdown_connections 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557cb7736db0 0x557cb77391a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557cb76bc370 0x557cb76c02a0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557cb76b2b40 0x557cb76bfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 --2- 172.21.15.16:0/1840987329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557cb76b15d0 0x557cb76c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a7d7fa640 0 client.19569 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 >> 172.21.15.16:0/1840987329 conn(0x557cb767a310 msgr2=0x557cb76b7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 shutdown_connections 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 1 -- 172.21.15.16:0/1840987329 wait complete. 2024-09-12T01:09:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9e1ba740 20 client.19569 trim_cache size 0 max 16384 2024-09-12T01:09:36.646 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:09:36.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 msgr2=0x7f6a800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 0x7f6a800a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c009bf0 tx=0x7f6a8c047000 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 msgr2=0x7f6a94198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000d950 tx=0x7f6a9000de10 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.631+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 shutdown_connections 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6a800a2cf0 0x7f6a800a51b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a94106f10 0x7f6a9419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a94102790 0x7f6a941989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 --2- 172.21.15.16:0/2465178983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a94101de0 0x7f6a94198470 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 >> 172.21.15.16:0/2465178983 conn(0x7f6a940fd8c0 msgr2=0x7f6a940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 shutdown_connections 2024-09-12T01:09:36.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:36.632+0000 7f6a9bde1640 1 -- 172.21.15.16:0/2465178983 wait complete. 2024-09-12T01:09:37.258 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:36 smithi016 ceph-mon[30972]: pgmap v421: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 20 op/s 2024-09-12T01:09:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:36 smithi099 ceph-mon[35654]: pgmap v421: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 20 op/s 2024-09-12T01:09:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:36 smithi159 ceph-mon[35566]: pgmap v421: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 86 KiB/s wr, 20 op/s 2024-09-12T01:09:37.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:09:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 -- 172.21.15.16:0/3241171094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c101400 msgr2=0x7f840c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 --2- 172.21.15.16:0/3241171094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c101400 0x7f840c10d930 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f8404009cd0 tx=0x7f840402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 -- 172.21.15.16:0/3241171094 shutdown_connections 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 --2- 172.21.15.16:0/3241171094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c101400 0x7f840c10d930 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 --2- 172.21.15.16:0/3241171094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c100a60 0x7f840c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 --2- 172.21.15.16:0/3241171094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c106a60 0x7f840c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 -- 172.21.15.16:0/3241171094 >> 172.21.15.16:0/3241171094 conn(0x7f840c0fc780 msgr2=0x7f840c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 -- 172.21.15.16:0/3241171094 shutdown_connections 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 -- 172.21.15.16:0/3241171094 wait complete. 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.280+0000 7f84146ce640 1 Processor -- start 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 -- start start 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c100a60 0x7f840c0fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 0x7f840c0fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f840c110e60 con 0x7f840c106a60 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f840c110ce0 con 0x7f840c100a60 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f84146ce640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f840c110fe0 con 0x7f840c101400 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59410/0 (socket says 172.21.15.16:59410) 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 -- 172.21.15.16:0/2609289495 learned_addr learned my addr 172.21.15.16:0/2609289495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:38.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f841246a640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c100a60 0x7f840c0fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8411c69640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 0x7f840c0fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 -- 172.21.15.16:0/2609289495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 msgr2=0x7f840c0fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 0x7f840c0fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 -- 172.21.15.16:0/2609289495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c100a60 msgr2=0x7f840c0fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c100a60 0x7f840c0fce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8412c6b640 1 -- 172.21.15.16:0/2609289495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c1a6880 con 0x7f840c106a60 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.281+0000 7f8411c69640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 0x7f840c0fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f8412c6b640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8404005cc0 tx=0x7f8404003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f84037fe640 1 -- 172.21.15.16:0/2609289495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004040 con 0x7f840c106a60 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f84037fe640 1 -- 172.21.15.16:0/2609289495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84040046c0 con 0x7f840c106a60 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f84037fe640 1 -- 172.21.15.16:0/2609289495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404046480 con 0x7f840c106a60 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c1a6b10 con 0x7f840c106a60 2024-09-12T01:09:38.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.282+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1a70a0 con 0x7f840c106a60 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.284+0000 7f84037fe640 1 -- 172.21.15.16:0/2609289495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f84040305a0 con 0x7f840c106a60 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.284+0000 7f8416aa7740 1 Processor -- start 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 -- start start 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf27522b40 0x55bf27521090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf2752e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf274872f0 con 0x55bf275215d0 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf2742ecf0 con 0x55bf27522b40 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8416aa7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf274832b0 con 0x55bf2752c370 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f84037fe640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59416/0 (socket says 172.21.15.16:59416) 2024-09-12T01:09:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 -- 172.21.15.16:0/2611595137 learned_addr learned my addr 172.21.15.16:0/2611595137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8411c69640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf27522b40 0x55bf27521090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8412c6b640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f8411c69640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf2752e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 -- 172.21.15.16:0/2611595137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 msgr2=0x55bf2752e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf2752e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f84037fe640 1 -- 172.21.15.16:0/2609289495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f840402e030 con 0x7f840c106a60 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 -- 172.21.15.16:0/2611595137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf27522b40 msgr2=0x55bf27521090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.285+0000 7f841246a640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf27522b40 0x55bf27521090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f841246a640 1 -- 172.21.15.16:0/2611595137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf274133a0 con 0x55bf275215d0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f8411c69640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf2752e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f841246a640 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f840800bc90 tx=0x7f84080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a187b41522637ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f8412c6b640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f84040bce10 tx=0x7f84040c1040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f8400ff9640 1 -- 172.21.15.16:0/2611595137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840801c480 con 0x55bf275215d0 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f8400ff9640 1 -- 172.21.15.16:0/2611595137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840801cb00 con 0x55bf275215d0 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.286+0000 7f8400ff9640 1 -- 172.21.15.16:0/2611595137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408011bb0 con 0x55bf275215d0 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- 172.21.15.16:0/2611595137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 msgr2=0x55bf27521a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f840800bc90 tx=0x7f84080107a0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- 172.21.15.16:0/2611595137 shutdown_connections 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf2752e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf275215d0 0x55bf27521a10 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 --2- 172.21.15.16:0/2611595137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf27522b40 0x55bf27521090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- 172.21.15.16:0/2611595137 >> 172.21.15.16:0/2611595137 conn(0x55bf274ea310 msgr2=0x55bf2751dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- 172.21.15.16:0/2611595137 shutdown_connections 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- 172.21.15.16:0/2611595137 wait complete. 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 -- start start 2024-09-12T01:09:38.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.287+0000 7f8416aa7740 1 Processor -- start 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf2752bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf2752f070 con 0x55bf27522b40 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf27442060 con 0x55bf275215d0 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf2752f1f0 con 0x55bf2752c370 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8416aa7740 10 client.0 _ll_register_callbacks cb 0x55bf271958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59418/0 (socket says 172.21.15.16:59418) 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 -- 172.21.15.16:0/3536110331 learned_addr learned my addr 172.21.15.16:0/3536110331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8411c69640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8411c69640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf2752bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 -- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 msgr2=0x55bf275294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 -- 172.21.15.16:0/3536110331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 msgr2=0x55bf2752bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf2752bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 -- 172.21.15.16:0/3536110331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf271f83c0 con 0x55bf27522b40 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f8411c69640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf2752bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f841246a640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8408010ca0 tx=0x7f840801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fec82c901872e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f83f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.288+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840801c950 con 0x55bf27522b40 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f8408011b40 con 0x55bf27522b40 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840801cb00 con 0x55bf27522b40 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8408011d40 con 0x55bf27522b40 2024-09-12T01:09:38.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 10 client.19581 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf2753aae0 con 0x55bf27522b40 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f840801cb00 con 0x55bf27522b40 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84080046c0 con 0x55bf27522b40 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 10 client.19581 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 10 client.19581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 20 client.19581 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf2753be30 0x55bf2753e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 4 client.19581 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.289+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf27494860 con 0x55bf2753be30 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.290+0000 7f8411c69640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf2753be30 0x55bf2753e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.290+0000 7f8411c69640 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf2753be30 0x55bf2753e220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88a2b63742492898 server_cookie=10c6c67911059cd6 in_seq=0 out_seq=0 2024-09-12T01:09:38.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.290+0000 7f83f5ffb640 10 client.19581 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f83f5ffb640 1 -- 172.21.15.16:0/3536110331 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf27494860 con 0x55bf2753be30 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f83f5ffb640 10 client.19581 handle_command_reply: tid=0 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 client.19581 shutdown 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 msgr2=0x55bf27528f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f8408010ca0 tx=0x7f840801c730 comp rx=0 tx=0).stop 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 shutdown_connections 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf2753be30 0x55bf2753e220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27528f90 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f83f5ffb640 0 client.19581 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 --2- 172.21.15.16:0/3536110331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf2752bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 >> 172.21.15.16:0/3536110331 conn(0x55bf274ea310 msgr2=0x55bf27523930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 shutdown_connections 2024-09-12T01:09:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 1 -- 172.21.15.16:0/3536110331 wait complete. 2024-09-12T01:09:38.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.291+0000 7f8416aa7740 20 client.19581 trim_cache size 0 max 16384 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 Processor -- start 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 -- start start 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf27538370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf27522b40 0x55bf2752af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1afa10 con 0x55bf2752c370 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1b1550 con 0x55bf275215d0 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.304+0000 7f8416aa7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1b00e0 con 0x55bf27522b40 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59426/0 (socket says 172.21.15.16:59426) 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 -- 172.21.15.16:0/1467168003 learned_addr learned my addr 172.21.15.16:0/1467168003 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:38.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f841246a640 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf27522b40 0x55bf2752af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf27538370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 -- 172.21.15.16:0/1467168003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf27522b40 msgr2=0x55bf2752af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf27522b40 0x55bf2752af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 -- 172.21.15.16:0/1467168003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 msgr2=0x55bf27538370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf27538370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 -- 172.21.15.16:0/1467168003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2751d510 con 0x55bf2752c370 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f8411c69640 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x55bf27522070 tx=0x7f83fc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79617b59f882897 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f83f7fff640 1 -- 172.21.15.16:0/1467168003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc012b20 con 0x55bf2752c370 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f83f7fff640 1 -- 172.21.15.16:0/1467168003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc024030 con 0x55bf2752c370 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.305+0000 7f83f7fff640 1 -- 172.21.15.16:0/1467168003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004cd0 con 0x55bf2752c370 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- 172.21.15.16:0/1467168003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 msgr2=0x55bf275439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x55bf27522070 tx=0x7f83fc014040 comp rx=0 tx=0).stop 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- 172.21.15.16:0/1467168003 shutdown_connections 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf2752c370 0x55bf275439e0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf27522b40 0x55bf2752af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1467168003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf27538370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- 172.21.15.16:0/1467168003 >> 172.21.15.16:0/1467168003 conn(0x55bf274ea310 msgr2=0x55bf2752a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- 172.21.15.16:0/1467168003 shutdown_connections 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- 172.21.15.16:0/1467168003 wait complete. 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 -- start start 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.306+0000 7f8416aa7740 1 Processor -- start 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf275347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1b0e80 con 0x55bf27522b40 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1b07b0 con 0x55bf275215d0 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1ada10 con 0x55bf2752c370 2024-09-12T01:09:38.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8416aa7740 10 client.0 _ll_register_callbacks cb 0x7f83fc00b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59430/0 (socket says 172.21.15.16:59430) 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 -- 172.21.15.16:0/1970100635 learned_addr learned my addr 172.21.15.16:0/1970100635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8411c69640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf275347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8411c69640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 -- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 msgr2=0x55bf275258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 -- 172.21.15.16:0/1970100635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 msgr2=0x55bf275347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf275347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 -- 172.21.15.16:0/1970100635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf27526120 con 0x55bf27522b40 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f8411c69640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:38.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f841246a640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8408000c00 tx=0x7f8408005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fffea79152bde4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.307+0000 7f83f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408005630 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f840800bd80 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc024030 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f840800bae0 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 10 client.19587 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf275a6590 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83fc024030 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84080035f0 con 0x55bf27522b40 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 10 client.19587 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 10 client.19587 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 20 client.19587 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf275a6c80 0x55bf275a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:38.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 4 client.19587 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf2748ed70 con 0x55bf275a6c80 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.308+0000 7f8411c69640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf275a6c80 0x55bf275a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f83f5ffb640 10 client.19587 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8411c69640 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf275a6c80 0x55bf275a90a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f62d895a768c5cd server_cookie=435f86addb89025b in_seq=0 out_seq=0 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f83f5ffb640 1 -- 172.21.15.16:0/1970100635 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf2748ed70 con 0x55bf275a6c80 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f83f5ffb640 10 client.19587 handle_command_reply: tid=0 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 client.19587 shutdown 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 msgr2=0x55bf27534ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8408000c00 tx=0x7f8408005850 comp rx=0 tx=0).stop 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 shutdown_connections 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf2752c370 0x55bf275258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55bf275a6c80 0x55bf275a90a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf27522b40 0x55bf27534ce0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 --2- 172.21.15.16:0/1970100635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf275215d0 0x55bf275347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 >> 172.21.15.16:0/1970100635 conn(0x55bf274ea310 msgr2=0x55bf27527f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f83f5ffb640 0 client.19587 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 shutdown_connections 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 1 -- 172.21.15.16:0/1970100635 wait complete. 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.309+0000 7f8416aa7740 20 client.19587 trim_cache size 0 max 16384 2024-09-12T01:09:38.316 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 msgr2=0x7f83f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f84040bce10 tx=0x7f84040c1040 comp rx=0 tx=0).stop 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 msgr2=0x7f840c0fd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f8404005cc0 tx=0x7f8404003970 comp rx=0 tx=0).stop 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 shutdown_connections 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83f80a2d70 0x7f83f80a5230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c106a60 0x7f840c0fd8e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f840c101400 0x7f840c0fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 --2- 172.21.15.16:0/2609289495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f840c100a60 0x7f840c0fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 >> 172.21.15.16:0/2609289495 conn(0x7f840c0fc780 msgr2=0x7f840c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 shutdown_connections 2024-09-12T01:09:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:38.310+0000 7f84146ce640 1 -- 172.21.15.16:0/2609289495 wait complete. 2024-09-12T01:09:38.749 INFO:tasks.workunit.client.0.smithi016.stdout:437/600: open fd = 3 2024-09-12T01:09:38.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.749 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.749 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.749 INFO:tasks.workunit.client.0.smithi016.stdout:438/600: open fd = 3 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:439/600: open fd = 3 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:440/600: open fd = 3 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:441/600: open fd = 3 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.750 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:442/600: open fd = 3 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:443/600: open fd = 3 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:444/600: open fd = 3 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:445/600: open fd = 3 2024-09-12T01:09:38.751 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:446/600: open fd = 3 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:447/600: open fd = 3 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:448/600: open fd = 3 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.752 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:449/600: open fd = 3 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:450/600: open fd = 3 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:451/600: open fd = 3 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.753 INFO:tasks.workunit.client.0.smithi016.stdout:452/600: open fd = 3 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:453/600: open fd = 3 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:454/600: open fd = 3 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.754 INFO:tasks.workunit.client.0.smithi016.stdout:455/600: open fd = 3 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:456/600: open fd = 3 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:457/600: open fd = 3 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:458/600: open fd = 3 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.755 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:459/600: open fd = 3 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:460/600: open fd = 3 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:461/600: open fd = 3 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.756 INFO:tasks.workunit.client.0.smithi016.stdout:462/600: open fd = 3 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:463/600: open fd = 3 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:464/600: open fd = 3 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:465/600: open fd = 3 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.757 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:466/600: open fd = 3 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:467/600: open fd = 3 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:468/600: open fd = 3 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.758 INFO:tasks.workunit.client.0.smithi016.stdout:469/600: open fd = 3 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:470/600: open fd = 3 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:471/600: open fd = 3 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:472/600: open fd = 3 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.759 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:473/600: open fd = 3 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:474/600: open fd = 3 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:475/600: open fd = 3 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.760 INFO:tasks.workunit.client.0.smithi016.stdout:476/600: open fd = 3 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:477/600: open fd = 3 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:478/600: open fd = 3 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.761 INFO:tasks.workunit.client.0.smithi016.stdout:479/600: open fd = 3 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:480/600: open fd = 3 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:481/600: open fd = 3 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.762 INFO:tasks.workunit.client.0.smithi016.stdout:482/600: open fd = 3 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:483/600: open fd = 3 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:484/600: open fd = 3 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:485/600: open fd = 3 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.763 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:486/600: open fd = 3 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:487/600: open fd = 3 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:488/600: open fd = 3 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.764 INFO:tasks.workunit.client.0.smithi016.stdout:489/600: open fd = 3 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:490/600: open fd = 3 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:491/600: open fd = 3 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:492/600: open fd = 3 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.765 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:493/600: open fd = 3 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:494/600: open fd = 3 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:495/600: open fd = 3 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.766 INFO:tasks.workunit.client.0.smithi016.stdout:496/600: open fd = 3 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:497/600: open fd = 3 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:498/600: open fd = 3 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:09:38.767 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:09:39.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:09:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:38 smithi099 ceph-mon[35654]: pgmap v422: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:38 smithi159 ceph-mon[35566]: pgmap v422: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:39.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:38 smithi016 ceph-mon[30972]: pgmap v422: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-12T01:09:39.441 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 -- 172.21.15.16:0/1242201959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 msgr2=0x7fce6810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 --2- 172.21.15.16:0/1242201959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6810eaf0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fce60009930 tx=0x7fce6002a420 comp rx=0 tx=0).stop 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 -- 172.21.15.16:0/1242201959 shutdown_connections 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 --2- 172.21.15.16:0/1242201959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6810eaf0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 --2- 172.21.15.16:0/1242201959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce68106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 --2- 172.21.15.16:0/1242201959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 0x7fce68100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 -- 172.21.15.16:0/1242201959 >> 172.21.15.16:0/1242201959 conn(0x7fce68077210 msgr2=0x7fce680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 -- 172.21.15.16:0/1242201959 shutdown_connections 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 -- 172.21.15.16:0/1242201959 wait complete. 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.980+0000 7fce6d79c640 1 Processor -- start 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 -- start start 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 0x7fce68198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce681989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce68111fa0 con 0x7fce68106f50 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce68111e20 con 0x7fce68100d00 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce6d79c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce68112120 con 0x7fce68100350 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46304/0 (socket says 172.21.15.16:46304) 2024-09-12T01:09:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 -- 172.21.15.16:0/273625121 learned_addr learned my addr 172.21.15.16:0/273625121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce66ffd640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 0x7fce68198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce667fc640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce681989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 -- 172.21.15.16:0/273625121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 msgr2=0x7fce68198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 0x7fce68198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 -- 172.21.15.16:0/273625121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 msgr2=0x7fce681989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce681989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 -- 172.21.15.16:0/273625121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6819d430 con 0x7fce68106f50 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce667fc640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce681989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.981+0000 7fce677fe640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fce60005ab0 tx=0x7fce600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.982+0000 7fce4ffff640 1 -- 172.21.15.16:0/273625121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6003f030 con 0x7fce68106f50 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.982+0000 7fce4ffff640 1 -- 172.21.15.16:0/273625121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce60003ce0 con 0x7fce68106f50 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.982+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6819d6c0 con 0x7fce68106f50 2024-09-12T01:09:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.982+0000 7fce4ffff640 1 -- 172.21.15.16:0/273625121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60035070 con 0x7fce68106f50 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.982+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce681a5000 con 0x7fce68106f50 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 Processor -- start 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce4ffff640 1 -- 172.21.15.16:0/273625121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fce60042030 con 0x7fce68106f50 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 -- start start 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482efb40 0x5600482ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600482542f0 con 0x5600482ee5d0 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5600481fbcf0 con 0x5600482f9370 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.985+0000 7fce6fb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600482502b0 con 0x5600482efb40 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46308/0 (socket says 172.21.15.16:46308) 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 -- 172.21.15.16:0/1821824227 learned_addr learned my addr 172.21.15.16:0/1821824227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce667fc640 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482efb40 0x5600482ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce667fc640 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 -- 172.21.15.16:0/1821824227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482efb40 msgr2=0x5600482ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482efb40 0x5600482ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 -- 172.21.15.16:0/1821824227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 msgr2=0x5600482fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 -- 172.21.15.16:0/1821824227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600481e03a0 con 0x5600482ee5d0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce4ffff640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 0x7fce500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce667fc640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 0x7fce500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce4ffff640 1 -- 172.21.15.16:0/273625121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fce600ef8d0 con 0x7fce68106f50 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fce580103f0 tx=0x7fce580108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7baa3e0de107b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.986+0000 7fce4d7fa640 1 -- 172.21.15.16:0/1821824227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58004d20 con 0x5600482ee5d0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce4d7fa640 1 -- 172.21.15.16:0/1821824227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce580129f0 con 0x5600482ee5d0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce4d7fa640 1 -- 172.21.15.16:0/1821824227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58005600 con 0x5600482ee5d0 2024-09-12T01:09:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 -- 172.21.15.16:0/1821824227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 msgr2=0x5600482eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce667fc640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 0x7fce500a51b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5600482ef070 tx=0x7fce5c005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fce580103f0 tx=0x7fce580108b0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 -- 172.21.15.16:0/1821824227 shutdown_connections 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482eea10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1821824227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482efb40 0x5600482ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 -- 172.21.15.16:0/1821824227 >> 172.21.15.16:0/1821824227 conn(0x5600482b7310 msgr2=0x5600482eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 -- 172.21.15.16:0/1821824227 shutdown_connections 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.987+0000 7fce6fb75740 1 -- 172.21.15.16:0/1821824227 wait complete. 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 -- start start 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 Processor -- start 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482ee5d0 0x5600482f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600482fc070 con 0x5600482efb40 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56004820f060 con 0x5600482f9370 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600482fc1f0 con 0x5600482ee5d0 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce6fb75740 10 client.0 _ll_register_callbacks cb 0x560047f628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46322/0 (socket says 172.21.15.16:46322) 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce667fc640 1 -- 172.21.15.16:0/1055317324 learned_addr learned my addr 172.21.15.16:0/1055317324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482ee5d0 0x5600482f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.988+0000 7fce677fe640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 -- 172.21.15.16:0/1055317324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482ee5d0 msgr2=0x5600482f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482ee5d0 0x5600482f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 -- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 msgr2=0x5600482f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 -- 172.21.15.16:0/1055317324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560047fc53c0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce667fc640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fce5c015fd0 tx=0x7fce5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6713c1e0e858d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c020440 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fce5c0205e0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce580129f0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce5c0207e0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce6fb75740 10 client.19599 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560048307b50 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce580129f0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.989+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce5c033ae0 con 0x5600482efb40 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 10 client.19599 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 10 client.19599 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 20 client.19599 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048308fb0 0x56004830b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 4 client.19599 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560048261860 con 0x560048308fb0 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048308fb0 0x56004830b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce4aff5640 10 client.19599 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.990+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048308fb0 0x56004830b3d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cb38d6a6d462871 server_cookie=4df610baf3b1dd9f in_seq=0 out_seq=0 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce4aff5640 1 -- 172.21.15.16:0/1055317324 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560048261860 con 0x560048308fb0 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce4aff5640 10 client.19599 handle_command_reply: tid=0 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 client.19599 shutdown 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 msgr2=0x5600482f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fce5c015fd0 tx=0x7fce5c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 shutdown_connections 2024-09-12T01:09:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048308fb0 0x56004830b3d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482f9370 0x5600482f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x5600482f6000 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce4aff5640 0 client.19599 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1055317324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482ee5d0 0x5600482f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.991+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 >> 172.21.15.16:0/1055317324 conn(0x5600482b7310 msgr2=0x5600482f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.992+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 shutdown_connections 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.992+0000 7fce6fb75740 1 -- 172.21.15.16:0/1055317324 wait complete. 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:39.992+0000 7fce6fb75740 20 client.19599 trim_cache size 0 max 16384 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 Processor -- start 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 -- start start 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482ee5d0 0x5600482eae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x560048310ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ad8f0 con 0x5600482efb40 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce681af430 con 0x5600482ee5d0 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce6fb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce681adfc0 con 0x5600482f9370 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46334/0 (socket says 172.21.15.16:46334) 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 -- 172.21.15.16:0/2282704359 learned_addr learned my addr 172.21.15.16:0/2282704359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce66ffd640 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482ee5d0 0x5600482eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 -- 172.21.15.16:0/2282704359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 msgr2=0x560048310ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x560048310ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 -- 172.21.15.16:0/2282704359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482ee5d0 msgr2=0x5600482eae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482ee5d0 0x5600482eae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.005+0000 7fce667fc640 1 -- 172.21.15.16:0/2282704359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600482f4550 con 0x5600482efb40 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce667fc640 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c4b0 tx=0x7fce5c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b207392729f623 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce4cff9640 1 -- 172.21.15.16:0/2282704359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c00fd70 con 0x5600482efb40 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce4cff9640 1 -- 172.21.15.16:0/2282704359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c020440 con 0x5600482efb40 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce4cff9640 1 -- 172.21.15.16:0/2282704359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c02a900 con 0x5600482efb40 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 -- 172.21.15.16:0/2282704359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 msgr2=0x560048305310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c4b0 tx=0x7fce5c00cf40 comp rx=0 tx=0).stop 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 -- 172.21.15.16:0/2282704359 shutdown_connections 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x560048310ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482efb40 0x560048305310 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 --2- 172.21.15.16:0/2282704359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482ee5d0 0x5600482eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 -- 172.21.15.16:0/2282704359 >> 172.21.15.16:0/2282704359 conn(0x5600482b7310 msgr2=0x5600482ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 -- 172.21.15.16:0/2282704359 shutdown_connections 2024-09-12T01:09:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.006+0000 7fce6fb75740 1 -- 172.21.15.16:0/2282704359 wait complete. 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 -- start start 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 Processor -- start 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482efb40 0x5600482f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x5600482f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce681aed60 con 0x5600482ee5d0 2024-09-12T01:09:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ae690 con 0x5600482efb40 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce681ab8f0 con 0x5600482f9370 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce6fb75740 10 client.0 _ll_register_callbacks cb 0x7fce5c00c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46348/0 (socket says 172.21.15.16:46348) 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.007+0000 7fce66ffd640 1 -- 172.21.15.16:0/1234469915 learned_addr learned my addr 172.21.15.16:0/1234469915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce677fe640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x5600482f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce667fc640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482efb40 0x5600482f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 -- 172.21.15.16:0/1234469915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 msgr2=0x5600482f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x5600482f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 -- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482efb40 msgr2=0x5600482f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482efb40 0x5600482f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 -- 172.21.15.16:0/1234469915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560048372dd0 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce677fe640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x5600482f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce66ffd640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fce580103f0 tx=0x7fce58005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff47ed1b19237471 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58003b80 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fce58003d20 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c020440 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce5802b020 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce6fb75740 10 client.19605 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560048374110 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.008+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5c020440 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fce5800f040 con 0x5600482ee5d0 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 10 client.19605 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 10 client.19605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 20 client.19605 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048374700 0x560048376b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 4 client.19605 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56004825bd70 con 0x560048374700 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce667fc640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048374700 0x560048376b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce667fc640 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048374700 0x560048376b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c029cc176f88193f server_cookie=8ea19bd16d759ce8 in_seq=0 out_seq=0 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.009+0000 7fce4aff5640 10 client.19605 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce4aff5640 1 -- 172.21.15.16:0/1234469915 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56004825bd70 con 0x560048374700 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce4aff5640 10 client.19605 handle_command_reply: tid=0 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 client.19605 shutdown 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 msgr2=0x5600482f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fce580103f0 tx=0x7fce58005240 comp rx=0 tx=0).stop 2024-09-12T01:09:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 shutdown_connections 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x560048374700 0x560048376b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600482f9370 0x5600482f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5600482efb40 0x5600482f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 --2- 172.21.15.16:0/1234469915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600482ee5d0 0x5600482f3180 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce4aff5640 0 client.19605 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 >> 172.21.15.16:0/1234469915 conn(0x5600482b7310 msgr2=0x5600482f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 shutdown_connections 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 1 -- 172.21.15.16:0/1234469915 wait complete. 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6fb75740 20 client.19605 trim_cache size 0 max 16384 2024-09-12T01:09:40.024 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:09:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 msgr2=0x7fce500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 0x7fce500a51b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x5600482ef070 tx=0x7fce5c005850 comp rx=0 tx=0).stop 2024-09-12T01:09:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 msgr2=0x7fce6819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.010+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fce60005ab0 tx=0x7fce600042f0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 shutdown_connections 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fce500a2cf0 0x7fce500a51b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce68106f50 0x7fce6819cd50 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce68100d00 0x7fce681989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 --2- 172.21.15.16:0/273625121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce68100350 0x7fce68198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 >> 172.21.15.16:0/273625121 conn(0x7fce68077210 msgr2=0x7fce68105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 shutdown_connections 2024-09-12T01:09:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:40.011+0000 7fce6d79c640 1 -- 172.21.15.16:0/273625121 wait complete. 2024-09-12T01:09:40.659 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T01:09:40.659 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T01:09:40.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:09:40.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:09:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:41.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:40 smithi016 ceph-mon[30972]: pgmap v423: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:40 smithi099 ceph-mon[35654]: pgmap v423: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:40 smithi159 ceph-mon[35566]: pgmap v423: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 89 KiB/s wr, 20 op/s 2024-09-12T01:09:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 -- 172.21.15.16:0/105077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0101de0 msgr2=0x7f47f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 --2- 172.21.15.16:0/105077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0101de0 0x7f47f01021c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009900 tx=0x7f47e002a410 comp rx=0 tx=0).stop 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 -- 172.21.15.16:0/105077028 shutdown_connections 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 --2- 172.21.15.16:0/105077028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 0x7f47f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 --2- 172.21.15.16:0/105077028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0102790 0x7f47f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 --2- 172.21.15.16:0/105077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0101de0 0x7f47f01021c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 -- 172.21.15.16:0/105077028 >> 172.21.15.16:0/105077028 conn(0x7f47f00fd8c0 msgr2=0x7f47f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 -- 172.21.15.16:0/105077028 shutdown_connections 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.430+0000 7f47f6212640 1 -- 172.21.15.16:0/105077028 wait complete. 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 Processor -- start 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 -- start start 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:41.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0111f70 con 0x7f47f0102790 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0111df0 con 0x7f47f0106f10 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.431+0000 7f47f6212640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47f01120f0 con 0x7f47f0101de0 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46360/0 (socket says 172.21.15.16:46360) 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 -- 172.21.15.16:0/2332222831 learned_addr learned my addr 172.21.15.16:0/2332222831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47ef7fe640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47effff640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 -- 172.21.15.16:0/2332222831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0101de0 msgr2=0x7f47f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 -- 172.21.15.16:0/2332222831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 msgr2=0x7f47f0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 -- 172.21.15.16:0/2332222831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f01a6970 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47eeffd640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f47e400e9f0 tx=0x7f47e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e400cd80 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4005500 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f01a6c60 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4010690 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.432+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a71a0 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.434+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f01021c0 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.435+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f47e40046c0 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.435+0000 7f47ecff9640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 0x7f47d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.435+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f47e40cf7d0 con 0x7f47f0102790 2024-09-12T01:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.435+0000 7f47ef7fe640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 0x7f47d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.436+0000 7f47ef7fe640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 0x7f47d80a5230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009230 tx=0x7f47e00058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.440+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47e409c3f0 con 0x7f47f0102790 2024-09-12T01:09:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:41.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.662+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47f010a0f0 con 0x7f47f0102790 2024-09-12T01:09:41.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.662+0000 7f47ecff9640 1 -- 172.21.15.16:0/2332222831 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f47e409bcf0 con 0x7f47f0102790 2024-09-12T01:09:41.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:09:41.661 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:09:41.662 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:09:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.664+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 msgr2=0x7f47d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 0x7f47d80a5230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009230 tx=0x7f47e00058d0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 msgr2=0x7f47f0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f47e400e9f0 tx=0x7f47e400edf0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 shutdown_connections 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f47d80a2d70 0x7f47d80a5230 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47f0106f10 0x7f47f0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47f0102790 0x7f47f0074a30 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 --2- 172.21.15.16:0/2332222831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f47f0101de0 0x7f47f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 >> 172.21.15.16:0/2332222831 conn(0x7f47f00fd8c0 msgr2=0x7f47f00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 shutdown_connections 2024-09-12T01:09:41.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:41.665+0000 7f47f6212640 1 -- 172.21.15.16:0/2332222831 wait complete. 2024-09-12T01:09:42.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:41 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2332222831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 -- 172.21.15.16:0/2458580191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8106f10 msgr2=0x7f09a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 --2- 172.21.15.16:0/2458580191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8106f10 0x7f09a810ea90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ca0 tx=0x7f09a002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 -- 172.21.15.16:0/2458580191 shutdown_connections 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 --2- 172.21.15.16:0/2458580191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8106f10 0x7f09a810ea90 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 --2- 172.21.15.16:0/2458580191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8102790 0x7f09a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 --2- 172.21.15.16:0/2458580191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a8101de0 0x7f09a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 -- 172.21.15.16:0/2458580191 >> 172.21.15.16:0/2458580191 conn(0x7f09a80fd8c0 msgr2=0x7f09a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 -- 172.21.15.16:0/2458580191 shutdown_connections 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.130+0000 7f09aff87640 1 -- 172.21.15.16:0/2458580191 wait complete. 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 Processor -- start 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 -- start start 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a8101de0 0x7f09a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 0x7f09a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111f70 con 0x7f09a8102790 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111df0 con 0x7f09a8106f10 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09aff87640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81120f0 con 0x7f09a8101de0 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46368/0 (socket says 172.21.15.16:46368) 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09ad522640 1 -- 172.21.15.16:0/536481954 learned_addr learned my addr 172.21.15.16:0/536481954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.131+0000 7f09ae524640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 0x7f09a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 -- 172.21.15.16:0/536481954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a8101de0 msgr2=0x7f09a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a8101de0 0x7f09a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 -- 172.21.15.16:0/536481954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 msgr2=0x7f09a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 0x7f09a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 -- 172.21.15.16:0/536481954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a6890 con 0x7f09a8102790 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ae524640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 0x7f09a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09ad522640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f099800d950 tx=0x7f099800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f099effd640 1 -- 172.21.15.16:0/536481954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998017070 con 0x7f09a8102790 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f099effd640 1 -- 172.21.15.16:0/536481954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998009c30 con 0x7f09a8102790 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a6b80 con 0x7f09a8102790 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f099effd640 1 -- 172.21.15.16:0/536481954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800f6b0 con 0x7f09a8102790 2024-09-12T01:09:42.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.132+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a70c0 con 0x7f09a8102790 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.134+0000 7f099effd640 1 -- 172.21.15.16:0/536481954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09980109d0 con 0x7f09a8102790 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 Processor -- start 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 -- start start 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f795b40 0x55660f794090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f7a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55660f6fa2f0 con 0x55660f79f370 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55660f6a1cf0 con 0x55660f795b40 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09b2360740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55660f6f62b0 con 0x55660f7945d0 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f7a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09add23640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f099effd640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09add23640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:43470/0 (socket says 172.21.15.16:43470) 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f7a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46372/0 (socket says 172.21.15.16:46372) 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09add23640 1 -- 172.21.15.16:0/1502619890 learned_addr learned my addr 172.21.15.16:0/1502619890 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.135+0000 7f09ae524640 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f795b40 0x55660f794090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09ae524640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 -- 172.21.15.16:0/1502619890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f795b40 msgr2=0x55660f794090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f795b40 0x55660f794090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f099effd640 1 -- 172.21.15.16:0/536481954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f09980cf0f0 con 0x7f09a8102790 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 -- 172.21.15.16:0/1502619890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 msgr2=0x55660f7a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f7a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 -- 172.21.15.16:0/1502619890 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55660f6863a0 con 0x55660f7945d0 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09add23640 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009c40 tx=0x7f09a400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12a782331c40a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f09ae524640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 0x7f09940a51b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005e30 tx=0x7f09a00095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f0993fff640 1 -- 172.21.15.16:0/1502619890 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4005020 con 0x55660f7945d0 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f0993fff640 1 -- 172.21.15.16:0/1502619890 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a40047a0 con 0x55660f7945d0 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.136+0000 7f0993fff640 1 -- 172.21.15.16:0/1502619890 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4010760 con 0x55660f7945d0 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- 172.21.15.16:0/1502619890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 msgr2=0x55660f794a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009c40 tx=0x7f09a400e720 comp rx=0 tx=0).stop 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- 172.21.15.16:0/1502619890 shutdown_connections 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f7a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f794a10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 --2- 172.21.15.16:0/1502619890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f795b40 0x55660f794090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- 172.21.15.16:0/1502619890 >> 172.21.15.16:0/1502619890 conn(0x55660f75d310 msgr2=0x55660f790c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- 172.21.15.16:0/1502619890 shutdown_connections 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- 172.21.15.16:0/1502619890 wait complete. 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.137+0000 7f09b2360740 1 -- start start 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 Processor -- start 2024-09-12T01:09:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f7945d0 0x55660f79eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 0x55660f79bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55660f7a2070 con 0x55660f79f370 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55660f6b5060 con 0x55660f7945d0 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55660f7a21f0 con 0x55660f795b40 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09b2360740 10 client.0 _ll_register_callbacks cb 0x55660f4088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09ae524640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09ae524640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46384/0 (socket says 172.21.15.16:46384) 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09ae524640 1 -- 172.21.15.16:0/3358797937 learned_addr learned my addr 172.21.15.16:0/3358797937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09ad522640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 0x55660f79bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.138+0000 7f09add23640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f7945d0 0x55660f79eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 -- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 msgr2=0x55660f79bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 0x55660f79bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 -- 172.21.15.16:0/3358797937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f7945d0 msgr2=0x55660f79eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f7945d0 0x55660f79eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 -- 172.21.15.16:0/3358797937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660f46b3c0 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ad522640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 0x55660f79bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09ae524640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f09a0035000 tx=0x7f09a002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebc87cf9c0182f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09917fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a004d070 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f09a0004240 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a40047a0 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09a0043070 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09b2360740 10 client.19623 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55660f7adab0 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.139+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a40047a0 con 0x55660f79f370 2024-09-12T01:09:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09a002fd10 con 0x55660f79f370 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 10 client.19623 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 10 client.19623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 20 client.19623 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f7aeec0 0x55660f7b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 4 client.19623 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55660f707860 con 0x55660f7aeec0 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09add23640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f7aeec0 0x55660f7b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09add23640 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f7aeec0 0x55660f7b1310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=585eb9300426791e server_cookie=793ea71f9ba373dd in_seq=0 out_seq=0 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.140+0000 7f09917fa640 10 client.19623 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09917fa640 1 -- 172.21.15.16:0/3358797937 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55660f707860 con 0x55660f7aeec0 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09917fa640 10 client.19623 handle_command_reply: tid=0 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 client.19623 shutdown 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 msgr2=0x55660f79c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f09a0035000 tx=0x7f09a002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 shutdown_connections 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f79f370 0x55660f79c490 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f795b40 0x55660f79bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f7aeec0 0x55660f7b1310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 --2- 172.21.15.16:0/3358797937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f7945d0 0x55660f79eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 >> 172.21.15.16:0/3358797937 conn(0x55660f75d310 msgr2=0x55660f796960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09917fa640 0 client.19623 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.141+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 shutdown_connections 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.142+0000 7f09b2360740 1 -- 172.21.15.16:0/3358797937 wait complete. 2024-09-12T01:09:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.142+0000 7f09b2360740 20 client.19623 trim_cache size 0 max 16384 2024-09-12T01:09:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 Processor -- start 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 -- start start 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f790e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81afa40 con 0x55660f795b40 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b1580 con 0x55660f79f370 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09b2360740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b0110 con 0x55660f7945d0 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46388/0 (socket says 172.21.15.16:46388) 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ad522640 1 -- 172.21.15.16:0/612489740 learned_addr learned my addr 172.21.15.16:0/612489740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09add23640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f790e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ad522640 1 -- 172.21.15.16:0/612489740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 msgr2=0x55660f790e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ad522640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f790e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09ad522640 1 -- 172.21.15.16:0/612489740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 msgr2=0x55660f7b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.155+0000 7f09ae524640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09ad522640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09ad522640 1 -- 172.21.15.16:0/612489740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55660f791dd0 con 0x55660f795b40 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09add23640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f790e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09ae524640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09ad522640 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f099809b540 tx=0x7f09980d1540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00a9da851c18df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09937fe640 1 -- 172.21.15.16:0/612489740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998020040 con 0x55660f795b40 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09937fe640 1 -- 172.21.15.16:0/612489740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998009de0 con 0x55660f795b40 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.156+0000 7f09937fe640 1 -- 172.21.15.16:0/612489740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980d10b0 con 0x55660f795b40 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- 172.21.15.16:0/612489740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 msgr2=0x55660f7ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f099809b540 tx=0x7f09980d1540 comp rx=0 tx=0).stop 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- 172.21.15.16:0/612489740 shutdown_connections 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7ab310 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 --2- 172.21.15.16:0/612489740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f790e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- 172.21.15.16:0/612489740 >> 172.21.15.16:0/612489740 conn(0x55660f75d310 msgr2=0x55660f790700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- 172.21.15.16:0/612489740 shutdown_connections 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- 172.21.15.16:0/612489740 wait complete. 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 -- start start 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.157+0000 7f09b2360740 1 Processor -- start 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f7a7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b0eb0 con 0x55660f795b40 2024-09-12T01:09:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b07e0 con 0x55660f79f370 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ada40 con 0x55660f7945d0 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 10 client.0 _ll_register_callbacks cb 0x7f09a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46402/0 (socket says 172.21.15.16:46402) 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 -- 172.21.15.16:0/3914092802 learned_addr learned my addr 172.21.15.16:0/3914092802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09add23640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f7a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ae524640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 -- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 msgr2=0x55660f7a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f7a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 -- 172.21.15.16:0/3914092802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 msgr2=0x55660f7987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 -- 172.21.15.16:0/3914092802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660f799010 con 0x55660f795b40 2024-09-12T01:09:42.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09add23640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f7a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09ad522640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0998009610 tx=0x7f099800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8926d3a203876b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09917fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998020070 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7f09980d1240 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.158+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0998009de0 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09980ccc70 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 10 client.19629 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55660f819660 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998009de0 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09980b4220 con 0x55660f795b40 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 10 client.19629 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 10 client.19629 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:09:42.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 20 client.19629 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f819d50 0x55660f81c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 4 client.19629 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55660f701d70 con 0x55660f819d50 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.159+0000 7f09add23640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f819d50 0x55660f81c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09add23640 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f819d50 0x55660f81c170 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=163ab70e78a5ae25 server_cookie=23ad52e481858b72 in_seq=0 out_seq=0 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09917fa640 10 client.19629 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09917fa640 1 -- 172.21.15.16:0/3914092802 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55660f701d70 con 0x55660f819d50 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09917fa640 10 client.19629 handle_command_reply: tid=0 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 client.19629 shutdown 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 msgr2=0x55660f7a7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f0998009610 tx=0x7f099800bf60 comp rx=0 tx=0).stop 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 shutdown_connections 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55660f819d50 0x55660f81c170 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55660f79f370 0x55660f7987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09917fa640 0 client.19629 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55660f795b40 0x55660f7a7c50 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 --2- 172.21.15.16:0/3914092802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55660f7945d0 0x55660f7a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 >> 172.21.15.16:0/3914092802 conn(0x55660f75d310 msgr2=0x55660f79dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 shutdown_connections 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 1 -- 172.21.15.16:0/3914092802 wait complete. 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.160+0000 7f09b2360740 20 client.19629 trim_cache size 0 max 16384 2024-09-12T01:09:42.166 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 msgr2=0x7f09940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 0x7f09940a51b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005e30 tx=0x7f09a00095c0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 msgr2=0x7f09a8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f099800d950 tx=0x7f099800de10 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 shutdown_connections 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09940a2cf0 0x7f09940a51b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a8106f10 0x7f09a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09a8102790 0x7f09a8074a30 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 --2- 172.21.15.16:0/536481954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09a8101de0 0x7f09a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 >> 172.21.15.16:0/536481954 conn(0x7f09a80fd8c0 msgr2=0x7f09a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 shutdown_connections 2024-09-12T01:09:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:42.161+0000 7f09aff87640 1 -- 172.21.15.16:0/536481954 wait complete. 2024-09-12T01:09:42.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:41 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2332222831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:42.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:41 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2332222831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:09:43.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:09:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:42 smithi099 ceph-mon[35654]: pgmap v424: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:43.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:42 smithi159 ceph-mon[35566]: pgmap v424: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:43.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:42 smithi016 ceph-mon[30972]: pgmap v424: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:43.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T01:09:43.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T01:09:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:44.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- 172.21.15.16:0/843556548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060103660 msgr2=0x7fb06010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 --2- 172.21.15.16:0/843556548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060103660 0x7fb06010bb50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402a480 comp rx=0 tx=0).stop 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- 172.21.15.16:0/843556548 shutdown_connections 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 --2- 172.21.15.16:0/843556548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb06010c170 0x7fb06010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 --2- 172.21.15.16:0/843556548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb060103660 0x7fb06010bb50 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 --2- 172.21.15.16:0/843556548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 0x7fb060103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- 172.21.15.16:0/843556548 >> 172.21.15.16:0/843556548 conn(0x7fb0600fc7a0 msgr2=0x7fb0600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- 172.21.15.16:0/843556548 shutdown_connections 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- 172.21.15.16:0/843556548 wait complete. 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 Processor -- start 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.095+0000 7fb06851f640 1 -- start start 2024-09-12T01:09:44.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 0x7fb060198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 0x7fb060198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111fb0 con 0x7fb06010c170 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb060111e30 con 0x7fb060102d40 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb06851f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb060112130 con 0x7fb060103660 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46416/0 (socket says 172.21.15.16:46416) 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 -- 172.21.15.16:0/472799886 learned_addr learned my addr 172.21.15.16:0/472799886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb0662bb640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 0x7fb060198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb065aba640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 0x7fb060198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 -- 172.21.15.16:0/472799886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 msgr2=0x7fb060198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 0x7fb060198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 -- 172.21.15.16:0/472799886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 msgr2=0x7fb060198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 0x7fb060198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 -- 172.21.15.16:0/472799886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06019d540 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb065aba640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 0x7fb060198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb066abc640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fb058009fd0 tx=0x7fb05800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb05f7fe640 1 -- 172.21.15.16:0/472799886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058011830 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.096+0000 7fb05f7fe640 1 -- 172.21.15.16:0/472799886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb058016070 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.097+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06019d830 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.097+0000 7fb05f7fe640 1 -- 172.21.15.16:0/472799886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05800f2e0 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.097+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0601a5110 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb05f7fe640 1 -- 172.21.15.16:0/472799886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb0580119d0 con 0x7fb06010c170 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 Processor -- start 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 -- start start 2024-09-12T01:09:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711dd370 0x5583711df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5583711382f0 con 0x5583711d3b40 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583710dfcf0 con 0x5583711d25d0 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.099+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5583711342b0 con 0x5583711dd370 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46432/0 (socket says 172.21.15.16:46432) 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 -- 172.21.15.16:0/3479026293 learned_addr learned my addr 172.21.15.16:0/3479026293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb05f7fe640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 0x7fb0480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb0662bb640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711dd370 0x5583711df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb066abc640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 0x7fb0480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 -- 172.21.15.16:0/3479026293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711dd370 msgr2=0x5583711df760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711dd370 0x5583711df760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 -- 172.21.15.16:0/3479026293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 msgr2=0x5583711d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 -- 172.21.15.16:0/3479026293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583710c43a0 con 0x5583711d3b40 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb0662bb640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb05f7fe640 1 -- 172.21.15.16:0/472799886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb0580ce990 con 0x7fb06010c170 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb065aba640 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x5583711d3070 tx=0x7fb054031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36283e22bf6e99f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb05cff9640 1 -- 172.21.15.16:0/3479026293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035050 con 0x5583711d3b40 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb066abc640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 0x7fb0480a51b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb0580cc6e0 tx=0x7fb0580bb040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb05cff9640 1 -- 172.21.15.16:0/3479026293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054031960 con 0x5583711d3b40 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.100+0000 7fb05cff9640 1 -- 172.21.15.16:0/3479026293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540048e0 con 0x5583711d3b40 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 -- 172.21.15.16:0/3479026293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 msgr2=0x5583711d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x5583711d3070 tx=0x7fb054031480 comp rx=0 tx=0).stop 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 -- 172.21.15.16:0/3479026293 shutdown_connections 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711dd370 0x5583711df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/3479026293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711d2090 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 -- 172.21.15.16:0/3479026293 >> 172.21.15.16:0/3479026293 conn(0x55837119b310 msgr2=0x5583711cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 -- 172.21.15.16:0/3479026293 shutdown_connections 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.101+0000 7fb06a8f8740 1 -- 172.21.15.16:0/3479026293 wait complete. 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 -- start start 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 Processor -- start 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711dcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5583711dffb0 con 0x5583711dd370 2024-09-12T01:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583710f3060 con 0x5583711d25d0 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5583711e0130 con 0x5583711d3b40 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb06a8f8740 10 client.0 _ll_register_callbacks cb 0x558370e468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46448/0 (socket says 172.21.15.16:46448) 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 -- 172.21.15.16:0/5185096 learned_addr learned my addr 172.21.15.16:0/5185096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb0662bb640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711dcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 -- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 msgr2=0x5583711d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 -- 172.21.15.16:0/5185096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 msgr2=0x5583711dcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711dcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.102+0000 7fb065aba640 1 -- 172.21.15.16:0/5185096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558370ea93c0 con 0x5583711dd370 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb065aba640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fb054000c00 tx=0x7fb054005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450ee829971da5ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035070 con 0x5583711dd370 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb054004c80 con 0x5583711dd370 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb054031960 con 0x5583711dd370 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb054004030 con 0x5583711dd370 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 10 client.19641 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583711eb990 con 0x5583711dd370 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054031960 con 0x5583711dd370 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb054003c60 con 0x5583711dd370 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 10 client.19641 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 10 client.19641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 20 client.19641 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5583711ecce0 0x5583711ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 4 client.19641 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.103+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558371145860 con 0x5583711ecce0 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.104+0000 7fb0662bb640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5583711ecce0 0x5583711ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.104+0000 7fb0662bb640 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5583711ecce0 0x5583711ef0d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8ced171411fa65 server_cookie=2482b5f370e4dba3 in_seq=0 out_seq=0 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.104+0000 7fb045ffb640 10 client.19641 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb045ffb640 1 -- 172.21.15.16:0/5185096 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558371145860 con 0x5583711ecce0 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb045ffb640 10 client.19641 handle_command_reply: tid=0 2024-09-12T01:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 client.19641 shutdown 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 msgr2=0x5583711da5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fb054000c00 tx=0x7fb054005650 comp rx=0 tx=0).stop 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 shutdown_connections 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5583711ecce0 0x5583711ef0d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711da5e0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb045ffb640 0 client.19641 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/5185096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711dcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 >> 172.21.15.16:0/5185096 conn(0x55837119b310 msgr2=0x5583711d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 shutdown_connections 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 1 -- 172.21.15.16:0/5185096 wait complete. 2024-09-12T01:09:44.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.105+0000 7fb06a8f8740 20 client.19641 trim_cache size 0 max 16384 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.118+0000 7fb06a8f8740 1 Processor -- start 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 -- start start 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 0x5583711e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711dd370 0x5583711f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ab670 con 0x5583711d3b40 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ad1b0 con 0x5583711dd370 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601abd40 con 0x5583711d25d0 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46462/0 (socket says 172.21.15.16:46462) 2024-09-12T01:09:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 -- 172.21.15.16:0/2027148374 learned_addr learned my addr 172.21.15.16:0/2027148374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb065aba640 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 0x5583711e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 -- 172.21.15.16:0/2027148374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 msgr2=0x5583711e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 0x5583711e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 -- 172.21.15.16:0/2027148374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711dd370 msgr2=0x5583711f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711dd370 0x5583711f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 -- 172.21.15.16:0/2027148374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583711d5120 con 0x5583711d3b40 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb065aba640 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 0x5583711e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fb050000f80 tx=0x7fb050009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb3636b37bd3c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.119+0000 7fb047fff640 1 -- 172.21.15.16:0/2027148374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05001f030 con 0x5583711d3b40 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb047fff640 1 -- 172.21.15.16:0/2027148374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050003e50 con 0x5583711d3b40 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb047fff640 1 -- 172.21.15.16:0/2027148374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500052a0 con 0x5583711d3b40 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2027148374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 msgr2=0x5583711f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fb050000f80 tx=0x7fb050009680 comp rx=0 tx=0).stop 2024-09-12T01:09:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2027148374 shutdown_connections 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711dd370 0x5583711f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711d3b40 0x5583711f4670 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2027148374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d25d0 0x5583711e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2027148374 >> 172.21.15.16:0/2027148374 conn(0x55837119b310 msgr2=0x5583711d88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2027148374 shutdown_connections 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.120+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2027148374 wait complete. 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 -- start start 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 Processor -- start 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601acae0 con 0x5583711dd370 2024-09-12T01:09:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601ac410 con 0x5583711d25d0 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0601a9670 con 0x5583711d3b40 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb06a8f8740 10 client.0 _ll_register_callbacks cb 0x7fb054009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46470/0 (socket says 172.21.15.16:46470) 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 -- 172.21.15.16:0/2017191286 learned_addr learned my addr 172.21.15.16:0/2017191286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 -- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 msgr2=0x5583711e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711e0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 -- 172.21.15.16:0/2017191286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 msgr2=0x5583711e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711e5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.121+0000 7fb065aba640 1 -- 172.21.15.16:0/2017191286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583711e15b0 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb065aba640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fb0540052c0 tx=0x7fb05405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bef8b1e2790c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035070 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb0540043d0 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb050003e50 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb054031620 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 10 client.19647 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583712572a0 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050003e50 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb054003710 con 0x5583711dd370 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 10 client.19647 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 10 client.19647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 20 client.19647 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558371257950 0x558371259d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 4 client.19647 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.122+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55837113fd70 con 0x558371257950 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.123+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558371257950 0x558371259d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.123+0000 7fb045ffb640 10 client.19647 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.123+0000 7fb0662bb640 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558371257950 0x558371259d70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3cb20de2a9f7df server_cookie=a26a6ee0e983a22e in_seq=0 out_seq=0 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb045ffb640 1 -- 172.21.15.16:0/2017191286 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55837113fd70 con 0x558371257950 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb045ffb640 10 client.19647 handle_command_reply: tid=0 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 client.19647 shutdown 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 msgr2=0x5583711e1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fb0540052c0 tx=0x7fb05405c040 comp rx=0 tx=0).stop 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 shutdown_connections 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558371257950 0x558371259d70 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb045ffb640 0 client.19647 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5583711dd370 0x5583711e1070 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5583711d3b40 0x5583711e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 --2- 172.21.15.16:0/2017191286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583711d25d0 0x5583711e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 >> 172.21.15.16:0/2017191286 conn(0x55837119b310 msgr2=0x5583711db850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 shutdown_connections 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 1 -- 172.21.15.16:0/2017191286 wait complete. 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06a8f8740 20 client.19647 trim_cache size 0 max 16384 2024-09-12T01:09:44.138 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 msgr2=0x7fb0480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 0x7fb0480a51b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb0580cc6e0 tx=0x7fb0580bb040 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 msgr2=0x7fb06019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fb058009fd0 tx=0x7fb05800b8e0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 shutdown_connections 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb0480a2cf0 0x7fb0480a51b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06010c170 0x7fb06019ce60 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb060103660 0x7fb060198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 --2- 172.21.15.16:0/472799886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb060102d40 0x7fb060198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.124+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 >> 172.21.15.16:0/472799886 conn(0x7fb0600fc7a0 msgr2=0x7fb0600fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.125+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 shutdown_connections 2024-09-12T01:09:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:44.125+0000 7fb06851f640 1 -- 172.21.15.16:0/472799886 wait complete. 2024-09-12T01:09:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:45.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:44 smithi099 ceph-mon[35654]: pgmap v425: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:45.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:44 smithi016 ceph-mon[30972]: pgmap v425: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:45.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:44 smithi159 ceph-mon[35566]: pgmap v425: 97 pgs: 97 active+clean; 22 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 -- 172.21.15.16:0/3447394699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030101de0 msgr2=0x7ff0301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 --2- 172.21.15.16:0/3447394699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030101de0 0x7ff0301021c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff02002a410 comp rx=0 tx=0).stop 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 -- 172.21.15.16:0/3447394699 shutdown_connections 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 --2- 172.21.15.16:0/3447394699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030106f10 0x7ff03010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 --2- 172.21.15.16:0/3447394699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030102790 0x7ff0301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 --2- 172.21.15.16:0/3447394699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030101de0 0x7ff0301021c0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 -- 172.21.15.16:0/3447394699 >> 172.21.15.16:0/3447394699 conn(0x7ff0300fd8c0 msgr2=0x7ff0300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.544+0000 7ff03589e640 1 -- 172.21.15.16:0/3447394699 shutdown_connections 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 -- 172.21.15.16:0/3447394699 wait complete. 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 Processor -- start 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 -- start start 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 0x7ff0301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030106f10 0x7ff03019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff030111f70 con 0x7ff030102790 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff030111df0 con 0x7ff030101de0 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.545+0000 7ff03589e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301120f0 con 0x7ff030106f10 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46480/0 (socket says 172.21.15.16:46480) 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 -- 172.21.15.16:0/4208393752 learned_addr learned my addr 172.21.15.16:0/4208393752 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff02effd640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 0x7ff0301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030106f10 0x7ff03019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 -- 172.21.15.16:0/4208393752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030106f10 msgr2=0x7ff03019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030106f10 0x7ff03019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 -- 172.21.15.16:0/4208393752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 msgr2=0x7ff0301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 0x7ff0301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 -- 172.21.15.16:0/4208393752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03019d460 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff02effd640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 0x7ff0301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff0267fc640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff01800d770 tx=0x7ff01800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff03489c640 1 -- 172.21.15.16:0/4208393752 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018017070 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff03489c640 1 -- 172.21.15.16:0/4208393752 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018005500 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff03489c640 1 -- 172.21.15.16:0/4208393752 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018004e00 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03019d750 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.546+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0301a5030 con 0x7ff030102790 2024-09-12T01:09:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff03489c640 1 -- 172.21.15.16:0/4208393752 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff018004fa0 con 0x7ff030102790 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 Processor -- start 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 -- start start 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x556493011b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x55649301ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556492f772a0 con 0x556493012080 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556492f1eb10 con 0x556493011760 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff037c77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556492f73260 con 0x55649301c940 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff02effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff02effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46490/0 (socket says 172.21.15.16:46490) 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff02effd640 1 -- 172.21.15.16:0/2877294470 learned_addr learned my addr 172.21.15.16:0/2877294470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.549+0000 7ff03489c640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 0x7ff01c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x556493011b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff0267fc640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x55649301ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 0x7ff01c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 -- 172.21.15.16:0/2877294470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 msgr2=0x55649301ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x55649301ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 -- 172.21.15.16:0/2877294470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 msgr2=0x556493011b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x556493011b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff03489c640 1 -- 172.21.15.16:0/4208393752 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ff0180cef80 con 0x7ff030102790 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 -- 172.21.15.16:0/2877294470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556492ce8b90 con 0x556493012080 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff0267fc640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x55649301ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x556493011b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02effd640 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff020033000 tx=0x7ff020032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad2ae5c1021e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 0x7ff01c0a51b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff028009fd0 tx=0x7ff028005bd0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff025ffb640 1 -- 172.21.15.16:0/2877294470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02003f070 con 0x556493012080 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff025ffb640 1 -- 172.21.15.16:0/2877294470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0200317b0 con 0x556493012080 2024-09-12T01:09:45.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.550+0000 7ff025ffb640 1 -- 172.21.15.16:0/2877294470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020035070 con 0x556493012080 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- 172.21.15.16:0/2877294470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 msgr2=0x55649301c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff020033000 tx=0x7ff020032cb0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- 172.21.15.16:0/2877294470 shutdown_connections 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x55649301ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649301c400 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 --2- 172.21.15.16:0/2877294470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x556493011b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- 172.21.15.16:0/2877294470 >> 172.21.15.16:0/2877294470 conn(0x55649300dcf0 msgr2=0x55649300e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- 172.21.15.16:0/2877294470 shutdown_connections 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- 172.21.15.16:0/2877294470 wait complete. 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.551+0000 7ff037c77740 1 -- start start 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 Processor -- start 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493019830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493019d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55649301f510 con 0x55649301c940 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556492eff6e0 con 0x556493012080 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55649301f690 con 0x556493011760 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff037c77740 10 client.0 _ll_register_callbacks cb 0x556492f23a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46494/0 (socket says 172.21.15.16:46494) 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 -- 172.21.15.16:0/4267207849 learned_addr learned my addr 172.21.15.16:0/4267207849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff0267fc640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493019d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02effd640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493019830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 -- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 msgr2=0x556493019830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493019830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 -- 172.21.15.16:0/4267207849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 msgr2=0x556493019d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493019d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.552+0000 7ff02f7fe640 1 -- 172.21.15.16:0/4267207849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556493028a00 con 0x55649301c940 2024-09-12T01:09:45.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.553+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff028010a20 tx=0x7ff028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0658bbaf415be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.553+0000 7ff00effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:45.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.553+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02800fb50 con 0x55649301c940 2024-09-12T01:09:45.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ff02800fcf0 con 0x55649301c940 2024-09-12T01:09:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0200317b0 con 0x55649301c940 2024-09-12T01:09:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff028027400 con 0x55649301c940 2024-09-12T01:09:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 10 client.19659 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55649302b3b0 con 0x55649301c940 2024-09-12T01:09:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0200317b0 con 0x55649301c940 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff028030830 con 0x55649301c940 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 10 client.19659 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 10 client.19659 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 20 client.19659 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649302c7c0 0x55649302ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 4 client.19659 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556492f849d0 con 0x55649302c7c0 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.554+0000 7ff02effd640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649302c7c0 0x55649302ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.555+0000 7ff02effd640 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649302c7c0 0x55649302ebd0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5feb27438f128ea5 server_cookie=5ec9e856b84315fa in_seq=0 out_seq=0 2024-09-12T01:09:45.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.555+0000 7ff00effd640 10 client.19659 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff00effd640 1 -- 172.21.15.16:0/4267207849 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556492f849d0 con 0x55649302c7c0 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff00effd640 10 client.19659 handle_command_reply: tid=0 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 client.19659 shutdown 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 msgr2=0x55649301a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff028010a20 tx=0x7ff028004a40 comp rx=0 tx=0).stop 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 shutdown_connections 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649302c7c0 0x55649302ebd0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x55649301a2b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493019d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 --2- 172.21.15.16:0/4267207849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493019830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff00effd640 0 client.19659 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 >> 172.21.15.16:0/4267207849 conn(0x55649300dcf0 msgr2=0x55649300ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 shutdown_connections 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 1 -- 172.21.15.16:0/4267207849 wait complete. 2024-09-12T01:09:45.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.556+0000 7ff037c77740 20 client.19659 trim_cache size 0 max 16384 2024-09-12T01:09:45.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.573+0000 7ff037c77740 1 Processor -- start 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.573+0000 7ff037c77740 1 -- start start 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.573+0000 7ff037c77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x5564930141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.573+0000 7ff037c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.573+0000 7ff037c77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x5564930344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff037c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301ad7a0 con 0x556493012080 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff037c77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301af2e0 con 0x556493011760 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff037c77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301ade70 con 0x55649301c940 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46502/0 (socket says 172.21.15.16:46502) 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 -- 172.21.15.16:0/1238997638 learned_addr learned my addr 172.21.15.16:0/1238997638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff02effd640 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x5564930141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x5564930344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 -- 172.21.15.16:0/1238997638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 msgr2=0x5564930344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x5564930344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 -- 172.21.15.16:0/1238997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 msgr2=0x5564930141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x5564930141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 -- 172.21.15.16:0/1238997638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556493009260 con 0x556493012080 2024-09-12T01:09:45.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0267fc640 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cc730 tx=0x7ff01806e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ed0f9971835b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.574+0000 7ff0257fa640 1 -- 172.21.15.16:0/1238997638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018020040 con 0x556493012080 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff0257fa640 1 -- 172.21.15.16:0/1238997638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018066530 con 0x556493012080 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff0257fa640 1 -- 172.21.15.16:0/1238997638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018099830 con 0x556493012080 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 -- 172.21.15.16:0/1238997638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 msgr2=0x55649300cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff0180cc730 tx=0x7ff01806e510 comp rx=0 tx=0).stop 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 -- 172.21.15.16:0/1238997638 shutdown_connections 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55649301c940 0x5564930344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556493012080 0x55649300cd60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 --2- 172.21.15.16:0/1238997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493011760 0x5564930141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 -- 172.21.15.16:0/1238997638 >> 172.21.15.16:0/1238997638 conn(0x55649300dcf0 msgr2=0x55649300c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 -- 172.21.15.16:0/1238997638 shutdown_connections 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.575+0000 7ff037c77740 1 -- 172.21.15.16:0/1238997638 wait complete. 2024-09-12T01:09:45.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 -- start start 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 Processor -- start 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493024ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493025000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301aec10 con 0x55649301c940 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301ae540 con 0x556493012080 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff037c77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0301ab7a0 con 0x556493011760 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.576+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff037c77740 10 client.0 _ll_register_callbacks cb 0x7ff028010670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46514/0 (socket says 172.21.15.16:46514) 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 -- 172.21.15.16:0/2562075079 learned_addr learned my addr 172.21.15.16:0/2562075079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02effd640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493024ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff0267fc640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493025000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 -- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 msgr2=0x556493024ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493024ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 -- 172.21.15.16:0/2562075079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 msgr2=0x556493025000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493025000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 -- 172.21.15.16:0/2562075079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55649300e980 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff02f7fe640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff028030c80 tx=0x7ff028012220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ce71f8e9ffc14e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff00effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02800f790 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7ff028030690 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.577+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018066530 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0280308d0 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 10 client.19665 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55649309c1d0 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018066530 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff028043030 con 0x55649301c940 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 10 client.19665 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 10 client.19665 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 20 client.19665 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649309c9c0 0x55649309ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 4 client.19665 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x556492f7ed20 con 0x55649309c9c0 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.578+0000 7ff02effd640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649309c9c0 0x55649309ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.579+0000 7ff00effd640 10 client.19665 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.579+0000 7ff02effd640 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649309c9c0 0x55649309ede0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535b5b89d8416ffe server_cookie=4080325c3ed4eb5a in_seq=0 out_seq=0 2024-09-12T01:09:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff00effd640 1 -- 172.21.15.16:0/2562075079 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556492f7ed20 con 0x55649309c9c0 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff00effd640 10 client.19665 handle_command_reply: tid=0 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff037c77740 1 client.19665 shutdown 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 msgr2=0x5564930218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff028030c80 tx=0x7ff028012220 comp rx=0 tx=0).stop 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.591+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 shutdown_connections 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55649309c9c0 0x55649309ede0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55649301c940 0x5564930218b0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556493012080 0x556493025000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff00effd640 0 client.19665 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 --2- 172.21.15.16:0/2562075079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556493011760 0x556493024ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 >> 172.21.15.16:0/2562075079 conn(0x55649300dcf0 msgr2=0x55649301a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 shutdown_connections 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 1 -- 172.21.15.16:0/2562075079 wait complete. 2024-09-12T01:09:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff037c77740 20 client.19665 trim_cache size 0 max 16384 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 50, 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "dadf246f": { 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 50, 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:09:45.592 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:45.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 msgr2=0x7ff01c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 0x7ff01c0a51b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff028009fd0 tx=0x7ff028005bd0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 msgr2=0x7ff0301989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.592+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff01800d770 tx=0x7ff01800dc30 comp rx=0 tx=0).stop 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 shutdown_connections 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff01c0a2cf0 0x7ff01c0a51b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff030106f10 0x7ff03019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff030102790 0x7ff0301989f0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 --2- 172.21.15.16:0/4208393752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff030101de0 0x7ff0301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 >> 172.21.15.16:0/4208393752 conn(0x7ff0300fd8c0 msgr2=0x7ff0300fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 shutdown_connections 2024-09-12T01:09:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:45.593+0000 7ff03589e640 1 -- 172.21.15.16:0/4208393752 wait complete. 2024-09-12T01:09:46.001 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:09:46.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:09:46.434 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dadf246f', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-12T01:09:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 -- 172.21.15.16:0/4092771928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10810c2f0 msgr2=0x7fd10810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/4092771928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10810c2f0 0x7fd10810e790 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd100009900 tx=0x7fd10002a410 comp rx=0 tx=0).stop 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 -- 172.21.15.16:0/4092771928 shutdown_connections 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/4092771928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10810c2f0 0x7fd10810e790 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/4092771928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1081037a0 0x7fd10810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/4092771928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd108102df0 0x7fd1081031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 -- 172.21.15.16:0/4092771928 >> 172.21.15.16:0/4092771928 conn(0x7fd1080fc7d0 msgr2=0x7fd1080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 -- 172.21.15.16:0/4092771928 shutdown_connections 2024-09-12T01:09:46.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 -- 172.21.15.16:0/4092771928 wait complete. 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.973+0000 7fd10d4cc640 1 Processor -- start 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 -- start start 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1081037a0 0x7fd108100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd10810c2f0 0x7fd1081014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd108112170 con 0x7fd108102df0 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd108111ff0 con 0x7fd10810c2f0 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081122f0 con 0x7fd1081037a0 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46532/0 (socket says 172.21.15.16:46532) 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 -- 172.21.15.16:0/654565998 learned_addr learned my addr 172.21.15.16:0/654565998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd1077fe640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd10810c2f0 0x7fd1081014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 -- 172.21.15.16:0/654565998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1081037a0 msgr2=0x7fd108100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1081037a0 0x7fd108100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 -- 172.21.15.16:0/654565998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd10810c2f0 msgr2=0x7fd1081014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd10810c2f0 0x7fd1081014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 -- 172.21.15.16:0/654565998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1081a6af0 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd106ffd640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800c970 tx=0x7fd0f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd0effff640 1 -- 172.21.15.16:0/654565998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8007bb0 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd0effff640 1 -- 172.21.15.16:0/654565998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f8007d50 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.974+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1081a6de0 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.975+0000 7fd0effff640 1 -- 172.21.15.16:0/654565998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800f450 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.975+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1081a7240 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd0effff640 1 -- 172.21.15.16:0/654565998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd0f8005020 con 0x7fd108102df0 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 Processor -- start 2024-09-12T01:09:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 -- start start 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610fa8a5280 con 0x5610fa93fb60 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5610fa84cb10 con 0x5610fa940480 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.977+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5610fa8a1240 con 0x5610fa94ad40 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46542/0 (socket says 172.21.15.16:46542) 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 -- 172.21.15.16:0/3183056893 learned_addr learned my addr 172.21.15.16:0/3183056893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd106ffd640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1067fc640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd0effff640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 0x7fd0f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 -- 172.21.15.16:0/3183056893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 msgr2=0x5610fa94d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 -- 172.21.15.16:0/3183056893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 msgr2=0x5610fa94a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 -- 172.21.15.16:0/3183056893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fa8591f0 con 0x5610fa93fb60 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd106ffd640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd0effff640 1 -- 172.21.15.16:0/654565998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fd0f80cf2f0 con 0x7fd108102df0 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 0x7fd0f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.978+0000 7fd1077fe640 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd100031680 tx=0x7fd1000316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c44e4526eaf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd0ed7fa640 1 -- 172.21.15.16:0/3183056893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10004d070 con 0x5610fa93fb60 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd0ed7fa640 1 -- 172.21.15.16:0/3183056893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd100003ad0 con 0x5610fa93fb60 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd0ed7fa640 1 -- 172.21.15.16:0/3183056893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100035070 con 0x5610fa93fb60 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 -- 172.21.15.16:0/3183056893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 msgr2=0x5610fa93ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fd100031680 tx=0x7fd1000316b0 comp rx=0 tx=0).stop 2024-09-12T01:09:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd1077fe640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 0x7fd0f00a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd10004a000 tx=0x7fd10004b000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 -- 172.21.15.16:0/3183056893 shutdown_connections 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/3183056893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa93ff40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 -- 172.21.15.16:0/3183056893 >> 172.21.15.16:0/3183056893 conn(0x5610fa93c0f0 msgr2=0x5610fa93c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 -- 172.21.15.16:0/3183056893 shutdown_connections 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.979+0000 7fd10f8a5740 1 -- 172.21.15.16:0/3183056893 wait complete. 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 -- start start 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 Processor -- start 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa947b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa94ad40 0x5610fa948600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5610fa94d910 con 0x5610fa940480 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5610fa85ff10 con 0x5610fa94ad40 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5610fa94da90 con 0x5610fa93fb60 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd10f8a5740 10 client.0 _ll_register_callbacks cb 0x5610fa828a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd1067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd1067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46544/0 (socket says 172.21.15.16:46544) 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd1067fc640 1 -- 172.21.15.16:0/2004827259 learned_addr learned my addr 172.21.15.16:0/2004827259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.980+0000 7fd106ffd640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa947b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1077fe640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa94ad40 0x5610fa948600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 -- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 msgr2=0x5610fa947b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa947b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 -- 172.21.15.16:0/2004827259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa94ad40 msgr2=0x5610fa948600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa94ad40 0x5610fa948600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 -- 172.21.15.16:0/2004827259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fa6166a0 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd1067fc640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00aa20 tx=0x7fd0fc00aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1cea85a061e178 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd0eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc018020 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fd0fc00eb70 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd100003ad0 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc023050 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd10f8a5740 10 client.19677 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610fa9594e0 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.981+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd100003ad0 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0fc01c3a0 con 0x5610fa940480 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 10 client.19677 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 10 client.19677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 20 client.19677 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa95aae0 0x5610fa95ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 4 client.19677 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610fa8b27f0 con 0x5610fa95aae0 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd106ffd640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa95aae0 0x5610fa95ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd106ffd640 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa95aae0 0x5610fa95ced0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d04df5dc1b6bf20 server_cookie=708dd6391dc3f6 in_seq=0 out_seq=0 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.982+0000 7fd0eaff5640 10 client.19677 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd0eaff5640 1 -- 172.21.15.16:0/2004827259 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610fa8b27f0 con 0x5610fa95aae0 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd0eaff5640 10 client.19677 handle_command_reply: tid=0 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 client.19677 shutdown 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 msgr2=0x5610fa9480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00aa20 tx=0x7fd0fc00aee0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 shutdown_connections 2024-09-12T01:09:47.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa95aae0 0x5610fa95ced0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa94ad40 0x5610fa948600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa940480 0x5610fa9480c0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd0eaff5640 0 client.19677 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/2004827259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa947b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 >> 172.21.15.16:0/2004827259 conn(0x5610fa93c0f0 msgr2=0x5610fa93b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 shutdown_connections 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 1 -- 172.21.15.16:0/2004827259 wait complete. 2024-09-12T01:09:47.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:46.983+0000 7fd10f8a5740 20 client.19677 trim_cache size 0 max 16384 2024-09-12T01:09:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 Processor -- start 2024-09-12T01:09:47.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 -- start start 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa9499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa94ad40 0x5610fa962790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081afaf0 con 0x5610fa94ad40 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081b1630 con 0x5610fa940480 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081b01c0 con 0x5610fa93fb60 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd1077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa94ad40 0x5610fa962790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd1067fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa9499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:43620/0 (socket says 172.21.15.16:43620) 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.002+0000 7fd106ffd640 1 -- 172.21.15.16:0/1439719824 learned_addr learned my addr 172.21.15.16:0/1439719824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 -- 172.21.15.16:0/1439719824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 msgr2=0x5610fa9499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa9499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 -- 172.21.15.16:0/1439719824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa94ad40 msgr2=0x5610fa962790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa94ad40 0x5610fa962790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 -- 172.21.15.16:0/1439719824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fa93bf20 con 0x5610fa93fb60 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd1067fc640 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa9499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd106ffd640 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80981e0 tx=0x7fd0f80d1800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912006535680ef0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd0ecff9640 1 -- 172.21.15.16:0/1439719824 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f801d040 con 0x5610fa93fb60 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd0ecff9640 1 -- 172.21.15.16:0/1439719824 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f809bf10 con 0x5610fa93fb60 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.003+0000 7fd0ecff9640 1 -- 172.21.15.16:0/1439719824 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8099940 con 0x5610fa93fb60 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- 172.21.15.16:0/1439719824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 msgr2=0x5610fa942610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80981e0 tx=0x7fd0f80d1800 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- 172.21.15.16:0/1439719824 shutdown_connections 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa94ad40 0x5610fa962790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa9499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/1439719824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa93fb60 0x5610fa942610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- 172.21.15.16:0/1439719824 >> 172.21.15.16:0/1439719824 conn(0x5610fa93c0f0 msgr2=0x5610fa9492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- 172.21.15.16:0/1439719824 shutdown_connections 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- 172.21.15.16:0/1439719824 wait complete. 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.004+0000 7fd10f8a5740 1 -- start start 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 Processor -- start 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081b0f60 con 0x5610fa93fb60 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081b0890 con 0x5610fa940480 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1081adaf0 con 0x5610fa94ad40 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46570/0 (socket says 172.21.15.16:46570) 2024-09-12T01:09:47.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 -- 172.21.15.16:0/4188590837 learned_addr learned my addr 172.21.15.16:0/4188590837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd10f8a5740 10 client.0 _ll_register_callbacks cb 0x7fd0fc0098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd1067fc640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 -- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 msgr2=0x5610fa94f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 -- 172.21.15.16:0/4188590837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 msgr2=0x5610fa94f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 -- 172.21.15.16:0/4188590837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fa936680 con 0x5610fa93fb60 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd1067fc640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd106ffd640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809b510 tx=0x7fd0f80d2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6578c2de20d8c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.005+0000 7fd0eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f801d070 con 0x5610fa93fb60 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fd0f8097e80 con 0x5610fa93fb60 2024-09-12T01:09:47.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f809bf10 con 0x5610fa93fb60 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0f809fbc0 con 0x5610fa93fb60 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 10 client.19683 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610fa9c5a70 con 0x5610fa93fb60 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f809bf10 con 0x5610fa93fb60 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0f8005310 con 0x5610fa93fb60 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 10 client.19683 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 10 client.19683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 20 client.19683 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa9c60e0 0x5610fa9c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 4 client.19683 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.006+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5610fa8acd00 con 0x5610fa9c60e0 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.007+0000 7fd1067fc640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa9c60e0 0x5610fa9c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.007+0000 7fd1067fc640 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa9c60e0 0x5610fa9c8510 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db6ae3a71f8a951 server_cookie=935ee4b1a8775284 in_seq=0 out_seq=0 2024-09-12T01:09:47.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.007+0000 7fd0eaff5640 10 client.19683 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.008+0000 7fd0eaff5640 1 -- 172.21.15.16:0/4188590837 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5610fa8acd00 con 0x5610fa9c60e0 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.008+0000 7fd0eaff5640 10 client.19683 handle_command_reply: tid=0 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.008+0000 7fd10f8a5740 1 client.19683 shutdown 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 msgr2=0x5610fa94ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fd0f809b510 tx=0x7fd0f80d2e40 comp rx=0 tx=0).stop 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 shutdown_connections 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5610fa9c60e0 0x5610fa9c8510 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610fa94ad40 0x5610fa94f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5610fa940480 0x5610fa94f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 --2- 172.21.15.16:0/4188590837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610fa93fb60 0x5610fa94ec30 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 >> 172.21.15.16:0/4188590837 conn(0x5610fa93c0f0 msgr2=0x5610fa93b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd0eaff5640 0 client.19683 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 shutdown_connections 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 1 -- 172.21.15.16:0/4188590837 wait complete. 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.009+0000 7fd10f8a5740 20 client.19683 trim_cache size 0 max 16384 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d462f3d3-1b6c-4635-a20c-de6049649f76", 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:09:47.015 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 msgr2=0x7fd0f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 0x7fd0f00a51b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd10004a000 tx=0x7fd10004b000 comp rx=0 tx=0).stop 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 msgr2=0x7fd108100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800c970 tx=0x7fd0f800ce30 comp rx=0 tx=0).stop 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 shutdown_connections 2024-09-12T01:09:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd0f00a2cf0 0x7fd0f00a51b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd10810c2f0 0x7fd1081014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1081037a0 0x7fd108100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 --2- 172.21.15.16:0/654565998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd108102df0 0x7fd108100a50 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 >> 172.21.15.16:0/654565998 conn(0x7fd1080fc7d0 msgr2=0x7fd1080fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 shutdown_connections 2024-09-12T01:09:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:47.010+0000 7fd10d4cc640 1 -- 172.21.15.16:0/654565998 wait complete. 2024-09-12T01:09:47.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:46 smithi099 ceph-mon[35654]: pgmap v426: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:47.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:46 smithi159 ceph-mon[35566]: pgmap v426: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:47.381 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:46 smithi016 ceph-mon[30972]: pgmap v426: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 19 op/s 2024-09-12T01:09:47.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:09:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:09:48.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:48 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:09:48.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:48 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:48 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:09:48.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:48 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:48 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:09:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:48 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:48 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:09:48.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:48 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:48 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:09:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:48 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:48 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:09:48.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:48 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 -- 172.21.15.16:0/2920853558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0106f10 msgr2=0x7fb4b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 --2- 172.21.15.16:0/2920853558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0106f10 0x7fb4b010ea90 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009ca0 tx=0x7fb4a802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 -- 172.21.15.16:0/2920853558 shutdown_connections 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 --2- 172.21.15.16:0/2920853558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0106f10 0x7fb4b010ea90 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 --2- 172.21.15.16:0/2920853558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0102790 0x7fb4b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 --2- 172.21.15.16:0/2920853558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0101de0 0x7fb4b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 -- 172.21.15.16:0/2920853558 >> 172.21.15.16:0/2920853558 conn(0x7fb4b00fd8c0 msgr2=0x7fb4b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 -- 172.21.15.16:0/2920853558 shutdown_connections 2024-09-12T01:09:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.606+0000 7fb4b835e640 1 -- 172.21.15.16:0/2920853558 wait complete. 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 Processor -- start 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 -- start start 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 0x7fb4b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0106f10 0x7fb4b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0111f70 con 0x7fb4b0102790 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0111df0 con 0x7fb4b0101de0 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b835e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01120f0 con 0x7fb4b0106f10 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44606/0 (socket says 172.21.15.16:44606) 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b58f9640 1 -- 172.21.15.16:0/3147009785 learned_addr learned my addr 172.21.15.16:0/3147009785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0106f10 0x7fb4b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 0x7fb4b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b58f9640 1 -- 172.21.15.16:0/3147009785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0106f10 msgr2=0x7fb4b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.607+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0106f10 0x7fb4b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b58f9640 1 -- 172.21.15.16:0/3147009785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 msgr2=0x7fb4b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 0x7fb4b00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b58f9640 1 -- 172.21.15.16:0/3147009785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b01a6890 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 0x7fb4b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d950 tx=0x7fb4a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4a77fe640 1 -- 172.21.15.16:0/3147009785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0017070 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4a77fe640 1 -- 172.21.15.16:0/3147009785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a0009c30 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b01a6b80 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4a77fe640 1 -- 172.21.15.16:0/3147009785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000f6b0 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.608+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b01a70c0 con 0x7fb4b0102790 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 Processor -- start 2024-09-12T01:09:48.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4a77fe640 1 -- 172.21.15.16:0/3147009785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4a00109d0 con 0x7fb4b0102790 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 -- start start 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c7192d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c71938140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c718932e0 con 0x558c71938680 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558c7183ace0 con 0x558c7192d4a0 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.611+0000 7fb4ba737740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558c7188f2a0 con 0x558c7192ddc0 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44608/0 (socket says 172.21.15.16:44608) 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 -- 172.21.15.16:0/2306095979 learned_addr learned my addr 172.21.15.16:0/2306095979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c7192d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c71938140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 -- 172.21.15.16:0/2306095979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 msgr2=0x558c71938140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c71938140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 -- 172.21.15.16:0/2306095979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 msgr2=0x558c7192d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c7192d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 -- 172.21.15.16:0/2306095979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7181f440 con 0x558c71938680 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4a77fe640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 0x7fb49c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 0x7fb49c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4a77fe640 1 -- 172.21.15.16:0/3147009785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb4a00cf0a0 con 0x7fb4b0102790 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.612+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00cc760 tx=0x7fb4a00d7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee73d81235221d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4a4dea640 1 -- 172.21.15.16:0/2306095979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0020030 con 0x558c71938680 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4a4dea640 1 -- 172.21.15.16:0/2306095979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a009bcc0 con 0x558c71938680 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4a4dea640 1 -- 172.21.15.16:0/2306095979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a009b390 con 0x558c71938680 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 -- 172.21.15.16:0/2306095979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 msgr2=0x558c7193aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 0x7fb49c0a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005e30 tx=0x7fb4a8005c30 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00cc760 tx=0x7fb4a00d7910 comp rx=0 tx=0).stop 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 -- 172.21.15.16:0/2306095979 shutdown_connections 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7193aa70 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c71938140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.613+0000 7fb4ba737740 1 --2- 172.21.15.16:0/2306095979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c7192d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- 172.21.15.16:0/2306095979 >> 172.21.15.16:0/2306095979 conn(0x558c71929a30 msgr2=0x558c71929e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- 172.21.15.16:0/2306095979 shutdown_connections 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- 172.21.15.16:0/2306095979 wait complete. 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- start start 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 Processor -- start 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192d4a0 0x558c71937ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c71935800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c7193b250 con 0x558c7192ddc0 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558c7184e050 con 0x558c71938680 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558c7193b3d0 con 0x558c7192d4a0 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4ba737740 10 client.0 _ll_register_callbacks cb 0x558c7188e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.614+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44612/0 (socket says 172.21.15.16:44612) 2024-09-12T01:09:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 -- 172.21.15.16:0/169011680 learned_addr learned my addr 172.21.15.16:0/169011680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192d4a0 0x558c71937ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c71935800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 -- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192d4a0 msgr2=0x558c71937ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192d4a0 0x558c71937ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 -- 172.21.15.16:0/169011680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 msgr2=0x558c71935800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c71935800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 -- 172.21.15.16:0/169011680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c716043c0 con 0x558c7192ddc0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c71935800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00979d0 tx=0x7fb4a00ccd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5626b7bab75fb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0020070 con 0x558c7192ddc0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb4a009b820 con 0x558c7192ddc0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a009bcc0 con 0x558c7192ddc0 2024-09-12T01:09:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4a009af20 con 0x558c7192ddc0 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4ba737740 10 client.19695 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.615+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c7193b250 con 0x558c7192ddc0 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a009bcc0 con 0x558c7192ddc0 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a00a3da0 con 0x558c7192ddc0 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 10 client.19695 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 20 client.19695 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c71947dd0 0x558c7194a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 4 client.19695 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c718a0850 con 0x558c71947dd0 2024-09-12T01:09:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.616+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c71947dd0 0x558c7194a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.617+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c71947dd0 0x558c7194a1c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f50b94bdc2f03f server_cookie=79d71d9e4c58708a in_seq=0 out_seq=0 2024-09-12T01:09:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.617+0000 7fb499ffb640 10 client.19695 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.617+0000 7fb499ffb640 1 -- 172.21.15.16:0/169011680 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c718a0850 con 0x558c71947dd0 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb499ffb640 10 client.19695 handle_command_reply: tid=0 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 client.19695 shutdown 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 msgr2=0x558c719352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00979d0 tx=0x7fb4a00ccd00 comp rx=0 tx=0).stop 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 shutdown_connections 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c71947dd0 0x558c7194a1c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c71935800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192ddc0 0x558c719352a0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 --2- 172.21.15.16:0/169011680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192d4a0 0x558c71937ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 >> 172.21.15.16:0/169011680 conn(0x558c71929a30 msgr2=0x558c7192f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb499ffb640 0 client.19695 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 shutdown_connections 2024-09-12T01:09:48.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 1 -- 172.21.15.16:0/169011680 wait complete. 2024-09-12T01:09:48.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.618+0000 7fb4ba737740 20 client.19695 trim_cache size 0 max 16384 2024-09-12T01:09:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 Processor -- start 2024-09-12T01:09:48.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 -- start start 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c719444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7194f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01afaf0 con 0x558c71938680 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01b1630 con 0x558c7192d4a0 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4ba737740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01b01c0 con 0x558c7192ddc0 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.635+0000 7fb4b68fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44628/0 (socket says 172.21.15.16:44628) 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 -- 172.21.15.16:0/3537734824 learned_addr learned my addr 172.21.15.16:0/3537734824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c719444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7194f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 -- 172.21.15.16:0/3537734824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 msgr2=0x558c7194f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7194f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 -- 172.21.15.16:0/3537734824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 msgr2=0x558c719444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c719444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 -- 172.21.15.16:0/3537734824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c71935b70 con 0x558c71938680 2024-09-12T01:09:48.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb4a802ade0 tx=0x7fb4a8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91ba1f8a5e30ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb49bfff640 1 -- 172.21.15.16:0/3537734824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8003e00 con 0x558c71938680 2024-09-12T01:09:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb49bfff640 1 -- 172.21.15.16:0/3537734824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8036040 con 0x558c71938680 2024-09-12T01:09:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.636+0000 7fb49bfff640 1 -- 172.21.15.16:0/3537734824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a802fbc0 con 0x558c71938680 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- 172.21.15.16:0/3537734824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 msgr2=0x558c7194fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb4a802ade0 tx=0x7fb4a8040000 comp rx=0 tx=0).stop 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- 172.21.15.16:0/3537734824 shutdown_connections 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c71938680 0x558c7194fd90 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7194f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 --2- 172.21.15.16:0/3537734824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c7192d4a0 0x558c719444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- 172.21.15.16:0/3537734824 >> 172.21.15.16:0/3537734824 conn(0x558c71929a30 msgr2=0x558c71930900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- 172.21.15.16:0/3537734824 shutdown_connections 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- 172.21.15.16:0/3537734824 wait complete. 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 -- start start 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.637+0000 7fb4ba737740 1 Processor -- start 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7193bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c7193c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01b0f60 con 0x558c7192d4a0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01b0890 con 0x558c71938680 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b01adaf0 con 0x558c7192ddc0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4ba737740 10 client.0 _ll_register_callbacks cb 0x7fb4a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44632/0 (socket says 172.21.15.16:44632) 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 -- 172.21.15.16:0/466971672 learned_addr learned my addr 172.21.15.16:0/466971672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b68fb640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c7193c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7193bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 -- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 msgr2=0x558c7193bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7193bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 -- 172.21.15.16:0/466971672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 msgr2=0x558c7193c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c7193c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 -- 172.21.15.16:0/466971672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7193cb30 con 0x558c7192d4a0 2024-09-12T01:09:48.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb4b60fa640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x558c7194a7f0 tx=0x7fb4ac014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2453772339cac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.638+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac005230 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a8036040 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8521+0+0 (secure 0 0 0) 0x7fb4ac0053d0 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4ac0055d0 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 10 client.19701 fetch_fsmap learned FSMap version 28 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c719b2700 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8036040 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4ac022020 con 0x558c7192d4a0 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 10 client.19701 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 10 client.19701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 20 client.19701 populate_metadata read hostname 'smithi016' 2024-09-12T01:09:48.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c719b2dd0 0x558c719b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:09:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 4 client.19701 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:09:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c7189ad60 con 0x558c719b2dd0 2024-09-12T01:09:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.639+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c719b2dd0 0x558c719b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:09:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4b58f9640 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c719b2dd0 0x558c719b51c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c26bb70755961f server_cookie=796775c3092c9824 in_seq=0 out_seq=0 2024-09-12T01:09:48.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb499ffb640 10 client.19701 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb499ffb640 1 -- 172.21.15.16:0/466971672 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c7189ad60 con 0x558c719b2dd0 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb499ffb640 10 client.19701 handle_command_reply: tid=0 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 client.19701 shutdown 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 msgr2=0x558c71940c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x558c7194a7f0 tx=0x7fb4ac014840 comp rx=0 tx=0).stop 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 shutdown_connections 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558c719b2dd0 0x558c719b51c0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.640+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c71938680 0x558c7193c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c7192ddc0 0x558c7193bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 1 --2- 172.21.15.16:0/466971672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c7192d4a0 0x558c71940c20 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 >> 172.21.15.16:0/466971672 conn(0x558c71929a30 msgr2=0x558c71928350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb499ffb640 0 client.19701 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 shutdown_connections 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 1 -- 172.21.15.16:0/466971672 wait complete. 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4ba737740 20 client.19701 trim_cache size 0 max 16384 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:09:48.644 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "d462f3d3-1b6c-4635-a20c-de6049649f76": { 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d462f3d3-1b6c-4635-a20c-de6049649f76", 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:09:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 msgr2=0x7fb49c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 0x7fb49c0a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8005e30 tx=0x7fb4a8005c30 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 msgr2=0x7fb4b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000d950 tx=0x7fb4a000de10 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 shutdown_connections 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb49c0a2cf0 0x7fb49c0a51b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b0106f10 0x7fb4b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4b0102790 0x7fb4b0074a30 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 --2- 172.21.15.16:0/3147009785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4b0101de0 0x7fb4b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 >> 172.21.15.16:0/3147009785 conn(0x7fb4b00fd8c0 msgr2=0x7fb4b00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 shutdown_connections 2024-09-12T01:09:48.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:09:48.641+0000 7fb4b835e640 1 -- 172.21.15.16:0/3147009785 wait complete. 2024-09-12T01:09:49.270 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:49 smithi016 ceph-mon[30972]: pgmap v427: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:49.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:49 smithi099 ceph-mon[35654]: pgmap v427: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:49.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:49 smithi159 ceph-mon[35566]: pgmap v427: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 13 op/s 2024-09-12T01:09:49.502 INFO:tasks.cephfs.filesystem:scrub status for tag:d462f3d3-1b6c-4635-a20c-de6049649f76 - {'path': '/', 'tag': 'd462f3d3-1b6c-4635-a20c-de6049649f76', 'options': 'recursive,force'} 2024-09-12T01:09:50.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:50 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:09:50.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:50 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:09:50.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:50 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:09:51.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:51 smithi016 ceph-mon[30972]: pgmap v428: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-09-12T01:09:51.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:51 smithi099 ceph-mon[35654]: pgmap v428: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-09-12T01:09:51.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:51 smithi159 ceph-mon[35566]: pgmap v428: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 66 KiB/s wr, 18 op/s 2024-09-12T01:09:53.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:53 smithi099 ceph-mon[35654]: pgmap v429: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:53.443 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:53 smithi016 ceph-mon[30972]: pgmap v429: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:53.443 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:53 smithi159 ceph-mon[35566]: pgmap v429: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:55.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:55 smithi099 ceph-mon[35654]: pgmap v430: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:55.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:55 smithi159 ceph-mon[35566]: pgmap v430: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:55.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:55 smithi016 ceph-mon[30972]: pgmap v430: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 11 op/s 2024-09-12T01:09:57.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:57 smithi099 ceph-mon[35654]: pgmap v431: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-12T01:09:57.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:57 smithi159 ceph-mon[35566]: pgmap v431: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-12T01:09:57.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:57 smithi016 ceph-mon[30972]: pgmap v431: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-09-12T01:09:59.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:09:59 smithi099 ceph-mon[35654]: pgmap v432: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-12T01:09:59.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:09:59 smithi159 ceph-mon[35566]: pgmap v432: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-12T01:09:59.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:09:59 smithi016 ceph-mon[30972]: pgmap v432: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 9 op/s 2024-09-12T01:10:00.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:00 smithi159 ceph-mon[35566]: overall HEALTH_OK 2024-09-12T01:10:00.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:00 smithi016 ceph-mon[30972]: overall HEALTH_OK 2024-09-12T01:10:00.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:00 smithi099 ceph-mon[35654]: overall HEALTH_OK 2024-09-12T01:10:01.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:01 smithi016 ceph-mon[30972]: pgmap v433: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T01:10:01.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:01 smithi159 ceph-mon[35566]: pgmap v433: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T01:10:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:01 smithi099 ceph-mon[35654]: pgmap v433: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 13 KiB/s wr, 14 op/s 2024-09-12T01:10:03.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:03 smithi159 ceph-mon[35566]: pgmap v434: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:03.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:03 smithi016 ceph-mon[30972]: pgmap v434: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:03 smithi099 ceph-mon[35654]: pgmap v434: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:05.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:05 smithi159 ceph-mon[35566]: pgmap v435: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:05.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:05 smithi016 ceph-mon[30972]: pgmap v435: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:05.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:05 smithi099 ceph-mon[35654]: pgmap v435: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.8 KiB/s wr, 8 op/s 2024-09-12T01:10:07.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:07 smithi159 ceph-mon[35566]: pgmap v436: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-09-12T01:10:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:07 smithi016 ceph-mon[30972]: pgmap v436: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-09-12T01:10:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:07 smithi099 ceph-mon[35654]: pgmap v436: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-09-12T01:10:09.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:09 smithi159 ceph-mon[35566]: pgmap v437: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T01:10:09.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:09 smithi016 ceph-mon[30972]: pgmap v437: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T01:10:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:09 smithi099 ceph-mon[35654]: pgmap v437: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 49 KiB/s wr, 25 op/s 2024-09-12T01:10:10.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:10:10.910 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:11.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:11 smithi016 ceph-mon[30972]: pgmap v438: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-12T01:10:11.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:11 smithi016 ceph-mon[30972]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:10:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:11 smithi159 ceph-mon[35566]: pgmap v438: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-12T01:10:11.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:11 smithi159 ceph-mon[35566]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/604600910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 msgr2=0x7fd6a010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/604600910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a010ec50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fd698009cd0 tx=0x7fd69802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/604600910 shutdown_connections 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/604600910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a010ec50 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/604600910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0103760 0x7fd6a0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/604600910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0102db0 0x7fd6a0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/604600910 >> 172.21.15.16:0/604600910 conn(0x7fd6a00fc7d0 msgr2=0x7fd6a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/604600910 shutdown_connections 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.399+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/604600910 wait complete. 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 Processor -- start 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 -- start start 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0102db0 0x7fd6a01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 0x7fd6a0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01120b0 con 0x7fd6a01082e0 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0111f30 con 0x7fd6a0103760 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd6a5dd4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0112230 con 0x7fd6a0102db0 2024-09-12T01:10:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59424/0 (socket says 172.21.15.16:59424) 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd69f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0102db0 0x7fd6a01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.400+0000 7fd69ffff640 1 -- 172.21.15.16:0/1812276631 learned_addr learned my addr 172.21.15.16:0/1812276631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69effd640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 0x7fd6a0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 -- 172.21.15.16:0/1812276631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0102db0 msgr2=0x7fd6a01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0102db0 0x7fd6a01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 -- 172.21.15.16:0/1812276631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 msgr2=0x7fd6a0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 0x7fd6a0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 -- 172.21.15.16:0/1812276631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a019d560 con 0x7fd6a01082e0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69effd640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 0x7fd6a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69ffff640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd698005e30 tx=0x7fd698003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698004040 con 0x7fd6a01082e0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a019d7f0 con 0x7fd6a01082e0 2024-09-12T01:10:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6980046c0 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69803a400 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.401+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a5130 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.403+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a0103190 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.405+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd698030630 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.405+0000 7fd69cff9640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 0x7fd6880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.405+0000 7fd69f7fe640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 0x7fd6880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.405+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7fd69802e030 con 0x7fd6a01082e0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.406+0000 7fd69f7fe640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 0x7fd6880a51b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd69000bbe0 tx=0x7fd69000b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.410+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6980bd190 con 0x7fd6a01082e0 2024-09-12T01:10:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:11 smithi099 ceph-mon[35654]: pgmap v438: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-09-12T01:10:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:11 smithi099 ceph-mon[35654]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T01:10:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.628+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a010a2c0 con 0x7fd6a01082e0 2024-09-12T01:10:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.629+0000 7fd69cff9640 1 -- 172.21.15.16:0/1812276631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fd6980bca90 con 0x7fd6a01082e0 2024-09-12T01:10:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:10:11.629 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:10:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 msgr2=0x7fd6880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 0x7fd6880a51b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd69000bbe0 tx=0x7fd69000b390 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 msgr2=0x7fd6a019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fd698005e30 tx=0x7fd698003970 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 shutdown_connections 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd6880a2cf0 0x7fd6880a51b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6a01082e0 0x7fd6a019ce80 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103760 0x7fd6a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 --2- 172.21.15.16:0/1812276631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0102db0 0x7fd6a01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 >> 172.21.15.16:0/1812276631 conn(0x7fd6a00fc7d0 msgr2=0x7fd6a0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 shutdown_connections 2024-09-12T01:10:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:11.632+0000 7fd6a5dd4640 1 -- 172.21.15.16:0/1812276631 wait complete. 2024-09-12T01:10:12.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:12 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1812276631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:12.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:12 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1812276631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:12.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id dadf246f --release --await' 2024-09-12T01:10:12.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id dadf246f --release --await 2024-09-12T01:10:12.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:12 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1812276631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:12.676 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 -- 172.21.15.16:0/1079933078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8106f10 msgr2=0x7f81c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 --2- 172.21.15.16:0/1079933078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8106f10 0x7f81c810ea90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009ca0 tx=0x7f81c002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 -- 172.21.15.16:0/1079933078 shutdown_connections 2024-09-12T01:10:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 --2- 172.21.15.16:0/1079933078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8106f10 0x7f81c810ea90 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 --2- 172.21.15.16:0/1079933078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 --2- 172.21.15.16:0/1079933078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8101de0 0x7f81c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 -- 172.21.15.16:0/1079933078 >> 172.21.15.16:0/1079933078 conn(0x7f81c80fd8c0 msgr2=0x7f81c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.191+0000 7f81d04a2640 1 -- 172.21.15.16:0/1079933078 shutdown_connections 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.192+0000 7f81d04a2640 1 -- 172.21.15.16:0/1079933078 wait complete. 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.192+0000 7f81d04a2640 1 Processor -- start 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.192+0000 7f81d04a2640 1 -- start start 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.192+0000 7f81d04a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.192+0000 7f81d04a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81d04a2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8106f10 0x7f81c8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81d04a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8111fb0 con 0x7f81c8101de0 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81d04a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8111e30 con 0x7f81c8102790 2024-09-12T01:10:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81d04a2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8112130 con 0x7f81c8106f10 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59432/0 (socket says 172.21.15.16:59432) 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 -- 172.21.15.16:0/3378501464 learned_addr learned my addr 172.21.15.16:0/3378501464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81cda3d640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81cea3f640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8106f10 0x7f81c8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 -- 172.21.15.16:0/3378501464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8106f10 msgr2=0x7f81c8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8106f10 0x7f81c8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 -- 172.21.15.16:0/3378501464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 msgr2=0x7f81c8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 -- 172.21.15.16:0/3378501464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c81a6910 con 0x7f81c8101de0 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81ce23e640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f81c400d770 tx=0x7f81c400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81bf7fe640 1 -- 172.21.15.16:0/3378501464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4017070 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81bf7fe640 1 -- 172.21.15.16:0/3378501464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c4009820 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81bf7fe640 1 -- 172.21.15.16:0/3378501464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c4004cd0 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.193+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81a6c00 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.194+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a70f0 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.196+0000 7f81bf7fe640 1 -- 172.21.15.16:0/3378501464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f81c40099d0 con 0x7f81c8101de0 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 Processor -- start 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 -- start start 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81bf7fe640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 0x7f81b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 0x55e69201d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 0x55e692027d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e691f82620 con 0x55e69201d060 2024-09-12T01:10:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e691f29e50 con 0x55e69201d980 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81d287b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e691f7e5e0 con 0x55e692028240 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81cea3f640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 0x7f81b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81ce23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 0x55e69201d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81bf7fe640 1 -- 172.21.15.16:0/3378501464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f81c40cee30 con 0x7f81c8101de0 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.197+0000 7f81ce23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 0x55e69201d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59440/0 (socket says 172.21.15.16:59440) 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81ce23e640 1 -- 172.21.15.16:0/2478876693 learned_addr learned my addr 172.21.15.16:0/2478876693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 0x55e692027d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cea3f640 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 -- 172.21.15.16:0/2478876693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 msgr2=0x55e69202a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 -- 172.21.15.16:0/2478876693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 msgr2=0x55e69201d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 0x55e69201d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 -- 172.21.15.16:0/2478876693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e691f1ad40 con 0x55e69201d980 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cea3f640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 0x7f81b40a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81c0004af0 tx=0x7f81c0031d70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 0x55e692027d00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009970 tx=0x7f81b8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa052769c8ab1c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81bcff9640 1 -- 172.21.15.16:0/2478876693 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8017070 con 0x55e69201d980 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81bcff9640 1 -- 172.21.15.16:0/2478876693 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b8013470 con 0x55e69201d980 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.198+0000 7f81bcff9640 1 -- 172.21.15.16:0/2478876693 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8012630 con 0x55e69201d980 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- 172.21.15.16:0/2478876693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 msgr2=0x55e692027d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 0x55e692027d00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009970 tx=0x7f81b8009e30 comp rx=0 tx=0).stop 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- 172.21.15.16:0/2478876693 shutdown_connections 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d980 0x55e692027d00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 --2- 172.21.15.16:0/2478876693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d060 0x55e69201d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- 172.21.15.16:0/2478876693 >> 172.21.15.16:0/2478876693 conn(0x55e6920195f0 msgr2=0x55e6920199d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- 172.21.15.16:0/2478876693 shutdown_connections 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- 172.21.15.16:0/2478876693 wait complete. 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.199+0000 7f81d287b740 1 -- start start 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 Processor -- start 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e69201d060 0x55e692025070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 0x55e692025af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e69202ae10 con 0x55e69201d980 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e691f0af10 con 0x55e692028240 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e69202af90 con 0x55e69201d060 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81d287b740 10 client.0 _ll_register_callbacks cb 0x55e691f2edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59446/0 (socket says 172.21.15.16:59446) 2024-09-12T01:10:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 -- 172.21.15.16:0/2065758231 learned_addr learned my addr 172.21.15.16:0/2065758231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cea3f640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 0x55e692025af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81ce23e640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e69201d060 0x55e692025070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 -- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e69201d060 msgr2=0x55e692025070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e69201d060 0x55e692025070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 -- 172.21.15.16:0/2065758231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 msgr2=0x55e692025af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 0x55e692025af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cda3d640 1 -- 172.21.15.16:0/2065758231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e691c91580 con 0x55e69201d980 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.200+0000 7f81cea3f640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 0x55e692025af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81cda3d640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f81b8013b60 tx=0x7f81b8013b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36548044f21224a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8017050 con 0x55e69201d980 2024-09-12T01:10:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f81b801c450 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b8013470 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b801c650 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 10 client.19719 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e691d5b960 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b8013470 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81b802d070 con 0x55e69201d980 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 10 client.19719 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 10 client.19719 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.201+0000 7f81d287b740 20 client.19719 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e692037d90 0x55e69203a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81d287b740 4 client.19719 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e691f8fb90 con 0x55e692037d90 2024-09-12T01:10:13.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81ce23e640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e692037d90 0x55e69203a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81b1ffb640 10 client.19719 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.202+0000 7f81ce23e640 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e692037d90 0x55e69203a180 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbd0cc0222f50f9 server_cookie=47bdd49760aa08d5 in_seq=0 out_seq=0 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81b1ffb640 1 -- 172.21.15.16:0/2065758231 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e691f8fb90 con 0x55e692037d90 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81b1ffb640 10 client.19719 handle_command_reply: tid=0 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 client.19719 shutdown 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 msgr2=0x55e6920255b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f81b8013b60 tx=0x7f81b8013b90 comp rx=0 tx=0).stop 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 shutdown_connections 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e692037d90 0x55e69203a180 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e692028240 0x55e692025af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e6920255b0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81b1ffb640 0 client.19719 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:13.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 --2- 172.21.15.16:0/2065758231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e69201d060 0x55e692025070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.203+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 >> 172.21.15.16:0/2065758231 conn(0x55e6920195f0 msgr2=0x55e692018750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.204+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 shutdown_connections 2024-09-12T01:10:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.204+0000 7f81d287b740 1 -- 172.21.15.16:0/2065758231 wait complete. 2024-09-12T01:10:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.204+0000 7f81d287b740 20 client.19719 trim_cache size 0 max 16384 2024-09-12T01:10:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 Processor -- start 2024-09-12T01:10:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 -- start start 2024-09-12T01:10:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69201fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69203f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81afa60 con 0x55e69201d980 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b15a0 con 0x55e69201d060 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81d287b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b0130 con 0x55e692028240 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59456/0 (socket says 172.21.15.16:59456) 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 -- 172.21.15.16:0/347782936 learned_addr learned my addr 172.21.15.16:0/347782936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81ce23e640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69201fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cea3f640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69203f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 -- 172.21.15.16:0/347782936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 msgr2=0x55e69203f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69203f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 -- 172.21.15.16:0/347782936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 msgr2=0x55e69201fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69201fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 -- 172.21.15.16:0/347782936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e692019420 con 0x55e69201d980 2024-09-12T01:10:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cea3f640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69203f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.221+0000 7f81cda3d640 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f81b800afd0 tx=0x7f81b8009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954d6ecd48179498 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81b3fff640 1 -- 172.21.15.16:0/347782936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8033070 con 0x55e69201d980 2024-09-12T01:10:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81b3fff640 1 -- 172.21.15.16:0/347782936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b8013c00 con 0x55e69201d980 2024-09-12T01:10:13.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81b3fff640 1 -- 172.21.15.16:0/347782936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8017070 con 0x55e69201d980 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 -- 172.21.15.16:0/347782936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 msgr2=0x55e692018640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f81b800afd0 tx=0x7f81b8009400 comp rx=0 tx=0).stop 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 -- 172.21.15.16:0/347782936 shutdown_connections 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69203f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e692018640 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 --2- 172.21.15.16:0/347782936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69201fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 -- 172.21.15.16:0/347782936 >> 172.21.15.16:0/347782936 conn(0x55e6920195f0 msgr2=0x55e692017f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 -- 172.21.15.16:0/347782936 shutdown_connections 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.222+0000 7f81d287b740 1 -- 172.21.15.16:0/347782936 wait complete. 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 -- start start 2024-09-12T01:10:13.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 Processor -- start 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69202bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b0ed0 con 0x55e69201d980 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b0800 con 0x55e69201d060 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81ada60 con 0x55e692028240 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81d287b740 10 client.0 _ll_register_callbacks cb 0x7f81b80139a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59468/0 (socket says 172.21.15.16:59468) 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 -- 172.21.15.16:0/1834297649 learned_addr learned my addr 172.21.15.16:0/1834297649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81ce23e640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69202bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cea3f640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 -- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 msgr2=0x55e69202ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 -- 172.21.15.16:0/1834297649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 msgr2=0x55e69202bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.223+0000 7f81cda3d640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69202bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81cda3d640 1 -- 172.21.15.16:0/1834297649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6920a17d0 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81ce23e640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69202bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81cda3d640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005d20 tx=0x7f81b80268c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18097b024cb03088 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8017070 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b8013c00 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f81b801ca00 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b8045050 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81d287b740 10 client.19725 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e6920a2a40 con 0x55e69201d980 2024-09-12T01:10:13.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.224+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b8013c00 con 0x55e69201d980 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f81b802d040 con 0x55e69201d980 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 10 client.19725 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 10 client.19725 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 20 client.19725 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e6920a3090 0x55e6920a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 4 client.19725 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dadf246f", "release": true, "await": true}] 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dadf246f", "release": true, "await": true}) -- 0x55e691f8a0a0 con 0x55e6920a3090 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81ce23e640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e6920a3090 0x55e6920a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81ce23e640 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e6920a3090 0x55e6920a5480 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef8e2d24803681e9 server_cookie=aa545365630fc7bf in_seq=0 out_seq=0 2024-09-12T01:10:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.225+0000 7f81b1ffb640 10 client.19725 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:13.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.228+0000 7f81b1ffb640 1 -- 172.21.15.16:0/1834297649 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e691f8a0a0 con 0x55e6920a3090 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.228+0000 7f81b1ffb640 10 client.19725 handle_command_reply: tid=0 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.228+0000 7f81d287b740 1 client.19725 shutdown 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 msgr2=0x55e69202c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005d20 tx=0x7f81b80268c0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 shutdown_connections 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e6920a3090 0x55e6920a5480 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e692028240 0x55e69202ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e69201d980 0x55e69202c4e0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 --2- 172.21.15.16:0/1834297649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e69201d060 0x55e69202bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 >> 172.21.15.16:0/1834297649 conn(0x55e6920195f0 msgr2=0x55e692027440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81b1ffb640 0 client.19725 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 shutdown_connections 2024-09-12T01:10:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 1 -- 172.21.15.16:0/1834297649 wait complete. 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d287b740 20 client.19725 trim_cache size 0 max 16384 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 52, 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "dadf246f": { 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 52, 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:10:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 msgr2=0x7f81b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 0x7f81b40a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81c0004af0 tx=0x7f81c0031d70 comp rx=0 tx=0).stop 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 msgr2=0x7f81c80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f81c400d770 tx=0x7f81c400db70 comp rx=0 tx=0).stop 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 shutdown_connections 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81b40a2cf0 0x7f81b40a51b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81c8106f10 0x7f81c8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.229+0000 7f81d04a2640 1 --2- 172.21.15.16:0/3378501464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81c8101de0 0x7f81c80744f0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.230+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 >> 172.21.15.16:0/3378501464 conn(0x7f81c80fd8c0 msgr2=0x7f81c80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.230+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 shutdown_connections 2024-09-12T01:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:13.230+0000 7f81d04a2640 1 -- 172.21.15.16:0/3378501464 wait complete. 2024-09-12T01:10:13.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:13 smithi159 ceph-mon[35566]: pgmap v440: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:13.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:13 smithi016 ceph-mon[30972]: pgmap v440: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:13 smithi099 ceph-mon[35654]: pgmap v440: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:14.052 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dadf246f', total seconds elapsed: 33.4 2024-09-12T01:10:14.052 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T01:10:14.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:10:14.316 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 -- 172.21.15.16:0/1704009544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201055b0 msgr2=0x7f4b2010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 --2- 172.21.15.16:0/1704009544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201055b0 0x7f4b2010d130 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009ca0 tx=0x7f4b1802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 -- 172.21.15.16:0/1704009544 shutdown_connections 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 --2- 172.21.15.16:0/1704009544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201055b0 0x7f4b2010d130 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 --2- 172.21.15.16:0/1704009544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100e30 0x7f4b20104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 --2- 172.21.15.16:0/1704009544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b20100480 0x7f4b20100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 -- 172.21.15.16:0/1704009544 >> 172.21.15.16:0/1704009544 conn(0x7f4b20077210 msgr2=0x7f4b200fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.865+0000 7f4b273b9640 1 -- 172.21.15.16:0/1704009544 shutdown_connections 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 -- 172.21.15.16:0/1704009544 wait complete. 2024-09-12T01:10:14.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 Processor -- start 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 -- start start 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 0x7f4b201984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 0x7f4b2019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20110ed0 con 0x7f4b20100e30 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20110d50 con 0x7f4b20100480 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b273b9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20111050 con 0x7f4b201055b0 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b24954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b24954640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59486/0 (socket says 172.21.15.16:59486) 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.866+0000 7f4b24954640 1 -- 172.21.15.16:0/490258481 learned_addr learned my addr 172.21.15.16:0/490258481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b25155640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 0x7f4b201984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b25956640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 0x7f4b2019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:14.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 -- 172.21.15.16:0/490258481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 msgr2=0x7f4b2019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 0x7f4b2019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 -- 172.21.15.16:0/490258481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 msgr2=0x7f4b201984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 0x7f4b201984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 -- 172.21.15.16:0/490258481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2019d490 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b25155640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 0x7f4b201984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b25956640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 0x7f4b2019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b24954640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d950 tx=0x7f4b1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10017030 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2019d780 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b10004020 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100047a0 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.867+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b201a5060 con 0x7f4b20100e30 2024-09-12T01:10:14.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.868+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b20100860 con 0x7f4b20100e30 2024-09-12T01:10:14.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.869+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4b100041d0 con 0x7f4b20100e30 2024-09-12T01:10:14.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.870+0000 7f4b167fc640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 0x7f4b0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:14.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.870+0000 7f4b25155640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 0x7f4b0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:14.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.871+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f4b100cf620 con 0x7f4b20100e30 2024-09-12T01:10:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.871+0000 7f4b25155640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 0x7f4b0c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006fd0 tx=0x7f4b1c008040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:14.874+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b1009c1b0 con 0x7f4b20100e30 2024-09-12T01:10:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.087+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b20109860 con 0x7f4b20100e30 2024-09-12T01:10:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.088+0000 7f4b167fc640 1 -- 172.21.15.16:0/490258481 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f4b1009bab0 con 0x7f4b20100e30 2024-09-12T01:10:15.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:10:15.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:10:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:10:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 msgr2=0x7f4b0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 0x7f4b0c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c006fd0 tx=0x7f4b1c008040 comp rx=0 tx=0).stop 2024-09-12T01:10:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 msgr2=0x7f4b20198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d950 tx=0x7f4b1000de10 comp rx=0 tx=0).stop 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 shutdown_connections 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4b0c0a2cf0 0x7f4b0c0a51b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b201055b0 0x7f4b2019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b20100e30 0x7f4b20198a20 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 --2- 172.21.15.16:0/490258481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b20100480 0x7f4b201984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 >> 172.21.15.16:0/490258481 conn(0x7f4b20077210 msgr2=0x7f4b2010dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 shutdown_connections 2024-09-12T01:10:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:15.090+0000 7f4b273b9640 1 -- 172.21.15.16:0/490258481 wait complete. 2024-09-12T01:10:15.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:15 smithi159 ceph-mon[35566]: pgmap v441: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:15.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:15 smithi016 ceph-mon[30972]: pgmap v441: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:15.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:15 smithi099 ceph-mon[35654]: pgmap v441: 97 pgs: 97 active+clean; 23 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 30 op/s 2024-09-12T01:10:15.823 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T01:10:15.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T01:10:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:16.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:16 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/490258481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:16.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:16 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/490258481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:16.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:16 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/490258481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 -- 172.21.15.16:0/348906244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754101de0 msgr2=0x7f97541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 --2- 172.21.15.16:0/348906244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754101de0 0x7f97541021c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402a3f0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 -- 172.21.15.16:0/348906244 shutdown_connections 2024-09-12T01:10:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 --2- 172.21.15.16:0/348906244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 0x7f975410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 --2- 172.21.15.16:0/348906244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754102790 0x7f97541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 --2- 172.21.15.16:0/348906244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754101de0 0x7f97541021c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 -- 172.21.15.16:0/348906244 >> 172.21.15.16:0/348906244 conn(0x7f97540fd8c0 msgr2=0x7f97540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 -- 172.21.15.16:0/348906244 shutdown_connections 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.661+0000 7f9758c15640 1 -- 172.21.15.16:0/348906244 wait complete. 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 Processor -- start 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 -- start start 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754101de0 0x7f975410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 0x7f97541a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9754111f30 con 0x7f9754102790 2024-09-12T01:10:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9754111db0 con 0x7f9754106f10 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9758c15640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97541120b0 con 0x7f9754101de0 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59494/0 (socket says 172.21.15.16:59494) 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f974bfff640 1 -- 172.21.15.16:0/2438365179 learned_addr learned my addr 172.21.15.16:0/2438365179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f975259c640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754101de0 0x7f975410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.662+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 0x7f97541a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 -- 172.21.15.16:0/2438365179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754101de0 msgr2=0x7f975410ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754101de0 0x7f975410ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 -- 172.21.15.16:0/2438365179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 msgr2=0x7f97541a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 0x7f97541a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 -- 172.21.15.16:0/2438365179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97541a39e0 con 0x7f9754102790 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974bfff640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c920 tx=0x7f973c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974b7fe640 1 -- 172.21.15.16:0/2438365179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c007c70 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974b7fe640 1 -- 172.21.15.16:0/2438365179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973c007e10 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97541a3cd0 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f974b7fe640 1 -- 172.21.15.16:0/2438365179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c018890 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.663+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97541ab590 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.665+0000 7f974b7fe640 1 -- 172.21.15.16:0/2438365179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f973c018a70 con 0x7f9754102790 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 Processor -- start 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 -- start start 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f974b7fe640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 0x7f97400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a304a080 0x5597a3054400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a3056d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5597a2faf2a0 con 0x5597a3049760 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597a2f56b10 con 0x5597a304a080 2024-09-12T01:10:16.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.666+0000 7f975afee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5597a2fab260 con 0x5597a3054940 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 0x7f97400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f974b7fe640 1 -- 172.21.15.16:0/2438365179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f973c0cf520 con 0x7f9754102790 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59504/0 (socket says 172.21.15.16:59504) 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 -- 172.21.15.16:0/2486072333 learned_addr learned my addr 172.21.15.16:0/2486072333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f974bfff640 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a304a080 0x5597a3054400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a3056d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 0x7f97400a51b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f974c00ad70 tx=0x7f974c00f790 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 -- 172.21.15.16:0/2486072333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 msgr2=0x5597a3056d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a3056d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 -- 172.21.15.16:0/2486072333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a304a080 msgr2=0x5597a3054400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a304a080 0x5597a3054400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 -- 172.21.15.16:0/2486072333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a2d20b90 con 0x5597a3049760 2024-09-12T01:10:16.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.667+0000 7f975259c640 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f97440316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36fed0004a436d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.668+0000 7f9748ff9640 1 -- 172.21.15.16:0/2486072333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744040070 con 0x5597a3049760 2024-09-12T01:10:16.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.668+0000 7f9748ff9640 1 -- 172.21.15.16:0/2486072333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9744005020 con 0x5597a3049760 2024-09-12T01:10:16.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.668+0000 7f9748ff9640 1 -- 172.21.15.16:0/2486072333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744035040 con 0x5597a3049760 2024-09-12T01:10:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.668+0000 7f975afee740 1 -- 172.21.15.16:0/2486072333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 msgr2=0x5597a3049b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.668+0000 7f975afee740 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f97440316e0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- 172.21.15.16:0/2486072333 shutdown_connections 2024-09-12T01:10:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a3056d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a304a080 0x5597a3054400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- 172.21.15.16:0/2486072333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3049760 0x5597a3049b40 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- 172.21.15.16:0/2486072333 >> 172.21.15.16:0/2486072333 conn(0x5597a3045cf0 msgr2=0x5597a30460d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- 172.21.15.16:0/2486072333 shutdown_connections 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- 172.21.15.16:0/2486072333 wait complete. 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- start start 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 Processor -- start 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a3051780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a304a080 0x5597a3051cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5597a3057510 con 0x5597a3054940 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597a2f376e0 con 0x5597a3049760 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5597a3057690 con 0x5597a304a080 2024-09-12T01:10:16.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.669+0000 7f975afee740 10 client.0 _ll_register_callbacks cb 0x5597a2f5ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59510/0 (socket says 172.21.15.16:59510) 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 -- 172.21.15.16:0/3576445676 learned_addr learned my addr 172.21.15.16:0/3576445676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f975259c640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a3051780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 -- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a304a080 msgr2=0x5597a3051cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a304a080 0x5597a3051cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 -- 172.21.15.16:0/3576445676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 msgr2=0x5597a3051780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a3051780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 -- 172.21.15.16:0/3576445676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a3060980 con 0x5597a3054940 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9752d9d640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f974c010de0 tx=0x7f974c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2893e2a5e7bb22e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c02a060 con 0x5597a3054940 2024-09-12T01:10:16.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f974c01b7a0 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.670+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9744005020 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f974c018070 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 10 client.19743 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597a3063340 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9744005020 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f974c024030 con 0x5597a3054940 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 10 client.19743 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 10 client.19743 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 20 client.19743 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a3064800 0x5597a3066c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 4 client.19743 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.671+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a2fbc9d0 con 0x5597a3064800 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.672+0000 7f975259c640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a3064800 0x5597a3066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.673+0000 7f975259c640 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a3064800 0x5597a3066c90 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2127cf9174b9d838 server_cookie=baa3bf893e5b86e2 in_seq=0 out_seq=0 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.673+0000 7f9739ffb640 10 client.19743 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f9739ffb640 1 -- 172.21.15.16:0/3576445676 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597a2fbc9d0 con 0x5597a3064800 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f9739ffb640 10 client.19743 handle_command_reply: tid=0 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 client.19743 shutdown 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 msgr2=0x5597a3052200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f974c010de0 tx=0x7f974c004ab0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 shutdown_connections 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a3064800 0x5597a3066c90 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a3054940 0x5597a3052200 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a304a080 0x5597a3051cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f9739ffb640 0 client.19743 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 --2- 172.21.15.16:0/3576445676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a3051780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 >> 172.21.15.16:0/3576445676 conn(0x5597a3045cf0 msgr2=0x5597a304ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 shutdown_connections 2024-09-12T01:10:16.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 1 -- 172.21.15.16:0/3576445676 wait complete. 2024-09-12T01:10:16.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.675+0000 7f975afee740 20 client.19743 trim_cache size 0 max 16384 2024-09-12T01:10:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 Processor -- start 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 -- start start 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a306c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a304e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b3ea0 con 0x5597a304a080 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b59e0 con 0x5597a3049760 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f975afee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b4570 con 0x5597a3054940 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59524/0 (socket says 172.21.15.16:59524) 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.692+0000 7f974bfff640 1 -- 172.21.15.16:0/1607475060 learned_addr learned my addr 172.21.15.16:0/1607475060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f9752d9d640 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a304e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f975259c640 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a306c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 -- 172.21.15.16:0/1607475060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 msgr2=0x5597a304e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a304e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 -- 172.21.15.16:0/1607475060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 msgr2=0x5597a306c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a306c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 -- 172.21.15.16:0/1607475060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a3045b20 con 0x5597a304a080 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f974bfff640 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f973c0983e0 tx=0x7f973c097d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9a5c2e2007581 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f973bfff640 1 -- 172.21.15.16:0/1607475060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0b3020 con 0x5597a304a080 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f973bfff640 1 -- 172.21.15.16:0/1607475060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973c0a4680 con 0x5597a304a080 2024-09-12T01:10:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f973bfff640 1 -- 172.21.15.16:0/1607475060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c098080 con 0x5597a304a080 2024-09-12T01:10:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.693+0000 7f975afee740 1 -- 172.21.15.16:0/1607475060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 msgr2=0x5597a306caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f973c0983e0 tx=0x7f973c097d60 comp rx=0 tx=0).stop 2024-09-12T01:10:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- 172.21.15.16:0/1607475060 shutdown_connections 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a304e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a306caa0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- 172.21.15.16:0/1607475060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a306c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- 172.21.15.16:0/1607475060 >> 172.21.15.16:0/1607475060 conn(0x5597a3045cf0 msgr2=0x5597a3050050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- 172.21.15.16:0/1607475060 shutdown_connections 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- 172.21.15.16:0/1607475060 wait complete. 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- start start 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 Processor -- start 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a30592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a30c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b5310 con 0x5597a304a080 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b4c40 con 0x5597a3049760 2024-09-12T01:10:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97541b1ea0 con 0x5597a3054940 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.694+0000 7f975afee740 10 client.0 _ll_register_callbacks cb 0x7f974c010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59536/0 (socket says 172.21.15.16:59536) 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 -- 172.21.15.16:0/2358255814 learned_addr learned my addr 172.21.15.16:0/2358255814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f975259c640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a30592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a30c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 -- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 msgr2=0x5597a30c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a30c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 -- 172.21.15.16:0/2358255814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 msgr2=0x5597a30592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a30592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 -- 172.21.15.16:0/2358255814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a3041260 con 0x5597a304a080 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9752d9d640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a30c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f974bfff640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f973c007590 tx=0x7f973c018d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26246585a6795383 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0b3030 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8565+0+0 (secure 0 0 0) 0x7f973c0ba300 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973c0a4680 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.695+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f973c09bc70 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 10 client.19749 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597a30cfe90 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973c0a4680 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f973c09ae60 con 0x5597a304a080 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 10 client.19749 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 10 client.19749 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 20 client.19749 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a30d05c0 0x5597a30d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 4 client.19749 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T01:10:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5597a2fb6d20 con 0x5597a30d05c0 2024-09-12T01:10:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.696+0000 7f975259c640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a30d05c0 0x5597a30d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.697+0000 7f975259c640 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a30d05c0 0x5597a30d29b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a07a1b94e1c272 server_cookie=84ad92bcac4533a0 in_seq=0 out_seq=0 2024-09-12T01:10:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.697+0000 7f9739ffb640 10 client.19749 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.709+0000 7f9739ffb640 1 -- 172.21.15.16:0/2358255814 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5597a2fb6d20 con 0x5597a30d05c0 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.709+0000 7f9739ffb640 10 client.19749 handle_command_reply: tid=0 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.709+0000 7f975afee740 1 client.19749 shutdown 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.709+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 msgr2=0x5597a30597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.709+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f973c007590 tx=0x7f973c018d60 comp rx=0 tx=0).stop 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 shutdown_connections 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5597a30d05c0 0x5597a30d29b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597a3054940 0x5597a30c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597a304a080 0x5597a30597f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9739ffb640 0 client.19749 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 --2- 172.21.15.16:0/2358255814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597a3049760 0x5597a30592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 >> 172.21.15.16:0/2358255814 conn(0x5597a3045cf0 msgr2=0x5597a3052fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 shutdown_connections 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 1 -- 172.21.15.16:0/2358255814 wait complete. 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f975afee740 20 client.19749 trim_cache size 0 max 16384 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 54, 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "563d9ea6": { 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 54, 2024-09-12T01:10:16.711 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-12T01:10:16.712 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:16.713 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 msgr2=0x7f97400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 0x7f97400a51b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f974c00ad70 tx=0x7f974c00f790 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 msgr2=0x7f975410e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c920 tx=0x7f973c00cde0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 shutdown_connections 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f97400a2cf0 0x7f97400a51b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9754106f10 0x7f97541a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9754102790 0x7f975410e2f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 --2- 172.21.15.16:0/2438365179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9754101de0 0x7f975410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.710+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 >> 172.21.15.16:0/2438365179 conn(0x7f97540fd8c0 msgr2=0x7f97540ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.711+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 shutdown_connections 2024-09-12T01:10:16.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:16.711+0000 7f9758c15640 1 -- 172.21.15.16:0/2438365179 wait complete. 2024-09-12T01:10:17.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:17 smithi159 ceph-mon[35566]: pgmap v442: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-12T01:10:17.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:17 smithi016 ceph-mon[30972]: pgmap v442: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-12T01:10:17.420 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '563d9ea6', quiesce took 3.4 seconds. Will release after: 26.6 2024-09-12T01:10:17.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:17 smithi099 ceph-mon[35654]: pgmap v442: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-09-12T01:10:18.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:18 smithi099 ceph-mon[35654]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:10:18.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:18 smithi099 ceph-mon[35654]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-12T01:10:18.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:18 smithi159 ceph-mon[35566]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:10:18.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:18 smithi159 ceph-mon[35566]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-12T01:10:18.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:18 smithi016 ceph-mon[30972]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T01:10:18.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:18 smithi016 ceph-mon[30972]: pgmap v444: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-09-12T01:10:19.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:10:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 -- 172.21.15.16:0/3796799692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 msgr2=0x7fa1081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 --2- 172.21.15.16:0/3796799692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa1081067e0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009900 tx=0x7fa0f802a410 comp rx=0 tx=0).stop 2024-09-12T01:10:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 -- 172.21.15.16:0/3796799692 shutdown_connections 2024-09-12T01:10:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 --2- 172.21.15.16:0/3796799692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa10810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 --2- 172.21.15.16:0/3796799692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa1081067e0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 --2- 172.21.15.16:0/3796799692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 -- 172.21.15.16:0/3796799692 >> 172.21.15.16:0/3796799692 conn(0x7fa1080fd8c0 msgr2=0x7fa1080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 -- 172.21.15.16:0/3796799692 shutdown_connections 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.311+0000 7fa10ece2640 1 -- 172.21.15.16:0/3796799692 wait complete. 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 Processor -- start 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 -- start start 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa108074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111f70 con 0x7fa108102790 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111df0 con 0x7fa108101de0 2024-09-12T01:10:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10ece2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081120f0 con 0x7fa108106f10 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54648/0 (socket says 172.21.15.16:54648) 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa0fffff640 1 -- 172.21.15.16:0/2683459262 learned_addr learned my addr 172.21.15.16:0/2683459262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.312+0000 7fa10d27f640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa108074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 -- 172.21.15.16:0/2683459262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 msgr2=0x7fa108074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa108074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 -- 172.21.15.16:0/2683459262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 msgr2=0x7fa1080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 -- 172.21.15.16:0/2683459262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1081aae40 con 0x7fa108102790 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa10d27f640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa108074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:20.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fffff640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fdffb640 1 -- 172.21.15.16:0/2683459262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f803f030 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fdffb640 1 -- 172.21.15.16:0/2683459262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8004440 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1081ab010 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa0fdffb640 1 -- 172.21.15.16:0/2683459262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8035070 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.313+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081ab540 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa0fdffb640 1 -- 172.21.15.16:0/2683459262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa0f8042030 con 0x7fa108102790 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 Processor -- start 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 -- start start 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 0x55886bb1d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55886ba782e0 con 0x55886bb124a0 2024-09-12T01:10:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55886ba1fce0 con 0x55886bb1d680 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa1110bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55886ba742a0 con 0x55886bb12dc0 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa10d27f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa0fdffb640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 0x7fa0f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa10d27f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54654/0 (socket says 172.21.15.16:54654) 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa10d27f640 1 -- 172.21.15.16:0/1334028387 learned_addr learned my addr 172.21.15.16:0/1334028387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.316+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 0x55886bb1d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10d27f640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 0x7fa0f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa0fdffb640 1 -- 172.21.15.16:0/2683459262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa0f80ef870 con 0x7fa108102790 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa0fffff640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 -- 172.21.15.16:0/1334028387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 msgr2=0x55886bb1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 -- 172.21.15.16:0/1334028387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 msgr2=0x55886bb12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 -- 172.21.15.16:0/1334028387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886ba04440 con 0x55886bb12dc0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10d27f640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 0x55886bb1d140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa104009c30 tx=0x7fa1040117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f23ab7e411d7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa10d27f640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 0x7fa0f00a5230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa100005c40 tx=0x7fa100004ab0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa0eeffd640 1 -- 172.21.15.16:0/1334028387 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104012450 con 0x55886bb12dc0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa0eeffd640 1 -- 172.21.15.16:0/1334028387 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104012ad0 con 0x55886bb12dc0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.317+0000 7fa0eeffd640 1 -- 172.21.15.16:0/1334028387 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400fb60 con 0x55886bb12dc0 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- 172.21.15.16:0/1334028387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 msgr2=0x55886bb1d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 0x55886bb1d140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa104009c30 tx=0x7fa1040117b0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- 172.21.15.16:0/1334028387 shutdown_connections 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb12dc0 0x55886bb1d140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1334028387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb124a0 0x55886bb12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- 172.21.15.16:0/1334028387 >> 172.21.15.16:0/1334028387 conn(0x55886bb0ea30 msgr2=0x55886bb0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- 172.21.15.16:0/1334028387 shutdown_connections 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- 172.21.15.16:0/1334028387 wait complete. 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.318+0000 7fa1110bb740 1 -- start start 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 Processor -- start 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55886bb20250 con 0x55886bb12dc0 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55886ba33050 con 0x55886bb1d680 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55886bb203d0 con 0x55886bb124a0 2024-09-12T01:10:20.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa1110bb740 10 client.0 _ll_register_callbacks cb 0x55886ba73160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa10ca7e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54660/0 (socket says 172.21.15.16:54660) 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa0fffff640 1 -- 172.21.15.16:0/3431638001 learned_addr learned my addr 172.21.15.16:0/3431638001 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa10ca7e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:56610/0 (socket says 172.21.15.16:56610) 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa10d27f640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.319+0000 7fa0fffff640 1 -- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 msgr2=0x55886bb1cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0fffff640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0fffff640 1 -- 172.21.15.16:0/3431638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 msgr2=0x55886bb1a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0fffff640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0fffff640 1 -- 172.21.15.16:0/3431638001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886b7e93c0 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0fffff640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80bbb50 tx=0x7fa0f8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f23c2610ccfdf58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8049070 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa0f80bc0b0 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa104012ad0 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0f80bc550 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa1110bb740 10 client.19761 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55886bb2bad0 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104012ad0 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f80c4630 con 0x55886bb12dc0 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.320+0000 7fa1110bb740 10 client.19761 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa1110bb740 10 client.19761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa1110bb740 20 client.19761 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb2ce70 0x55886bb2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa1110bb740 4 client.19761 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886ba85850 con 0x55886bb2ce70 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb2ce70 0x55886bb2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb2ce70 0x55886bb2f2c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32851418b3c10fb3 server_cookie=917b841ac2506194 in_seq=0 out_seq=0 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.321+0000 7fa0e7fff640 10 client.19761 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa0e7fff640 1 -- 172.21.15.16:0/3431638001 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55886ba85850 con 0x55886bb2ce70 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa0e7fff640 10 client.19761 handle_command_reply: tid=0 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 client.19761 shutdown 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 msgr2=0x55886bb1a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80bbb50 tx=0x7fa0f8009650 comp rx=0 tx=0).stop 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 shutdown_connections 2024-09-12T01:10:20.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb2ce70 0x55886bb2f2c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb1d680 0x55886bb1a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1a1c0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa0e7fff640 0 client.19761 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3431638001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb1cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.322+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 >> 172.21.15.16:0/3431638001 conn(0x55886bb0ea30 msgr2=0x55886bb0db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.323+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 shutdown_connections 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.323+0000 7fa1110bb740 1 -- 172.21.15.16:0/3431638001 wait complete. 2024-09-12T01:10:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.323+0000 7fa1110bb740 20 client.19761 trim_cache size 0 max 16384 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.346+0000 7fa1110bb740 1 Processor -- start 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.346+0000 7fa1110bb740 1 -- start start 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb34ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 0x55886bb350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b3ee0 con 0x55886bb0ea30 2024-09-12T01:10:20.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b5a20 con 0x55886bb12dc0 2024-09-12T01:10:20.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa1110bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b45b0 con 0x55886bb124a0 2024-09-12T01:10:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54670/0 (socket says 172.21.15.16:54670) 2024-09-12T01:10:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 -- 172.21.15.16:0/3717185073 learned_addr learned my addr 172.21.15.16:0/3717185073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa0fffff640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb34ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10d27f640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 0x55886bb350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 -- 172.21.15.16:0/3717185073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 msgr2=0x55886bb34ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb34ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 -- 172.21.15.16:0/3717185073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 msgr2=0x55886bb350e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 0x55886bb350e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 -- 172.21.15.16:0/3717185073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886bb1d280 con 0x55886bb0ea30 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10d27f640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 0x55886bb350e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa104009b50 tx=0x7fa104012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18be3be77fa52273 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa0ee7fc640 1 -- 172.21.15.16:0/3717185073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400f560 con 0x55886bb0ea30 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa0ee7fc640 1 -- 172.21.15.16:0/3717185073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104005520 con 0x55886bb0ea30 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.347+0000 7fa0ee7fc640 1 -- 172.21.15.16:0/3717185073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10400fdd0 con 0x55886bb0ea30 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 -- 172.21.15.16:0/3717185073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 msgr2=0x55886bb294b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa104009b50 tx=0x7fa104012f70 comp rx=0 tx=0).stop 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 -- 172.21.15.16:0/3717185073 shutdown_connections 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb12dc0 0x55886bb350e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb124a0 0x55886bb34ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 --2- 172.21.15.16:0/3717185073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb0ea30 0x55886bb294b0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 -- 172.21.15.16:0/3717185073 >> 172.21.15.16:0/3717185073 conn(0x55886bb1a860 msgr2=0x55886bb18a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 -- 172.21.15.16:0/3717185073 shutdown_connections 2024-09-12T01:10:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.348+0000 7fa1110bb740 1 -- 172.21.15.16:0/3717185073 wait complete. 2024-09-12T01:10:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 -- start start 2024-09-12T01:10:20.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 Processor -- start 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 0x55886bb1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 0x55886bb1ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b5350 con 0x55886bb12dc0 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b4c80 con 0x55886bb124a0 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081b1ee0 con 0x55886bb0ea30 2024-09-12T01:10:20.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa1110bb740 10 client.0 _ll_register_callbacks cb 0x7fa0f8005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa10d27f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa10d27f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54680/0 (socket says 172.21.15.16:54680) 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa10d27f640 1 -- 172.21.15.16:0/1022024282 learned_addr learned my addr 172.21.15.16:0/1022024282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.349+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 0x55886bb1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0fffff640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 0x55886bb1ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 -- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 msgr2=0x55886bb1e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 0x55886bb1e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 -- 172.21.15.16:0/1022024282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 msgr2=0x55886bb1ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 0x55886bb1ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 -- 172.21.15.16:0/1022024282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886bb23fc0 con 0x55886bb12dc0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 0x55886bb1e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0fffff640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 0x55886bb1ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa10d27f640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa1000106b0 tx=0x7fa100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913caacc06f2237d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10002e060 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fa10001f800 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa104005520 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa10001fa00 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa1110bb740 10 client.19767 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.350+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55886bb98030 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa104005520 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa100029ce0 con 0x55886bb12dc0 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 10 client.19767 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 10 client.19767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 20 client.19767 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb1d680 0x55886bb9a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 4 client.19767 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:10:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55886ba7fd60 con 0x55886bb1d680 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.351+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb1d680 0x55886bb9a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa0e7fff640 10 client.19767 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa10ca7e640 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb1d680 0x55886bb9a7c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142eaeb4b8e575ad server_cookie=9e3261f902b52aca in_seq=0 out_seq=0 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa0e7fff640 1 -- 172.21.15.16:0/1022024282 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55886ba7fd60 con 0x55886bb1d680 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa0e7fff640 10 client.19767 handle_command_reply: tid=0 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 client.19767 shutdown 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 msgr2=0x55886bb1f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fa1000106b0 tx=0x7fa100004a40 comp rx=0 tx=0).stop 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 shutdown_connections 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55886bb1d680 0x55886bb9a7c0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55886bb12dc0 0x55886bb1f450 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55886bb124a0 0x55886bb1ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 --2- 172.21.15.16:0/1022024282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55886bb0ea30 0x55886bb1e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 >> 172.21.15.16:0/1022024282 conn(0x55886bb1a860 msgr2=0x55886bb0db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa0e7fff640 0 client.19767 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 shutdown_connections 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 1 -- 172.21.15.16:0/1022024282 wait complete. 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.352+0000 7fa1110bb740 20 client.19767 trim_cache size 0 max 16384 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:10:20.362 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 msgr2=0x7fa0f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 0x7fa0f00a5230 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa100005c40 tx=0x7fa100004ab0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 msgr2=0x7fa108074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f8004040 comp rx=0 tx=0).stop 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 shutdown_connections 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa0f00a2d70 0x7fa0f00a5230 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa108106f10 0x7fa108074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa108102790 0x7fa108074a30 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 --2- 172.21.15.16:0/2683459262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa108101de0 0x7fa1080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:20.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 >> 172.21.15.16:0/2683459262 conn(0x7fa1080fd8c0 msgr2=0x7fa1080fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:20.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 shutdown_connections 2024-09-12T01:10:20.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:20.353+0000 7fa10ece2640 1 -- 172.21.15.16:0/2683459262 wait complete. 2024-09-12T01:10:20.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:20 smithi016 ceph-mon[30972]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:10:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:20 smithi099 ceph-mon[35654]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:10:21.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:10:21.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:10:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:20 smithi159 ceph-mon[35566]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-12T01:10:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.875+0000 7fee7fd53640 1 -- 172.21.15.16:0/3729251396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 msgr2=0x7fee781006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.875+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3729251396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781006c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fee68009930 tx=0x7fee6802a420 comp rx=0 tx=0).stop 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 -- 172.21.15.16:0/3729251396 shutdown_connections 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3729251396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee7810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3729251396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781006c0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3729251396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee78106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 -- 172.21.15.16:0/3729251396 >> 172.21.15.16:0/3729251396 conn(0x7fee780fc5c0 msgr2=0x7fee780fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 -- 172.21.15.16:0/3729251396 shutdown_connections 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 -- 172.21.15.16:0/3729251396 wait complete. 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 Processor -- start 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 -- start start 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.876+0000 7fee7fd53640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee781989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee7819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810fd90 con 0x7fee781002a0 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810fc10 con 0x7fee781062b0 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810ff10 con 0x7fee78100c00 2024-09-12T01:10:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54706/0 (socket says 172.21.15.16:54706) 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 -- 172.21.15.16:0/3288167472 learned_addr learned my addr 172.21.15.16:0/3288167472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7e2f0640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee7819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7d2ee640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee781989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 -- 172.21.15.16:0/3288167472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 msgr2=0x7fee781989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee781989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 -- 172.21.15.16:0/3288167472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 msgr2=0x7fee7819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee7819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 -- 172.21.15.16:0/3288167472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7819d450 con 0x7fee781002a0 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7e2f0640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee7819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7d2ee640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7daef640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fee7400ddc0 tx=0x7fee7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7401b070 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7400be40 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7819d740 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74016bb0 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.877+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a5020 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.878+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee781006c0 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.880+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee7401e070 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.880+0000 7fee6effd640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 0x7fee640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.880+0000 7fee7d2ee640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 0x7fee640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.880+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fee740cf300 con 0x7fee781002a0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.881+0000 7fee7d2ee640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 0x7fee640a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee68000c00 tx=0x7fee68005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:21.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:21.883+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee7409c030 con 0x7fee781002a0 2024-09-12T01:10:22.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.093+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee78199800 con 0x7fee781002a0 2024-09-12T01:10:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.094+0000 7fee6effd640 1 -- 172.21.15.16:0/3288167472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fee7409b930 con 0x7fee781002a0 2024-09-12T01:10:22.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:10:22.094 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:10:22.094 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 msgr2=0x7fee640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 0x7fee640a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee68000c00 tx=0x7fee68005a20 comp rx=0 tx=0).stop 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 msgr2=0x7fee781984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fee7400ddc0 tx=0x7fee7400a620 comp rx=0 tx=0).stop 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 shutdown_connections 2024-09-12T01:10:22.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fee640a2d70 0x7fee640a5230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee781062b0 0x7fee7819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee78100c00 0x7fee781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 --2- 172.21.15.16:0/3288167472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee781002a0 0x7fee781984a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 >> 172.21.15.16:0/3288167472 conn(0x7fee780fc5c0 msgr2=0x7fee780feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 shutdown_connections 2024-09-12T01:10:22.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:22.096+0000 7fee7fd53640 1 -- 172.21.15.16:0/3288167472 wait complete. 2024-09-12T01:10:22.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:22 smithi016 ceph-mon[30972]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:22.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:22 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3288167472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:22.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:10:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:22 smithi099 ceph-mon[35654]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:22 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3288167472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:23.049 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:22 smithi159 ceph-mon[35566]: pgmap v446: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:22 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3288167472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 -- 172.21.15.16:0/2743014765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c106f10 msgr2=0x7f6e4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2743014765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c106f10 0x7f6e4c10ea90 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009ca0 tx=0x7f6e4402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 -- 172.21.15.16:0/2743014765 shutdown_connections 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2743014765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c106f10 0x7f6e4c10ea90 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2743014765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c102790 0x7f6e4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2743014765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e4c101de0 0x7f6e4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 -- 172.21.15.16:0/2743014765 >> 172.21.15.16:0/2743014765 conn(0x7f6e4c0fd8c0 msgr2=0x7f6e4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 -- 172.21.15.16:0/2743014765 shutdown_connections 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.553+0000 7f6e520ed640 1 -- 172.21.15.16:0/2743014765 wait complete. 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 Processor -- start 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 -- start start 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 0x7f6e4c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e4c106f10 0x7f6e4c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c111f70 con 0x7f6e4c102790 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c111df0 con 0x7f6e4c101de0 2024-09-12T01:10:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.554+0000 7f6e520ed640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1120f0 con 0x7f6e4c106f10 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54714/0 (socket says 172.21.15.16:54714) 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 -- 172.21.15.16:0/2156763133 learned_addr learned my addr 172.21.15.16:0/2156763133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 0x7f6e4c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 -- 172.21.15.16:0/2156763133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e4c106f10 msgr2=0x7f6e4c19cc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e4c106f10 0x7f6e4c19cc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 -- 172.21.15.16:0/2156763133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 msgr2=0x7f6e4c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 0x7f6e4c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 -- 172.21.15.16:0/2156763133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4c19d300 con 0x7f6e4c102790 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 0x7f6e4c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e43fff640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c920 tx=0x7f6e3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e497fa640 1 -- 172.21.15.16:0/2156763133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34007c70 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e497fa640 1 -- 172.21.15.16:0/2156763133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e34007e10 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4c19d5f0 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e497fa640 1 -- 172.21.15.16:0/2156763133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34018890 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.555+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4c1a4ed0 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.557+0000 7f6e497fa640 1 -- 172.21.15.16:0/2156763133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e34018a50 con 0x7f6e4c102790 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 Processor -- start 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 -- start start 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f708232f0 con 0x557f708beb40 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f707cacf0 con 0x557f708bd5d0 2024-09-12T01:10:23.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e544c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f7081f2b0 con 0x557f708cc780 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e497fa640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 0x7f6e380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54722/0 (socket says 172.21.15.16:54722) 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e4bfff640 1 -- 172.21.15.16:0/2499558822 learned_addr learned my addr 172.21.15.16:0/2499558822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e43fff640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.558+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e497fa640 1 -- 172.21.15.16:0/2156763133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6e340cf630 con 0x7f6e4c102790 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 0x7f6e380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 -- 172.21.15.16:0/2499558822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 msgr2=0x557f708ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 -- 172.21.15.16:0/2499558822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 msgr2=0x557f708bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 -- 172.21.15.16:0/2499558822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f707af3a0 con 0x557f708beb40 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6e44005cc0 tx=0x7f6e44031d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ecb8dd85097ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 0x7f6e380a51b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6e44031f80 tx=0x7f6e44048000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e41ffb640 1 -- 172.21.15.16:0/2499558822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4404f070 con 0x557f708beb40 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e41ffb640 1 -- 172.21.15.16:0/2499558822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e44036040 con 0x557f708beb40 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.559+0000 7f6e41ffb640 1 -- 172.21.15.16:0/2499558822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4404a450 con 0x557f708beb40 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 -- 172.21.15.16:0/2499558822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 msgr2=0x557f708bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f6e44005cc0 tx=0x7f6e44031d60 comp rx=0 tx=0).stop 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 -- 172.21.15.16:0/2499558822 shutdown_connections 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 --2- 172.21.15.16:0/2499558822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708bd090 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 -- 172.21.15.16:0/2499558822 >> 172.21.15.16:0/2499558822 conn(0x557f70886310 msgr2=0x557f708b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 -- 172.21.15.16:0/2499558822 shutdown_connections 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.560+0000 7f6e544c6740 1 -- 172.21.15.16:0/2499558822 wait complete. 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 -- start start 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 Processor -- start 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708beb40 0x557f708c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f708cf2a0 con 0x557f708cc780 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557f707de060 con 0x557f708bd5d0 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f708cf420 con 0x557f708beb40 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e544c6740 10 client.0 _ll_register_callbacks cb 0x557f705318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54738/0 (socket says 172.21.15.16:54738) 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e4bfff640 1 -- 172.21.15.16:0/3717485678 learned_addr learned my addr 172.21.15.16:0/3717485678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.561+0000 7f6e43fff640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708beb40 0x557f708c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 -- 172.21.15.16:0/3717485678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708beb40 msgr2=0x557f708c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708beb40 0x557f708c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 -- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 msgr2=0x557f708cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 -- 172.21.15.16:0/3717485678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f705943c0 con 0x557f708cc780 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009ca0 tx=0x7f6e4402fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498e06e2f09578aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44057070 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e44036040 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6e4405c460 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e4404f050 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e544c6740 10 client.19785 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f708cf2a0 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.562+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e44036040 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e44065b90 con 0x557f708cc780 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 10 client.19785 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 10 client.19785 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 20 client.19785 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f708dbf70 0x557f708de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 4 client.19785 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f70830860 con 0x557f708dbf70 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f708dbf70 0x557f708de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f708dbf70 0x557f708de360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c868f438a352dfe4 server_cookie=ebd0458d3dcf05c7 in_seq=0 out_seq=0 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.563+0000 7f6e2affd640 10 client.19785 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.564+0000 7f6e2affd640 1 -- 172.21.15.16:0/3717485678 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f70830860 con 0x557f708dbf70 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.564+0000 7f6e2affd640 10 client.19785 handle_command_reply: tid=0 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.564+0000 7f6e544c6740 1 client.19785 shutdown 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.564+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 msgr2=0x557f708c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.564+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009ca0 tx=0x7f6e4402fc90 comp rx=0 tx=0).stop 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 shutdown_connections 2024-09-12T01:10:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f708dbf70 0x557f708de360 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708cc780 0x557f708c9c60 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708beb40 0x557f708c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e2affd640 0 client.19785 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3717485678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 >> 172.21.15.16:0/3717485678 conn(0x557f70886310 msgr2=0x557f708bf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 shutdown_connections 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 1 -- 172.21.15.16:0/3717485678 wait complete. 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.565+0000 7f6e544c6740 20 client.19785 trim_cache size 0 max 16384 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 Processor -- start 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 -- start start 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1ab670 con 0x557f708beb40 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1ad1b0 con 0x557f708bd5d0 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e544c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1abd40 con 0x557f708cc780 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54744/0 (socket says 172.21.15.16:54744) 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 -- 172.21.15.16:0/935806746 learned_addr learned my addr 172.21.15.16:0/935806746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 -- 172.21.15.16:0/935806746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 msgr2=0x557f708e3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708e3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 -- 172.21.15.16:0/935806746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 msgr2=0x557f708c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 -- 172.21.15.16:0/935806746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f708c7a20 con 0x557f708beb40 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.578+0000 7f6e43fff640 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f6e3409b7f0 tx=0x7f6e340d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd0eb0326e09b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e417fa640 1 -- 172.21.15.16:0/935806746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3409fea0 con 0x557f708beb40 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e417fa640 1 -- 172.21.15.16:0/935806746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e340a4760 con 0x557f708beb40 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e417fa640 1 -- 172.21.15.16:0/935806746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e340982f0 con 0x557f708beb40 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 -- 172.21.15.16:0/935806746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 msgr2=0x557f708c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f6e3409b7f0 tx=0x7f6e340d1040 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 -- 172.21.15.16:0/935806746 shutdown_connections 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708cc780 0x557f708e3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c84b0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 --2- 172.21.15.16:0/935806746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708bd5d0 0x557f708c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 -- 172.21.15.16:0/935806746 >> 172.21.15.16:0/935806746 conn(0x557f70886310 msgr2=0x557f708c7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 -- 172.21.15.16:0/935806746 shutdown_connections 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.579+0000 7f6e544c6740 1 -- 172.21.15.16:0/935806746 wait complete. 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 -- start start 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 Processor -- start 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708bd5d0 0x557f708c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 0x557f708c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1acae0 con 0x557f708beb40 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1ac410 con 0x557f708cc780 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4c1a9690 con 0x557f708bd5d0 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e544c6740 10 client.0 _ll_register_callbacks cb 0x7f6e4404d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e43fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54754/0 (socket says 172.21.15.16:54754) 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.580+0000 7f6e43fff640 1 -- 172.21.15.16:0/3198753759 learned_addr learned my addr 172.21.15.16:0/3198753759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708bd5d0 0x557f708c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 0x557f708c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 -- 172.21.15.16:0/3198753759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708bd5d0 msgr2=0x557f708c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708bd5d0 0x557f708c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 -- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 msgr2=0x557f708c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 0x557f708c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 -- 172.21.15.16:0/3198753759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f70945b30 con 0x557f708beb40 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e4bfff640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 0x557f708c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e43fff640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6e3409ba30 tx=0x7f6e3409a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9836f0a59d08a804 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3406a070 con 0x557f708beb40 2024-09-12T01:10:23.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f6e340ccfa0 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e340a4760 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6e340ba300 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e544c6740 10 client.19791 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f70946d90 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.581+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e340a4760 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6e3409af40 con 0x557f708beb40 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 10 client.19791 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 10 client.19791 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 20 client.19791 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f709474a0 0x557f709498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 4 client.19791 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f7082ad70 con 0x557f709474a0 2024-09-12T01:10:23.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f709474a0 0x557f709498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e4b7fe640 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f709474a0 0x557f709498c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=114ddff05a1b206c server_cookie=b0317d8b418f6ebf in_seq=0 out_seq=0 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.582+0000 7f6e2affd640 10 client.19791 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e2affd640 1 -- 172.21.15.16:0/3198753759 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f7082ad70 con 0x557f709474a0 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e2affd640 10 client.19791 handle_command_reply: tid=0 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 client.19791 shutdown 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 msgr2=0x557f708c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f6e3409ba30 tx=0x7f6e3409a000 comp rx=0 tx=0).stop 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 shutdown_connections 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x557f709474a0 0x557f709498c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557f708cc780 0x557f708c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f708beb40 0x557f708c69a0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e2affd640 0 client.19791 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:10:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 --2- 172.21.15.16:0/3198753759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f708bd5d0 0x557f708c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 >> 172.21.15.16:0/3198753759 conn(0x557f70886310 msgr2=0x557f708b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 shutdown_connections 2024-09-12T01:10:23.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 1 -- 172.21.15.16:0/3198753759 wait complete. 2024-09-12T01:10:23.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e544c6740 20 client.19791 trim_cache size 0 max 16384 2024-09-12T01:10:23.601 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 msgr2=0x7f6e380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 0x7f6e380a51b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6e44031f80 tx=0x7f6e44048000 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 msgr2=0x7f6e4c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.583+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c920 tx=0x7f6e3400cde0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 shutdown_connections 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f6e380a2cf0 0x7f6e380a51b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6e4c106f10 0x7f6e4c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c102790 0x7f6e4c198890 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 --2- 172.21.15.16:0/2156763133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4c101de0 0x7f6e4c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 >> 172.21.15.16:0/2156763133 conn(0x7f6e4c0fd8c0 msgr2=0x7f6e4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 shutdown_connections 2024-09-12T01:10:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:23.584+0000 7f6e520ed640 1 -- 172.21.15.16:0/2156763133 wait complete. 2024-09-12T01:10:24.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:10:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:24 smithi016 ceph-mon[30972]: pgmap v447: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:10:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:10:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:10:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:24 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:10:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:24 smithi099 ceph-mon[35654]: pgmap v447: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:10:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:10:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:10:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:24 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:10:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:24 smithi159 ceph-mon[35566]: pgmap v447: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-09-12T01:10:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:10:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:10:25.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:10:25.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:24 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:10:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3287304022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 msgr2=0x7fafd0110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3287304022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd0110870 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ca0 tx=0x7fafc802a8c0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3287304022 shutdown_connections 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3287304022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd0110870 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3287304022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd0101280 0x7fafd010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3287304022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01008d0 0x7fafd0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3287304022 >> 172.21.15.16:0/3287304022 conn(0x7fafd00fc7a0 msgr2=0x7fafd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3287304022 shutdown_connections 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3287304022 wait complete. 2024-09-12T01:10:25.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 Processor -- start 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.148+0000 7fafd7ef4640 1 -- start start 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 0x7fafd019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0101280 0x7fafd019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0114160 con 0x7fafd010e480 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0113fe0 con 0x7fafd0101280 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd7ef4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01142e0 con 0x7fafd01008d0 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54770/0 (socket says 172.21.15.16:54770) 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 -- 172.21.15.16:0/3447746539 learned_addr learned my addr 172.21.15.16:0/3447746539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd548f640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0101280 0x7fafd019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 0x7fafd019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 -- 172.21.15.16:0/3447746539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 msgr2=0x7fafd019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 0x7fafd019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 -- 172.21.15.16:0/3447746539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0101280 msgr2=0x7fafd019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0101280 0x7fafd019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 -- 172.21.15.16:0/3447746539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019f670 con 0x7fafd010e480 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 0x7fafd019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.149+0000 7fafd6491640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fafc802ade0 tx=0x7fafc80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.150+0000 7fafc6ffd640 1 -- 172.21.15.16:0/3447746539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8003c30 con 0x7fafd010e480 2024-09-12T01:10:25.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.150+0000 7fafc6ffd640 1 -- 172.21.15.16:0/3447746539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc8003dd0 con 0x7fafd010e480 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.150+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019f900 con 0x7fafd010e480 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.150+0000 7fafc6ffd640 1 -- 172.21.15.16:0/3447746539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc802fd60 con 0x7fafd010e480 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.150+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a7240 con 0x7fafd010e480 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafc6ffd640 1 -- 172.21.15.16:0/3447746539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fafc8004050 con 0x7fafd010e480 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 Processor -- start 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 -- start start 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab511090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab51e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559aab4772f0 con 0x559aab5115d0 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559aab41ecf0 con 0x559aab51c370 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.153+0000 7fafda2cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559aab4732b0 con 0x559aab512b40 2024-09-12T01:10:25.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54786/0 (socket says 172.21.15.16:54786) 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 -- 172.21.15.16:0/3878206755 learned_addr learned my addr 172.21.15.16:0/3878206755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd548f640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab511090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd548f640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab51e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 -- 172.21.15.16:0/3878206755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 msgr2=0x559aab511090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab511090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 -- 172.21.15.16:0/3878206755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 msgr2=0x559aab51e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab51e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 -- 172.21.15.16:0/3878206755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aab4033a0 con 0x559aab5115d0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd548f640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab51e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafc6ffd640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 0x7fafbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fafcc008c60 tx=0x7fafcc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bbc533b633c302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafd548f640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 0x7fafbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafc6ffd640 1 -- 172.21.15.16:0/3447746539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fafc80f0720 con 0x7fafd010e480 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafbbfff640 1 -- 172.21.15.16:0/3878206755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc014070 con 0x559aab5115d0 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafbbfff640 1 -- 172.21.15.16:0/3878206755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc0047a0 con 0x559aab5115d0 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.154+0000 7fafbbfff640 1 -- 172.21.15.16:0/3878206755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc010760 con 0x559aab5115d0 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 -- 172.21.15.16:0/3878206755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 msgr2=0x559aab511a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafd548f640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 0x7fafbc0a51b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x559aab512070 tx=0x7fafc000b410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fafcc008c60 tx=0x7fafcc00e720 comp rx=0 tx=0).stop 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 -- 172.21.15.16:0/3878206755 shutdown_connections 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab51e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab511a10 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3878206755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab511090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 -- 172.21.15.16:0/3878206755 >> 172.21.15.16:0/3878206755 conn(0x559aab4da310 msgr2=0x559aab50dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 -- 172.21.15.16:0/3878206755 shutdown_connections 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.155+0000 7fafda2cd740 1 -- 172.21.15.16:0/3878206755 wait complete. 2024-09-12T01:10:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 -- start start 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 Processor -- start 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab51bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab5197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559aab51f070 con 0x559aab512b40 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559aab432060 con 0x559aab51c370 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559aab51f1f0 con 0x559aab5115d0 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafda2cd740 10 client.0 _ll_register_callbacks cb 0x559aab1858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafd548f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafd548f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54790/0 (socket says 172.21.15.16:54790) 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafd548f640 1 -- 172.21.15.16:0/3395937321 learned_addr learned my addr 172.21.15.16:0/3395937321 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.156+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab51bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd6491640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab5197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 -- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 msgr2=0x559aab51bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab51bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 -- 172.21.15.16:0/3395937321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 msgr2=0x559aab5197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab5197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 -- 172.21.15.16:0/3395937321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aab1e83c0 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab51bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafd548f640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005940 tx=0x7fafc000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80469e983793ef0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc001e900 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fafc001eaa0 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafcc0047a0 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafc001eca0 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafda2cd740 10 client.19803 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559aab52ab30 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.157+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafcc0047a0 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafc0028470 con 0x559aab512b40 2024-09-12T01:10:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 10 client.19803 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 10 client.19803 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 20 client.19803 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab52bf40 0x559aab52e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 4 client.19803 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aab484860 con 0x559aab52bf40 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab52bf40 0x559aab52e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafb97fa640 10 client.19803 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.158+0000 7fafd5c90640 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab52bf40 0x559aab52e3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ce34185c439f906 server_cookie=d4d3e74df4a13158 in_seq=0 out_seq=0 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafb97fa640 1 -- 172.21.15.16:0/3395937321 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559aab484860 con 0x559aab52bf40 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafb97fa640 10 client.19803 handle_command_reply: tid=0 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 client.19803 shutdown 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 msgr2=0x559aab519060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fafc0005940 tx=0x7fafc000b650 comp rx=0 tx=0).stop 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 shutdown_connections 2024-09-12T01:10:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab5197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab52bf40 0x559aab52e3b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab519060 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 --2- 172.21.15.16:0/3395937321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab51bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafb97fa640 0 client.19803 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 >> 172.21.15.16:0/3395937321 conn(0x559aab4da310 msgr2=0x559aab513960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 shutdown_connections 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 1 -- 172.21.15.16:0/3395937321 wait complete. 2024-09-12T01:10:25.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.159+0000 7fafda2cd740 20 client.19803 trim_cache size 0 max 16384 2024-09-12T01:10:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 Processor -- start 2024-09-12T01:10:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 -- start start 2024-09-12T01:10:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab52e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab50dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01afba0 con 0x559aab512b40 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b16e0 con 0x559aab51c370 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafda2cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b0270 con 0x559aab5115d0 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafd548f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafd548f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54804/0 (socket says 172.21.15.16:54804) 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafd548f640 1 -- 172.21.15.16:0/1167158226 learned_addr learned my addr 172.21.15.16:0/1167158226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafd5c90640 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab52e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.173+0000 7fafd6491640 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab50dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 -- 172.21.15.16:0/1167158226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 msgr2=0x559aab52e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab52e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 -- 172.21.15.16:0/1167158226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 msgr2=0x559aab50dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab50dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 -- 172.21.15.16:0/1167158226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aab50e180 con 0x559aab512b40 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafd548f640 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fafc0012e40 tx=0x7fafc000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff7a5bb17665426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafbb7fe640 1 -- 172.21.15.16:0/1167158226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0028470 con 0x559aab512b40 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafbb7fe640 1 -- 172.21.15.16:0/1167158226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc0023070 con 0x559aab512b40 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.174+0000 7fafbb7fe640 1 -- 172.21.15.16:0/1167158226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00049c0 con 0x559aab512b40 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- 172.21.15.16:0/1167158226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 msgr2=0x559aab50d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fafc0012e40 tx=0x7fafc000b560 comp rx=0 tx=0).stop 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- 172.21.15.16:0/1167158226 shutdown_connections 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab50dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab512b40 0x559aab50d700 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 --2- 172.21.15.16:0/1167158226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab5115d0 0x559aab52e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- 172.21.15.16:0/1167158226 >> 172.21.15.16:0/1167158226 conn(0x559aab4da310 msgr2=0x559aab508a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- 172.21.15.16:0/1167158226 shutdown_connections 2024-09-12T01:10:25.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- 172.21.15.16:0/1167158226 wait complete. 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 -- start start 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.175+0000 7fafda2cd740 1 Processor -- start 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab515cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab516230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b1010 con 0x559aab5115d0 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01b0940 con 0x559aab51c370 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafd01adba0 con 0x559aab512b40 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafda2cd740 10 client.0 _ll_register_callbacks cb 0x7fafc000bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54806/0 (socket says 172.21.15.16:54806) 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 -- 172.21.15.16:0/4027995297 learned_addr learned my addr 172.21.15.16:0/4027995297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd6491640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab516230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd548f640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab515cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 -- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 msgr2=0x559aab515cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab515cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 -- 172.21.15.16:0/4027995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 msgr2=0x559aab516230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab516230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 -- 172.21.15.16:0/4027995297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aab516940 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafd5c90640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005e10 tx=0x7fafcc008770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98608d55743e030f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.176+0000 7fafb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc014040 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7fafcc025890 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc0023070 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafcc004820 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 10 client.19809 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559aab596bf0 con 0x559aab5115d0 2024-09-12T01:10:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc0023070 con 0x559aab5115d0 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fafcc02ebf0 con 0x559aab5115d0 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 10 client.19809 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 10 client.19809 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 20 client.19809 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab597040 0x559aab599460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 4 client.19809 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aab47ed70 con 0x559aab597040 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.177+0000 7fafd548f640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab597040 0x559aab599460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafd548f640 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab597040 0x559aab599460 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e292e19ea7f542a server_cookie=24302d339d0f13ce in_seq=0 out_seq=0 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafb97fa640 10 client.19809 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafb97fa640 1 -- 172.21.15.16:0/4027995297 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aab47ed70 con 0x559aab597040 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafb97fa640 10 client.19809 handle_command_reply: tid=0 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 client.19809 shutdown 2024-09-12T01:10:25.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 msgr2=0x559aab524a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005e10 tx=0x7fafcc008770 comp rx=0 tx=0).stop 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 shutdown_connections 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aab51c370 0x559aab516230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559aab512b40 0x559aab515cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x559aab597040 0x559aab599460 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 --2- 172.21.15.16:0/4027995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559aab5115d0 0x559aab524a90 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafb97fa640 0 client.19809 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 >> 172.21.15.16:0/4027995297 conn(0x559aab4da310 msgr2=0x559aab517520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 shutdown_connections 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 1 -- 172.21.15.16:0/4027995297 wait complete. 2024-09-12T01:10:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.178+0000 7fafda2cd740 20 client.19809 trim_cache size 0 max 16384 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 msgr2=0x7fafbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 0x7fafbc0a51b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x559aab512070 tx=0x7fafc000b410 comp rx=0 tx=0).stop 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 msgr2=0x7fafd019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fafc802ade0 tx=0x7fafc80319f0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 shutdown_connections 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fafbc0a2cf0 0x7fafbc0a51b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd010e480 0x7fafd019ef90 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd0101280 0x7fafd019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 --2- 172.21.15.16:0/3447746539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafd01008d0 0x7fafd019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 >> 172.21.15.16:0/3447746539 conn(0x7fafd00fc7a0 msgr2=0x7fafd00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 shutdown_connections 2024-09-12T01:10:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:25.179+0000 7fafd7ef4640 1 -- 172.21.15.16:0/3447746539 wait complete. 2024-09-12T01:10:25.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:10:26.257 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1531617227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8102790 msgr2=0x7ffbe81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1531617227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8102790 0x7ffbe81067e0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009900 tx=0x7ffbd802a410 comp rx=0 tx=0).stop 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1531617227 shutdown_connections 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1531617227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8106f10 0x7ffbe810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1531617227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8102790 0x7ffbe81067e0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1531617227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 0x7ffbe81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1531617227 >> 172.21.15.16:0/1531617227 conn(0x7ffbe80fd8c0 msgr2=0x7ffbe80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1531617227 shutdown_connections 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.755+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1531617227 wait complete. 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 Processor -- start 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 -- start start 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 0x7ffbe8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8102790 0x7ffbe8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8111f90 con 0x7ffbe8106f10 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8111e10 con 0x7ffbe8101de0 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbeff2e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112110 con 0x7ffbe8102790 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbee4cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbee4cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54828/0 (socket says 172.21.15.16:54828) 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbee4cb640 1 -- 172.21.15.16:0/1521113857 learned_addr learned my addr 172.21.15.16:0/1521113857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8102790 0x7ffbe8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.756+0000 7ffbedcca640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 0x7ffbe8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 -- 172.21.15.16:0/1521113857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8102790 msgr2=0x7ffbe8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8102790 0x7ffbe8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 -- 172.21.15.16:0/1521113857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 msgr2=0x7ffbe8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 0x7ffbe8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 -- 172.21.15.16:0/1521113857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819d3c0 con 0x7ffbe8106f10 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbee4cb640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000cce0 tx=0x7ffbe000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbdeffd640 1 -- 172.21.15.16:0/1521113857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe000f450 con 0x7ffbe8106f10 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe819d6b0 con 0x7ffbe8106f10 2024-09-12T01:10:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbdeffd640 1 -- 172.21.15.16:0/1521113857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe000fad0 con 0x7ffbe8106f10 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbdeffd640 1 -- 172.21.15.16:0/1521113857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0018c10 con 0x7ffbe8106f10 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.757+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a4f90 con 0x7ffbe8106f10 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.759+0000 7ffbdeffd640 1 -- 172.21.15.16:0/1521113857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffbe000f5f0 con 0x7ffbe8106f10 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 Processor -- start 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 -- start start 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 0x55f8d8d20b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8c752f0 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8c1ccf0 con 0x55f8d8d10b40 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbf2307740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8c712b0 con 0x55f8d8d1e780 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbdeffd640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 0x7ffbd40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbedcca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbedcca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54840/0 (socket says 172.21.15.16:54840) 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbedcca640 1 -- 172.21.15.16:0/2641266624 learned_addr learned my addr 172.21.15.16:0/2641266624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 0x55f8d8d20b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.760+0000 7ffbee4cb640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 0x7ffbd40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbdeffd640 1 -- 172.21.15.16:0/1521113857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7ffbe00cfb40 con 0x7ffbe8106f10 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 -- 172.21.15.16:0/2641266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 msgr2=0x55f8d8d20b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 0x55f8d8d20b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 -- 172.21.15.16:0/2641266624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 msgr2=0x55f8d8d0f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d0f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 -- 172.21.15.16:0/2641266624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d8c013a0 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 0x55f8d8d20b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbedcca640 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400b820 tx=0x7ffbe400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff2e8cf2959ed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbee4cb640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 0x7ffbd40a51b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00cd340 tx=0x7ffbe006f0d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbd3fff640 1 -- 172.21.15.16:0/2641266624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400d950 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbd3fff640 1 -- 172.21.15.16:0/2641266624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe4019070 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.761+0000 7ffbd3fff640 1 -- 172.21.15.16:0/2641266624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40123e0 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- 172.21.15.16:0/2641266624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 msgr2=0x55f8d8d0fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400b820 tx=0x7ffbe400bce0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- 172.21.15.16:0/2641266624 shutdown_connections 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d1e780 0x55f8d8d20b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0fa10 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2641266624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d0f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- 172.21.15.16:0/2641266624 >> 172.21.15.16:0/2641266624 conn(0x55f8d8cd8310 msgr2=0x55f8d8d0bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- 172.21.15.16:0/2641266624 shutdown_connections 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- 172.21.15.16:0/2641266624 wait complete. 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.762+0000 7ffbf2307740 1 -- start start 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 Processor -- start 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d1b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8d21480 con 0x55f8d8d1e780 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8c30060 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f8d8d21600 con 0x55f8d8d10b40 2024-09-12T01:10:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbf2307740 10 client.0 _ll_register_callbacks cb 0x55f8d89838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54846/0 (socket says 172.21.15.16:54846) 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 -- 172.21.15.16:0/303375940 learned_addr learned my addr 172.21.15.16:0/303375940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbedcca640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d1b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 -- 172.21.15.16:0/303375940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 msgr2=0x55f8d8d1b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d1b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 -- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d0f5d0 msgr2=0x55f8d8d1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbed4c9640 1 -- 172.21.15.16:0/303375940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d89e63c0 con 0x55f8d8d1e780 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.763+0000 7ffbedcca640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d1b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802a930 tx=0x7ffbd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47d64051670e2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8044030 con 0x55f8d8d1e780 2024-09-12T01:10:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7ffbd8005020 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbe4019070 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbd8052050 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 10 client.19821 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8d8d2ceb0 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe4019070 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbd80035f0 con 0x55f8d8d1e780 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 10 client.19821 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 10 client.19821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 20 client.19821 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d2e2c0 0x55f8d8d306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 4 client.19821 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.764+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8d8c82860 con 0x55f8d8d2e2c0 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.765+0000 7ffbedcca640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d2e2c0 0x55f8d8d306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.765+0000 7ffbedcca640 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d2e2c0 0x55f8d8d306b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a3e14f17aab5788 server_cookie=2cb9141422c6baf4 in_seq=0 out_seq=0 2024-09-12T01:10:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.765+0000 7ffbd17fa640 10 client.19821 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbd17fa640 1 -- 172.21.15.16:0/303375940 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8d8c82860 con 0x55f8d8d2e2c0 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbd17fa640 10 client.19821 handle_command_reply: tid=0 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 client.19821 shutdown 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 msgr2=0x55f8d8d1b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802a930 tx=0x7ffbd8009260 comp rx=0 tx=0).stop 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 shutdown_connections 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d2e2c0 0x55f8d8d306b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d1b940 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d1b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 --2- 172.21.15.16:0/303375940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbd17fa640 0 client.19821 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:10:26.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 >> 172.21.15.16:0/303375940 conn(0x55f8d8cd8310 msgr2=0x55f8d8d11960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 shutdown_connections 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 1 -- 172.21.15.16:0/303375940 wait complete. 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.766+0000 7ffbf2307740 20 client.19821 trim_cache size 0 max 16384 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 Processor -- start 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 -- start start 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d2a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d1e780 0x55f8d8d35f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ab670 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ad1b0 con 0x55f8d8d1e780 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbf2307740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81abd40 con 0x55f8d8d10b40 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.779+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54858/0 (socket says 172.21.15.16:54858) 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 -- 172.21.15.16:0/2412585430 learned_addr learned my addr 172.21.15.16:0/2412585430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d1e780 0x55f8d8d35f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 -- 172.21.15.16:0/2412585430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 msgr2=0x55f8d8d2a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbedcca640 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d2a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d2a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 -- 172.21.15.16:0/2412585430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d1e780 msgr2=0x55f8d8d35f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d1e780 0x55f8d8d35f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 -- 172.21.15.16:0/2412585430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8d8d0f2d0 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80095c0 tx=0x7ffbd802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e97ac383503340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbd37fe640 1 -- 172.21.15.16:0/2412585430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8044040 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbd37fe640 1 -- 172.21.15.16:0/2412585430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd80037f0 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.780+0000 7ffbd37fe640 1 -- 172.21.15.16:0/2412585430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8035040 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- 172.21.15.16:0/2412585430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 msgr2=0x55f8d8d0be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80095c0 tx=0x7ffbd802a410 comp rx=0 tx=0).stop 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- 172.21.15.16:0/2412585430 shutdown_connections 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d1e780 0x55f8d8d35f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d10b40 0x55f8d8d2a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 --2- 172.21.15.16:0/2412585430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d0be10 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- 172.21.15.16:0/2412585430 >> 172.21.15.16:0/2412585430 conn(0x55f8d8cd8310 msgr2=0x55f8d8d0b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- 172.21.15.16:0/2412585430 shutdown_connections 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- 172.21.15.16:0/2412585430 wait complete. 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 -- start start 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.781+0000 7ffbf2307740 1 Processor -- start 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d22e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81acae0 con 0x55f8d8d1e780 2024-09-12T01:10:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ac410 con 0x55f8d8d10b40 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81a9670 con 0x55f8d8d0f5d0 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbf2307740 10 client.0 _ll_register_callbacks cb 0x7ffbd8032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54870/0 (socket says 172.21.15.16:54870) 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 -- 172.21.15.16:0/3917252740 learned_addr learned my addr 172.21.15.16:0/3917252740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbedcca640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d22e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 -- 172.21.15.16:0/3917252740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d0f5d0 msgr2=0x55f8d8d228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d228d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 -- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 msgr2=0x55f8d8d22e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d22e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 -- 172.21.15.16:0/3917252740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8d8d981d0 con 0x55f8d8d1e780 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbedcca640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d22e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbed4c9640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802ac00 tx=0x7ffbd8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84b7481c271b258 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.782+0000 7ffbd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8035070 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7ffbd80035f0 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd80037f0 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbd8044040 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 10 client.19827 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8d8d994b0 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd80037f0 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffbd8004960 con 0x55f8d8d1e780 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 10 client.19827 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 10 client.19827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 20 client.19827 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d99bc0 0x55f8d8d9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 4 client.19827 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:10:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8d8c7cd70 con 0x55f8d8d99bc0 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.783+0000 7ffbedcca640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d99bc0 0x55f8d8d9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbedcca640 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d99bc0 0x55f8d8d9bfe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf66b6be8337b3cc server_cookie=df2e1dd51c9c55a2 in_seq=0 out_seq=0 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbd17fa640 10 client.19827 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbd17fa640 1 -- 172.21.15.16:0/3917252740 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8d8c7cd70 con 0x55f8d8d99bc0 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbd17fa640 10 client.19827 handle_command_reply: tid=0 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 client.19827 shutdown 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 msgr2=0x55f8d8d23350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ffbd802ac00 tx=0x7ffbd8041040 comp rx=0 tx=0).stop 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 shutdown_connections 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55f8d8d99bc0 0x55f8d8d9bfe0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8d8d1e780 0x55f8d8d23350 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8d8d10b40 0x55f8d8d22e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbd17fa640 0 client.19827 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 --2- 172.21.15.16:0/3917252740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8d8d0f5d0 0x55f8d8d228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 >> 172.21.15.16:0/3917252740 conn(0x55f8d8cd8310 msgr2=0x55f8d8d0a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 shutdown_connections 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 1 -- 172.21.15.16:0/3917252740 wait complete. 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.784+0000 7ffbf2307740 20 client.19827 trim_cache size 0 max 16384 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 msgr2=0x7ffbd40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 0x7ffbd40a51b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00cd340 tx=0x7ffbe006f0d0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 msgr2=0x7ffbe819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000cce0 tx=0x7ffbe000a620 comp rx=0 tx=0).stop 2024-09-12T01:10:26.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 shutdown_connections 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffbd40a2cf0 0x7ffbd40a51b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe8106f10 0x7ffbe819cce0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbe8102790 0x7ffbe8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 --2- 172.21.15.16:0/1521113857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbe8101de0 0x7ffbe8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 >> 172.21.15.16:0/1521113857 conn(0x7ffbe80fd8c0 msgr2=0x7ffbe80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 shutdown_connections 2024-09-12T01:10:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:26.785+0000 7ffbeff2e640 1 -- 172.21.15.16:0/1521113857 wait complete. 2024-09-12T01:10:26.918 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:26 smithi159 ceph-mon[35566]: pgmap v448: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:26 smithi099 ceph-mon[35654]: pgmap v448: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:27.100 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:26 smithi016 ceph-mon[30972]: pgmap v448: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:27.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:10:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.351+0000 7f049feab640 1 -- 172.21.15.16:0/3201105421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498103740 msgr2=0x7f0498107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.351+0000 7f049feab640 1 --2- 172.21.15.16:0/3201105421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498103740 0x7f0498107b80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f048c009930 tx=0x7f048c02a420 comp rx=0 tx=0).stop 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.351+0000 7f049feab640 1 -- 172.21.15.16:0/3201105421 shutdown_connections 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- 172.21.15.16:0/3201105421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498108290 0x7f049810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- 172.21.15.16:0/3201105421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498103740 0x7f0498107b80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- 172.21.15.16:0/3201105421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498102d90 0x7f0498103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- 172.21.15.16:0/3201105421 >> 172.21.15.16:0/3201105421 conn(0x7f04980fc7d0 msgr2=0x7f04980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- 172.21.15.16:0/3201105421 shutdown_connections 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- 172.21.15.16:0/3201105421 wait complete. 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 Processor -- start 2024-09-12T01:10:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- start start 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498103740 0x7f0498198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498108290 0x7f049819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112090 con 0x7f0498102d90 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0498111f10 con 0x7f0498108290 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.352+0000 7f049feab640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0498112210 con 0x7f0498103740 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54888/0 (socket says 172.21.15.16:54888) 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 -- 172.21.15.16:0/1510355196 learned_addr learned my addr 172.21.15.16:0/1510355196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049d446640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498103740 0x7f0498198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049e448640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498108290 0x7f049819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 -- 172.21.15.16:0/1510355196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498103740 msgr2=0x7f0498198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498103740 0x7f0498198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 -- 172.21.15.16:0/1510355196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498108290 msgr2=0x7f049819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498108290 0x7f049819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 -- 172.21.15.16:0/1510355196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049819d4d0 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049dc47640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f048800cce0 tx=0x7f048800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f0496ffd640 1 -- 172.21.15.16:0/1510355196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048800f650 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049819d7c0 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f0496ffd640 1 -- 172.21.15.16:0/1510355196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048800fcd0 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f0496ffd640 1 -- 172.21.15.16:0/1510355196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488018c70 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.353+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04981a50a0 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.356+0000 7f0496ffd640 1 -- 172.21.15.16:0/1510355196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f048800f7f0 con 0x7f0498102d90 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.356+0000 7f04a2284740 1 Processor -- start 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.356+0000 7f04a2284740 1 -- start start 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x555950640090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950640a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5559505a62f0 con 0x55595064f780 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55595054dcf0 con 0x555950641b40 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f04a2284740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559505a22b0 con 0x5559506405d0 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f0496ffd640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 0x7f04800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54900/0 (socket says 172.21.15.16:54900) 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 -- 172.21.15.16:0/4271584499 learned_addr learned my addr 172.21.15.16:0/4271584499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049e448640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 0x7f04800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049e448640 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x555950640090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f0496ffd640 1 -- 172.21.15.16:0/1510355196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f04880cf8a0 con 0x7f0498102d90 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049dc47640 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950640a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 -- 172.21.15.16:0/4271584499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 msgr2=0x555950640a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950640a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 -- 172.21.15.16:0/4271584499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 msgr2=0x555950640090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x555950640090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 -- 172.21.15.16:0/4271584499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559505323a0 con 0x55595064f780 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049d446640 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f04980697d0 tx=0x7f048c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef625a6906b5a509 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.357+0000 7f049e448640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 0x7f04800a51b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0490012890 tx=0x7f04900046c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f047ffff640 1 -- 172.21.15.16:0/4271584499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c040070 con 0x55595064f780 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f047ffff640 1 -- 172.21.15.16:0/4271584499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048c005020 con 0x55595064f780 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f047ffff640 1 -- 172.21.15.16:0/4271584499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c035040 con 0x55595064f780 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 -- 172.21.15.16:0/4271584499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 msgr2=0x555950651b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f04980697d0 tx=0x7f048c031630 comp rx=0 tx=0).stop 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 -- 172.21.15.16:0/4271584499 shutdown_connections 2024-09-12T01:10:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55595064f780 0x555950651b70 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950640a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 --2- 172.21.15.16:0/4271584499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x555950640090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.358+0000 7f04a2284740 1 -- 172.21.15.16:0/4271584499 >> 172.21.15.16:0/4271584499 conn(0x555950609310 msgr2=0x55595063cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- 172.21.15.16:0/4271584499 shutdown_connections 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- 172.21.15.16:0/4271584499 wait complete. 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- start start 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 Processor -- start 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x55595064c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 0x55595064c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555950652420 con 0x5559506405d0 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555950561060 con 0x55595064f780 2024-09-12T01:10:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559506525a0 con 0x555950641b40 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.359+0000 7f04a2284740 10 client.0 _ll_register_callbacks cb 0x5559502b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54914/0 (socket says 172.21.15.16:54914) 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 -- 172.21.15.16:0/1203706322 learned_addr learned my addr 172.21.15.16:0/1203706322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049d446640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x55595064c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049e448640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 0x55595064c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 -- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 msgr2=0x55595064c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x55595064c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 -- 172.21.15.16:0/1203706322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 msgr2=0x55595064c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 0x55595064c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 -- 172.21.15.16:0/1203706322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559503173c0 con 0x5559506405d0 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049e448640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 0x55595064c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f049dc47640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f048809ba60 tx=0x7f048809b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb6fa08a2d859c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f047d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04880cd420 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f04880a4740 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f048c005020 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04880a0110 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f04a2284740 10 client.19839 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.360+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55595065de10 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048c005020 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f048809b2d0 con 0x5559506405d0 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 10 client.19839 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 10 client.19839 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 20 client.19839 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595065f2d0 0x555950661720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 4 client.19839 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559505b3860 con 0x55595065f2d0 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f049d446640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595065f2d0 0x555950661720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f047d7fa640 10 client.19839 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.361+0000 7f049d446640 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595065f2d0 0x555950661720 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2eda14e123b9df65 server_cookie=1ea126aed1611ca8 in_seq=0 out_seq=0 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.362+0000 7f047d7fa640 1 -- 172.21.15.16:0/1203706322 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559505b3860 con 0x55595065f2d0 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.362+0000 7f047d7fa640 10 client.19839 handle_command_reply: tid=0 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.362+0000 7f04a2284740 1 client.19839 shutdown 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.362+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 msgr2=0x55595064f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.362+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f048809ba60 tx=0x7f048809b780 comp rx=0 tx=0).stop 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 shutdown_connections 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595065f2d0 0x555950661720 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55595064f780 0x55595064c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x55595064c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 --2- 172.21.15.16:0/1203706322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559506405d0 0x55595064f040 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f047d7fa640 0 client.19839 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 >> 172.21.15.16:0/1203706322 conn(0x555950609310 msgr2=0x555950642960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 shutdown_connections 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 1 -- 172.21.15.16:0/1203706322 wait complete. 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.363+0000 7f04a2284740 20 client.19839 trim_cache size 0 max 16384 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 Processor -- start 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 -- start start 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 0x555950647720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x555950650b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04981ab640 con 0x555950609310 2024-09-12T01:10:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04981ad180 con 0x5559506405d0 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f04a2284740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04981abd10 con 0x555950641b40 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54930/0 (socket says 172.21.15.16:54930) 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 -- 172.21.15.16:0/3713807720 learned_addr learned my addr 172.21.15.16:0/3713807720 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049e448640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x555950650b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049d446640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 0x555950647720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 -- 172.21.15.16:0/3713807720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 msgr2=0x555950650b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x555950650b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 -- 172.21.15.16:0/3713807720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 msgr2=0x555950647720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 0x555950647720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049dc47640 1 -- 172.21.15.16:0/3713807720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555950647ef0 con 0x555950609310 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.376+0000 7f049d446640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 0x555950647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f049dc47640 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f048800ccb0 tx=0x7f0488007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3860ab1943803f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f047f7fe640 1 -- 172.21.15.16:0/3713807720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488098000 con 0x555950609310 2024-09-12T01:10:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f047f7fe640 1 -- 172.21.15.16:0/3713807720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0488018e10 con 0x555950609310 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f047f7fe640 1 -- 172.21.15.16:0/3713807720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048809b410 con 0x555950609310 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 -- 172.21.15.16:0/3713807720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 msgr2=0x55595065b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f048800ccb0 tx=0x7f0488007f30 comp rx=0 tx=0).stop 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 -- 172.21.15.16:0/3713807720 shutdown_connections 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555950641b40 0x555950650b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559506405d0 0x555950647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 --2- 172.21.15.16:0/3713807720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x55595065b6c0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 -- 172.21.15.16:0/3713807720 >> 172.21.15.16:0/3713807720 conn(0x55595064c990 msgr2=0x55595064abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 -- 172.21.15.16:0/3713807720 shutdown_connections 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.377+0000 7f04a2284740 1 -- 172.21.15.16:0/3713807720 wait complete. 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 -- start start 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 Processor -- start 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950657d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x5559506527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04981acab0 con 0x555950609310 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04981ac3e0 con 0x555950641b40 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04981a9620 con 0x5559506405d0 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f04a2284740 10 client.0 _ll_register_callbacks cb 0x7f048809b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f049dc47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54936/0 (socket says 172.21.15.16:54936) 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.378+0000 7f049dc47640 1 -- 172.21.15.16:0/3284239518 learned_addr learned my addr 172.21.15.16:0/3284239518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049d446640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950657d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049e448640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x5559506527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 -- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 msgr2=0x555950657d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950657d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 -- 172.21.15.16:0/3284239518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 msgr2=0x5559506527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x5559506527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 -- 172.21.15.16:0/3284239518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555950652d30 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049d446640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950657d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f049dc47640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f048809ac70 tx=0x7f048809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923168020a837814 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f047d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488085070 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f048809b1b0 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0488018e10 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04880a49d0 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f04a2284740 10 client.19845 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559506ca570 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.379+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0488018e10 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0488081030 con 0x555950609310 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 10 client.19845 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 10 client.19845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 20 client.19845 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595064f780 0x5559506cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 4 client.19845 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559505add70 con 0x55595064f780 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f049d446640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595064f780 0x5559506cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f047d7fa640 10 client.19845 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.380+0000 7f049d446640 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595064f780 0x5559506cca50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fb6fb6cad8e8b3a server_cookie=cb310582d200c022 in_seq=0 out_seq=0 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f047d7fa640 1 -- 172.21.15.16:0/3284239518 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559505add70 con 0x55595064f780 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f047d7fa640 10 client.19845 handle_command_reply: tid=0 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 client.19845 shutdown 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 msgr2=0x555950657840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f048809ac70 tx=0x7f048809b950 comp rx=0 tx=0).stop 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 shutdown_connections 2024-09-12T01:10:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55595064f780 0x5559506cca50 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555950641b40 0x5559506527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f047d7fa640 0 client.19845 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559506405d0 0x555950657d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 --2- 172.21.15.16:0/3284239518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555950609310 0x555950657840 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 >> 172.21.15.16:0/3284239518 conn(0x55595064c990 msgr2=0x555950642110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 shutdown_connections 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 1 -- 172.21.15.16:0/3284239518 wait complete. 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.381+0000 7f04a2284740 20 client.19845 trim_cache size 0 max 16384 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 msgr2=0x7f04800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 0x7f04800a51b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0490012890 tx=0x7f04900046c0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 msgr2=0x7f0498198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f048800cce0 tx=0x7f048800a620 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 shutdown_connections 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f04800a2cf0 0x7f04800a51b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0498108290 0x7f049819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0498103740 0x7f0498198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 --2- 172.21.15.16:0/1510355196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0498102d90 0x7f0498198520 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 >> 172.21.15.16:0/1510355196 conn(0x7f04980fc7d0 msgr2=0x7f0498106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 shutdown_connections 2024-09-12T01:10:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:28.382+0000 7f049feab640 1 -- 172.21.15.16:0/1510355196 wait complete. 2024-09-12T01:10:29.022 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:28 smithi016 ceph-mon[30972]: pgmap v449: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:28 smithi099 ceph-mon[35654]: pgmap v449: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:29.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:10:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:28 smithi159 ceph-mon[35566]: pgmap v449: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-09-12T01:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 -- 172.21.15.16:0/1919648135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c103660 msgr2=0x7f2a8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 --2- 172.21.15.16:0/1919648135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c103660 0x7f2a8c10bb50 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002a480 comp rx=0 tx=0).stop 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 -- 172.21.15.16:0/1919648135 shutdown_connections 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 --2- 172.21.15.16:0/1919648135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c10c170 0x7f2a8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 --2- 172.21.15.16:0/1919648135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c103660 0x7f2a8c10bb50 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 --2- 172.21.15.16:0/1919648135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c102d40 0x7f2a8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 -- 172.21.15.16:0/1919648135 >> 172.21.15.16:0/1919648135 conn(0x7f2a8c0fc7a0 msgr2=0x7f2a8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 -- 172.21.15.16:0/1919648135 shutdown_connections 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.812+0000 7f2a91b68640 1 -- 172.21.15.16:0/1919648135 wait complete. 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 Processor -- start 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 -- start start 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c103660 0x7f2a8c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 0x7f2a8c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111f70 con 0x7f2a8c102d40 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c111df0 con 0x7f2a8c10c170 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a91b68640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1120f0 con 0x7f2a8c103660 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36610/0 (socket says 172.21.15.16:36610) 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/507044728 learned_addr learned my addr 172.21.15.16:0/507044728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.813+0000 7f2a8affd640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c103660 0x7f2a8c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 0x7f2a8c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/507044728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c103660 msgr2=0x7f2a8c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c103660 0x7f2a8c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/507044728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 msgr2=0x7f2a8c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 0x7f2a8c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/507044728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c19d420 con 0x7f2a8c102d40 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 0x7f2a8c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d950 tx=0x7f2a7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a88ff9640 1 -- 172.21.15.16:0/507044728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c017030 con 0x7f2a8c102d40 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a88ff9640 1 -- 172.21.15.16:0/507044728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c004020 con 0x7f2a8c102d40 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c19d710 con 0x7f2a8c102d40 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a88ff9640 1 -- 172.21.15.16:0/507044728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0047a0 con 0x7f2a8c102d40 2024-09-12T01:10:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.814+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a4ff0 con 0x7f2a8c102d40 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.817+0000 7f2a88ff9640 1 -- 172.21.15.16:0/507044728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a7c0041d0 con 0x7f2a8c102d40 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.817+0000 7f2a93f41740 1 Processor -- start 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.817+0000 7f2a93f41740 1 -- start start 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.817+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d75d0 0x55f08a0d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a93f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f08a03d2f0 con 0x55f08a0d8b40 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a93f41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f089fe4cf0 con 0x55f08a0e2370 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a93f41740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f08a0392b0 con 0x55f08a0d75d0 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36624/0 (socket says 172.21.15.16:36624) 2024-09-12T01:10:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 -- 172.21.15.16:0/2798602793 learned_addr learned my addr 172.21.15.16:0/2798602793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a88ff9640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 0x7f2a740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8affd640 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 -- 172.21.15.16:0/2798602793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d75d0 msgr2=0x55f08a0d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d75d0 0x55f08a0d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 -- 172.21.15.16:0/2798602793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 msgr2=0x55f08a0e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 -- 172.21.15.16:0/2798602793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f089fc93a0 con 0x55f08a0d8b40 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a88ff9640 1 -- 172.21.15.16:0/507044728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f2a7c0cee30 con 0x7f2a8c102d40 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 0x7f2a740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.818+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400a2b0 tx=0x7f2a8400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52acdb625a64f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a71ffb640 1 -- 172.21.15.16:0/2798602793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84022070 con 0x55f08a0d8b40 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a71ffb640 1 -- 172.21.15.16:0/2798602793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84018070 con 0x55f08a0d8b40 2024-09-12T01:10:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a71ffb640 1 -- 172.21.15.16:0/2798602793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8401cd30 con 0x55f08a0d8b40 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 -- 172.21.15.16:0/2798602793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 msgr2=0x55f08a0d7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 0x7f2a740a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a8401f000 tx=0x7f2a84020000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400a2b0 tx=0x7f2a8400e7e0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 -- 172.21.15.16:0/2798602793 shutdown_connections 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d75d0 0x55f08a0d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2798602793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0d7090 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 -- 172.21.15.16:0/2798602793 >> 172.21.15.16:0/2798602793 conn(0x55f08a0a0310 msgr2=0x55f08a0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 -- 172.21.15.16:0/2798602793 shutdown_connections 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.819+0000 7f2a93f41740 1 -- 172.21.15.16:0/2798602793 wait complete. 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 -- start start 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 Processor -- start 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0d75d0 0x55f08a0e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0e2370 0x55f08a0df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f08a0e5070 con 0x55f08a0d8b40 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f089ff8060 con 0x55f08a0d75d0 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f08a0e51f0 con 0x55f08a0e2370 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a93f41740 10 client.0 _ll_register_callbacks cb 0x55f089d4b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36626/0 (socket says 172.21.15.16:36626) 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a8affd640 1 -- 172.21.15.16:0/1694421156 learned_addr learned my addr 172.21.15.16:0/1694421156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0e2370 0x55f08a0df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.820+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0d75d0 0x55f08a0e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 -- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0e2370 msgr2=0x55f08a0df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0e2370 0x55f08a0df850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 -- 172.21.15.16:0/1694421156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0d75d0 msgr2=0x55f08a0e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0d75d0 0x55f08a0e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 -- 172.21.15.16:0/1694421156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f089dae3c0 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a8affd640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2a8002a9a0 tx=0x7f2a80031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275468ab902184 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80035040 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f2a80004240 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a84018070 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a80004440 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a93f41740 10 client.19857 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f08a0f0bc0 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.821+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84018070 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a800456f0 con 0x55f08a0d8b40 2024-09-12T01:10:29.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 10 client.19857 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 10 client.19857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 20 client.19857 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a0f2080 0x55f08a0f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 4 client.19857 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f08a04a860 con 0x55f08a0f2080 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a0f2080 0x55f08a0f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a6affd640 10 client.19857 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.822+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a0f2080 0x55f08a0f4490 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30eab579beb698be server_cookie=a59300299595ff86 in_seq=0 out_seq=0 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a6affd640 1 -- 172.21.15.16:0/1694421156 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f08a04a860 con 0x55f08a0f2080 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a6affd640 10 client.19857 handle_command_reply: tid=0 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 client.19857 shutdown 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 msgr2=0x55f08a0df0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2a8002a9a0 tx=0x7f2a80031920 comp rx=0 tx=0).stop 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 shutdown_connections 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a0f2080 0x55f08a0f4490 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0e2370 0x55f08a0df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d8b40 0x55f08a0df0c0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 --2- 172.21.15.16:0/1694421156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0d75d0 0x55f08a0e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 >> 172.21.15.16:0/1694421156 conn(0x55f08a0a0310 msgr2=0x55f08a0d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a6affd640 0 client.19857 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 shutdown_connections 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.823+0000 7f2a93f41740 1 -- 172.21.15.16:0/1694421156 wait complete. 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.824+0000 7f2a93f41740 20 client.19857 trim_cache size 0 max 16384 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.836+0000 7f2a93f41740 1 Processor -- start 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.836+0000 7f2a93f41740 1 -- start start 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.836+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.836+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a93f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ad9b0 con 0x55f08a0d75d0 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a93f41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1af4f0 con 0x55f08a0e2370 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a93f41740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ae080 con 0x55f08a0d8b40 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36636/0 (socket says 172.21.15.16:36636) 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2152315530 learned_addr learned my addr 172.21.15.16:0/2152315530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8affd640 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2152315530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 msgr2=0x55f08a0da120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0da120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2152315530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 msgr2=0x55f08a0da660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0da660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2152315530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08a0d4960 con 0x55f08a0d75d0 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005250 tx=0x7f2a7c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95807c62f6cd0283 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a717fa640 1 -- 172.21.15.16:0/2152315530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c080e80 con 0x55f08a0d75d0 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a717fa640 1 -- 172.21.15.16:0/2152315530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c09b350 con 0x55f08a0d75d0 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.837+0000 7f2a717fa640 1 -- 172.21.15.16:0/2152315530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c09a7a0 con 0x55f08a0d75d0 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- 172.21.15.16:0/2152315530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 msgr2=0x55f08a0de0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005250 tx=0x7f2a7c0ce040 comp rx=0 tx=0).stop 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- 172.21.15.16:0/2152315530 shutdown_connections 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0da120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2152315530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0de0a0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- 172.21.15.16:0/2152315530 >> 172.21.15.16:0/2152315530 conn(0x55f08a0a0310 msgr2=0x55f08a0dd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- 172.21.15.16:0/2152315530 shutdown_connections 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- 172.21.15.16:0/2152315530 wait complete. 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.838+0000 7f2a93f41740 1 -- start start 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 Processor -- start 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1aee20 con 0x55f08a0d75d0 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ae750 con 0x55f08a0e2370 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ab9b0 con 0x55f08a0d8b40 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a93f41740 10 client.0 _ll_register_callbacks cb 0x7f2a80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36650/0 (socket says 172.21.15.16:36650) 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2954205330 learned_addr learned my addr 172.21.15.16:0/2954205330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8affd640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 msgr2=0x55f08a0eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2954205330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 msgr2=0x55f08a0e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 -- 172.21.15.16:0/2954205330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f08a0d4dd0 con 0x55f08a0d75d0 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8bfff640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.839+0000 7f2a8b7fe640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005fd0 tx=0x7f2a7c0cf030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77395d32d175301 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0b9060 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f2a7c0979c0 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a7c09b350 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c09f700 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 10 client.19863 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f08a15c710 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c09b350 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7c0bb0b0 con 0x55f08a0d75d0 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 10 client.19863 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 10 client.19863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 20 client.19863 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a15cce0 0x55f08a15f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 4 client.19863 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:10:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.840+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f08a044d70 con 0x55f08a15cce0 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a8affd640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a15cce0 0x55f08a15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a8affd640 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a15cce0 0x55f08a15f100 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510524420d0b9ac8 server_cookie=b5c10746508ac7b in_seq=0 out_seq=0 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a6affd640 10 client.19863 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a6affd640 1 -- 172.21.15.16:0/2954205330 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f08a044d70 con 0x55f08a15cce0 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a6affd640 10 client.19863 handle_command_reply: tid=0 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.841+0000 7f2a93f41740 1 client.19863 shutdown 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 msgr2=0x55f08a0ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005fd0 tx=0x7f2a7c0cf030 comp rx=0 tx=0).stop 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 shutdown_connections 2024-09-12T01:10:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f08a15cce0 0x55f08a15f100 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f08a0e2370 0x55f08a0e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f08a0d8b40 0x55f08a0eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 --2- 172.21.15.16:0/2954205330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f08a0d75d0 0x55f08a0ea790 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 >> 172.21.15.16:0/2954205330 conn(0x55f08a0a0310 msgr2=0x55f08a0de1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a6affd640 0 client.19863 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 shutdown_connections 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 1 -- 172.21.15.16:0/2954205330 wait complete. 2024-09-12T01:10:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a93f41740 20 client.19863 trim_cache size 0 max 16384 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 msgr2=0x7f2a740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 0x7f2a740a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a8401f000 tx=0x7f2a84020000 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 msgr2=0x7f2a8c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d950 tx=0x7f2a7c00de10 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 shutdown_connections 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2a740a2cf0 0x7f2a740a51b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a8c10c170 0x7f2a8c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a8c103660 0x7f2a8c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 --2- 172.21.15.16:0/507044728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a8c102d40 0x7f2a8c198470 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.842+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 >> 172.21.15.16:0/507044728 conn(0x7f2a8c0fc7a0 msgr2=0x7f2a8c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.843+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 shutdown_connections 2024-09-12T01:10:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:29.843+0000 7f2a91b68640 1 -- 172.21.15.16:0/507044728 wait complete. 2024-09-12T01:10:30.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:30 smithi016 ceph-mon[30972]: pgmap v450: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T01:10:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:30 smithi099 ceph-mon[35654]: pgmap v450: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T01:10:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:30 smithi159 ceph-mon[35566]: pgmap v450: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-12T01:10:31.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:10:31.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:10:31.925 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 -- 172.21.15.16:0/1792039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3410e500 msgr2=0x7f4c34110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/1792039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3410e500 0x7f4c34110910 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009900 tx=0x7f4c2c02a410 comp rx=0 tx=0).stop 2024-09-12T01:10:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 -- 172.21.15.16:0/1792039346 shutdown_connections 2024-09-12T01:10:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/1792039346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c3410e500 0x7f4c34110910 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/1792039346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c34101440 0x7f4c3410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/1792039346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c34100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 -- 172.21.15.16:0/1792039346 >> 172.21.15.16:0/1792039346 conn(0x7f4c340fc7d0 msgr2=0x7f4c340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 -- 172.21.15.16:0/1792039346 shutdown_connections 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.422+0000 7f4c3baf6640 1 -- 172.21.15.16:0/1792039346 wait complete. 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 Processor -- start 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 -- start start 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c341a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 0x7f4c3419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341142d0 con 0x7f4c34101440 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34114150 con 0x7f4c34100b20 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3baf6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34114450 con 0x7f4c3410e500 2024-09-12T01:10:32.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36666/0 (socket says 172.21.15.16:36666) 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 -- 172.21.15.16:0/4100181916 learned_addr learned my addr 172.21.15.16:0/4100181916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3a093640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 0x7f4c3419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39892640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c341a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 -- 172.21.15.16:0/4100181916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 msgr2=0x7f4c3419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 0x7f4c3419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 -- 172.21.15.16:0/4100181916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 msgr2=0x7f4c341a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c341a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39091640 1 -- 172.21.15.16:0/4100181916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3419acd0 con 0x7f4c34101440 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c3a093640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 0x7f4c3419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.423+0000 7f4c39892640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c341a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c39091640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b450 tx=0x7f4c2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c32ffd640 1 -- 172.21.15.16:0/4100181916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28015020 con 0x7f4c34101440 2024-09-12T01:10:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c32ffd640 1 -- 172.21.15.16:0/4100181916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c28011d70 con 0x7f4c34101440 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c32ffd640 1 -- 172.21.15.16:0/4100181916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800f340 con 0x7f4c34101440 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3419afc0 con 0x7f4c34101440 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.424+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341a72b0 con 0x7f4c34101440 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.426+0000 7f4c32ffd640 1 -- 172.21.15.16:0/4100181916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4c28018020 con 0x7f4c34101440 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.426+0000 7f4c3decf740 1 Processor -- start 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 -- start start 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae6fb60 0x56434ae6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56434add5280 con 0x56434ae7ad40 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56434ad7cb10 con 0x56434ae70480 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3decf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56434add1240 con 0x56434ae6fb60 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c32ffd640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 0x7f4c1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3a093640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 0x7f4c1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c3a093640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae6fb60 0x56434ae6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36672/0 (socket says 172.21.15.16:36672) 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c32ffd640 1 -- 172.21.15.16:0/4100181916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f4c280ceee0 con 0x7f4c34101440 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 -- 172.21.15.16:0/1604471232 learned_addr learned my addr 172.21.15.16:0/1604471232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39892640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 -- 172.21.15.16:0/1604471232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae6fb60 msgr2=0x56434ae6ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae6fb60 0x56434ae6ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 -- 172.21.15.16:0/1604471232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 msgr2=0x56434ae7a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.427+0000 7f4c39091640 1 -- 172.21.15.16:0/1604471232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434ad891f0 con 0x56434ae7ad40 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c3a093640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 0x7f4c1c0a51b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005940 tx=0x7f4c2c032cb0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c39091640 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f4c280a3c50 tx=0x7f4c280909a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb76a9abebd012 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c1bfff640 1 -- 172.21.15.16:0/1604471232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28009280 con 0x56434ae7ad40 2024-09-12T01:10:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c1bfff640 1 -- 172.21.15.16:0/1604471232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2809bc10 con 0x56434ae7ad40 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c1bfff640 1 -- 172.21.15.16:0/1604471232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2801f200 con 0x56434ae7ad40 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c3decf740 1 -- 172.21.15.16:0/1604471232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 msgr2=0x56434ae7d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.428+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f4c280a3c50 tx=0x7f4c280909a0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 -- 172.21.15.16:0/1604471232 shutdown_connections 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae7ad40 0x56434ae7d130 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1604471232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae6fb60 0x56434ae6ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 -- 172.21.15.16:0/1604471232 >> 172.21.15.16:0/1604471232 conn(0x56434ae6c0f0 msgr2=0x56434ae6c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 -- 172.21.15.16:0/1604471232 shutdown_connections 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 -- 172.21.15.16:0/1604471232 wait complete. 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 -- start start 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 Processor -- start 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.429+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56434ae7d910 con 0x56434ae70480 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56434ad8ff10 con 0x56434ae6fb60 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56434ae7da90 con 0x56434ae7ad40 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 10 client.0 _ll_register_callbacks cb 0x56434ad58a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36682/0 (socket says 172.21.15.16:36682) 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 -- 172.21.15.16:0/1876776313 learned_addr learned my addr 172.21.15.16:0/1876776313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39892640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 -- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 msgr2=0x56434ae786d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 -- 172.21.15.16:0/1876776313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 msgr2=0x56434ae77c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae77c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 -- 172.21.15.16:0/1876776313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434ab466a0 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39892640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c39091640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4c2809b4a0 tx=0x7f4c280cc6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9610b2dbec5cad11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280b3070 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f4c28097b80 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2809bc10 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c2809b510 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 10 client.19875 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.430+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56434ae7d910 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2809bc10 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c280b4260 con 0x56434ae70480 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 10 client.19875 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 10 client.19875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 20 client.19875 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434ae8a6c0 0x56434ae8cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 4 client.19875 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434ade27f0 con 0x56434ae8a6c0 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.431+0000 7f4c39892640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434ae8a6c0 0x56434ae8cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c197fa640 10 client.19875 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c39892640 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434ae8a6c0 0x56434ae8cab0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce24945a4e9661c server_cookie=a850e8760e5a4df3 in_seq=0 out_seq=0 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c197fa640 1 -- 172.21.15.16:0/1876776313 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56434ade27f0 con 0x56434ae8a6c0 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c197fa640 10 client.19875 handle_command_reply: tid=0 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c3decf740 1 client.19875 shutdown 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.432+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 msgr2=0x56434ae78190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f4c2809b4a0 tx=0x7f4c280cc6b0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 shutdown_connections 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434ae8a6c0 0x56434ae8cab0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae786d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae78190 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 --2- 172.21.15.16:0/1876776313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae77c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 >> 172.21.15.16:0/1876776313 conn(0x56434ae6c0f0 msgr2=0x56434ae71e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c197fa640 0 client.19875 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 shutdown_connections 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 1 -- 172.21.15.16:0/1876776313 wait complete. 2024-09-12T01:10:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.433+0000 7f4c3decf740 20 client.19875 trim_cache size 0 max 16384 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.450+0000 7f4c3decf740 1 Processor -- start 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.450+0000 7f4c3decf740 1 -- start start 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.450+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae92320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.450+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae74eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c3decf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341afb10 con 0x56434ae70480 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c3decf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341b1650 con 0x56434ae6fb60 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c3decf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341b01e0 con 0x56434ae7ad40 2024-09-12T01:10:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36692/0 (socket says 172.21.15.16:36692) 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 -- 172.21.15.16:0/2493574013 learned_addr learned my addr 172.21.15.16:0/2493574013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39892640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae92320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c3a093640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae74eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 -- 172.21.15.16:0/2493574013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 msgr2=0x56434ae74eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae74eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 -- 172.21.15.16:0/2493574013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 msgr2=0x56434ae92320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae92320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 -- 172.21.15.16:0/2493574013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434ae8d090 con 0x56434ae70480 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39892640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae92320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c39091640 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4c2809b210 tx=0x7f4c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7976c200e19c5c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c1b7fe640 1 -- 172.21.15.16:0/2493574013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280b3070 con 0x56434ae70480 2024-09-12T01:10:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c1b7fe640 1 -- 172.21.15.16:0/2493574013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2809bc10 con 0x56434ae70480 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.451+0000 7f4c1b7fe640 1 -- 172.21.15.16:0/2493574013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280b4280 con 0x56434ae70480 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- 172.21.15.16:0/2493574013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 msgr2=0x56434ae92860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4c2809b210 tx=0x7f4c280046c0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- 172.21.15.16:0/2493574013 shutdown_connections 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae74eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae70480 0x56434ae92860 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 --2- 172.21.15.16:0/2493574013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae6fb60 0x56434ae92320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- 172.21.15.16:0/2493574013 >> 172.21.15.16:0/2493574013 conn(0x56434ae6c0f0 msgr2=0x56434ae79bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- 172.21.15.16:0/2493574013 shutdown_connections 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- 172.21.15.16:0/2493574013 wait complete. 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.452+0000 7f4c3decf740 1 -- start start 2024-09-12T01:10:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 Processor -- start 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341b0f80 con 0x56434ae6fb60 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341b08b0 con 0x56434ae70480 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c341adb10 con 0x56434ae7ad40 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3decf740 10 client.0 _ll_register_callbacks cb 0x7f4c24009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36694/0 (socket says 172.21.15.16:36694) 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 -- 172.21.15.16:0/4244276032 learned_addr learned my addr 172.21.15.16:0/4244276032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c3a093640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39091640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 -- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 msgr2=0x56434ae829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 -- 172.21.15.16:0/4244276032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 msgr2=0x56434ae7f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.453+0000 7f4c39892640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c39892640 1 -- 172.21.15.16:0/4244276032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434ae83040 con 0x56434ae6fb60 2024-09-12T01:10:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c39892640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4c24013570 tx=0x7f4c24003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3709a44fd8aac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24024070 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2809bc10 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f4c24004240 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c24004440 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c3decf740 10 client.19881 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56434aef5e40 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.454+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c2809bc10 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c2400f040 con 0x56434ae6fb60 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 10 client.19881 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 10 client.19881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 20 client.19881 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434aef6510 0x56434aef8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 4 client.19881 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56434addcd00 con 0x56434aef6510 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.455+0000 7f4c39091640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434aef6510 0x56434aef8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.456+0000 7f4c39091640 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434aef6510 0x56434aef8930 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e460caeb8bd4e6f8 server_cookie=772b3cb992943c9a in_seq=0 out_seq=0 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.456+0000 7f4c197fa640 10 client.19881 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.456+0000 7f4c197fa640 1 -- 172.21.15.16:0/4244276032 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56434addcd00 con 0x56434aef6510 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.456+0000 7f4c197fa640 10 client.19881 handle_command_reply: tid=0 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.456+0000 7f4c3decf740 1 client.19881 shutdown 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 msgr2=0x56434ae7f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4c24013570 tx=0x7f4c24003e00 comp rx=0 tx=0).stop 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 shutdown_connections 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x56434aef6510 0x56434aef8930 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56434ae7ad40 0x56434ae829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56434ae70480 0x56434ae7f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c197fa640 0 client.19881 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 --2- 172.21.15.16:0/4244276032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56434ae6fb60 0x56434ae7f3a0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 >> 172.21.15.16:0/4244276032 conn(0x56434ae6c0f0 msgr2=0x56434ae6aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 shutdown_connections 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 1 -- 172.21.15.16:0/4244276032 wait complete. 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.457+0000 7f4c3decf740 20 client.19881 trim_cache size 0 max 16384 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d6966039-df1e-4e66-9386-5783de07c2a7", 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:10:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 msgr2=0x7f4c1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 0x7f4c1c0a51b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c005940 tx=0x7f4c2c032cb0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 msgr2=0x7f4c341a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b450 tx=0x7f4c2800b910 comp rx=0 tx=0).stop 2024-09-12T01:10:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 shutdown_connections 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f4c1c0a2cf0 0x7f4c1c0a51b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3410e500 0x7f4c3419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c34101440 0x7f4c341a0920 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 --2- 172.21.15.16:0/4100181916 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c34100b20 0x7f4c341a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 >> 172.21.15.16:0/4100181916 conn(0x7f4c340fc7d0 msgr2=0x7f4c340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 shutdown_connections 2024-09-12T01:10:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:32.458+0000 7f4c3baf6640 1 -- 172.21.15.16:0/4100181916 wait complete. 2024-09-12T01:10:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:32 smithi099 ceph-mon[35654]: pgmap v451: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:32 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:10:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:32 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:32 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:10:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:32 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:32 smithi016 ceph-mon[30972]: pgmap v451: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:33.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:32 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:10:33.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:32 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:32 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:10:33.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:32 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:32 smithi159 ceph-mon[35566]: pgmap v451: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:32 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:10:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:32 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:32 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:10:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:32 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:10:33.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:10:33.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- 172.21.15.16:0/164757861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0100e20 msgr2=0x7f93b0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 --2- 172.21.15.16:0/164757861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0100e20 0x7f93b0101200 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009900 tx=0x7f93a002a480 comp rx=0 tx=0).stop 2024-09-12T01:10:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- 172.21.15.16:0/164757861 shutdown_connections 2024-09-12T01:10:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 --2- 172.21.15.16:0/164757861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0102ef0 0x7f93b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 --2- 172.21.15.16:0/164757861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102570 0x7f93b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 --2- 172.21.15.16:0/164757861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0100e20 0x7f93b0101200 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- 172.21.15.16:0/164757861 >> 172.21.15.16:0/164757861 conn(0x7f93b00fc7d0 msgr2=0x7f93b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- 172.21.15.16:0/164757861 shutdown_connections 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- 172.21.15.16:0/164757861 wait complete. 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 Processor -- start 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.912+0000 7f93b68de640 1 -- start start 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 0x7f93b0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102ef0 0x7f93b019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0110fa0 con 0x7f93b0102570 2024-09-12T01:10:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0110e20 con 0x7f93b0102ef0 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0111120 con 0x7f93b0100e20 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36714/0 (socket says 172.21.15.16:36714) 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 -- 172.21.15.16:0/2338644655 learned_addr learned my addr 172.21.15.16:0/2338644655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93affff640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 0x7f93b0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102ef0 0x7f93b019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 -- 172.21.15.16:0/2338644655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 msgr2=0x7f93b0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 0x7f93b0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 -- 172.21.15.16:0/2338644655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102ef0 msgr2=0x7f93b019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102ef0 0x7f93b019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 -- 172.21.15.16:0/2338644655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b019d4f0 con 0x7f93b0102570 2024-09-12T01:10:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93affff640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 0x7f93b0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f939800d950 tx=0x7f939800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93adffb640 1 -- 172.21.15.16:0/2338644655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398017030 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93adffb640 1 -- 172.21.15.16:0/2338644655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9398004020 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b019d7e0 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.913+0000 7f93adffb640 1 -- 172.21.15.16:0/2338644655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980047a0 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.914+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a50c0 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93adffb640 1 -- 172.21.15.16:0/2338644655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f93980041d0 con 0x7f93b0102570 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 Processor -- start 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 -- start start 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e0429a22e0 con 0x55e042a4ba90 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e042949ce0 con 0x55e042a3cca0 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.916+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e04299e2a0 con 0x55e042a3c380 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93adffb640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 0x7f939c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36724/0 (socket says 172.21.15.16:36724) 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 -- 172.21.15.16:0/2415812962 learned_addr learned my addr 172.21.15.16:0/2415812962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93affff640 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 0x7f939c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93adffb640 1 -- 172.21.15.16:0/2338644655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f93980ceec0 con 0x7f93b0102570 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 -- 172.21.15.16:0/2415812962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 msgr2=0x55e042a3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a3c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 -- 172.21.15.16:0/2415812962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 msgr2=0x55e042a4b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 -- 172.21.15.16:0/2415812962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04292e440 con 0x55e042a4ba90 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9398097840 tx=0x7f93980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c195ec87ad7541d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 0x7f939c0a51b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009900 tx=0x7f93a8009650 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a67fc640 1 -- 172.21.15.16:0/2415812962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980a4050 con 0x55e042a4ba90 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a67fc640 1 -- 172.21.15.16:0/2415812962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939809bae0 con 0x55e042a4ba90 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.917+0000 7f93a67fc640 1 -- 172.21.15.16:0/2415812962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939809a830 con 0x55e042a4ba90 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2415812962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 msgr2=0x55e042a4de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f9398097840 tx=0x7f93980046c0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2415812962 shutdown_connections 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a4de80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2415812962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2415812962 >> 172.21.15.16:0/2415812962 conn(0x55e042a38a30 msgr2=0x55e042a38e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2415812962 shutdown_connections 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.918+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2415812962 wait complete. 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 -- start start 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 Processor -- start 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 0x55e042a4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a48d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e042a4e600 con 0x55e042a3cca0 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e04295d050 con 0x55e042a3c380 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e042a4e780 con 0x55e042a4ba90 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b8cb7740 10 client.0 _ll_register_callbacks cb 0x55e04299d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36730/0 (socket says 172.21.15.16:36730) 2024-09-12T01:10:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93a7fff640 1 -- 172.21.15.16:0/2680342206 learned_addr learned my addr 172.21.15.16:0/2680342206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a48d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93affff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 0x55e042a4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.919+0000 7f93a7fff640 1 -- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 msgr2=0x55e042a48d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a48d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93a7fff640 1 -- 172.21.15.16:0/2680342206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 msgr2=0x55e042a4b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 0x55e042a4b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93a7fff640 1 -- 172.21.15.16:0/2680342206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0427133c0 con 0x55e042a3cca0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a48d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93affff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 0x55e042a4b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f93980cc6c0 tx=0x7f93980ccac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf31f2d513e1baf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939809f790 con 0x55e042a3cca0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f9398097130 con 0x55e042a3cca0 2024-09-12T01:10:33.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939809bae0 con 0x55e042a3cca0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9398097330 con 0x55e042a3cca0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 10 client.19893 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e042a5a0a0 con 0x55e042a3cca0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f939809bae0 con 0x55e042a3cca0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93980a0b20 con 0x55e042a3cca0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 10 client.19893 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 10 client.19893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 20 client.19893 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.920+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042a5b3f0 0x55e042a5d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.921+0000 7f93b8cb7740 4 client.19893 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.921+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0429af850 con 0x55e042a5b3f0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.921+0000 7f93affff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042a5b3f0 0x55e042a5d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.921+0000 7f938f7fe640 10 client.19893 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.921+0000 7f93affff640 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042a5b3f0 0x55e042a5d7e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c165e13f09cc26d server_cookie=54e439a3d378f315 in_seq=0 out_seq=0 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f938f7fe640 1 -- 172.21.15.16:0/2680342206 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0429af850 con 0x55e042a5b3f0 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f938f7fe640 10 client.19893 handle_command_reply: tid=0 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 client.19893 shutdown 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 msgr2=0x55e042a48730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f93980cc6c0 tx=0x7f93980ccac0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 shutdown_connections 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042a5b3f0 0x55e042a5d7e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a48d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3cca0 0x55e042a48730 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f938f7fe640 0 client.19893 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.922+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2680342206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3c380 0x55e042a4b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.923+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 >> 172.21.15.16:0/2680342206 conn(0x55e042a38a30 msgr2=0x55e042a3e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.923+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 shutdown_connections 2024-09-12T01:10:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.923+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2680342206 wait complete. 2024-09-12T01:10:33.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.923+0000 7f93b8cb7740 20 client.19893 trim_cache size 0 max 16384 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 Processor -- start 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 -- start start 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a47000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ab6f0 con 0x55e042a4ba90 2024-09-12T01:10:33.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ad230 con 0x55e042a3cca0 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01abdc0 con 0x55e042a3c380 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b4e7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b4e7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36742/0 (socket says 172.21.15.16:36742) 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93b4e7b640 1 -- 172.21.15.16:0/3915928473 learned_addr learned my addr 172.21.15.16:0/3915928473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.940+0000 7f93affff640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a47000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93a7fff640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 -- 172.21.15.16:0/3915928473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 msgr2=0x55e042a47000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a47000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 -- 172.21.15.16:0/3915928473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 msgr2=0x55e042a4a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 -- 172.21.15.16:0/3915928473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e042a4b7f0 con 0x55e042a4ba90 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93affff640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a47000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:33.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93a7fff640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f93a800bf60 tx=0x7f93a8009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd28a2b8f93fb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93a5ffb640 1 -- 172.21.15.16:0/3915928473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a802a070 con 0x55e042a4ba90 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93a5ffb640 1 -- 172.21.15.16:0/3915928473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a800aae0 con 0x55e042a4ba90 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.941+0000 7f93a5ffb640 1 -- 172.21.15.16:0/3915928473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8014040 con 0x55e042a4ba90 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- 172.21.15.16:0/3915928473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 msgr2=0x55e042a45990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f93a800bf60 tx=0x7f93a8009ab0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- 172.21.15.16:0/3915928473 shutdown_connections 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a4ba90 0x55e042a45990 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a4a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/3915928473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a3c380 0x55e042a47000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- 172.21.15.16:0/3915928473 >> 172.21.15.16:0/3915928473 conn(0x55e042a38a30 msgr2=0x55e042a49f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- 172.21.15.16:0/3915928473 shutdown_connections 2024-09-12T01:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- 172.21.15.16:0/3915928473 wait complete. 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 -- start start 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.942+0000 7f93b8cb7740 1 Processor -- start 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a5ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01acb60 con 0x55e042a3c380 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ac490 con 0x55e042a3cca0 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01a96f0 con 0x55e042a4ba90 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b8cb7740 10 client.0 _ll_register_callbacks cb 0x7f93a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36756/0 (socket says 172.21.15.16:36756) 2024-09-12T01:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 -- 172.21.15.16:0/2624677995 learned_addr learned my addr 172.21.15.16:0/2624677995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93b4e7b640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a5ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 -- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 msgr2=0x55e042a5ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a5ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 -- 172.21.15.16:0/2624677995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 msgr2=0x55e042a5e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a5e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 -- 172.21.15.16:0/2624677995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e042ac4d30 con 0x55e042a3c380 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f93affff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f93a002a9a0 tx=0x7f93a0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac534d9470f0e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.943+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0035040 con 0x55e042a3c380 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f93a0005450 con 0x55e042a3c380 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a800aae0 con 0x55e042a3c380 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93a0005650 con 0x55e042a3c380 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 10 client.19899 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e042ac5e90 con 0x55e042a3c380 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a800aae0 con 0x55e042a3c380 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93a0003b10 con 0x55e042a3c380 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 10 client.19899 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 10 client.19899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 20 client.19899 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042ac65c0 0x55e042ac89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 4 client.19899 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0429a9d60 con 0x55e042ac65c0 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.944+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042ac65c0 0x55e042ac89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93a7fff640 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042ac65c0 0x55e042ac89e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d40e08de7aa552d server_cookie=8f1b27fac3511c3e in_seq=0 out_seq=0 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f938f7fe640 10 client.19899 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f938f7fe640 1 -- 172.21.15.16:0/2624677995 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e0429a9d60 con 0x55e042ac65c0 2024-09-12T01:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f938f7fe640 10 client.19899 handle_command_reply: tid=0 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 client.19899 shutdown 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 msgr2=0x55e042a610d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f93a002a9a0 tx=0x7f93a0004500 comp rx=0 tx=0).stop 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 shutdown_connections 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55e042ac65c0 0x55e042ac89e0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e042a4ba90 0x55e042a5ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e042a3cca0 0x55e042a5e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 --2- 172.21.15.16:0/2624677995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e042a3c380 0x55e042a610d0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 >> 172.21.15.16:0/2624677995 conn(0x55e042a38a30 msgr2=0x55e042a4a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f938f7fe640 0 client.19899 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 shutdown_connections 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 1 -- 172.21.15.16:0/2624677995 wait complete. 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.945+0000 7f93b8cb7740 20 client.19899 trim_cache size 0 max 16384 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "d6966039-df1e-4e66-9386-5783de07c2a7": { 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:10:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d6966039-df1e-4e66-9386-5783de07c2a7", 2024-09-12T01:10:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:10:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:33.958 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 msgr2=0x7f939c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 0x7f939c0a51b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009900 tx=0x7f93a8009650 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 msgr2=0x7f93b0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f939800d950 tx=0x7f939800de10 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 shutdown_connections 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f939c0a2cf0 0x7f939c0a51b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93b0102ef0 0x7f93b019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93b0102570 0x7f93b0198a80 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 --2- 172.21.15.16:0/2338644655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0100e20 0x7f93b0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 >> 172.21.15.16:0/2338644655 conn(0x7f93b00fc7d0 msgr2=0x7f93b006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 shutdown_connections 2024-09-12T01:10:33.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:33.946+0000 7f93b68de640 1 -- 172.21.15.16:0/2338644655 wait complete. 2024-09-12T01:10:34.606 INFO:tasks.cephfs.filesystem:scrub status for tag:d6966039-df1e-4e66-9386-5783de07c2a7 - {'path': '/', 'tag': 'd6966039-df1e-4e66-9386-5783de07c2a7', 'options': 'recursive,force'} 2024-09-12T01:10:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:34 smithi016 ceph-mon[30972]: pgmap v452: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:34 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:10:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:34 smithi099 ceph-mon[35654]: pgmap v452: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:34 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:10:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:34 smithi159 ceph-mon[35566]: pgmap v452: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T01:10:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:34 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:10:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:36 smithi099 ceph-mon[35654]: pgmap v453: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-09-12T01:10:37.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:36 smithi159 ceph-mon[35566]: pgmap v453: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-09-12T01:10:37.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:36 smithi016 ceph-mon[30972]: pgmap v453: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-09-12T01:10:39.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:38 smithi099 ceph-mon[35654]: pgmap v454: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:39.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:38 smithi159 ceph-mon[35566]: pgmap v454: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:39.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:38 smithi016 ceph-mon[30972]: pgmap v454: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:40 smithi099 ceph-mon[35654]: pgmap v455: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:40 smithi159 ceph-mon[35566]: pgmap v455: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:41.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:40 smithi016 ceph-mon[30972]: pgmap v455: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:42 smithi099 ceph-mon[35654]: pgmap v456: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:42 smithi159 ceph-mon[35566]: pgmap v456: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:42 smithi016 ceph-mon[30972]: pgmap v456: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:44.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:10:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.760+0000 7f33872ff640 1 -- 172.21.15.16:0/1680368729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 msgr2=0x7f3380110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.760+0000 7f33872ff640 1 --2- 172.21.15.16:0/1680368729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f3380110810 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f3378009ca0 tx=0x7f337802a950 comp rx=0 tx=0).stop 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 -- 172.21.15.16:0/1680368729 shutdown_connections 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 --2- 172.21.15.16:0/1680368729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f3380110810 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 --2- 172.21.15.16:0/1680368729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074f40 0x7f338010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 --2- 172.21.15.16:0/1680368729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074620 0x7f3380074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 -- 172.21.15.16:0/1680368729 >> 172.21.15.16:0/1680368729 conn(0x7f33800fd8c0 msgr2=0x7f33800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 -- 172.21.15.16:0/1680368729 shutdown_connections 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 -- 172.21.15.16:0/1680368729 wait complete. 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 Processor -- start 2024-09-12T01:10:44.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 -- start start 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074620 0x7f338019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.761+0000 7f33872ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074f40 0x7f338019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3380114160 con 0x7f338010e3c0 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113fe0 con 0x7f3380074620 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33801142e0 con 0x7f3380074f40 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53262/0 (socket says 172.21.15.16:53262) 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 -- 172.21.15.16:0/797345282 learned_addr learned my addr 172.21.15.16:0/797345282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338509b640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074620 0x7f338019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338489a640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074f40 0x7f338019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 -- 172.21.15.16:0/797345282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074f40 msgr2=0x7f338019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074f40 0x7f338019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 -- 172.21.15.16:0/797345282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074620 msgr2=0x7f338019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074620 0x7f338019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 -- 172.21.15.16:0/797345282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338019f5e0 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f338589c640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3378004af0 tx=0x7f337802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378034040 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378030920 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338019f870 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378004670 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.762+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33801a71b0 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.763+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3380075cd0 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.764+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3378037020 con 0x7f338010e3c0 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.765+0000 7f33767fc640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 0x7f336c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:44.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.765+0000 7f338509b640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 0x7f336c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.765+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f33780f0120 con 0x7f338010e3c0 2024-09-12T01:10:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.766+0000 7f338509b640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 0x7f336c0a51b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f337c009bd0 tx=0x7f337c005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:44.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.769+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33780bcdf0 con 0x7f338010e3c0 2024-09-12T01:10:44.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.979+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f338019b990 con 0x7f338010e3c0 2024-09-12T01:10:44.978 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:44 smithi016 ceph-mon[30972]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.980+0000 7f33767fc640 1 -- 172.21.15.16:0/797345282 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f33780bc6f0 con 0x7f338010e3c0 2024-09-12T01:10:44.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:10:44.979 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:10:44.979 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:10:44.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 msgr2=0x7f336c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 0x7f336c0a51b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f337c009bd0 tx=0x7f337c005f90 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 msgr2=0x7f338019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f3378004af0 tx=0x7f337802f3a0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 shutdown_connections 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f336c0a2cf0 0x7f336c0a51b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338010e3c0 0x7f338019ef00 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3380074f40 0x7f338019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 --2- 172.21.15.16:0/797345282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3380074620 0x7f338019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 >> 172.21.15.16:0/797345282 conn(0x7f33800fd8c0 msgr2=0x7f33800fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:44.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 shutdown_connections 2024-09-12T01:10:44.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:44.982+0000 7f33872ff640 1 -- 172.21.15.16:0/797345282 wait complete. 2024-09-12T01:10:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:44 smithi099 ceph-mon[35654]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:44 smithi159 ceph-mon[35566]: pgmap v457: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T01:10:45.853 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --set-id 563d9ea6 --release --await' 2024-09-12T01:10:45.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --set-id 563d9ea6 --release --await 2024-09-12T01:10:46.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:45 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/797345282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:46.115 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:10:46.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:45 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/797345282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:46.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:45 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/797345282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.619+0000 7f089f7cc640 1 -- 172.21.15.16:0/2499255682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 msgr2=0x7f08981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.619+0000 7f089f7cc640 1 --2- 172.21.15.16:0/2499255682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f08981067e0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f0888009900 tx=0x7f088802a410 comp rx=0 tx=0).stop 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 -- 172.21.15.16:0/2499255682 shutdown_connections 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 --2- 172.21.15.16:0/2499255682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898106f10 0x7f089810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 --2- 172.21.15.16:0/2499255682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f08981067e0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 --2- 172.21.15.16:0/2499255682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898101de0 0x7f08981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 -- 172.21.15.16:0/2499255682 >> 172.21.15.16:0/2499255682 conn(0x7f08980fd8c0 msgr2=0x7f08980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 -- 172.21.15.16:0/2499255682 shutdown_connections 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 -- 172.21.15.16:0/2499255682 wait complete. 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 Processor -- start 2024-09-12T01:10:46.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.620+0000 7f089f7cc640 1 -- start start 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898101de0 0x7f08980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 0x7f0898074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0898112010 con 0x7f0898102790 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111e90 con 0x7f0898101de0 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0898112190 con 0x7f0898106f10 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53280/0 (socket says 172.21.15.16:53280) 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 -- 172.21.15.16:0/1719135669 learned_addr learned my addr 172.21.15.16:0/1719135669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089d568640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898101de0 0x7f08980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089dd69640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 0x7f0898074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 -- 172.21.15.16:0/1719135669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 msgr2=0x7f0898074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 0x7f0898074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 -- 172.21.15.16:0/1719135669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898101de0 msgr2=0x7f08980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898101de0 0x7f08980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 -- 172.21.15.16:0/1719135669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08981a6a50 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089dd69640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 0x7f0898074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089cd67640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f0888005ab0 tx=0x7f08880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f088e7fc640 1 -- 172.21.15.16:0/1719135669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088803f030 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f088e7fc640 1 -- 172.21.15.16:0/1719135669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0888003ce0 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.621+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08981a6ce0 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.622+0000 7f088e7fc640 1 -- 172.21.15.16:0/1719135669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888035070 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.622+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08981a71c0 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f088e7fc640 1 -- 172.21.15.16:0/1719135669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0888042030 con 0x7f0898102790 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 Processor -- start 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 -- start start 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c060 0x55f9c089c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 0x55f9c08a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9c0801620 con 0x55f9c08a7240 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f9c07a8e50 con 0x55f9c089c980 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.624+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f9c07fd5e0 con 0x55f9c089c060 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53282/0 (socket says 172.21.15.16:53282) 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089cd67640 1 -- 172.21.15.16:0/4280414195 learned_addr learned my addr 172.21.15.16:0/4280414195 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 0x55f9c08a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089dd69640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c060 0x55f9c089c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f088e7fc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 0x7f08840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089dd69640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 0x7f08840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 -- 172.21.15.16:0/4280414195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c060 msgr2=0x55f9c089c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c060 0x55f9c089c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 -- 172.21.15.16:0/4280414195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 msgr2=0x55f9c08a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f088e7fc640 1 -- 172.21.15.16:0/1719135669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f08880efd50 con 0x7f0898102790 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 -- 172.21.15.16:0/4280414195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c0799d40 con 0x55f9c089c980 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089cd67640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089dd69640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 0x7f08840a5230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0890013870 tx=0x7f0890004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.625+0000 7f089d568640 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 0x55f9c08a6d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08940099a0 tx=0x7f0894009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409866eba8b98b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.626+0000 7f08837fe640 1 -- 172.21.15.16:0/4280414195 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894017070 con 0x55f9c089c980 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.626+0000 7f08837fe640 1 -- 172.21.15.16:0/4280414195 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894013470 con 0x55f9c089c980 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.626+0000 7f08837fe640 1 -- 172.21.15.16:0/4280414195 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894012610 con 0x55f9c089c980 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.626+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4280414195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 msgr2=0x55f9c08a6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.626+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 0x55f9c08a6d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08940099a0 tx=0x7f0894009e60 comp rx=0 tx=0).stop 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4280414195 shutdown_connections 2024-09-12T01:10:46.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c980 0x55f9c08a6d00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4280414195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c060 0x55f9c089c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4280414195 >> 172.21.15.16:0/4280414195 conn(0x55f9c08985f0 msgr2=0x55f9c08989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4280414195 shutdown_connections 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4280414195 wait complete. 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- start start 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 Processor -- start 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f9c08a9e10 con 0x55f9c08a7240 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f9c0789f10 con 0x55f9c089c060 2024-09-12T01:10:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.627+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f9c08a9f90 con 0x55f9c089c980 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f08a1ba5740 10 client.0 _ll_register_callbacks cb 0x55f9c07addb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53290/0 (socket says 172.21.15.16:53290) 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 -- 172.21.15.16:0/4124486175 learned_addr learned my addr 172.21.15.16:0/4124486175 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089d568640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089cd67640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 -- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 msgr2=0x55f9c08a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 -- 172.21.15.16:0/4124486175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 msgr2=0x55f9c08a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 -- 172.21.15.16:0/4124486175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c0510580 con 0x55f9c08a7240 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f089dd69640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f089000d3b0 tx=0x7f0890004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559e4c5c40d6c6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f0880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890022070 con 0x55f9c08a7240 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f089001e800 con 0x55f9c08a7240 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.628+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0894013470 con 0x55f9c08a7240 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f089002e050 con 0x55f9c08a7240 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 10 client.19917 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c05da960 con 0x55f9c08a7240 2024-09-12T01:10:46.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0894013470 con 0x55f9c08a7240 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f089001d070 con 0x55f9c08a7240 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 10 client.19917 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 10 client.19917 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 20 client.19917 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c08b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 4 client.19917 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c080eb90 con 0x55f9c08b6d60 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.629+0000 7f089d568640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c08b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.630+0000 7f089d568640 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c08b9200 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06c7495ff063e1c server_cookie=f97465eeabd461b in_seq=0 out_seq=0 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.630+0000 7f0880ff9640 10 client.19917 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.630+0000 7f0880ff9640 1 -- 172.21.15.16:0/4124486175 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9c080eb90 con 0x55f9c08b6d60 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.630+0000 7f0880ff9640 10 client.19917 handle_command_reply: tid=0 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.630+0000 7f08a1ba5740 1 client.19917 shutdown 2024-09-12T01:10:46.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 msgr2=0x55f9c08a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f089000d3b0 tx=0x7f0890004b50 comp rx=0 tx=0).stop 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 shutdown_connections 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c08b9200 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c08a7240 0x55f9c08a4b80 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f0880ff9640 0 client.19917 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/4124486175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 >> 172.21.15.16:0/4124486175 conn(0x55f9c08985f0 msgr2=0x55f9c089e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 shutdown_connections 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 1 -- 172.21.15.16:0/4124486175 wait complete. 2024-09-12T01:10:46.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.631+0000 7f08a1ba5740 20 client.19917 trim_cache size 0 max 16384 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 Processor -- start 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 -- start start 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 0x55f9c08a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08981afb80 con 0x55f9c089c980 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08981b16c0 con 0x55f9c089c060 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08981b0250 con 0x55f9c08a7240 2024-09-12T01:10:46.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f089cd67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53294/0 (socket says 172.21.15.16:53294) 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f089cd67640 1 -- 172.21.15.16:0/2594001695 learned_addr learned my addr 172.21.15.16:0/2594001695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f089d568640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.648+0000 7f089dd69640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 0x55f9c08a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 -- 172.21.15.16:0/2594001695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 msgr2=0x55f9c08a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 0x55f9c08a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 -- 172.21.15.16:0/2594001695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 msgr2=0x55f9c08beb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08beb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 -- 172.21.15.16:0/2594001695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c0898090 con 0x55f9c089c980 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089dd69640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 0x55f9c08a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089d568640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08beb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f089cd67640 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0888006010 tx=0x7f08880f38a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed925fdb13608c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f0882ffd640 1 -- 172.21.15.16:0/2594001695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888049040 con 0x55f9c089c980 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f0882ffd640 1 -- 172.21.15.16:0/2594001695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bc970 con 0x55f9c089c980 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.649+0000 7f0882ffd640 1 -- 172.21.15.16:0/2594001695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08880c0620 con 0x55f9c089c980 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- 172.21.15.16:0/2594001695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 msgr2=0x55f9c08bf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0888006010 tx=0x7f08880f38a0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- 172.21.15.16:0/2594001695 shutdown_connections 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c08a7240 0x55f9c08a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c980 0x55f9c08bf060 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/2594001695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c089c060 0x55f9c08beb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- 172.21.15.16:0/2594001695 >> 172.21.15.16:0/2594001695 conn(0x55f9c08985f0 msgr2=0x55f9c08a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- 172.21.15.16:0/2594001695 shutdown_connections 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- 172.21.15.16:0/2594001695 wait complete. 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 -- start start 2024-09-12T01:10:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.650+0000 7f08a1ba5740 1 Processor -- start 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 0x55f9c08ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08981b0ff0 con 0x55f9c089c060 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08981b0920 con 0x55f9c08a7240 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f08981adb80 con 0x55f9c089c980 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 10 client.0 _ll_register_callbacks cb 0x7f0890010ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53298/0 (socket says 172.21.15.16:53298) 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 -- 172.21.15.16:0/506162022 learned_addr learned my addr 172.21.15.16:0/506162022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089dd69640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 0x55f9c08ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089cd67640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 -- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 msgr2=0x55f9c08ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08ac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 -- 172.21.15.16:0/506162022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 msgr2=0x55f9c08ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 0x55f9c08ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 -- 172.21.15.16:0/506162022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c08acc60 con 0x55f9c089c060 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089dd69640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 0x55f9c08ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089cd67640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f089d568640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f08940095d0 tx=0x7f0894013ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f48683418436b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f0880ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:10:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0894017040 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8521+0+0 (secure 0 0 0) 0x7f08940059c0 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.651+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08880bc970 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0894035050 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 10 client.19923 fetch_fsmap learned FSMap version 28 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9c09227c0 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bc970 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08940264a0 con 0x55f9c089c060 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 10 client.19923 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 10 client.19923 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 20 client.19923 populate_metadata read hostname 'smithi016' 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c0924c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 4 client.19923 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "563d9ea6", "release": true, "await": true}] 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "set_id": "563d9ea6", "release": true, "await": true}) -- 0x55f9c08090a0 con 0x55f9c08b6d60 2024-09-12T01:10:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.652+0000 7f089cd67640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c0924c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:10:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.653+0000 7f089cd67640 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c0924c70 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67c4b45ba750a8d server_cookie=48f250f1157a9aa in_seq=0 out_seq=0 2024-09-12T01:10:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.653+0000 7f0880ff9640 10 client.19923 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.655+0000 7f0880ff9640 1 -- 172.21.15.16:0/506162022 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f9c08090a0 con 0x55f9c08b6d60 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.655+0000 7f0880ff9640 10 client.19923 handle_command_reply: tid=0 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.655+0000 7f08a1ba5740 1 client.19923 shutdown 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 msgr2=0x55f9c08af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f08940095d0 tx=0x7f0894013ec0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 shutdown_connections 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55f9c08b6d60 0x55f9c0924c70 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f9c08a7240 0x55f9c08ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9c089c980 0x55f9c08ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 --2- 172.21.15.16:0/506162022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9c089c060 0x55f9c08af690 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 >> 172.21.15.16:0/506162022 conn(0x55f9c08985f0 msgr2=0x55f9c08a40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f0880ff9640 0 client.19923 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 shutdown_connections 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 1 -- 172.21.15.16:0/506162022 wait complete. 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.656+0000 7f08a1ba5740 20 client.19923 trim_cache size 0 max 16384 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 56, 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "563d9ea6": { 2024-09-12T01:10:46.659 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 56, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.0, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-12T01:10:46.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:10:46.661 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 msgr2=0x7f08840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 0x7f08840a5230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0890013870 tx=0x7f0890004a40 comp rx=0 tx=0).stop 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 msgr2=0x7f0898074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f0888005ab0 tx=0x7f08880042f0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 shutdown_connections 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f08840a2d70 0x7f08840a5230 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0898106f10 0x7f0898074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898102790 0x7f0898074a30 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 --2- 172.21.15.16:0/1719135669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0898101de0 0x7f08980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:10:46.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 >> 172.21.15.16:0/1719135669 conn(0x7f08980fd8c0 msgr2=0x7f08980fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:10:46.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 shutdown_connections 2024-09-12T01:10:46.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:10:46.657+0000 7f089f7cc640 1 -- 172.21.15.16:0/1719135669 wait complete. 2024-09-12T01:10:46.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:46 smithi016 ceph-mon[30972]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:46 smithi099 ceph-mon[35654]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:46 smithi159 ceph-mon[35566]: pgmap v458: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-12T01:10:47.472 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '563d9ea6', total seconds elapsed: 33.4 2024-09-12T01:10:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:48 smithi099 ceph-mon[35654]: pgmap v459: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:10:49.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:48 smithi159 ceph-mon[35566]: pgmap v459: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:10:49.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:48 smithi016 ceph-mon[30972]: pgmap v459: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:10:50.957 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:50 smithi016 ceph-mon[30972]: pgmap v460: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T01:10:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:50 smithi099 ceph-mon[35654]: pgmap v460: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T01:10:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:50 smithi159 ceph-mon[35566]: pgmap v460: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-09-12T01:10:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:52 smithi099 ceph-mon[35654]: pgmap v461: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:53.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:52 smithi159 ceph-mon[35566]: pgmap v461: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:53.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:52 smithi016 ceph-mon[30972]: pgmap v461: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:54 smithi099 ceph-mon[35654]: pgmap v462: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:54 smithi159 ceph-mon[35566]: pgmap v462: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:55.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:54 smithi016 ceph-mon[30972]: pgmap v462: 97 pgs: 97 active+clean; 23 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2024-09-12T01:10:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:56 smithi099 ceph-mon[35654]: pgmap v463: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-12T01:10:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:56 smithi159 ceph-mon[35566]: pgmap v463: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-12T01:10:57.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:56 smithi016 ceph-mon[30972]: pgmap v463: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 47 KiB/s wr, 16 op/s 2024-09-12T01:10:59.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:10:58 smithi099 ceph-mon[35654]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-12T01:10:59.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:10:58 smithi159 ceph-mon[35566]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-12T01:10:59.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:10:58 smithi016 ceph-mon[30972]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 11 op/s 2024-09-12T01:11:01.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:00 smithi016 ceph-mon[30972]: pgmap v465: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 18 op/s 2024-09-12T01:11:01.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:00 smithi099 ceph-mon[35654]: pgmap v465: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 18 op/s 2024-09-12T01:11:01.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:00 smithi159 ceph-mon[35566]: pgmap v465: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 75 KiB/s wr, 18 op/s 2024-09-12T01:11:03.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:02 smithi099 ceph-mon[35654]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:03.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:02 smithi159 ceph-mon[35566]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:03.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:02 smithi016 ceph-mon[30972]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:04.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:11:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:04.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:04 smithi016 ceph-mon[30972]: pgmap v467: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:05.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:04 smithi099 ceph-mon[35654]: pgmap v467: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:05.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:04 smithi159 ceph-mon[35566]: pgmap v467: 97 pgs: 97 active+clean; 24 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 12 op/s 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 -- 172.21.15.16:0/1025203872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 msgr2=0x7f23c01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 --2- 172.21.15.16:0/1025203872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c01029b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009900 tx=0x7f23a802a480 comp rx=0 tx=0).stop 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 -- 172.21.15.16:0/1025203872 shutdown_connections 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 --2- 172.21.15.16:0/1025203872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0102ef0 0x7f23c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 --2- 172.21.15.16:0/1025203872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c01029b0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 --2- 172.21.15.16:0/1025203872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0100e20 0x7f23c0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 -- 172.21.15.16:0/1025203872 >> 172.21.15.16:0/1025203872 conn(0x7f23c00fc7d0 msgr2=0x7f23c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 -- 172.21.15.16:0/1025203872 shutdown_connections 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 -- 172.21.15.16:0/1025203872 wait complete. 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.406+0000 7f23c51dd640 1 Processor -- start 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 -- start start 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0100e20 0x7f23c010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0102ef0 0x7f23c010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01120a0 con 0x7f23c0102570 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0111f20 con 0x7f23c0102ef0 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23c51dd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0112220 con 0x7f23c0100e20 2024-09-12T01:11:05.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 -- 172.21.15.16:0/3549362140 learned_addr learned my addr 172.21.15.16:0/3549362140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23bed9d640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0100e20 0x7f23c010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0102ef0 0x7f23c010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 -- 172.21.15.16:0/3549362140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0100e20 msgr2=0x7f23c010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0100e20 0x7f23c010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 -- 172.21.15.16:0/3549362140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0102ef0 msgr2=0x7f23c010ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0102ef0 0x7f23c010ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.407+0000 7f23b7fff640 1 -- 172.21.15.16:0/3549362140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01a6980 con 0x7f23c0102570 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23b7fff640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f23a8006010 tx=0x7f23a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23bcd99640 1 -- 172.21.15.16:0/3549362140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a803f070 con 0x7f23c0102570 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23bcd99640 1 -- 172.21.15.16:0/3549362140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23a8031c30 con 0x7f23c0102570 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a6c10 con 0x7f23c0102570 2024-09-12T01:11:05.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23bcd99640 1 -- 172.21.15.16:0/3549362140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8035030 con 0x7f23c0102570 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.408+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a7140 con 0x7f23c0102570 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.410+0000 7f23bcd99640 1 -- 172.21.15.16:0/3549362140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f23a8042070 con 0x7f23c0102570 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.410+0000 7f23c75b6740 1 Processor -- start 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.410+0000 7f23c75b6740 1 -- start start 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34e3680 0x55faf34e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf343e2e0 con 0x55faf34d84a0 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55faf33e5ce0 con 0x55faf34e3680 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23c75b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55faf343a2a0 con 0x55faf34d8dc0 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bcd99640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 0x7f23ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41790/0 (socket says 172.21.15.16:41790) 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 -- 172.21.15.16:0/402676437 learned_addr learned my addr 172.21.15.16:0/402676437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 0x7f23ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bed9d640 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23b7fff640 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34e3680 0x55faf34e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bcd99640 1 -- 172.21.15.16:0/3549362140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f23a80efbb0 con 0x7f23c0102570 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 -- 172.21.15.16:0/402676437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 msgr2=0x55faf34e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 -- 172.21.15.16:0/402676437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34e3680 msgr2=0x55faf34e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34e3680 0x55faf34e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.411+0000 7f23bf59e640 1 -- 172.21.15.16:0/402676437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf33ca440 con 0x55faf34d84a0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.412+0000 7f23bf59e640 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f23b8014df0 tx=0x7f23b80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42138ead1be1f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.412+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 0x7f23ac0a5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011cb0 tx=0x7f23b801c000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.412+0000 7f23b57fa640 1 -- 172.21.15.16:0/402676437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8014770 con 0x55faf34d84a0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.412+0000 7f23b57fa640 1 -- 172.21.15.16:0/402676437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b8003e50 con 0x55faf34d84a0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.412+0000 7f23b57fa640 1 -- 172.21.15.16:0/402676437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8004810 con 0x55faf34d84a0 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- 172.21.15.16:0/402676437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 msgr2=0x55faf34d8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f23b8014df0 tx=0x7f23b80117b0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- 172.21.15.16:0/402676437 shutdown_connections 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34e3680 0x55faf34e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- 172.21.15.16:0/402676437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34d84a0 0x55faf34d8880 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- 172.21.15.16:0/402676437 >> 172.21.15.16:0/402676437 conn(0x55faf34d4a30 msgr2=0x55faf34d4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- 172.21.15.16:0/402676437 shutdown_connections 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- 172.21.15.16:0/402676437 wait complete. 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- start start 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 Processor -- start 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 0x55faf34e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d8dc0 0x55faf34e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf34e6250 con 0x55faf34e3680 2024-09-12T01:11:05.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55faf33f9050 con 0x55faf34d8dc0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55faf34e63d0 con 0x55faf34d84a0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.413+0000 7f23c75b6740 10 client.0 _ll_register_callbacks cb 0x55faf3439160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41806/0 (socket says 172.21.15.16:41806) 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 -- 172.21.15.16:0/2397794918 learned_addr learned my addr 172.21.15.16:0/2397794918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bed9d640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 0x55faf34e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23b7fff640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d8dc0 0x55faf34e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 -- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 msgr2=0x55faf34e2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 0x55faf34e2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 -- 172.21.15.16:0/2397794918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d8dc0 msgr2=0x55faf34e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d8dc0 0x55faf34e0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 -- 172.21.15.16:0/2397794918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf31af3c0 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bed9d640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 0x55faf34e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23bf59e640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006010 tx=0x7f23b8014a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48837c153e81ee0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8033060 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f23b8004750 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b8003e50 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23b8014050 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23c75b6740 10 client.19935 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.414+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55faf34f1bd0 con 0x55faf34e3680 2024-09-12T01:11:05.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b8003e50 con 0x55faf34e3680 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23b801d050 con 0x55faf34e3680 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 10 client.19935 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 10 client.19935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 20 client.19935 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf34f3030 0x55faf34f5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 4 client.19935 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf344b850 con 0x55faf34f3030 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.415+0000 7f23bed9d640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf34f3030 0x55faf34f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23bed9d640 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf34f3030 0x55faf34f5420 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a417793371d118e5 server_cookie=dbc33ab76d7e5174 in_seq=0 out_seq=0 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23a67fc640 10 client.19935 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23a67fc640 1 -- 172.21.15.16:0/2397794918 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faf344b850 con 0x55faf34f3030 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23a67fc640 10 client.19935 handle_command_reply: tid=0 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23c75b6740 1 client.19935 shutdown 2024-09-12T01:11:05.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 msgr2=0x55faf34e0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.416+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006010 tx=0x7f23b8014a50 comp rx=0 tx=0).stop 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 shutdown_connections 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf34f3030 0x55faf34f5420 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e0780 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d8dc0 0x55faf34e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23a67fc640 0 client.19935 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 --2- 172.21.15.16:0/2397794918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d84a0 0x55faf34e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 >> 172.21.15.16:0/2397794918 conn(0x55faf34d4a30 msgr2=0x55faf34da710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 shutdown_connections 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 1 -- 172.21.15.16:0/2397794918 wait complete. 2024-09-12T01:11:05.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.417+0000 7f23c75b6740 20 client.19935 trim_cache size 0 max 16384 2024-09-12T01:11:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.434+0000 7f23c75b6740 1 Processor -- start 2024-09-12T01:11:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.434+0000 7f23c75b6740 1 -- start start 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34faf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01afaf0 con 0x55faf34e3680 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b1630 con 0x55faf34d84a0 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23c75b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b01c0 con 0x55faf34d8dc0 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41820/0 (socket says 172.21.15.16:41820) 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 -- 172.21.15.16:0/609382904 learned_addr learned my addr 172.21.15.16:0/609382904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23b7fff640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bed9d640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 -- 172.21.15.16:0/609382904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 msgr2=0x55faf34faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 -- 172.21.15.16:0/609382904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 msgr2=0x55faf34fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 -- 172.21.15.16:0/609382904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf34d5ae0 con 0x55faf34e3680 2024-09-12T01:11:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23b7fff640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23bf59e640 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f23b800a340 tx=0x7f23b80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acf2d354406d5b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.435+0000 7f23b4ff9640 1 -- 172.21.15.16:0/609382904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8033070 con 0x55faf34e3680 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.436+0000 7f23b4ff9640 1 -- 172.21.15.16:0/609382904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b80141f0 con 0x55faf34e3680 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.436+0000 7f23b4ff9640 1 -- 172.21.15.16:0/609382904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8013460 con 0x55faf34e3680 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- 172.21.15.16:0/609382904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 msgr2=0x55faf34dd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f23b800a340 tx=0x7f23b80117b0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- 172.21.15.16:0/609382904 shutdown_connections 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34dd4a0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 --2- 172.21.15.16:0/609382904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- 172.21.15.16:0/609382904 >> 172.21.15.16:0/609382904 conn(0x55faf34d4a30 msgr2=0x55faf34e22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- 172.21.15.16:0/609382904 shutdown_connections 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- 172.21.15.16:0/609382904 wait complete. 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 -- start start 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.437+0000 7f23c75b6740 1 Processor -- start 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b0f60 con 0x55faf34e3680 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01b0890 con 0x55faf34d84a0 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01adaf0 con 0x55faf34d8dc0 2024-09-12T01:11:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23c75b6740 10 client.0 _ll_register_callbacks cb 0x7f23a8005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41822/0 (socket says 172.21.15.16:41822) 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 -- 172.21.15.16:0/3638257809 learned_addr learned my addr 172.21.15.16:0/3638257809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:42030/0 (socket says 172.21.15.16:42030) 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23b7fff640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 -- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 msgr2=0x55faf34e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 -- 172.21.15.16:0/3638257809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 msgr2=0x55faf34e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bf59e640 1 -- 172.21.15.16:0/3638257809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf355c9a0 con 0x55faf34e3680 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.438+0000 7f23bed9d640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23bf59e640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f23b8021040 tx=0x7f23b8014ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f470358e168762 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8033070 con 0x55faf34e3680 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f23b8013660 con 0x55faf34e3680 2024-09-12T01:11:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b80141f0 con 0x55faf34e3680 2024-09-12T01:11:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23b80043a0 con 0x55faf34e3680 2024-09-12T01:11:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 10 client.19941 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55faf355dc60 con 0x55faf34e3680 2024-09-12T01:11:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b80141f0 con 0x55faf34e3680 2024-09-12T01:11:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23b801d070 con 0x55faf34e3680 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 10 client.19941 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 10 client.19941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 20 client.19941 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf355e2b0 0x55faf35606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 4 client.19941 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.439+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55faf3445d60 con 0x55faf355e2b0 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23bed9d640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf355e2b0 0x55faf35606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23bed9d640 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf355e2b0 0x55faf35606a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58bf099f6951184 server_cookie=86c0b5b390b1b627 in_seq=0 out_seq=0 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23a67fc640 10 client.19941 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23a67fc640 1 -- 172.21.15.16:0/3638257809 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55faf3445d60 con 0x55faf355e2b0 2024-09-12T01:11:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23a67fc640 10 client.19941 handle_command_reply: tid=0 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 client.19941 shutdown 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 msgr2=0x55faf34e9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f23b8021040 tx=0x7f23b8014ad0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 shutdown_connections 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55faf355e2b0 0x55faf35606a0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf34e3680 0x55faf34e9df0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55faf34d8dc0 0x55faf34e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 --2- 172.21.15.16:0/3638257809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faf34d84a0 0x55faf34e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 >> 172.21.15.16:0/3638257809 conn(0x55faf34d4a30 msgr2=0x55faf34d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.440+0000 7f23a67fc640 0 client.19941 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 shutdown_connections 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c75b6740 1 -- 172.21.15.16:0/3638257809 wait complete. 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c75b6740 20 client.19941 trim_cache size 0 max 16384 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:11:05.454 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:11:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 msgr2=0x7f23ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 0x7f23ac0a5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f23b8011cb0 tx=0x7f23b801c000 comp rx=0 tx=0).stop 2024-09-12T01:11:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 msgr2=0x7f23c010ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f23a8006010 tx=0x7f23a8003c10 comp rx=0 tx=0).stop 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 shutdown_connections 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f23ac0a2d70 0x7f23ac0a5230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23c0102ef0 0x7f23c010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c0102570 0x7f23c010ea10 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 --2- 172.21.15.16:0/3549362140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23c0100e20 0x7f23c010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 >> 172.21.15.16:0/3549362140 conn(0x7f23c00fc7d0 msgr2=0x7f23c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 shutdown_connections 2024-09-12T01:11:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:05.441+0000 7f23c51dd640 1 -- 172.21.15.16:0/3549362140 wait complete. 2024-09-12T01:11:06.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:05 smithi099 ceph-mon[35654]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:11:06.037 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:05 smithi016 ceph-mon[30972]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:11:06.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:05 smithi159 ceph-mon[35566]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T01:11:06.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:11:06.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:11:06.534 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:06.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:06 smithi016 ceph-mon[30972]: pgmap v469: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:07.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:06 smithi099 ceph-mon[35654]: pgmap v469: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 -- 172.21.15.16:0/3694584555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d68101280 msgr2=0x7f7d6810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 --2- 172.21.15.16:0/3694584555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d68101280 0x7f7d6810df40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009900 tx=0x7f7d5802a3f0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 -- 172.21.15.16:0/3694584555 shutdown_connections 2024-09-12T01:11:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 --2- 172.21.15.16:0/3694584555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 0x7f7d68110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 --2- 172.21.15.16:0/3694584555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d68101280 0x7f7d6810df40 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 --2- 172.21.15.16:0/3694584555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d681008d0 0x7f7d68100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 -- 172.21.15.16:0/3694584555 >> 172.21.15.16:0/3694584555 conn(0x7f7d680fc7a0 msgr2=0x7f7d680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 -- 172.21.15.16:0/3694584555 shutdown_connections 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.012+0000 7f7d701d0640 1 -- 172.21.15.16:0/3694584555 wait complete. 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 Processor -- start 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 -- start start 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d68101280 0x7f7d6819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 0x7f7d6819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68114190 con 0x7f7d681008d0 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68114010 con 0x7f7d68101280 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.013+0000 7f7d701d0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68114310 con 0x7f7d6810e480 2024-09-12T01:11:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41848/0 (socket says 172.21.15.16:41848) 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 -- 172.21.15.16:0/1553680008 learned_addr learned my addr 172.21.15.16:0/1553680008 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6e76d640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 0x7f7d6819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6d76b640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d68101280 0x7f7d6819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 -- 172.21.15.16:0/1553680008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 msgr2=0x7f7d6819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 0x7f7d6819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 -- 172.21.15.16:0/1553680008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d68101280 msgr2=0x7f7d6819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d68101280 0x7f7d6819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 -- 172.21.15.16:0/1553680008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6819f540 con 0x7f7d681008d0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d6df6c640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400c920 tx=0x7f7d6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64007c70 con 0x7f7d681008d0 2024-09-12T01:11:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6819f830 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d64007e10 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64018890 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.014+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d681a7110 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.015+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d68100cb0 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.016+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d64018a50 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.017+0000 7f7d5effd640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 0x7f7d540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.017+0000 7f7d6d76b640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 0x7f7d540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.017+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f7d640cf920 con 0x7f7d681008d0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.018+0000 7f7d6d76b640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 0x7f7d540a51b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d580096c0 tx=0x7f7d58000bc0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.020+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d64014030 con 0x7f7d681008d0 2024-09-12T01:11:07.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:06 smithi159 ceph-mon[35566]: pgmap v469: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.232+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6819b8f0 con 0x7f7d681008d0 2024-09-12T01:11:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.233+0000 7f7d5effd640 1 -- 172.21.15.16:0/1553680008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f7d6409c5a0 con 0x7f7d681008d0 2024-09-12T01:11:07.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:11:07.233 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:11:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:11:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.235+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 msgr2=0x7f7d540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.235+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 0x7f7d540a51b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d580096c0 tx=0x7f7d58000bc0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.235+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 msgr2=0x7f7d6819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.235+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400c920 tx=0x7f7d6400cde0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 shutdown_connections 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7d540a2cf0 0x7f7d540a51b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6810e480 0x7f7d6819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d68101280 0x7f7d6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 --2- 172.21.15.16:0/1553680008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d681008d0 0x7f7d6819a590 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 >> 172.21.15.16:0/1553680008 conn(0x7f7d680fc7a0 msgr2=0x7f7d680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 shutdown_connections 2024-09-12T01:11:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:07.236+0000 7f7d701d0640 1 -- 172.21.15.16:0/1553680008 wait complete. 2024-09-12T01:11:07.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:07 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1553680008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:08.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:07 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1553680008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:08.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:11:08.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:07 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1553680008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:08.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 -- 172.21.15.16:0/2203765672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710100e20 msgr2=0x7fb710101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2203765672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710100e20 0x7fb710101200 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb700009ca0 tx=0x7fb70002a950 comp rx=0 tx=0).stop 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 -- 172.21.15.16:0/2203765672 shutdown_connections 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2203765672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102ef0 0x7fb71010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2203765672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710102570 0x7fb7101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2203765672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710100e20 0x7fb710101200 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 -- 172.21.15.16:0/2203765672 >> 172.21.15.16:0/2203765672 conn(0x7fb7100fc7d0 msgr2=0x7fb7100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 -- 172.21.15.16:0/2203765672 shutdown_connections 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.840+0000 7fb714ed3640 1 -- 172.21.15.16:0/2203765672 wait complete. 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 Processor -- start 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 -- start start 2024-09-12T01:11:08.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710100e20 0x7fb710199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102570 0x7fb7101998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb710110fc0 con 0x7fb710102ef0 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb710110e40 con 0x7fb710100e20 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb714ed3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb710111140 con 0x7fb710102570 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44474/0 (socket says 172.21.15.16:44474) 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 -- 172.21.15.16:0/2362837570 learned_addr learned my addr 172.21.15.16:0/2362837570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102570 0x7fb7101998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70e59c640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710100e20 0x7fb710199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 -- 172.21.15.16:0/2362837570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102570 msgr2=0x7fb7101998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102570 0x7fb7101998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 -- 172.21.15.16:0/2362837570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710100e20 msgr2=0x7fb710199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710100e20 0x7fb710199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.841+0000 7fb70ed9d640 1 -- 172.21.15.16:0/2362837570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7101a4950 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb70ed9d640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c970 tx=0x7fb70800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb6ff7fe640 1 -- 172.21.15.16:0/2362837570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708007bb0 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb6ff7fe640 1 -- 172.21.15.16:0/2362837570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708007d50 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7101a4b20 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb6ff7fe640 1 -- 172.21.15.16:0/2362837570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70800f450 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.842+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7101a5060 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.844+0000 7fb6ff7fe640 1 -- 172.21.15.16:0/2362837570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb708004f20 con 0x7fb710102ef0 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.844+0000 7fb7172ac740 1 Processor -- start 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 -- start start 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 0x56462e732090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e741780 0x56462e743b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56462e6982f0 con 0x56462e7325d0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56462e63fcf0 con 0x56462e733b40 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb7172ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56462e6942b0 con 0x56462e741780 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb6ff7fe640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 0x7fb6f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44478/0 (socket says 172.21.15.16:44478) 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70e59c640 1 -- 172.21.15.16:0/1533736150 learned_addr learned my addr 172.21.15.16:0/1533736150 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70ed9d640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 0x7fb6f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 0x56462e732090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb6ff7fe640 1 -- 172.21.15.16:0/2362837570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7fb7080cf1b0 con 0x7fb710102ef0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e741780 0x56462e743b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 -- 172.21.15.16:0/1533736150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e741780 msgr2=0x56462e743b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e741780 0x56462e743b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 -- 172.21.15.16:0/1533736150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 msgr2=0x56462e732a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70dd9b640 1 -- 172.21.15.16:0/1533736150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462e6243a0 con 0x56462e733b40 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.845+0000 7fb70e59c640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.846+0000 7fb70ed9d640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 0x7fb6f80a5230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb708098010 tx=0x7fb70801baf0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.846+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 0x56462e732090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb71006aac0 tx=0x7fb70400a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dec87e46feafef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.846+0000 7fb6fcff9640 1 -- 172.21.15.16:0/1533736150 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704016dc0 con 0x56462e733b40 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.846+0000 7fb6fcff9640 1 -- 172.21.15.16:0/1533736150 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb704018910 con 0x56462e733b40 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.846+0000 7fb6fcff9640 1 -- 172.21.15.16:0/1533736150 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704017880 con 0x56462e733b40 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- 172.21.15.16:0/1533736150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 msgr2=0x56462e732090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 0x56462e732090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb71006aac0 tx=0x7fb70400a230 comp rx=0 tx=0).stop 2024-09-12T01:11:08.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- 172.21.15.16:0/1533736150 shutdown_connections 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e741780 0x56462e743b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e732a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- 172.21.15.16:0/1533736150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e733b40 0x56462e732090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- 172.21.15.16:0/1533736150 >> 172.21.15.16:0/1533736150 conn(0x56462e6fb310 msgr2=0x56462e72ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- 172.21.15.16:0/1533736150 shutdown_connections 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- 172.21.15.16:0/1533736150 wait complete. 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- start start 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 Processor -- start 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e7325d0 0x56462e741140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56462e744420 con 0x56462e741780 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56462e653060 con 0x56462e7325d0 2024-09-12T01:11:08.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56462e7445a0 con 0x56462e733b40 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.847+0000 7fb7172ac740 10 client.0 _ll_register_callbacks cb 0x56462e3a68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44494/0 (socket says 172.21.15.16:44494) 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 -- 172.21.15.16:0/3159509526 learned_addr learned my addr 172.21.15.16:0/3159509526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e7325d0 0x56462e741140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 -- 172.21.15.16:0/3159509526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 msgr2=0x56462e73e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 -- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e7325d0 msgr2=0x56462e741140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e7325d0 0x56462e741140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 -- 172.21.15.16:0/3159509526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462e4093c0 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb7040175c0 tx=0x7fb704006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7700f4bfe62245 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040186f0 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb704018910 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7fb704017660 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.848+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb704035050 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 10 client.19959 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56462e74feb0 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb704018910 con 0x56462e741780 2024-09-12T01:11:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb704024040 con 0x56462e741780 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 10 client.19959 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 10 client.19959 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 20 client.19959 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e751200 0x56462e7535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 4 client.19959 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56462e6a5860 con 0x56462e751200 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.849+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e751200 0x56462e7535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.850+0000 7fb6f5ffb640 10 client.19959 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.850+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e751200 0x56462e7535f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6975b8e639a6d8e7 server_cookie=7553221b747daf0a in_seq=0 out_seq=0 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.850+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3159509526 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56462e6a5860 con 0x56462e751200 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.850+0000 7fb6f5ffb640 10 client.19959 handle_command_reply: tid=0 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 client.19959 shutdown 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 msgr2=0x56462e73e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb7040175c0 tx=0x7fb704006a30 comp rx=0 tx=0).stop 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 shutdown_connections 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e751200 0x56462e7535f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e741780 0x56462e73e9e0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb6f5ffb640 0 client.19959 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3159509526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e7325d0 0x56462e741140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 >> 172.21.15.16:0/3159509526 conn(0x56462e6fb310 msgr2=0x56462e734930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 shutdown_connections 2024-09-12T01:11:08.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 1 -- 172.21.15.16:0/3159509526 wait complete. 2024-09-12T01:11:08.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.851+0000 7fb7172ac740 20 client.19959 trim_cache size 0 max 16384 2024-09-12T01:11:08.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.865+0000 7fb7172ac740 1 Processor -- start 2024-09-12T01:11:08.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.865+0000 7fb7172ac740 1 -- start start 2024-09-12T01:11:08.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e758c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101ab840 con 0x56462e7325d0 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101ad380 con 0x56462e741780 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb7172ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101abf10 con 0x56462e733b40 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44506/0 (socket says 172.21.15.16:44506) 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 -- 172.21.15.16:0/80373622 learned_addr learned my addr 172.21.15.16:0/80373622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e758c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70e59c640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 -- 172.21.15.16:0/80373622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 msgr2=0x56462e73d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 -- 172.21.15.16:0/80373622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 msgr2=0x56462e758c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e758c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 -- 172.21.15.16:0/80373622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462e739fd0 con 0x56462e7325d0 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70e59c640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb70400aad0 tx=0x7fb70400a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7ac20e42ccf550 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.866+0000 7fb6f7fff640 1 -- 172.21.15.16:0/80373622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70400ae60 con 0x56462e7325d0 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb6f7fff640 1 -- 172.21.15.16:0/80373622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb70402c070 con 0x56462e7325d0 2024-09-12T01:11:08.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb6f7fff640 1 -- 172.21.15.16:0/80373622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040175f0 con 0x56462e7325d0 2024-09-12T01:11:08.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 -- 172.21.15.16:0/80373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 msgr2=0x56462e74d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb70400aad0 tx=0x7fb70400a760 comp rx=0 tx=0).stop 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 -- 172.21.15.16:0/80373622 shutdown_connections 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e758c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e733b40 0x56462e73d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 --2- 172.21.15.16:0/80373622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e7325d0 0x56462e74d6f0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 -- 172.21.15.16:0/80373622 >> 172.21.15.16:0/80373622 conn(0x56462e6fb310 msgr2=0x56462e73ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 -- 172.21.15.16:0/80373622 shutdown_connections 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.867+0000 7fb7172ac740 1 -- 172.21.15.16:0/80373622 wait complete. 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 -- start start 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 Processor -- start 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e7325d0 0x56462e7456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e746120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101accb0 con 0x56462e733b40 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101ac5e0 con 0x56462e741780 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb7101a9840 con 0x56462e7325d0 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb7172ac740 10 client.0 _ll_register_callbacks cb 0x7fb700009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44512/0 (socket says 172.21.15.16:44512) 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 -- 172.21.15.16:0/3703169394 learned_addr learned my addr 172.21.15.16:0/3703169394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e7325d0 0x56462e7456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e746120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 -- 172.21.15.16:0/3703169394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e7325d0 msgr2=0x56462e7456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e7325d0 0x56462e7456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.868+0000 7fb70e59c640 1 -- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 msgr2=0x56462e746120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e746120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb70e59c640 1 -- 172.21.15.16:0/3703169394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462e7bac40 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e746120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb70e59c640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb700005fc0 tx=0x7fb70002fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df8d8d940e7c4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700034040 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7fb700003970 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb70402c070 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb700055050 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb7172ac740 10 client.19965 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56462e7bbf20 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb70402c070 con 0x56462e733b40 2024-09-12T01:11:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.869+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb700046420 con 0x56462e733b40 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 10 client.19965 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 10 client.19965 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 20 client.19965 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e7bc6d0 0x56462e7beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 4 client.19965 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56462e69fd70 con 0x56462e7bc6d0 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e7bc6d0 0x56462e7beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb70dd9b640 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e7bc6d0 0x56462e7beaf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=520ff3fde278dc32 server_cookie=3fda8161c55f5120 in_seq=0 out_seq=0 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb6f5ffb640 10 client.19965 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb6f5ffb640 1 -- 172.21.15.16:0/3703169394 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56462e69fd70 con 0x56462e7bc6d0 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb6f5ffb640 10 client.19965 handle_command_reply: tid=0 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 1 client.19965 shutdown 2024-09-12T01:11:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 msgr2=0x56462e745be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.870+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb700005fc0 tx=0x7fb70002fa90 comp rx=0 tx=0).stop 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 shutdown_connections 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x56462e7bc6d0 0x56462e7beaf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56462e741780 0x56462e746120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56462e733b40 0x56462e745be0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 --2- 172.21.15.16:0/3703169394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56462e7325d0 0x56462e7456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 >> 172.21.15.16:0/3703169394 conn(0x56462e6fb310 msgr2=0x56462e73d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb6f5ffb640 0 client.19965 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 shutdown_connections 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 1 -- 172.21.15.16:0/3703169394 wait complete. 2024-09-12T01:11:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb7172ac740 20 client.19965 trim_cache size 0 max 16384 2024-09-12T01:11:08.877 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 msgr2=0x7fb6f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 0x7fb6f80a5230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb708098010 tx=0x7fb70801baf0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 msgr2=0x7fb7101a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c970 tx=0x7fb70800ce30 comp rx=0 tx=0).stop 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 shutdown_connections 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6f80a2d70 0x7fb6f80a5230 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710102ef0 0x7fb7101a4410 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb710102570 0x7fb7101998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 --2- 172.21.15.16:0/2362837570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb710100e20 0x7fb710199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 >> 172.21.15.16:0/2362837570 conn(0x7fb7100fc7d0 msgr2=0x7fb7100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 shutdown_connections 2024-09-12T01:11:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:08.871+0000 7fb714ed3640 1 -- 172.21.15.16:0/2362837570 wait complete. 2024-09-12T01:11:09.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:08 smithi099 ceph-mon[35654]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:09.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:08 smithi159 ceph-mon[35566]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:09.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:08 smithi016 ceph-mon[30972]: pgmap v470: 97 pgs: 97 active+clean; 24 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-12T01:11:09.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:11:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 -- 172.21.15.16:0/2554333303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c102790 msgr2=0x7f309c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/2554333303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c102790 0x7f309c1067e0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3090009900 tx=0x7f309002a410 comp rx=0 tx=0).stop 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 -- 172.21.15.16:0/2554333303 shutdown_connections 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/2554333303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c106f10 0x7f309c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/2554333303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c102790 0x7f309c1067e0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/2554333303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c101de0 0x7f309c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 -- 172.21.15.16:0/2554333303 >> 172.21.15.16:0/2554333303 conn(0x7f309c0fd8c0 msgr2=0x7f309c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 -- 172.21.15.16:0/2554333303 shutdown_connections 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.506+0000 7f30a3fe6640 1 -- 172.21.15.16:0/2554333303 wait complete. 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 Processor -- start 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 -- start start 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c101de0 0x7f309c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c102790 0x7f309c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f309c111f70 con 0x7f309c106f10 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f309c111df0 con 0x7f309c102790 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a3fe6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1120f0 con 0x7f309c101de0 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a2583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a2583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44522/0 (socket says 172.21.15.16:44522) 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a2583640 1 -- 172.21.15.16:0/3887954846 learned_addr learned my addr 172.21.15.16:0/3887954846 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:10.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a1581640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c102790 0x7f309c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.507+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c101de0 0x7f309c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 -- 172.21.15.16:0/3887954846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c101de0 msgr2=0x7f309c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c101de0 0x7f309c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 -- 172.21.15.16:0/3887954846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c102790 msgr2=0x7f309c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c102790 0x7f309c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 -- 172.21.15.16:0/3887954846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309c19f5c0 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a2583640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f309400d750 tx=0x7f309400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f308effd640 1 -- 172.21.15.16:0/3887954846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094017070 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f308effd640 1 -- 172.21.15.16:0/3887954846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005500 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309c19f8b0 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f308effd640 1 -- 172.21.15.16:0/3887954846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004d90 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.508+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f309c1a7190 con 0x7f309c106f10 2024-09-12T01:11:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.511+0000 7f308effd640 1 -- 172.21.15.16:0/3887954846 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3094004f30 con 0x7f309c106f10 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.511+0000 7f30a63bf740 1 Processor -- start 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.511+0000 7f30a63bf740 1 -- start start 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56236a9502f0 con 0x56236a9ea5d0 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56236a8f7cf0 con 0x56236a9ebb40 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a63bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56236a94c2b0 con 0x56236a9f5370 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44532/0 (socket says 172.21.15.16:44532) 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 -- 172.21.15.16:0/3569084475 learned_addr learned my addr 172.21.15.16:0/3569084475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1581640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1581640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f308effd640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 0x7f30880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 -- 172.21.15.16:0/3569084475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 msgr2=0x56236a9f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 -- 172.21.15.16:0/3569084475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 msgr2=0x56236a9ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 -- 172.21.15.16:0/3569084475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236a8dc3a0 con 0x56236a9ea5d0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1581640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a2583640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 0x7f30880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f308effd640 1 -- 172.21.15.16:0/3887954846 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f30940ced20 con 0x7f309c106f10 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.512+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f309800af40 tx=0x7f309800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d87555029c7ef3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f3087fff640 1 -- 172.21.15.16:0/3569084475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309801d070 con 0x56236a9ea5d0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f3087fff640 1 -- 172.21.15.16:0/3569084475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098010de0 con 0x56236a9ea5d0 2024-09-12T01:11:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f3087fff640 1 -- 172.21.15.16:0/3569084475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098016340 con 0x56236a9ea5d0 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f30a2583640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 0x7f30880a51b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f30940cc4f0 tx=0x7f30940bb040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f30a63bf740 1 -- 172.21.15.16:0/3569084475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 msgr2=0x56236a9eaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.513+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f309800af40 tx=0x7f309800f850 comp rx=0 tx=0).stop 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 -- 172.21.15.16:0/3569084475 shutdown_connections 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eaa10 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3569084475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 -- 172.21.15.16:0/3569084475 >> 172.21.15.16:0/3569084475 conn(0x56236a9b3310 msgr2=0x56236a9e6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 -- 172.21.15.16:0/3569084475 shutdown_connections 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 -- 172.21.15.16:0/3569084475 wait complete. 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 -- start start 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.514+0000 7f30a63bf740 1 Processor -- start 2024-09-12T01:11:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ea5d0 0x56236a9f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56236a9f8070 con 0x56236a9ebb40 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56236a90b060 con 0x56236a9ea5d0 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56236a9f81f0 con 0x56236a9f5370 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a63bf740 10 client.0 _ll_register_callbacks cb 0x56236a65e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44538/0 (socket says 172.21.15.16:44538) 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 -- 172.21.15.16:0/3546021045 learned_addr learned my addr 172.21.15.16:0/3546021045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1581640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ea5d0 0x56236a9f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 -- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 msgr2=0x56236a9f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1581640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 -- 172.21.15.16:0/3546021045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ea5d0 msgr2=0x56236a9f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ea5d0 0x56236a9f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 -- 172.21.15.16:0/3546021045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236a6c13c0 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1581640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30a1d82640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3098018b70 tx=0x7f309800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99483d90c66567b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.515+0000 7f30857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098026020 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f3098010700 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3098010de0 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3098031050 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 10 client.19977 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56236aa03af0 con 0x56236a9ebb40 2024-09-12T01:11:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098010de0 con 0x56236a9ebb40 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3098005720 con 0x56236a9ebb40 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 10 client.19977 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 10 client.19977 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 20 client.19977 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa04f50 0x56236aa07340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 4 client.19977 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.516+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236a95d860 con 0x56236aa04f50 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.517+0000 7f30a1581640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa04f50 0x56236aa07340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.517+0000 7f30a1581640 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa04f50 0x56236aa07340 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=850f7a01e1ea5cf6 server_cookie=671527239f262cbe in_seq=0 out_seq=0 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.517+0000 7f30857fa640 10 client.19977 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30857fa640 1 -- 172.21.15.16:0/3546021045 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56236a95d860 con 0x56236aa04f50 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30857fa640 10 client.19977 handle_command_reply: tid=0 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 client.19977 shutdown 2024-09-12T01:11:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 msgr2=0x56236a9f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3098018b70 tx=0x7f309800a330 comp rx=0 tx=0).stop 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 shutdown_connections 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236a9f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa04f50 0x56236aa07340 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ebb40 0x56236a9f2000 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 --2- 172.21.15.16:0/3546021045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ea5d0 0x56236a9f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30857fa640 0 client.19977 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 >> 172.21.15.16:0/3546021045 conn(0x56236a9b3310 msgr2=0x56236a9ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 shutdown_connections 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 1 -- 172.21.15.16:0/3546021045 wait complete. 2024-09-12T01:11:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.518+0000 7f30a63bf740 20 client.19977 trim_cache size 0 max 16384 2024-09-12T01:11:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 Processor -- start 2024-09-12T01:11:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 -- start start 2024-09-12T01:11:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236aa0c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1afb70 con 0x56236a9ea5d0 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b16b0 con 0x56236a9ebb40 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a63bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b0240 con 0x56236a9f5370 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a1581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a1581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44552/0 (socket says 172.21.15.16:44552) 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a1581640 1 -- 172.21.15.16:0/1516945461 learned_addr learned my addr 172.21.15.16:0/1516945461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a1d82640 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.532+0000 7f30a1581640 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236aa0c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 -- 172.21.15.16:0/1516945461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 msgr2=0x56236aa0c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236aa0c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 -- 172.21.15.16:0/1516945461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 msgr2=0x56236a9f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 -- 172.21.15.16:0/1516945461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236a9e5310 con 0x56236a9ea5d0 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30a1581640 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f309002a930 tx=0x7f3090000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674018be25321313 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30877fe640 1 -- 172.21.15.16:0/1516945461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309004f070 con 0x56236a9ea5d0 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30877fe640 1 -- 172.21.15.16:0/1516945461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3090005500 con 0x56236a9ea5d0 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.533+0000 7f30877fe640 1 -- 172.21.15.16:0/1516945461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090035040 con 0x56236a9ea5d0 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- 172.21.15.16:0/1516945461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 msgr2=0x56236aa01310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f309002a930 tx=0x7f3090000bc0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- 172.21.15.16:0/1516945461 shutdown_connections 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9f5370 0x56236aa0c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9ebb40 0x56236a9f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 --2- 172.21.15.16:0/1516945461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236aa01310 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- 172.21.15.16:0/1516945461 >> 172.21.15.16:0/1516945461 conn(0x56236a9b3310 msgr2=0x56236a9f3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- 172.21.15.16:0/1516945461 shutdown_connections 2024-09-12T01:11:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- 172.21.15.16:0/1516945461 wait complete. 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 -- start start 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.534+0000 7f30a63bf740 1 Processor -- start 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9ebb40 0x56236a9ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9f5370 0x56236a9ef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b0fe0 con 0x56236a9ea5d0 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1b0910 con 0x56236a9f5370 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f309c1adb70 con 0x56236a9ebb40 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a63bf740 10 client.0 _ll_register_callbacks cb 0x7f309801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44558/0 (socket says 172.21.15.16:44558) 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 -- 172.21.15.16:0/2743552696 learned_addr learned my addr 172.21.15.16:0/2743552696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9f5370 0x56236a9ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1d82640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9ebb40 0x56236a9ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 -- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9ebb40 msgr2=0x56236a9ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9ebb40 0x56236a9ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 -- 172.21.15.16:0/2743552696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9f5370 msgr2=0x56236a9ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9f5370 0x56236a9ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.535+0000 7f30a1581640 1 -- 172.21.15.16:0/2743552696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236a9ed120 con 0x56236a9ea5d0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a1581640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f309004c040 tx=0x7f3090005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a84f872d5941e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309004f030 con 0x56236a9ea5d0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f3090004110 con 0x56236a9ea5d0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3090005500 con 0x56236a9ea5d0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3090035070 con 0x56236a9ea5d0 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 10 client.19983 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56236aa6f980 con 0x56236a9ea5d0 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3090005500 con 0x56236a9ea5d0 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30900039f0 con 0x56236a9ea5d0 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 10 client.19983 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 10 client.19983 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 20 client.19983 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa70020 0x56236aa72410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 4 client.19983 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.536+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56236a957d70 con 0x56236aa70020 2024-09-12T01:11:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30a1d82640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa70020 0x56236aa72410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30a1d82640 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa70020 0x56236aa72410 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15eb93dd62c36eaa server_cookie=84eb51f656f8e6fe in_seq=0 out_seq=0 2024-09-12T01:11:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30857fa640 10 client.19983 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30857fa640 1 -- 172.21.15.16:0/2743552696 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56236a957d70 con 0x56236aa70020 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30857fa640 10 client.19983 handle_command_reply: tid=0 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.537+0000 7f30a63bf740 1 client.19983 shutdown 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 msgr2=0x56236a9eef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f309004c040 tx=0x7f3090005a20 comp rx=0 tx=0).stop 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 shutdown_connections 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56236a9f5370 0x56236a9ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56236a9ebb40 0x56236a9ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x56236aa70020 0x56236aa72410 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 --2- 172.21.15.16:0/2743552696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56236a9ea5d0 0x56236a9eef00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 >> 172.21.15.16:0/2743552696 conn(0x56236a9b3310 msgr2=0x56236a9f4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30857fa640 0 client.19983 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 shutdown_connections 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 1 -- 172.21.15.16:0/2743552696 wait complete. 2024-09-12T01:11:10.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a63bf740 20 client.19983 trim_cache size 0 max 16384 2024-09-12T01:11:10.544 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 msgr2=0x7f30880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 0x7f30880a51b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f30940cc4f0 tx=0x7f30940bb040 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 msgr2=0x7f309c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f309400d750 tx=0x7f309400dc10 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 shutdown_connections 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f30880a2cf0 0x7f30880a51b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f309c106f10 0x7f309c19eee0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f309c102790 0x7f309c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 --2- 172.21.15.16:0/3887954846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f309c101de0 0x7f309c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.538+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 >> 172.21.15.16:0/3887954846 conn(0x7f309c0fd8c0 msgr2=0x7f309c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.539+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 shutdown_connections 2024-09-12T01:11:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:10.539+0000 7f30a3fe6640 1 -- 172.21.15.16:0/3887954846 wait complete. 2024-09-12T01:11:11.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:10 smithi099 ceph-mon[35654]: pgmap v471: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:11.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:10 smithi159 ceph-mon[35566]: pgmap v471: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:11.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:10 smithi016 ceph-mon[30972]: pgmap v471: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:11.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:11:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 -- 172.21.15.16:0/1271447286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50101420 msgr2=0x7f5c5010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 --2- 172.21.15.16:0/1271447286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50101420 0x7f5c5010d950 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009cd0 tx=0x7f5c4802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 -- 172.21.15.16:0/1271447286 shutdown_connections 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 --2- 172.21.15.16:0/1271447286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50101420 0x7f5c5010d950 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 --2- 172.21.15.16:0/1271447286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c50100a80 0x7f5c50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 --2- 172.21.15.16:0/1271447286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50106a80 0x7f5c50106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 -- 172.21.15.16:0/1271447286 >> 172.21.15.16:0/1271447286 conn(0x7f5c500fc7a0 msgr2=0x7f5c500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 -- 172.21.15.16:0/1271447286 shutdown_connections 2024-09-12T01:11:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.057+0000 7f5c54b86640 1 -- 172.21.15.16:0/1271447286 wait complete. 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 Processor -- start 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 -- start start 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 0x7f5c500fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c50106a80 0x7f5c500fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50110ea0 con 0x7f5c50100a80 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50110d20 con 0x7f5c50101420 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c54b86640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c50111020 con 0x7f5c50106a80 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44572/0 (socket says 172.21.15.16:44572) 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 -- 172.21.15.16:0/696970723 learned_addr learned my addr 172.21.15.16:0/696970723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 0x7f5c500fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 -- 172.21.15.16:0/696970723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c50106a80 msgr2=0x7f5c500fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c50106a80 0x7f5c500fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 -- 172.21.15.16:0/696970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 msgr2=0x7f5c500fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 0x7f5c500fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4e59c640 1 -- 172.21.15.16:0/696970723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c501a69c0 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.058+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 0x7f5c500fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c890 tx=0x7f5c4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c3f7fe640 1 -- 172.21.15.16:0/696970723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40004220 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c3f7fe640 1 -- 172.21.15.16:0/696970723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c400043c0 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c501a6cb0 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c3f7fe640 1 -- 172.21.15.16:0/696970723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c400049e0 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.059+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c501a71a0 con 0x7f5c50100a80 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 Processor -- start 2024-09-12T01:11:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c3f7fe640 1 -- 172.21.15.16:0/696970723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5c400079b0 con 0x7f5c50100a80 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 -- start start 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 0x55adc1857a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc1864760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55adc17bd2f0 con 0x55adc1858b40 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adc1764cf0 con 0x55adc1862370 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.062+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55adc17b92b0 con 0x55adc18575d0 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44574/0 (socket says 172.21.15.16:44574) 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/1485026404 learned_addr learned my addr 172.21.15.16:0/1485026404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 0x55adc1857a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc1864760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/1485026404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 msgr2=0x55adc1857a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 0x55adc1857a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/1485026404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 msgr2=0x55adc1864760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc1864760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/1485026404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc17493a0 con 0x55adc1858b40 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 0x55adc1857a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c3f7fe640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 0x7f5c380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005fc0 tx=0x7f5c48009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1941221da180b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 0x7f5c380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c3f7fe640 1 -- 172.21.15.16:0/696970723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f5c400cf590 con 0x7f5c50100a80 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.063+0000 7f5c3cff9640 1 -- 172.21.15.16:0/1485026404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c480318c0 con 0x55adc1858b40 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c3cff9640 1 -- 172.21.15.16:0/1485026404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48036040 con 0x55adc1858b40 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c3cff9640 1 -- 172.21.15.16:0/1485026404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004640 con 0x55adc1858b40 2024-09-12T01:11:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c4e59c640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 0x7f5c380a51b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c400ccd60 tx=0x7f5c40005300 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1485026404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 msgr2=0x55adc1857090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005fc0 tx=0x7f5c48009a20 comp rx=0 tx=0).stop 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1485026404 shutdown_connections 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc1864760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc18575d0 0x55adc1857a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1485026404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1857090 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.064+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1485026404 >> 172.21.15.16:0/1485026404 conn(0x55adc1820310 msgr2=0x55adc1853c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1485026404 shutdown_connections 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1485026404 wait complete. 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- start start 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 Processor -- start 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1858b40 0x55adc185efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc185f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55adc1865010 con 0x55adc18575d0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55adc1778060 con 0x55adc1862370 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55adc1865190 con 0x55adc1858b40 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c56f5f740 10 client.0 _ll_register_callbacks cb 0x55adc14cb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44582/0 (socket says 172.21.15.16:44582) 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/4094174050 learned_addr learned my addr 172.21.15.16:0/4094174050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.065+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1858b40 0x55adc185efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc185f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/4094174050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1858b40 msgr2=0x55adc185efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1858b40 0x55adc185efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 msgr2=0x55adc185f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc185f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/4094174050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc152e3c0 con 0x55adc18575d0 2024-09-12T01:11:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f5c48033010 tx=0x7f5c48009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f10197c37bf477 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4804e040 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f5c48003b70 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c48036040 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c48004840 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c56f5f740 10 client.19995 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adc1870a70 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.066+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c48036040 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c48030070 con 0x55adc18575d0 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 10 client.19995 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 10 client.19995 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 20 client.19995 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc1871dc0 0x55adc18741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 4 client.19995 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc17ca860 con 0x55adc1871dc0 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc1871dc0 0x55adc18741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c35ffb640 10 client.19995 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.067+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc1871dc0 0x55adc18741b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa6c21c7388ead2d server_cookie=19fa73e9abb69bf1 in_seq=0 out_seq=0 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c35ffb640 1 -- 172.21.15.16:0/4094174050 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adc17ca860 con 0x55adc1871dc0 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c35ffb640 10 client.19995 handle_command_reply: tid=0 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 client.19995 shutdown 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 msgr2=0x55adc1861d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f5c48033010 tx=0x7f5c48009800 comp rx=0 tx=0).stop 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 shutdown_connections 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc1871dc0 0x55adc18741b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc1862370 0x55adc185f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1858b40 0x55adc185efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/4094174050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc18575d0 0x55adc1861d30 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c35ffb640 0 client.19995 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 >> 172.21.15.16:0/4094174050 conn(0x55adc1820310 msgr2=0x55adc1859960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 shutdown_connections 2024-09-12T01:11:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.068+0000 7f5c56f5f740 1 -- 172.21.15.16:0/4094174050 wait complete. 2024-09-12T01:11:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.069+0000 7f5c56f5f740 20 client.19995 trim_cache size 0 max 16384 2024-09-12T01:11:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.081+0000 7f5c56f5f740 1 Processor -- start 2024-09-12T01:11:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.081+0000 7f5c56f5f740 1 -- start start 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc18771d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc185c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501afc50 con 0x55adc1858b40 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501b1790 con 0x55adc18575d0 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501b0320 con 0x55adc1862370 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44588/0 (socket says 172.21.15.16:44588) 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 -- 172.21.15.16:0/1945573185 learned_addr learned my addr 172.21.15.16:0/1945573185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc18771d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc185c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 -- 172.21.15.16:0/1945573185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 msgr2=0x55adc185c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc185c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 -- 172.21.15.16:0/1945573185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 msgr2=0x55adc18771d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc18771d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 -- 172.21.15.16:0/1945573185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc1853e30 con 0x55adc1858b40 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400ca90 tx=0x7f5c44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e60a9ef5f22df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c37fff640 1 -- 172.21.15.16:0/1945573185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440158d0 con 0x55adc1858b40 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c37fff640 1 -- 172.21.15.16:0/1945573185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4401f030 con 0x55adc1858b40 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.082+0000 7f5c37fff640 1 -- 172.21.15.16:0/1945573185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44023c30 con 0x55adc1858b40 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1945573185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 msgr2=0x55adc1877710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400ca90 tx=0x7f5c44008040 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1945573185 shutdown_connections 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc185c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1858b40 0x55adc1877710 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/1945573185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc18771d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1945573185 >> 172.21.15.16:0/1945573185 conn(0x55adc1820310 msgr2=0x55adc185a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1945573185 shutdown_connections 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- 172.21.15.16:0/1945573185 wait complete. 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 -- start start 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.083+0000 7f5c56f5f740 1 Processor -- start 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc1875230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc1875770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501b10c0 con 0x55adc1875d00 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501b09f0 con 0x55adc18575d0 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c501adc50 con 0x55adc1862370 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c56f5f740 10 client.0 _ll_register_callbacks cb 0x7f5c4803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44596/0 (socket says 172.21.15.16:44596) 2024-09-12T01:11:12.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/35873144 learned_addr learned my addr 172.21.15.16:0/35873144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc1875230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc1875770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/35873144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 msgr2=0x55adc1875770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc1875770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 msgr2=0x55adc1875230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc1875230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 -- 172.21.15.16:0/35873144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc18dbe40 con 0x55adc1875d00 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c4ed9d640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f5c48002230 tx=0x7f5c48031bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95674d3e0c6572b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.084+0000 7f5c35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48030ea0 con 0x55adc1875d00 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8565+0+0 (secure 0 0 0) 0x7f5c48004b40 con 0x55adc1875d00 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4401f030 con 0x55adc1875d00 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c4806b050 con 0x55adc1875d00 2024-09-12T01:11:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 10 client.20001 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55adc18dd140 con 0x55adc1875d00 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c4401f030 con 0x55adc1875d00 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c48003d20 con 0x55adc1875d00 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 10 client.20001 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 10 client.20001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 20 client.20001 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc18dd7b0 0x55adc18dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 4 client.20001 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.085+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adc17c4d70 con 0x55adc18dd7b0 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc18dd7b0 0x55adc18dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c4dd9b640 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc18dd7b0 0x55adc18dfba0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9321541a95ebcd4 server_cookie=2b706ddb96b7c3ce in_seq=0 out_seq=0 2024-09-12T01:11:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c35ffb640 10 client.20001 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c35ffb640 1 -- 172.21.15.16:0/35873144 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adc17c4d70 con 0x55adc18dd7b0 2024-09-12T01:11:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c35ffb640 10 client.20001 handle_command_reply: tid=0 2024-09-12T01:11:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 client.20001 shutdown 2024-09-12T01:11:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 msgr2=0x55adc18db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f5c48002230 tx=0x7f5c48031bd0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 shutdown_connections 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55adc18dd7b0 0x55adc18dfba0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55adc1875d00 0x55adc18db700 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc1862370 0x55adc1875770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 --2- 172.21.15.16:0/35873144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55adc18575d0 0x55adc1875230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c35ffb640 0 client.20001 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.086+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 >> 172.21.15.16:0/35873144 conn(0x55adc1820310 msgr2=0x55adc18524e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 shutdown_connections 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c56f5f740 1 -- 172.21.15.16:0/35873144 wait complete. 2024-09-12T01:11:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c56f5f740 20 client.20001 trim_cache size 0 max 16384 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 msgr2=0x7f5c380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 0x7f5c380a51b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c400ccd60 tx=0x7f5c40005300 comp rx=0 tx=0).stop 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 msgr2=0x7f5c500fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c890 tx=0x7f5c4000cd50 comp rx=0 tx=0).stop 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 shutdown_connections 2024-09-12T01:11:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f5c380a2cf0 0x7f5c380a51b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c50106a80 0x7f5c500fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c50101420 0x7f5c500fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 --2- 172.21.15.16:0/696970723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c50100a80 0x7f5c500fcd40 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 >> 172.21.15.16:0/696970723 conn(0x7f5c500fc7a0 msgr2=0x7f5c5006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 shutdown_connections 2024-09-12T01:11:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:12.087+0000 7f5c54b86640 1 -- 172.21.15.16:0/696970723 wait complete. 2024-09-12T01:11:12.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:12 smithi016 ceph-mon[30972]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:12.861 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:12 smithi016 ceph-mon[30972]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:11:12.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:11:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:12 smithi099 ceph-mon[35654]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:13.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:12 smithi099 ceph-mon[35654]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:11:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:12 smithi159 ceph-mon[35566]: pgmap v472: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 133 KiB/s wr, 36 op/s 2024-09-12T01:11:13.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:12 smithi159 ceph-mon[35566]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T01:11:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:13.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- 172.21.15.16:0/3744789768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a01002f0 msgr2=0x7fb4a01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/3744789768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a01002f0 0x7fb4a01006d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb490009930 tx=0x7fb49002a420 comp rx=0 tx=0).stop 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- 172.21.15.16:0/3744789768 shutdown_connections 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/3744789768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a0106f50 0x7fb4a010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/3744789768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 0x7fb4a0106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/3744789768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a01002f0 0x7fb4a01006d0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- 172.21.15.16:0/3744789768 >> 172.21.15.16:0/3744789768 conn(0x7fb4a0077210 msgr2=0x7fb4a00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- 172.21.15.16:0/3744789768 shutdown_connections 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- 172.21.15.16:0/3744789768 wait complete. 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 Processor -- start 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.633+0000 7fb4a65f2640 1 -- start start 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 0x7fb4a0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 0x7fb4a0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0111fa0 con 0x7fb4a0106f50 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0111e20 con 0x7fb4a0100ca0 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112120 con 0x7fb4a01002f0 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44624/0 (socket says 172.21.15.16:44624) 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 -- 172.21.15.16:0/2533778258 learned_addr learned my addr 172.21.15.16:0/2533778258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb497fff640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 0x7fb4a0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb49ffff640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 0x7fb4a0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 -- 172.21.15.16:0/2533778258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 msgr2=0x7fb4a0198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 0x7fb4a0198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 -- 172.21.15.16:0/2533778258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 msgr2=0x7fb4a0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 0x7fb4a0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 -- 172.21.15.16:0/2533778258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a019d510 con 0x7fb4a0106f50 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb49ffff640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 0x7fb4a0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a4b8f640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb49800eaa0 tx=0x7fb49800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb49dffb640 1 -- 172.21.15.16:0/2533778258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800cdb0 con 0x7fb4a0106f50 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb49dffb640 1 -- 172.21.15.16:0/2533778258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb498005020 con 0x7fb4a0106f50 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a019d800 con 0x7fb4a0106f50 2024-09-12T01:11:13.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.634+0000 7fb49dffb640 1 -- 172.21.15.16:0/2533778258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980116b0 con 0x7fb4a0106f50 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.635+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a50e0 con 0x7fb4a0106f50 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb49dffb640 1 -- 172.21.15.16:0/2533778258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb498005560 con 0x7fb4a0106f50 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 Processor -- start 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 -- start start 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c85f1e2f0 con 0x564c85fc7780 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c85ec5cf0 con 0x564c85fb9b40 2024-09-12T01:11:13.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c85f1a2b0 con 0x564c85fb85d0 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.637+0000 7fb49dffb640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 0x7fb48c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44640/0 (socket says 172.21.15.16:44640) 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 -- 172.21.15.16:0/3045655078 learned_addr learned my addr 172.21.15.16:0/3045655078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb4a4b8f640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 0x7fb48c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb49dffb640 1 -- 172.21.15.16:0/2533778258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fb4980cf630 con 0x7fb4a0106f50 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 -- 172.21.15.16:0/3045655078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 msgr2=0x564c85fb8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fb8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 -- 172.21.15.16:0/3045655078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 msgr2=0x564c85fb8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fb8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 -- 172.21.15.16:0/3045655078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c85eaa3a0 con 0x564c85fc7780 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb49ffff640 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb4a4b8f640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 0x7fb48c0a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb498098520 tx=0x7fb498097ee0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb497fff640 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fb488012770 tx=0x7fb488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64f33d5da6e981e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.638+0000 7fb4967fc640 1 -- 172.21.15.16:0/3045655078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488012cf0 con 0x564c85fc7780 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4967fc640 1 -- 172.21.15.16:0/3045655078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb488010770 con 0x564c85fc7780 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4967fc640 1 -- 172.21.15.16:0/3045655078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488003850 con 0x564c85fc7780 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3045655078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 msgr2=0x564c85fc9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fb488012770 tx=0x7fb488004730 comp rx=0 tx=0).stop 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3045655078 shutdown_connections 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc9b70 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3045655078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fb8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3045655078 >> 172.21.15.16:0/3045655078 conn(0x564c85f81310 msgr2=0x564c85fb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3045655078 shutdown_connections 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.639+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3045655078 wait complete. 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 -- start start 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 Processor -- start 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fc7780 0x564c85fc4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c85fca480 con 0x564c85fb85d0 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564c85ed9060 con 0x564c85fc7780 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c85fca600 con 0x564c85fb9b40 2024-09-12T01:11:13.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb4a89cb740 10 client.0 _ll_register_callbacks cb 0x564c85c2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44648/0 (socket says 172.21.15.16:44648) 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb497fff640 1 -- 172.21.15.16:0/944603310 learned_addr learned my addr 172.21.15.16:0/944603310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.640+0000 7fb49ffff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 -- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 msgr2=0x564c85fc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fc4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 -- 172.21.15.16:0/944603310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fc7780 msgr2=0x564c85fc4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fc7780 0x564c85fc4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 -- 172.21.15.16:0/944603310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c85c8f3c0 con 0x564c85fb85d0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb497fff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb488010ec0 tx=0x7fb488005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7cb149c0c78f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488005220 con 0x564c85fb85d0 2024-09-12T01:11:13.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fb4880053c0 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb488010770 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4880055c0 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb4a89cb740 10 client.20013 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.641+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c85fd5f50 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb488010770 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4880309e0 con 0x564c85fb85d0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 10 client.20013 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 10 client.20013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 20 client.20013 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c85fd72a0 0x564c85fd9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 4 client.20013 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c85f2b860 con 0x564c85fd72a0 2024-09-12T01:11:13.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb49ffff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c85fd72a0 0x564c85fd9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb49ffff640 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c85fd72a0 0x564c85fd9690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2584be3ec5b9a359 server_cookie=49a990780ff27cd8 in_seq=0 out_seq=0 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.642+0000 7fb47f7fe640 10 client.20013 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb47f7fe640 1 -- 172.21.15.16:0/944603310 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c85f2b860 con 0x564c85fd72a0 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb47f7fe640 10 client.20013 handle_command_reply: tid=0 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 client.20013 shutdown 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 msgr2=0x564c85fc7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb488010ec0 tx=0x7fb488005850 comp rx=0 tx=0).stop 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 shutdown_connections 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c85fd72a0 0x564c85fd9690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fc7780 0x564c85fc4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb47f7fe640 0 client.20013 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:13.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/944603310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fb85d0 0x564c85fc7190 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.643+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 >> 172.21.15.16:0/944603310 conn(0x564c85f81310 msgr2=0x564c85fba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.644+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 shutdown_connections 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.644+0000 7fb4a89cb740 1 -- 172.21.15.16:0/944603310 wait complete. 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.644+0000 7fb4a89cb740 20 client.20013 trim_cache size 0 max 16384 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.657+0000 7fb4a89cb740 1 Processor -- start 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.657+0000 7fb4a89cb740 1 -- start start 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.657+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb85d0 0x564c85fb4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.657+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ab780 con 0x564c85fc7780 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ad2c0 con 0x564c85fb85d0 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01abe50 con 0x564c85fb9b40 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb497fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb85d0 0x564c85fb4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:33998/0 (socket says 172.21.15.16:33998) 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 -- 172.21.15.16:0/2335144320 learned_addr learned my addr 172.21.15.16:0/2335144320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 -- 172.21.15.16:0/2335144320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb85d0 msgr2=0x564c85fb4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb85d0 0x564c85fb4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 -- 172.21.15.16:0/2335144320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 msgr2=0x564c85fc3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 -- 172.21.15.16:0/2335144320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c85fb9300 con 0x564c85fb9b40 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.658+0000 7fb49ffff640 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb490009930 tx=0x7fb490032cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496ac09159672ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb495ffb640 1 -- 172.21.15.16:0/2335144320 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490035040 con 0x564c85fb9b40 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb495ffb640 1 -- 172.21.15.16:0/2335144320 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb490049070 con 0x564c85fb9b40 2024-09-12T01:11:13.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb495ffb640 1 -- 172.21.15.16:0/2335144320 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490004920 con 0x564c85fb9b40 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 -- 172.21.15.16:0/2335144320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 msgr2=0x564c85fd3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb490009930 tx=0x7fb490032cf0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 -- 172.21.15.16:0/2335144320 shutdown_connections 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fc3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb9b40 0x564c85fd3720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/2335144320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb85d0 0x564c85fb4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 -- 172.21.15.16:0/2335144320 >> 172.21.15.16:0/2335144320 conn(0x564c85f81310 msgr2=0x564c85fc2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 -- 172.21.15.16:0/2335144320 shutdown_connections 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.659+0000 7fb4a89cb740 1 -- 172.21.15.16:0/2335144320 wait complete. 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 -- start start 2024-09-12T01:11:13.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 Processor -- start 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fbc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fbc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01acbf0 con 0x564c85fc7780 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01ac520 con 0x564c85fb9b40 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01a9780 con 0x564c85fb85d0 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb4a89cb740 10 client.0 _ll_register_callbacks cb 0x7fb48800be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44670/0 (socket says 172.21.15.16:44670) 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb497fff640 1 -- 172.21.15.16:0/3762111843 learned_addr learned my addr 172.21.15.16:0/3762111843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb49ffff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fbc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.660+0000 7fb497fff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fbc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 -- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 msgr2=0x564c85fbc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fbc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 -- 172.21.15.16:0/3762111843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 msgr2=0x564c85fbc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fbc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 -- 172.21.15.16:0/3762111843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c85fceb40 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb497fff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb48801f540 tx=0x7fb488010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291d13aec37aa969 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb47f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488005270 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fb488030560 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb490049070 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb488045050 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb4a89cb740 10 client.20019 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.661+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c85fcfe40 con 0x564c85fc7780 2024-09-12T01:11:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb490049070 con 0x564c85fc7780 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb48803c070 con 0x564c85fc7780 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 10 client.20019 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 10 client.20019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 20 client.20019 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c860464b0 0x564c860488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 4 client.20019 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c85f25d70 con 0x564c860464b0 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.662+0000 7fb49ffff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c860464b0 0x564c860488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb47f7fe640 10 client.20019 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb49ffff640 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c860464b0 0x564c860488a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eae8e96e96cac889 server_cookie=dcac4fa47693a5e5 in_seq=0 out_seq=0 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb47f7fe640 1 -- 172.21.15.16:0/3762111843 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c85f25d70 con 0x564c860464b0 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb47f7fe640 10 client.20019 handle_command_reply: tid=0 2024-09-12T01:11:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 client.20019 shutdown 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 msgr2=0x564c85fbcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb48801f540 tx=0x7fb488010f50 comp rx=0 tx=0).stop 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 shutdown_connections 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c85fc7780 0x564c85fbcbc0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x564c860464b0 0x564c860488a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564c85fb9b40 0x564c85fbc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 --2- 172.21.15.16:0/3762111843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c85fb85d0 0x564c85fbc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 >> 172.21.15.16:0/3762111843 conn(0x564c85f81310 msgr2=0x564c85fc3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb47f7fe640 0 client.20019 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 shutdown_connections 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 1 -- 172.21.15.16:0/3762111843 wait complete. 2024-09-12T01:11:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.663+0000 7fb4a89cb740 20 client.20019 trim_cache size 0 max 16384 2024-09-12T01:11:13.679 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 msgr2=0x7fb48c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 0x7fb48c0a51b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb498098520 tx=0x7fb498097ee0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 msgr2=0x7fb4a019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb49800eaa0 tx=0x7fb49800ef60 comp rx=0 tx=0).stop 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 shutdown_connections 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb48c0a2cf0 0x7fb48c0a51b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4a0106f50 0x7fb4a019ce30 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4a0100ca0 0x7fb4a0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 --2- 172.21.15.16:0/2533778258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4a01002f0 0x7fb4a0198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 >> 172.21.15.16:0/2533778258 conn(0x7fb4a0077210 msgr2=0x7fb4a00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 shutdown_connections 2024-09-12T01:11:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:13.664+0000 7fb4a65f2640 1 -- 172.21.15.16:0/2533778258 wait complete. 2024-09-12T01:11:14.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:11:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:15.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:14 smithi099 ceph-mon[35654]: pgmap v474: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-09-12T01:11:15.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:14 smithi016 ceph-mon[30972]: pgmap v474: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-09-12T01:11:15.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:14 smithi159 ceph-mon[35566]: pgmap v474: 97 pgs: 97 active+clean; 25 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 8 op/s 2024-09-12T01:11:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 -- 172.21.15.16:0/2525677992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc106f10 msgr2=0x7f2bcc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/2525677992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc106f10 0x7f2bcc10ea90 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009ca0 tx=0x7f2bc402a8c0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 -- 172.21.15.16:0/2525677992 shutdown_connections 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/2525677992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc106f10 0x7f2bcc10ea90 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/2525677992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 0x7f2bcc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/2525677992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc101de0 0x7f2bcc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 -- 172.21.15.16:0/2525677992 >> 172.21.15.16:0/2525677992 conn(0x7f2bcc0fd8c0 msgr2=0x7f2bcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 -- 172.21.15.16:0/2525677992 shutdown_connections 2024-09-12T01:11:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.192+0000 7f2bd0f77640 1 -- 172.21.15.16:0/2525677992 wait complete. 2024-09-12T01:11:15.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 Processor -- start 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 -- start start 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 0x7f2bcc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 0x7f2bcc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc111f70 con 0x7f2bcc101de0 2024-09-12T01:11:15.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc111df0 con 0x7f2bcc106f10 2024-09-12T01:11:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bd0f77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1120f0 con 0x7f2bcc102790 2024-09-12T01:11:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44702/0 (socket says 172.21.15.16:44702) 2024-09-12T01:11:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bca59c640 1 -- 172.21.15.16:0/703494265 learned_addr learned my addr 172.21.15.16:0/703494265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:15.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 0x7f2bcc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.193+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 0x7f2bcc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 -- 172.21.15.16:0/703494265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 msgr2=0x7f2bcc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 0x7f2bcc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 -- 172.21.15.16:0/703494265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 msgr2=0x7f2bcc19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 0x7f2bcc19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 -- 172.21.15.16:0/703494265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc19d2e0 con 0x7f2bcc101de0 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 0x7f2bcc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bca59c640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c920 tx=0x7f2bbc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bbb7fe640 1 -- 172.21.15.16:0/703494265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc007c70 con 0x7f2bcc101de0 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc19d5d0 con 0x7f2bcc101de0 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bbb7fe640 1 -- 172.21.15.16:0/703494265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc007e10 con 0x7f2bcc101de0 2024-09-12T01:11:15.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bbb7fe640 1 -- 172.21.15.16:0/703494265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc018890 con 0x7f2bcc101de0 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.194+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc1a4eb0 con 0x7f2bcc101de0 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.196+0000 7f2bbb7fe640 1 -- 172.21.15.16:0/703494265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2bbc018b30 con 0x7f2bcc101de0 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 Processor -- start 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 -- start start 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bbb7fe640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 0x7f2bb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 0x558ff64b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64be760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff64172f0 con 0x558ff64b15d0 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff63becf0 con 0x558ff64bc370 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bd3350740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ff64132b0 con 0x558ff64b2b40 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44714/0 (socket says 172.21.15.16:44714) 2024-09-12T01:11:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.197+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3069528328 learned_addr learned my addr 172.21.15.16:0/3069528328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bbb7fe640 1 -- 172.21.15.16:0/703494265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f2bbc0cf5a0 con 0x7f2bcc101de0 2024-09-12T01:11:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 0x7f2bb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 0x558ff64b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3069528328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 msgr2=0x558ff64b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 0x558ff64b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3069528328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 msgr2=0x558ff64be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3069528328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff63a33a0 con 0x558ff64b15d0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 0x558ff64b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 0x7f2bb40a51b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4006010 tx=0x7f2bc4031d60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b690 tx=0x7f2bc000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489a42896ca4e951 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bb8ff9640 1 -- 172.21.15.16:0/3069528328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0003bd0 con 0x558ff64b15d0 2024-09-12T01:11:15.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bb8ff9640 1 -- 172.21.15.16:0/3069528328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc0015070 con 0x558ff64b15d0 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.198+0000 7f2bb8ff9640 1 -- 172.21.15.16:0/3069528328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0004d30 con 0x558ff64b15d0 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- 172.21.15.16:0/3069528328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 msgr2=0x558ff64b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b690 tx=0x7f2bc000bb50 comp rx=0 tx=0).stop 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- 172.21.15.16:0/3069528328 shutdown_connections 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64b1a10 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3069528328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b2b40 0x558ff64b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- 172.21.15.16:0/3069528328 >> 172.21.15.16:0/3069528328 conn(0x558ff647a310 msgr2=0x558ff64adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- 172.21.15.16:0/3069528328 shutdown_connections 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- 172.21.15.16:0/3069528328 wait complete. 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.199+0000 7f2bd3350740 1 -- start start 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 Processor -- start 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b15d0 0x558ff64bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff64bf070 con 0x558ff64b2b40 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff63d2060 con 0x558ff64bc370 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ff64bf1f0 con 0x558ff64b15d0 2024-09-12T01:11:15.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bd3350740 10 client.0 _ll_register_callbacks cb 0x558ff61258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44728/0 (socket says 172.21.15.16:44728) 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/2268095048 learned_addr learned my addr 172.21.15.16:0/2268095048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bca59c640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b15d0 0x558ff64bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b15d0 msgr2=0x558ff64bbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b15d0 0x558ff64bbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/2268095048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 msgr2=0x558ff64b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.200+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/2268095048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff61883c0 con 0x558ff64b2b40 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000bfd0 tx=0x7f2bc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765969da77052a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:15.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0028060 con 0x558ff64b2b40 2024-09-12T01:11:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f2bc0011640 con 0x558ff64b2b40 2024-09-12T01:11:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc0015070 con 0x558ff64b2b40 2024-09-12T01:11:15.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0003e60 con 0x558ff64b2b40 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 10 client.20031 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ff64cab40 con 0x558ff64b2b40 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc0015070 con 0x558ff64b2b40 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc0022070 con 0x558ff64b2b40 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 10 client.20031 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:15.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 10 client.20031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 20 client.20031 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff64cbf40 0x558ff64ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 4 client.20031 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.201+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff6424860 con 0x558ff64cbf40 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.202+0000 7f2bca59c640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff64cbf40 0x558ff64ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.202+0000 7f2bca59c640 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff64cbf40 0x558ff64ce330 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff3d57462b7a9c9 server_cookie=df13e1dd285717d7 in_seq=0 out_seq=0 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.202+0000 7f2bb1ffb640 10 client.20031 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:15.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/2268095048 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ff6424860 con 0x558ff64cbf40 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bb1ffb640 10 client.20031 handle_command_reply: tid=0 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 client.20031 shutdown 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 msgr2=0x558ff64b9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000bfd0 tx=0x7f2bc00046c0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 shutdown_connections 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff64cbf40 0x558ff64ce330 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64bc370 0x558ff64b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64b9030 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bb1ffb640 0 client.20031 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 --2- 172.21.15.16:0/2268095048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64b15d0 0x558ff64bbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 >> 172.21.15.16:0/2268095048 conn(0x558ff647a310 msgr2=0x558ff64b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 shutdown_connections 2024-09-12T01:11:15.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 1 -- 172.21.15.16:0/2268095048 wait complete. 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.203+0000 7f2bd3350740 20 client.20031 trim_cache size 0 max 16384 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 Processor -- start 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 -- start start 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b2b40 0x558ff64b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1ad780 con 0x558ff64b15d0 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1af2c0 con 0x558ff64b2b40 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bd3350740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1ade50 con 0x558ff64bc370 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44730/0 (socket says 172.21.15.16:44730) 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.216+0000 7f2bca59c640 1 -- 172.21.15.16:0/432376108 learned_addr learned my addr 172.21.15.16:0/432376108 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:15.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b2b40 0x558ff64b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 -- 172.21.15.16:0/432376108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 msgr2=0x558ff64b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 -- 172.21.15.16:0/432376108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b2b40 msgr2=0x558ff64b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b2b40 0x558ff64b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 -- 172.21.15.16:0/432376108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff64ae960 con 0x558ff64b15d0 2024-09-12T01:11:15.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bca59c640 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ccd50 tx=0x7f2bbc097d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28e4c554e354a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bb3fff640 1 -- 172.21.15.16:0/432376108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc099b60 con 0x558ff64b15d0 2024-09-12T01:11:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bb3fff640 1 -- 172.21.15.16:0/432376108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc099dc0 con 0x558ff64b15d0 2024-09-12T01:11:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.217+0000 7f2bb3fff640 1 -- 172.21.15.16:0/432376108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc09bd20 con 0x558ff64b15d0 2024-09-12T01:11:15.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- 172.21.15.16:0/432376108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 msgr2=0x558ff64c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ccd50 tx=0x7f2bbc097d60 comp rx=0 tx=0).stop 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- 172.21.15.16:0/432376108 shutdown_connections 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b2b40 0x558ff64b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- 172.21.15.16:0/432376108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b15d0 0x558ff64c8310 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- 172.21.15.16:0/432376108 >> 172.21.15.16:0/432376108 conn(0x558ff647a310 msgr2=0x558ff64b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- 172.21.15.16:0/432376108 shutdown_connections 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- 172.21.15.16:0/432376108 wait complete. 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- start start 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 Processor -- start 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 0x558ff64c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1aebf0 con 0x558ff64b2b40 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1ae520 con 0x558ff64b15d0 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.218+0000 7f2bd3350740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1ab780 con 0x558ff64bc370 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bd3350740 10 client.0 _ll_register_callbacks cb 0x7f2bbc00a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44738/0 (socket says 172.21.15.16:44738) 2024-09-12T01:11:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3900474493 learned_addr learned my addr 172.21.15.16:0/3900474493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 0x558ff64c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bcad9d640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 msgr2=0x558ff64b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3900474493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 msgr2=0x558ff64c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 0x558ff64c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 -- 172.21.15.16:0/3900474493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff64ad4f0 con 0x558ff64b2b40 2024-09-12T01:11:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 0x558ff64c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bc9d9b640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b690 tx=0x7f2bc0019910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bf126777aa1abb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0028040 con 0x558ff64b2b40 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f2bc0011650 con 0x558ff64b2b40 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.219+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc099dc0 con 0x558ff64b2b40 2024-09-12T01:11:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bc003c050 con 0x558ff64b2b40 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 10 client.20037 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ff65362b0 con 0x558ff64b2b40 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc099dc0 con 0x558ff64b2b40 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2bc0022070 con 0x558ff64b2b40 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 10 client.20037 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 10 client.20037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 20 client.20037 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff6536910 0x558ff6538d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 4 client.20037 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff641ed70 con 0x558ff6536910 2024-09-12T01:11:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.220+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff6536910 0x558ff6538d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bca59c640 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff6536910 0x558ff6538d30 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8deafa4bc4035970 server_cookie=85172c0f5440bd65 in_seq=0 out_seq=0 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bb1ffb640 10 client.20037 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bb1ffb640 1 -- 172.21.15.16:0/3900474493 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff641ed70 con 0x558ff6536910 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bb1ffb640 10 client.20037 handle_command_reply: tid=0 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 client.20037 shutdown 2024-09-12T01:11:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 msgr2=0x558ff64c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000b690 tx=0x7f2bc0019910 comp rx=0 tx=0).stop 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 shutdown_connections 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558ff6536910 0x558ff6538d30 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff64bc370 0x558ff64b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff64b2b40 0x558ff64c4c30 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bb1ffb640 0 client.20037 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 --2- 172.21.15.16:0/3900474493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff64b15d0 0x558ff64c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 >> 172.21.15.16:0/3900474493 conn(0x558ff647a310 msgr2=0x558ff64b7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 shutdown_connections 2024-09-12T01:11:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 1 -- 172.21.15.16:0/3900474493 wait complete. 2024-09-12T01:11:15.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.221+0000 7f2bd3350740 20 client.20037 trim_cache size 0 max 16384 2024-09-12T01:11:15.252 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 msgr2=0x7f2bb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 0x7f2bb40a51b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4006010 tx=0x7f2bc4031d60 comp rx=0 tx=0).stop 2024-09-12T01:11:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 msgr2=0x7f2bcc198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c920 tx=0x7f2bbc00cde0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 shutdown_connections 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2bb40a2cf0 0x7f2bb40a51b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bcc106f10 0x7f2bcc19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bcc102790 0x7f2bcc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 --2- 172.21.15.16:0/703494265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc101de0 0x7f2bcc198330 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 >> 172.21.15.16:0/703494265 conn(0x7f2bcc0fd8c0 msgr2=0x7f2bcc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 shutdown_connections 2024-09-12T01:11:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:15.222+0000 7f2bd0f77640 1 -- 172.21.15.16:0/703494265 wait complete. 2024-09-12T01:11:16.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:16 smithi016 ceph-mon[30972]: pgmap v475: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:17.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:16 smithi099 ceph-mon[35654]: pgmap v475: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:17.050 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:11:17.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:11:17.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:16 smithi159 ceph-mon[35566]: pgmap v475: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 -- 172.21.15.16:0/1530232898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 msgr2=0x7f431810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 --2- 172.21.15.16:0/1530232898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f431810da40 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f4310009900 tx=0x7f431002a480 comp rx=0 tx=0).stop 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 -- 172.21.15.16:0/1530232898 shutdown_connections 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 --2- 172.21.15.16:0/1530232898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f431810da40 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 --2- 172.21.15.16:0/1530232898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 0x7f43181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 --2- 172.21.15.16:0/1530232898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 0x7f4318101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 -- 172.21.15.16:0/1530232898 >> 172.21.15.16:0/1530232898 conn(0x7f43180fc7d0 msgr2=0x7f43180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 -- 172.21.15.16:0/1530232898 shutdown_connections 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.812+0000 7f431cff7640 1 -- 172.21.15.16:0/1530232898 wait complete. 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 Processor -- start 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 -- start start 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 0x7f43180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 0x7f43180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4318110fc0 con 0x7f4318102ef0 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4318110e40 con 0x7f4318102570 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431cff7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4318111140 con 0x7f4318100e20 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44754/0 (socket says 172.21.15.16:44754) 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 -- 172.21.15.16:0/844170427 learned_addr learned my addr 172.21.15.16:0/844170427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f431659c640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 0x7f43180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4315d9b640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 0x7f43180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 -- 172.21.15.16:0/844170427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 msgr2=0x7f43180fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 0x7f43180fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 -- 172.21.15.16:0/844170427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 msgr2=0x7f43180fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 0x7f43180fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.813+0000 7f4316d9d640 1 -- 172.21.15.16:0/844170427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181a69e0 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f4316d9d640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f4310004770 tx=0x7f4310003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f43077fe640 1 -- 172.21.15.16:0/844170427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431003f070 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181a6c70 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f43077fe640 1 -- 172.21.15.16:0/844170427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4310031c30 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f43077fe640 1 -- 172.21.15.16:0/844170427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310035030 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.814+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a7150 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.816+0000 7f43077fe640 1 -- 172.21.15.16:0/844170427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4310042070 con 0x7f4318102ef0 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.816+0000 7f431f3d0740 1 Processor -- start 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 -- start start 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d94d40 0x562170d97130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562170cef280 con 0x562170d8a480 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562170c96b10 con 0x562170d94d40 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431f3d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562170ceb240 con 0x562170d89b60 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f43077fe640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 0x7f43000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44768/0 (socket says 172.21.15.16:44768) 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 -- 172.21.15.16:0/2307526323 learned_addr learned my addr 172.21.15.16:0/2307526323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d94d40 0x562170d97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f4316d9d640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 0x7f43000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f43077fe640 1 -- 172.21.15.16:0/844170427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f43100eff40 con 0x7f4318102ef0 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 -- 172.21.15.16:0/2307526323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 msgr2=0x562170d89f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d89f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 -- 172.21.15.16:0/2307526323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d94d40 msgr2=0x562170d97130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d94d40 0x562170d97130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f431659c640 1 -- 172.21.15.16:0/2307526323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562170ca31f0 con 0x562170d8a480 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.817+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d89f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f431659c640 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f43080103f0 tx=0x7f43080108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4899bc3da95caf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f4316d9d640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 0x7f43000a51b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f43100bc100 tx=0x7f43100dc040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f4304ff9640 1 -- 172.21.15.16:0/2307526323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308016070 con 0x562170d8a480 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f4304ff9640 1 -- 172.21.15.16:0/2307526323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308004d20 con 0x562170d8a480 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f4304ff9640 1 -- 172.21.15.16:0/2307526323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308005450 con 0x562170d8a480 2024-09-12T01:11:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f431f3d0740 1 -- 172.21.15.16:0/2307526323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 msgr2=0x562170d94800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.818+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f43080103f0 tx=0x7f43080108b0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- 172.21.15.16:0/2307526323 shutdown_connections 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d94d40 0x562170d97130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d94800 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2307526323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d89f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- 172.21.15.16:0/2307526323 >> 172.21.15.16:0/2307526323 conn(0x562170d860f0 msgr2=0x562170d864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- 172.21.15.16:0/2307526323 shutdown_connections 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- 172.21.15.16:0/2307526323 wait complete. 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- start start 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 Processor -- start 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562170d97910 con 0x562170d94d40 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562170ca9f10 con 0x562170d8a480 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562170d97a90 con 0x562170d89b60 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f431f3d0740 10 client.0 _ll_register_callbacks cb 0x562170c72a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.819+0000 7f4315d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44780/0 (socket says 172.21.15.16:44780) 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 -- 172.21.15.16:0/3215039313 learned_addr learned my addr 172.21.15.16:0/3215039313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f431659c640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 -- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 msgr2=0x562170d91bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d91bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 -- 172.21.15.16:0/3215039313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 msgr2=0x562170d920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 -- 172.21.15.16:0/3215039313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562170a606a0 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f430c01aa60 tx=0x7f430c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668d9bd29bad73c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f42fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c01a110 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f430c013bf0 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308004d20 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f430c029050 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f431f3d0740 10 client.20049 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.820+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562170da3530 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308004d20 con 0x562170d94d40 2024-09-12T01:11:17.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f430c017440 con 0x562170d94d40 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 10 client.20049 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 10 client.20049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 20 client.20049 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170da49f0 0x562170da6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 4 client.20049 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562170cfc7f0 con 0x562170da49f0 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.821+0000 7f431659c640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170da49f0 0x562170da6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.822+0000 7f431659c640 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170da49f0 0x562170da6e20 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53971525f97feb53 server_cookie=b80aa59125d01a88 in_seq=0 out_seq=0 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.822+0000 7f42fdffb640 10 client.20049 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.822+0000 7f42fdffb640 1 -- 172.21.15.16:0/3215039313 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562170cfc7f0 con 0x562170da49f0 2024-09-12T01:11:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f42fdffb640 10 client.20049 handle_command_reply: tid=0 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 client.20049 shutdown 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 msgr2=0x562170d92630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f430c01aa60 tx=0x7f430c0046c0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 shutdown_connections 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170da49f0 0x562170da6e20 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d92630 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3215039313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d91bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f42fdffb640 0 client.20049 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 >> 172.21.15.16:0/3215039313 conn(0x562170d860f0 msgr2=0x562170d85270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 shutdown_connections 2024-09-12T01:11:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 1 -- 172.21.15.16:0/3215039313 wait complete. 2024-09-12T01:11:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.823+0000 7f431f3d0740 20 client.20049 trim_cache size 0 max 16384 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 Processor -- start 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 -- start start 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d89b60 0x562170d8c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d94d40 0x562170dac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43181afb20 con 0x562170d8a480 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b1660 con 0x562170d89b60 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431f3d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b01f0 con 0x562170d94d40 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.840+0000 7f431659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44796/0 (socket says 172.21.15.16:44796) 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 -- 172.21.15.16:0/3222258204 learned_addr learned my addr 172.21.15.16:0/3222258204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:17.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d89b60 0x562170d8c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f4315d9b640 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d94d40 0x562170dac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 -- 172.21.15.16:0/3222258204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d94d40 msgr2=0x562170dac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d94d40 0x562170dac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 -- 172.21.15.16:0/3222258204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d89b60 msgr2=0x562170d8c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d89b60 0x562170d8c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 -- 172.21.15.16:0/3222258204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562170d941b0 con 0x562170d8a480 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f431659c640 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4308009cc0 tx=0x7f4308009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274a1dd4c8b10886 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f42fffff640 1 -- 172.21.15.16:0/3222258204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308016070 con 0x562170d8a480 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f42fffff640 1 -- 172.21.15.16:0/3222258204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308003b70 con 0x562170d8a480 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.841+0000 7f42fffff640 1 -- 172.21.15.16:0/3222258204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43080042c0 con 0x562170d8a480 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- 172.21.15.16:0/3222258204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 msgr2=0x562170d91490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4308009cc0 tx=0x7f4308009680 comp rx=0 tx=0).stop 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- 172.21.15.16:0/3222258204 shutdown_connections 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d94d40 0x562170dac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d8a480 0x562170d91490 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 --2- 172.21.15.16:0/3222258204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d89b60 0x562170d8c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- 172.21.15.16:0/3222258204 >> 172.21.15.16:0/3222258204 conn(0x562170d860f0 msgr2=0x562170d85270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- 172.21.15.16:0/3222258204 shutdown_connections 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- 172.21.15.16:0/3222258204 wait complete. 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 -- start start 2024-09-12T01:11:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.842+0000 7f431f3d0740 1 Processor -- start 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d98ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b0f90 con 0x562170d94d40 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b08c0 con 0x562170d8a480 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43181adb20 con 0x562170d89b60 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 10 client.0 _ll_register_callbacks cb 0x7f430c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44806/0 (socket says 172.21.15.16:44806) 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 -- 172.21.15.16:0/2495790761 learned_addr learned my addr 172.21.15.16:0/2495790761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431659c640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d98ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 -- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 msgr2=0x562170d98990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d98990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 -- 172.21.15.16:0/2495790761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 msgr2=0x562170d98ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d98ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 -- 172.21.15.16:0/2495790761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562170d8a2b0 con 0x562170d94d40 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f4315d9b640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f430c017a00 tx=0x7f430c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec79863c8b11062 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f42fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c036060 con 0x562170d94d40 2024-09-12T01:11:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f430c013bf0 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.843+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4308003b70 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f430c009620 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 10 client.20055 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562170e0f730 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4308003b70 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f430c018a20 con 0x562170d94d40 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 10 client.20055 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 10 client.20055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 20 client.20055 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170e0fdf0 0x562170e12210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 4 client.20055 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562170cf6d00 con 0x562170e0fdf0 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.844+0000 7f431659c640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170e0fdf0 0x562170e12210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.845+0000 7f431659c640 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170e0fdf0 0x562170e12210 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ea6eeca28ad2cf server_cookie=213561a8ea681991 in_seq=0 out_seq=0 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.845+0000 7f42fdffb640 10 client.20055 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f42fdffb640 1 -- 172.21.15.16:0/2495790761 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562170cf6d00 con 0x562170e0fdf0 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f42fdffb640 10 client.20055 handle_command_reply: tid=0 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 client.20055 shutdown 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 msgr2=0x562170d99410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f430c017a00 tx=0x7f430c0047d0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 shutdown_connections 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x562170e0fdf0 0x562170e12210 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562170d94d40 0x562170d99410 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562170d8a480 0x562170d98ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 --2- 172.21.15.16:0/2495790761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562170d89b60 0x562170d98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 >> 172.21.15.16:0/2495790761 conn(0x562170d860f0 msgr2=0x562170d84a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f42fdffb640 0 client.20055 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 shutdown_connections 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 1 -- 172.21.15.16:0/2495790761 wait complete. 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.846+0000 7f431f3d0740 20 client.20055 trim_cache size 0 max 16384 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8eadc9b0-d493-41f1-8151-bcaad963ab5f", 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:11:17.851 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 msgr2=0x7f43000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 0x7f43000a51b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f43100bc100 tx=0x7f43100dc040 comp rx=0 tx=0).stop 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 msgr2=0x7f43180fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f4310004770 tx=0x7f4310003c10 comp rx=0 tx=0).stop 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 shutdown_connections 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f43000a2cf0 0x7f43000a51b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4318102ef0 0x7f43180fd700 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4318102570 0x7f43180fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 --2- 172.21.15.16:0/844170427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4318100e20 0x7f43180fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:17.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 >> 172.21.15.16:0/844170427 conn(0x7f43180fc7d0 msgr2=0x7f431806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:17.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 shutdown_connections 2024-09-12T01:11:17.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:17.847+0000 7f431cff7640 1 -- 172.21.15.16:0/844170427 wait complete. 2024-09-12T01:11:18.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:11:18.885 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:18 smithi016 ceph-mon[30972]: pgmap v476: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:18 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:11:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:18 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:18 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:11:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:18 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:18 smithi099 ceph-mon[35654]: pgmap v476: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:18 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:11:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:18 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:18 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:11:19.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:18 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:18 smithi159 ceph-mon[35566]: pgmap v476: 97 pgs: 97 active+clean; 25 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 15 op/s 2024-09-12T01:11:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:18 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:11:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:18 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:18 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:11:19.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:18 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:11:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 -- 172.21.15.16:0/1217308532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458106f10 msgr2=0x7fe45810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 --2- 172.21.15.16:0/1217308532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458106f10 0x7fe45810ea90 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe450009ca0 tx=0x7fe45002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 -- 172.21.15.16:0/1217308532 shutdown_connections 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 --2- 172.21.15.16:0/1217308532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458106f10 0x7fe45810ea90 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 --2- 172.21.15.16:0/1217308532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458102790 0x7fe4581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 --2- 172.21.15.16:0/1217308532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458101de0 0x7fe4581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 -- 172.21.15.16:0/1217308532 >> 172.21.15.16:0/1217308532 conn(0x7fe4580fd8c0 msgr2=0x7fe4580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 -- 172.21.15.16:0/1217308532 shutdown_connections 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.384+0000 7fe45deb8640 1 -- 172.21.15.16:0/1217308532 wait complete. 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.385+0000 7fe45deb8640 1 Processor -- start 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.385+0000 7fe45deb8640 1 -- start start 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 0x7fe458198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458106f10 0x7fe45819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe458111f70 con 0x7fe458101de0 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe458111df0 con 0x7fe458102790 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe45deb8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581120f0 con 0x7fe458106f10 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44772/0 (socket says 172.21.15.16:44772) 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 -- 172.21.15.16:0/3767966886 learned_addr learned my addr 172.21.15.16:0/3767966886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe457fff640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458106f10 0x7fe45819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe44effd640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 0x7fe458198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 -- 172.21.15.16:0/3767966886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458106f10 msgr2=0x7fe45819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458106f10 0x7fe45819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 -- 172.21.15.16:0/3767966886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 msgr2=0x7fe458198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 0x7fe458198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 -- 172.21.15.16:0/3767966886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45819d2e0 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe44effd640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 0x7fe458198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4577fe640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c920 tx=0x7fe44800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.386+0000 7fe4557fa640 1 -- 172.21.15.16:0/3767966886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448007c70 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.387+0000 7fe4557fa640 1 -- 172.21.15.16:0/3767966886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448007e10 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.387+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45819d5d0 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.387+0000 7fe4557fa640 1 -- 172.21.15.16:0/3767966886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448018890 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.387+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581a4eb0 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe4557fa640 1 -- 172.21.15.16:0/3767966886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe448018a70 con 0x7fe458101de0 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 Processor -- start 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 -- start start 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f264a0 0x55eed2f26880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 0x55eed2f33a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eed2e8c2e0 con 0x55eed2f26dc0 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eed2e33ce0 con 0x55eed2f31680 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe460291740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eed2e882a0 con 0x55eed2f264a0 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.389+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44786/0 (socket says 172.21.15.16:44786) 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 -- 172.21.15.16:0/602368887 learned_addr learned my addr 172.21.15.16:0/602368887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe457fff640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f264a0 0x55eed2f26880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe44effd640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 0x55eed2f33a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4557fa640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 0x7fe4440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 -- 172.21.15.16:0/602368887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f264a0 msgr2=0x55eed2f26880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f264a0 0x55eed2f26880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 -- 172.21.15.16:0/602368887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 msgr2=0x55eed2f33a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 0x55eed2f33a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 -- 172.21.15.16:0/602368887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed2e18440 con 0x55eed2f26dc0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe457fff640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 0x7fe4440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe44effd640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 0x55eed2f33a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4557fa640 1 -- 172.21.15.16:0/3767966886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe4480cf5a0 con 0x7fe458101de0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe4577fe640 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ccc90 tx=0x7fe448090f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3a9517813a80fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe44dffb640 1 -- 172.21.15.16:0/602368887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480666b0 con 0x55eed2f26dc0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.390+0000 7fe44dffb640 1 -- 172.21.15.16:0/602368887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448066b50 con 0x55eed2f26dc0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe44dffb640 1 -- 172.21.15.16:0/602368887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44809bac0 con 0x55eed2f26dc0 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 -- 172.21.15.16:0/602368887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 msgr2=0x55eed2f31140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ccc90 tx=0x7fe448090f50 comp rx=0 tx=0).stop 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe457fff640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 0x7fe4440a51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe450005e30 tx=0x7fe4500095c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 -- 172.21.15.16:0/602368887 shutdown_connections 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f31680 0x55eed2f33a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f26dc0 0x55eed2f31140 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 --2- 172.21.15.16:0/602368887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f264a0 0x55eed2f26880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 -- 172.21.15.16:0/602368887 >> 172.21.15.16:0/602368887 conn(0x55eed2f22a30 msgr2=0x55eed2f22e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 -- 172.21.15.16:0/602368887 shutdown_connections 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.391+0000 7fe460291740 1 -- 172.21.15.16:0/602368887 wait complete. 2024-09-12T01:11:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 -- start start 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 Processor -- start 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f30f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eed2f34250 con 0x55eed2f31680 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55eed2e47050 con 0x55eed2f264a0 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eed2f343d0 con 0x55eed2f26dc0 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe460291740 10 client.0 _ll_register_callbacks cb 0x55eed2e87160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44802/0 (socket says 172.21.15.16:44802) 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe457fff640 1 -- 172.21.15.16:0/400061687 learned_addr learned my addr 172.21.15.16:0/400061687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe44effd640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f2e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.392+0000 7fe4577fe640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f30f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 -- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 msgr2=0x55eed2f2e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f2e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 -- 172.21.15.16:0/400061687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 msgr2=0x55eed2f30f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f30f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 -- 172.21.15.16:0/400061687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed2bfd3c0 con 0x55eed2f31680 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe4577fe640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f30f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe457fff640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe450035000 tx=0x7fe45002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb29ac5a4d62b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4500047c0 con 0x55eed2f31680 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe45002f610 con 0x55eed2f31680 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe448066b50 con 0x55eed2f31680 2024-09-12T01:11:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe45002f810 con 0x55eed2f31680 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe460291740 10 client.20067 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eed2f3fb70 con 0x55eed2f31680 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448066b50 con 0x55eed2f31680 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe450051020 con 0x55eed2f31680 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe460291740 10 client.20067 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.393+0000 7fe460291740 10 client.20067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe460291740 20 client.20067 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f40f80 0x55eed2f43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe460291740 4 client.20067 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed2e99850 con 0x55eed2f40f80 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe4577fe640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f40f80 0x55eed2f43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe4577fe640 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f40f80 0x55eed2f43400 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409541148b739645 server_cookie=bd1fdb10f851d519 in_seq=0 out_seq=0 2024-09-12T01:11:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.394+0000 7fe436ffd640 10 client.20067 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe436ffd640 1 -- 172.21.15.16:0/400061687 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eed2e99850 con 0x55eed2f40f80 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe436ffd640 10 client.20067 handle_command_reply: tid=0 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 client.20067 shutdown 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 msgr2=0x55eed2f2e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe450035000 tx=0x7fe45002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 shutdown_connections 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f40f80 0x55eed2f43400 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f31680 0x55eed2f2e710 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f2e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe436ffd640 0 client.20067 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 --2- 172.21.15.16:0/400061687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f30f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 >> 172.21.15.16:0/400061687 conn(0x55eed2f22a30 msgr2=0x55eed2f28760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 shutdown_connections 2024-09-12T01:11:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 1 -- 172.21.15.16:0/400061687 wait complete. 2024-09-12T01:11:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.395+0000 7fe460291740 20 client.20067 trim_cache size 0 max 16384 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.412+0000 7fe460291740 1 Processor -- start 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.412+0000 7fe460291740 1 -- start start 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f48d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581ad800 con 0x55eed2f22a30 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581af340 con 0x55eed2f264a0 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe460291740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581aded0 con 0x55eed2f26dc0 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44818/0 (socket says 172.21.15.16:44818) 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 -- 172.21.15.16:0/525434921 learned_addr learned my addr 172.21.15.16:0/525434921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe44effd640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f48d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 -- 172.21.15.16:0/525434921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 msgr2=0x55eed2f49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe457fff640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f49250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 -- 172.21.15.16:0/525434921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 msgr2=0x55eed2f48d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f48d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 -- 172.21.15.16:0/525434921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed2f2c720 con 0x55eed2f22a30 2024-09-12T01:11:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe44effd640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f48d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe4577fe640 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe44809b7c0 tx=0x7fe44800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4758b69d0518702a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe44d7fa640 1 -- 172.21.15.16:0/525434921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a4730 con 0x55eed2f22a30 2024-09-12T01:11:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe44d7fa640 1 -- 172.21.15.16:0/525434921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44809bac0 con 0x55eed2f22a30 2024-09-12T01:11:19.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.413+0000 7fe44d7fa640 1 -- 172.21.15.16:0/525434921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480ccf70 con 0x55eed2f22a30 2024-09-12T01:11:19.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 -- 172.21.15.16:0/525434921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 msgr2=0x55eed2f30140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe44809b7c0 tx=0x7fe44800cf10 comp rx=0 tx=0).stop 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 -- 172.21.15.16:0/525434921 shutdown_connections 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f26dc0 0x55eed2f49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f264a0 0x55eed2f48d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 --2- 172.21.15.16:0/525434921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f22a30 0x55eed2f30140 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 -- 172.21.15.16:0/525434921 >> 172.21.15.16:0/525434921 conn(0x55eed2f2e870 msgr2=0x55eed2f2fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 -- 172.21.15.16:0/525434921 shutdown_connections 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.414+0000 7fe460291740 1 -- 172.21.15.16:0/525434921 wait complete. 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 -- start start 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 Processor -- start 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 0x55eed2f39ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 0x55eed2f379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581aec70 con 0x55eed2f264a0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581ae5a0 con 0x55eed2f26dc0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581ab800 con 0x55eed2f22a30 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe460291740 10 client.0 _ll_register_callbacks cb 0x7fe450031e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe44effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe44effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44828/0 (socket says 172.21.15.16:44828) 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe44effd640 1 -- 172.21.15.16:0/761870500 learned_addr learned my addr 172.21.15.16:0/761870500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.415+0000 7fe4577fe640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 0x55eed2f39ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe457fff640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 0x55eed2f379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 -- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 msgr2=0x55eed2f39ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 0x55eed2f39ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 -- 172.21.15.16:0/761870500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 msgr2=0x55eed2f379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:19.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 0x55eed2f379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 -- 172.21.15.16:0/761870500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed2faacf0 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe457fff640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 0x55eed2f379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe4577fe640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 0x55eed2f39ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe44effd640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe440005fd0 tx=0x7fe440013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56580248fe9d6344 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440021060 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe440011650 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe44809bac0 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe440011890 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe460291740 10 client.20073 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eed2fabee0 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.416+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44809bac0 con 0x55eed2f264a0 2024-09-12T01:11:19.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe440010c90 con 0x55eed2f264a0 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 10 client.20073 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 10 client.20073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 20 client.20073 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f31680 0x55eed2fae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 4 client.20073 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eed2e93d60 con 0x55eed2f31680 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe4577fe640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f31680 0x55eed2fae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe4577fe640 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f31680 0x55eed2fae6d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6778ae66aa4e27c server_cookie=421d1809ffb75b4e in_seq=0 out_seq=0 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe436ffd640 10 client.20073 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe436ffd640 1 -- 172.21.15.16:0/761870500 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eed2e93d60 con 0x55eed2f31680 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe436ffd640 10 client.20073 handle_command_reply: tid=0 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.417+0000 7fe460291740 1 client.20073 shutdown 2024-09-12T01:11:19.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 msgr2=0x55eed2f37210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fe440005fd0 tx=0x7fe440013830 comp rx=0 tx=0).stop 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 shutdown_connections 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55eed2f31680 0x55eed2fae6d0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55eed2f26dc0 0x55eed2f379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe436ffd640 0 client.20073 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eed2f264a0 0x55eed2f37210 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 --2- 172.21.15.16:0/761870500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eed2f22a30 0x55eed2f39ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 >> 172.21.15.16:0/761870500 conn(0x55eed2f2e870 msgr2=0x55eed2f30250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 shutdown_connections 2024-09-12T01:11:19.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 1 -- 172.21.15.16:0/761870500 wait complete. 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe460291740 20 client.20073 trim_cache size 0 max 16384 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "8eadc9b0-d493-41f1-8151-bcaad963ab5f": { 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "8eadc9b0-d493-41f1-8151-bcaad963ab5f", 2024-09-12T01:11:19.424 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 msgr2=0x7fe4440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 0x7fe4440a51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe450005e30 tx=0x7fe4500095c0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 msgr2=0x7fe458198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:19.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.418+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c920 tx=0x7fe44800cde0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 shutdown_connections 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe4440a2cf0 0x7fe4440a51b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe458106f10 0x7fe45819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe458102790 0x7fe458198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 --2- 172.21.15.16:0/3767966886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe458101de0 0x7fe458198330 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 >> 172.21.15.16:0/3767966886 conn(0x7fe4580fd8c0 msgr2=0x7fe4580febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 shutdown_connections 2024-09-12T01:11:19.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:19.419+0000 7fe45deb8640 1 -- 172.21.15.16:0/3767966886 wait complete. 2024-09-12T01:11:20.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:19 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:11:20.096 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:19 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:11:20.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:19 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:11:20.126 INFO:tasks.cephfs.filesystem:scrub status for tag:8eadc9b0-d493-41f1-8151-bcaad963ab5f - {'path': '/', 'tag': '8eadc9b0-d493-41f1-8151-bcaad963ab5f', 'options': 'recursive,force'} 2024-09-12T01:11:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:20 smithi099 ceph-mon[35654]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:20 smithi159 ceph-mon[35566]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:21.139 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:20 smithi016 ceph-mon[30972]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:23.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:22 smithi099 ceph-mon[35654]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:23.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:22 smithi159 ceph-mon[35566]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:22 smithi016 ceph-mon[30972]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 79 KiB/s wr, 16 op/s 2024-09-12T01:11:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:24 smithi099 ceph-mon[35654]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-09-12T01:11:25.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:24 smithi159 ceph-mon[35566]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-09-12T01:11:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:24 smithi016 ceph-mon[30972]: pgmap v479: 97 pgs: 97 active+clean; 26 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-09-12T01:11:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:25 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:11:26.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:25 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:11:26.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:25 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:11:26.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:25 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:11:26.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:25 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:11:26.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:25 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:11:26.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:25 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:11:26.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:25 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:11:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:25 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:11:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:25 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:11:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:25 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:11:26.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:25 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:11:27.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:27 smithi099 ceph-mon[35654]: pgmap v480: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 20 op/s 2024-09-12T01:11:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:27 smithi159 ceph-mon[35566]: pgmap v480: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 20 op/s 2024-09-12T01:11:27.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:27 smithi016 ceph-mon[30972]: pgmap v480: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 102 KiB/s wr, 20 op/s 2024-09-12T01:11:29.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:29 smithi099 ceph-mon[35654]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-09-12T01:11:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:29 smithi159 ceph-mon[35566]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-09-12T01:11:29.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:29 smithi016 ceph-mon[30972]: pgmap v481: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-09-12T01:11:31.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:31 smithi099 ceph-mon[35654]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-09-12T01:11:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:31 smithi159 ceph-mon[35566]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-09-12T01:11:31.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:31 smithi016 ceph-mon[30972]: pgmap v482: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 99 KiB/s wr, 20 op/s 2024-09-12T01:11:33.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:33 smithi099 ceph-mon[35654]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:33 smithi159 ceph-mon[35566]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:33 smithi016 ceph-mon[30972]: pgmap v483: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:35.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:35 smithi099 ceph-mon[35654]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:35.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:35 smithi159 ceph-mon[35566]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:35.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:35 smithi016 ceph-mon[30972]: pgmap v484: 97 pgs: 97 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:11:37.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:37 smithi099 ceph-mon[35654]: pgmap v485: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:37.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:37 smithi159 ceph-mon[35566]: pgmap v485: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:37.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:37 smithi016 ceph-mon[30972]: pgmap v485: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:499/600: open fd = 3 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:500/600: open fd = 3 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:501/600: open fd = 3 2024-09-12T01:11:38.089 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:502/600: open fd = 3 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:503/600: open fd = 3 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:504/600: open fd = 3 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.090 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:505/600: open fd = 3 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:506/600: open fd = 3 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:507/600: open fd = 3 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.091 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.092 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.092 INFO:tasks.workunit.client.0.smithi016.stdout:508/600: open fd = 3 2024-09-12T01:11:38.092 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.092 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.092 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:509/600: open fd = 3 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:510/600: open fd = 3 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.093 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:511/600: open fd = 3 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:512/600: open fd = 3 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.094 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:513/600: open fd = 3 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:514/600: open fd = 3 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.095 INFO:tasks.workunit.client.0.smithi016.stdout:515/600: open fd = 3 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:516/600: open fd = 3 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.096 INFO:tasks.workunit.client.0.smithi016.stdout:517/600: open fd = 3 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:518/600: open fd = 3 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:519/600: open fd = 3 2024-09-12T01:11:38.097 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:520/600: open fd = 3 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:521/600: open fd = 3 2024-09-12T01:11:38.098 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:522/600: open fd = 3 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:523/600: open fd = 3 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.099 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:524/600: open fd = 3 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:525/600: open fd = 3 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.100 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:526/600: open fd = 3 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:527/600: open fd = 3 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.101 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:528/600: open fd = 3 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:529/600: open fd = 3 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.102 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:530/600: open fd = 3 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:531/600: open fd = 3 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.103 INFO:tasks.workunit.client.0.smithi016.stdout:532/600: open fd = 3 2024-09-12T01:11:38.104 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.104 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.104 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.104 INFO:tasks.workunit.client.0.smithi016.stdout:533/600: open fd = 3 2024-09-12T01:11:38.104 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:534/600: open fd = 3 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:535/600: open fd = 3 2024-09-12T01:11:38.105 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:536/600: open fd = 3 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:537/600: open fd = 3 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:538/600: open fd = 3 2024-09-12T01:11:38.106 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:539/600: open fd = 3 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:540/600: open fd = 3 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:541/600: open fd = 3 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.107 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:542/600: open fd = 3 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:543/600: open fd = 3 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:544/600: open fd = 3 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:545/600: open fd = 3 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.108 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:546/600: open fd = 3 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:547/600: open fd = 3 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:548/600: open fd = 3 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.109 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:549/600: open fd = 3 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:550/600: open fd = 3 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:551/600: open fd = 3 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:552/600: open fd = 3 2024-09-12T01:11:38.110 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:553/600: open fd = 3 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:554/600: open fd = 3 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:555/600: open fd = 3 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:556/600: open fd = 3 2024-09-12T01:11:38.111 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:557/600: open fd = 3 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:558/600: open fd = 3 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:559/600: open fd = 3 2024-09-12T01:11:38.112 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:560/600: open fd = 3 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:11:38.113 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:11:39.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:39 smithi099 ceph-mon[35654]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:39.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:39 smithi159 ceph-mon[35566]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:39.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:39 smithi016 ceph-mon[30972]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:41.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:41 smithi099 ceph-mon[35654]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:41.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:41 smithi159 ceph-mon[35566]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:41.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:41 smithi016 ceph-mon[30972]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:43.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:43 smithi099 ceph-mon[35654]: pgmap v488: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:43.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:43 smithi159 ceph-mon[35566]: pgmap v488: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:43.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:43 smithi016 ceph-mon[30972]: pgmap v488: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:45.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:45 smithi099 ceph-mon[35654]: pgmap v489: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:45.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:45 smithi159 ceph-mon[35566]: pgmap v489: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:45.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:45 smithi016 ceph-mon[30972]: pgmap v489: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:47.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:47 smithi099 ceph-mon[35654]: pgmap v490: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 100 KiB/s wr, 19 op/s 2024-09-12T01:11:47.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:47 smithi159 ceph-mon[35566]: pgmap v490: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 100 KiB/s wr, 19 op/s 2024-09-12T01:11:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:47 smithi016 ceph-mon[30972]: pgmap v490: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 100 KiB/s wr, 19 op/s 2024-09-12T01:11:49.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:49 smithi099 ceph-mon[35654]: pgmap v491: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-09-12T01:11:49.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:49 smithi159 ceph-mon[35566]: pgmap v491: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-09-12T01:11:49.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:49 smithi016 ceph-mon[30972]: pgmap v491: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 12 op/s 2024-09-12T01:11:50.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:11:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.057+0000 7f45360f6640 1 -- 172.21.15.16:0/3746351870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102570 msgr2=0x7f45301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.057+0000 7f45360f6640 1 --2- 172.21.15.16:0/3746351870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102570 0x7f45301029b0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f452402a480 comp rx=0 tx=0).stop 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 -- 172.21.15.16:0/3746351870 shutdown_connections 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 --2- 172.21.15.16:0/3746351870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530102ef0 0x7f453010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 --2- 172.21.15.16:0/3746351870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102570 0x7f45301029b0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 --2- 172.21.15.16:0/3746351870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530100e20 0x7f4530101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 -- 172.21.15.16:0/3746351870 >> 172.21.15.16:0/3746351870 conn(0x7f45300fc7d0 msgr2=0x7f45300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 -- 172.21.15.16:0/3746351870 shutdown_connections 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 -- 172.21.15.16:0/3746351870 wait complete. 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.058+0000 7f45360f6640 1 Processor -- start 2024-09-12T01:11:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 -- start start 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530100e20 0x7f45301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530102570 0x7f45301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110f80 con 0x7f4530102ef0 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4530110e00 con 0x7f4530102570 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f45360f6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4530111100 con 0x7f4530100e20 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f452ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f452ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46082/0 (socket says 172.21.15.16:46082) 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f452ffff640 1 -- 172.21.15.16:0/1498149204 learned_addr learned my addr 172.21.15.16:0/1498149204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f452f7fe640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530100e20 0x7f45301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.059+0000 7f452effd640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530102570 0x7f45301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 -- 172.21.15.16:0/1498149204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530100e20 msgr2=0x7f45301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530100e20 0x7f45301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 -- 172.21.15.16:0/1498149204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530102570 msgr2=0x7f45301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530102570 0x7f45301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 -- 172.21.15.16:0/1498149204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453019d460 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452ffff640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f452800cce0 tx=0x7f452800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452cff9640 1 -- 172.21.15.16:0/1498149204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452800f450 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452cff9640 1 -- 172.21.15.16:0/1498149204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f452800fad0 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453019d750 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f452cff9640 1 -- 172.21.15.16:0/1498149204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528018c10 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.060+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301a5030 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.063+0000 7f452cff9640 1 -- 172.21.15.16:0/1498149204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f452800f5f0 con 0x7f4530102ef0 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.063+0000 7f45384cf740 1 Processor -- start 2024-09-12T01:11:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.063+0000 7f45384cf740 1 -- start start 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x56135394a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x56135395be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5613538b02e0 con 0x56135394aca0 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561353857ce0 con 0x56135394a380 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f45384cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5613538ac2a0 con 0x561353959a90 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46094/0 (socket says 172.21.15.16:46094) 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 -- 172.21.15.16:0/1554512553 learned_addr learned my addr 172.21.15.16:0/1554512553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452effd640 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x56135394a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452cff9640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 0x7f45180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 -- 172.21.15.16:0/1554512553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 msgr2=0x56135395be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x56135395be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 -- 172.21.15.16:0/1554512553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 msgr2=0x56135394a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x56135394a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 -- 172.21.15.16:0/1554512553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135383c440 con 0x56135394aca0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452ffff640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 0x7f45180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452cff9640 1 -- 172.21.15.16:0/1498149204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f45280cf9d0 con 0x7f4530102ef0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.064+0000 7f452f7fe640 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f452000bd10 tx=0x7f45200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0dec98101b3db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f4515ffb640 1 -- 172.21.15.16:0/1554512553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520004030 con 0x56135394aca0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f4515ffb640 1 -- 172.21.15.16:0/1554512553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45200041d0 con 0x56135394aca0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f4515ffb640 1 -- 172.21.15.16:0/1554512553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520011820 con 0x56135394aca0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f45384cf740 1 -- 172.21.15.16:0/1554512553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 msgr2=0x561353959550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f45384cf740 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f452000bd10 tx=0x7f45200107a0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.065+0000 7f452ffff640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 0x7f45180a51b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f45280cd1d0 tx=0x7f45280185b0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 -- 172.21.15.16:0/1554512553 shutdown_connections 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x56135395be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353959550 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 --2- 172.21.15.16:0/1554512553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x56135394a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 -- 172.21.15.16:0/1554512553 >> 172.21.15.16:0/1554512553 conn(0x561353946a30 msgr2=0x561353946e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 -- 172.21.15.16:0/1554512553 shutdown_connections 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 -- 172.21.15.16:0/1554512553 wait complete. 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 -- start start 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.066+0000 7f45384cf740 1 Processor -- start 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x5613539594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x561353956f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56135395c660 con 0x56135394aca0 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56135386b050 con 0x56135394a380 2024-09-12T01:11:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56135395c7e0 con 0x561353959a90 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f45384cf740 10 client.0 _ll_register_callbacks cb 0x5613538ab160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46102/0 (socket says 172.21.15.16:46102) 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 -- 172.21.15.16:0/330991930 learned_addr learned my addr 172.21.15.16:0/330991930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452effd640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x5613539594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452effd640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x561353956f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 -- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 msgr2=0x561353956f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x561353956f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 -- 172.21.15.16:0/330991930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 msgr2=0x5613539594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x5613539594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 -- 172.21.15.16:0/330991930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613536213c0 con 0x56135394aca0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f452f7fe640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f4520004000 tx=0x7f4520004030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb26327a44e26622 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.067+0000 7f450effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520029060 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f45200035f0 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45200041d0 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f452000b8e0 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 10 client.20085 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561353968150 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45200041d0 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4520011cb0 con 0x56135394aca0 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 10 client.20085 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 10 client.20085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 20 client.20085 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539695b0 0x56135396b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 4 client.20085 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.068+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613538bd850 con 0x5613539695b0 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.069+0000 7f452effd640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539695b0 0x56135396b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.069+0000 7f450effd640 10 client.20085 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.069+0000 7f452effd640 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539695b0 0x56135396b9a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14a26e311591fd8 server_cookie=c7becca05fe1ed9 in_seq=0 out_seq=0 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f450effd640 1 -- 172.21.15.16:0/330991930 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613538bd850 con 0x5613539695b0 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f450effd640 10 client.20085 handle_command_reply: tid=0 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 client.20085 shutdown 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 msgr2=0x561353956780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f4520004000 tx=0x7f4520004030 comp rx=0 tx=0).stop 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 shutdown_connections 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539695b0 0x56135396b9a0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f450effd640 0 client.20085 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561353959a90 0x561353956f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394aca0 0x561353956780 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 --2- 172.21.15.16:0/330991930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56135394a380 0x5613539594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 >> 172.21.15.16:0/330991930 conn(0x561353946a30 msgr2=0x56135394c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 shutdown_connections 2024-09-12T01:11:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 1 -- 172.21.15.16:0/330991930 wait complete. 2024-09-12T01:11:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.070+0000 7f45384cf740 20 client.20085 trim_cache size 0 max 16384 2024-09-12T01:11:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 Processor -- start 2024-09-12T01:11:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 -- start start 2024-09-12T01:11:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x561353951d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ab570 con 0x56135394a380 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ad0b0 con 0x561353959a90 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f45384cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45301abc40 con 0x56135394aca0 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.088+0000 7f452effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46118/0 (socket says 172.21.15.16:46118) 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 -- 172.21.15.16:0/4079733285 learned_addr learned my addr 172.21.15.16:0/4079733285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452f7fe640 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x561353951d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 -- 172.21.15.16:0/4079733285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 msgr2=0x561353951d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x561353951d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 -- 172.21.15.16:0/4079733285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 msgr2=0x56135396e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 -- 172.21.15.16:0/4079733285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561353957280 con 0x56135394a380 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f452effd640 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f45240329e0 tx=0x7f4524009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd7abb031346bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f45157fa640 1 -- 172.21.15.16:0/4079733285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524035040 con 0x56135394a380 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f45157fa640 1 -- 172.21.15.16:0/4079733285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4524041030 con 0x56135394a380 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.089+0000 7f45157fa640 1 -- 172.21.15.16:0/4079733285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524004da0 con 0x56135394a380 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 -- 172.21.15.16:0/4079733285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 msgr2=0x561353956040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f45240329e0 tx=0x7f4524009260 comp rx=0 tx=0).stop 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 -- 172.21.15.16:0/4079733285 shutdown_connections 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x561353951d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 --2- 172.21.15.16:0/4079733285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x561353956040 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 -- 172.21.15.16:0/4079733285 >> 172.21.15.16:0/4079733285 conn(0x561353946a30 msgr2=0x561353946e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 -- 172.21.15.16:0/4079733285 shutdown_connections 2024-09-12T01:11:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.090+0000 7f45384cf740 1 -- 172.21.15.16:0/4079733285 wait complete. 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.091+0000 7f45384cf740 1 -- start start 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.091+0000 7f45384cf740 1 Processor -- start 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x56135396f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ac9e0 con 0x56135394a380 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45301ac310 con 0x561353959a90 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45301a9590 con 0x56135394aca0 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46126/0 (socket says 172.21.15.16:46126) 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 -- 172.21.15.16:0/2370066551 learned_addr learned my addr 172.21.15.16:0/2370066551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452f7fe640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x56135396f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f45384cf740 10 client.0 _ll_register_callbacks cb 0x7f452000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 -- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 msgr2=0x56135396f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x56135396f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 -- 172.21.15.16:0/2370066551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 msgr2=0x56135396bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452effd640 1 -- 172.21.15.16:0/2370066551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135396c790 con 0x56135394a380 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.092+0000 7f452f7fe640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x56135396f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f452effd640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f452402a9a0 tx=0x7f45240049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669f29889c448c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f450effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452405d060 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4524041030 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f4524004250 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4524035070 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f45384cf740 10 client.20091 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613539d3a00 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.093+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4524041030 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4524003b10 con 0x56135394a380 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 10 client.20091 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 10 client.20091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 20 client.20091 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539d4140 0x5613539d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 4 client.20091 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613538b7d60 con 0x5613539d4140 2024-09-12T01:11:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.094+0000 7f452f7fe640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539d4140 0x5613539d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f452f7fe640 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539d4140 0x5613539d6560 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d127b0e3c55f11 server_cookie=d65421400158c8d6 in_seq=0 out_seq=0 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f450effd640 10 client.20091 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f450effd640 1 -- 172.21.15.16:0/2370066551 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5613538b7d60 con 0x5613539d4140 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f450effd640 10 client.20091 handle_command_reply: tid=0 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 client.20091 shutdown 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 msgr2=0x56135396eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f452402a9a0 tx=0x7f45240049c0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 shutdown_connections 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5613539d4140 0x5613539d6560 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561353959a90 0x56135396bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56135394aca0 0x56135396f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 --2- 172.21.15.16:0/2370066551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56135394a380 0x56135396eb30 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 >> 172.21.15.16:0/2370066551 conn(0x561353946a30 msgr2=0x561353965290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f450effd640 0 client.20091 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 shutdown_connections 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.095+0000 7f45384cf740 1 -- 172.21.15.16:0/2370066551 wait complete. 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45384cf740 20 client.20091 trim_cache size 0 max 16384 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:11:51.104 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:11:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 msgr2=0x7f45180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 0x7f45180a51b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f45280cd1d0 tx=0x7f45280185b0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 msgr2=0x7f453019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f452800cce0 tx=0x7f452800a620 comp rx=0 tx=0).stop 2024-09-12T01:11:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 shutdown_connections 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f45180a2cf0 0x7f45180a51b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4530102ef0 0x7f453019cd80 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4530102570 0x7f45301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 --2- 172.21.15.16:0/1498149204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4530100e20 0x7f45301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 >> 172.21.15.16:0/1498149204 conn(0x7f45300fc7d0 msgr2=0x7f4530069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 shutdown_connections 2024-09-12T01:11:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:51.096+0000 7f45360f6640 1 -- 172.21.15.16:0/1498149204 wait complete. 2024-09-12T01:11:51.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:51 smithi016 ceph-mon[30972]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:51.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:51 smithi099 ceph-mon[35654]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:51.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:51 smithi159 ceph-mon[35566]: pgmap v492: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 94 KiB/s wr, 19 op/s 2024-09-12T01:11:51.984 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:11:51.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:11:52.244 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 -- 172.21.15.16:0/2275204563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102ef0 msgr2=0x7f816010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 --2- 172.21.15.16:0/2275204563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102ef0 0x7f816010da40 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f8158009ca0 tx=0x7f815802a950 comp rx=0 tx=0).stop 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 -- 172.21.15.16:0/2275204563 shutdown_connections 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 --2- 172.21.15.16:0/2275204563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102ef0 0x7f816010da40 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 --2- 172.21.15.16:0/2275204563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160102570 0x7f81601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 --2- 172.21.15.16:0/2275204563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160100e20 0x7f8160101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 -- 172.21.15.16:0/2275204563 >> 172.21.15.16:0/2275204563 conn(0x7f81600fc7d0 msgr2=0x7f81600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 -- 172.21.15.16:0/2275204563 shutdown_connections 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 -- 172.21.15.16:0/2275204563 wait complete. 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.743+0000 7f8165942640 1 Processor -- start 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 -- start start 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 0x7f8160199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160102ef0 0x7f81601a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8160110fa0 con 0x7f8160102570 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8160110e20 con 0x7f8160100e20 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f8165942640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8160111120 con 0x7f8160102ef0 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46150/0 (socket says 172.21.15.16:46150) 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 -- 172.21.15.16:0/755039883 learned_addr learned my addr 172.21.15.16:0/755039883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815f7fe640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160102ef0 0x7f81601a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 -- 172.21.15.16:0/755039883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160102ef0 msgr2=0x7f81601a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160102ef0 0x7f81601a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815effd640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 0x7f8160199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.744+0000 7f815e7fc640 1 -- 172.21.15.16:0/755039883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 msgr2=0x7f8160199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f815e7fc640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 0x7f8160199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f815e7fc640 1 -- 172.21.15.16:0/755039883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81601a48d0 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f815effd640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 0x7f8160199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f815e7fc640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f815400d950 tx=0x7f815400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154017070 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8154009c30 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81601a4aa0 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815400f6b0 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.745+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81601a5030 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.746+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8160101200 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.747+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f81540109d0 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.748+0000 7f8164940640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.748+0000 7f815effd640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.748+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f81540cf090 con 0x7f8160102570 2024-09-12T01:11:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.748+0000 7f815effd640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 0x7f81480a5230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8150006fd0 tx=0x7f8150008040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.752+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8154009de0 con 0x7f8160102570 2024-09-12T01:11:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.964+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81601a5200 con 0x7f8160102570 2024-09-12T01:11:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.965+0000 7f8164940640 1 -- 172.21.15.16:0/755039883 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f815409bcb0 con 0x7f8160102570 2024-09-12T01:11:52.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:11:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:11:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 msgr2=0x7f81480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 0x7f81480a5230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8150006fd0 tx=0x7f8150008040 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 msgr2=0x7f8160199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f815400d950 tx=0x7f815400de10 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 shutdown_connections 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8160102ef0 0x7f81601a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8160102570 0x7f8160199850 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 --2- 172.21.15.16:0/755039883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8160100e20 0x7f8160199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 >> 172.21.15.16:0/755039883 conn(0x7f81600fc7d0 msgr2=0x7f8160106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 shutdown_connections 2024-09-12T01:11:52.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:52.967+0000 7f8165942640 1 -- 172.21.15.16:0/755039883 wait complete. 2024-09-12T01:11:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:53 smithi099 ceph-mon[35654]: pgmap v493: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:53.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:53 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/755039883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:53 smithi159 ceph-mon[35566]: pgmap v493: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:53.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:53 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/755039883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:53 smithi016 ceph-mon[30972]: pgmap v493: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:53.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:53 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/755039883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:11:53.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:11:54.172 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.668+0000 7f69d379f640 1 -- 172.21.15.16:0/3940287991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 msgr2=0x7f69cc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.668+0000 7f69d379f640 1 --2- 172.21.15.16:0/3940287991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc10e610 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009ca0 tx=0x7f69c402a950 comp rx=0 tx=0).stop 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 -- 172.21.15.16:0/3940287991 shutdown_connections 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 --2- 172.21.15.16:0/3940287991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc10e610 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 --2- 172.21.15.16:0/3940287991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 0x7f69cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 --2- 172.21.15.16:0/3940287991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 0x7f69cc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 -- 172.21.15.16:0/3940287991 >> 172.21.15.16:0/3940287991 conn(0x7f69cc0fc7a0 msgr2=0x7f69cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 -- 172.21.15.16:0/3940287991 shutdown_connections 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 -- 172.21.15.16:0/3940287991 wait complete. 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 Processor -- start 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 -- start start 2024-09-12T01:11:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 0x7f69cc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.669+0000 7f69d379f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 0x7f69cc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d379f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d379f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc111f50 con 0x7f69cc10c170 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d379f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc111dd0 con 0x7f69cc102d40 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d379f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1120d0 con 0x7f69cc103660 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46172/0 (socket says 172.21.15.16:46172) 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 -- 172.21.15.16:0/4061084926 learned_addr learned my addr 172.21.15.16:0/4061084926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d153b640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 0x7f69cc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 0x7f69cc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 -- 172.21.15.16:0/4061084926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 msgr2=0x7f69cc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 0x7f69cc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 -- 172.21.15.16:0/4061084926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 msgr2=0x7f69cc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 0x7f69cc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 -- 172.21.15.16:0/4061084926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc19d320 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d1d3c640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f69c402ae70 tx=0x7f69c402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69c27fc640 1 -- 172.21.15.16:0/4061084926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4034040 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69c27fc640 1 -- 172.21.15.16:0/4061084926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4030920 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc19d5b0 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.670+0000 7f69c27fc640 1 -- 172.21.15.16:0/4061084926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c40047b0 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.671+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc1a4ef0 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69c27fc640 1 -- 172.21.15.16:0/4061084926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f69c4037020 con 0x7f69cc10c170 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 Processor -- start 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 -- start start 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e463d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e463da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3e45a32f0 con 0x55f3e464c780 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69c27fc640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 0x7f69b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.673+0000 7f69d5b78740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f3e454acf0 con 0x55f3e463d5d0 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d5b78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3e459f2b0 con 0x55f3e463eb40 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46180/0 (socket says 172.21.15.16:46180) 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 -- 172.21.15.16:0/2859274162 learned_addr learned my addr 172.21.15.16:0/2859274162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d1d3c640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 0x7f69b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e463d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69c27fc640 1 -- 172.21.15.16:0/4061084926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f69c40f0100 con 0x7f69cc10c170 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d153b640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e463da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 -- 172.21.15.16:0/2859274162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 msgr2=0x55f3e463d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e463d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 -- 172.21.15.16:0/2859274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 msgr2=0x55f3e463da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e463da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 -- 172.21.15.16:0/2859274162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e452f3a0 con 0x55f3e464c780 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d153b640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e463da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0126f0 tx=0x7f69bc00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32aef8bf2fde5b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69d1d3c640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 0x7f69b80a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69c40eda10 tx=0x7f69c40b1bc0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69b77fe640 1 -- 172.21.15.16:0/2859274162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc012c70 con 0x55f3e464c780 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69b77fe640 1 -- 172.21.15.16:0/2859274162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc012e10 con 0x55f3e464c780 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.674+0000 7f69b77fe640 1 -- 172.21.15.16:0/2859274162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0049e0 con 0x55f3e464c780 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 -- 172.21.15.16:0/2859274162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 msgr2=0x55f3e464eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0126f0 tx=0x7f69bc00b6d0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 -- 172.21.15.16:0/2859274162 shutdown_connections 2024-09-12T01:11:54.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e464eb70 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e463da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 --2- 172.21.15.16:0/2859274162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e463d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 -- 172.21.15.16:0/2859274162 >> 172.21.15.16:0/2859274162 conn(0x55f3e4606310 msgr2=0x55f3e4639c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 -- 172.21.15.16:0/2859274162 shutdown_connections 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.675+0000 7f69d5b78740 1 -- 172.21.15.16:0/2859274162 wait complete. 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 -- start start 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 Processor -- start 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e464c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e464c780 0x55f3e4649a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f3e464f480 con 0x55f3e463eb40 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f3e455e060 con 0x55f3e463d5d0 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3e464f600 con 0x55f3e464c780 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d5b78740 10 client.0 _ll_register_callbacks cb 0x55f3e42b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d153b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d153b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46192/0 (socket says 172.21.15.16:46192) 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d153b640 1 -- 172.21.15.16:0/1931722432 learned_addr learned my addr 172.21.15.16:0/1931722432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e464c780 0x55f3e4649a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.676+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e464c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 -- 172.21.15.16:0/1931722432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e464c780 msgr2=0x55f3e4649a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e464c780 0x55f3e4649a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 -- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 msgr2=0x55f3e464c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e464c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 -- 172.21.15.16:0/1931722432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e43143c0 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e464c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d153b640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f69c80127d0 tx=0x7f69c8012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5b5995bd41bf2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c801f060 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc012e10 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f69c8010dc0 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69c801a430 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d5b78740 10 client.20109 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3e465b000 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc012e10 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.677+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f69c800e610 con 0x55f3e463eb40 2024-09-12T01:11:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 10 client.20109 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 10 client.20109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 20 client.20109 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e465e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 4 client.20109 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e45b0860 con 0x55f3e465c410 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e465e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e465e840 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7cfdcfe582c2514b server_cookie=b012195ea839729c in_seq=0 out_seq=0 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.678+0000 7f69b4ff9640 10 client.20109 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69b4ff9640 1 -- 172.21.15.16:0/1931722432 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3e45b0860 con 0x55f3e465c410 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69b4ff9640 10 client.20109 handle_command_reply: tid=0 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 client.20109 shutdown 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 msgr2=0x55f3e4649440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f69c80127d0 tx=0x7f69c8012c90 comp rx=0 tx=0).stop 2024-09-12T01:11:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 shutdown_connections 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e465e840 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e464c780 0x55f3e4649a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463eb40 0x55f3e4649440 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69b4ff9640 0 client.20109 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 --2- 172.21.15.16:0/1931722432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e464c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 >> 172.21.15.16:0/1931722432 conn(0x55f3e4606310 msgr2=0x55f3e463f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.679+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 shutdown_connections 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.680+0000 7f69d5b78740 1 -- 172.21.15.16:0/1931722432 wait complete. 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.680+0000 7f69d5b78740 20 client.20109 trim_cache size 0 max 16384 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.692+0000 7f69d5b78740 1 Processor -- start 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 -- start start 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4658720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e464c780 0x55f3e4664110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ab530 con 0x55f3e463d5d0 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ad070 con 0x55f3e464c780 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d5b78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1abc00 con 0x55f3e463eb40 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46208/0 (socket says 172.21.15.16:46208) 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 -- 172.21.15.16:0/619878322 learned_addr learned my addr 172.21.15.16:0/619878322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 -- 172.21.15.16:0/619878322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 msgr2=0x55f3e4658720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4658720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 -- 172.21.15.16:0/619878322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e464c780 msgr2=0x55f3e4664110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e464c780 0x55f3e4664110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 -- 172.21.15.16:0/619878322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e463d2d0 con 0x55f3e463d5d0 2024-09-12T01:11:54.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.693+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0055c0 tx=0x7f69bc005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64243440f9360cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69b6ffd640 1 -- 172.21.15.16:0/619878322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc005110 con 0x55f3e463d5d0 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69b6ffd640 1 -- 172.21.15.16:0/619878322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc005700 con 0x55f3e463d5d0 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69b6ffd640 1 -- 172.21.15.16:0/619878322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0038b0 con 0x55f3e463d5d0 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 -- 172.21.15.16:0/619878322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 msgr2=0x55f3e4639e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0055c0 tx=0x7f69bc005550 comp rx=0 tx=0).stop 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 -- 172.21.15.16:0/619878322 shutdown_connections 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e464c780 0x55f3e4664110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4658720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 --2- 172.21.15.16:0/619878322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e463d5d0 0x55f3e4639e10 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 -- 172.21.15.16:0/619878322 >> 172.21.15.16:0/619878322 conn(0x55f3e4606310 msgr2=0x55f3e4639700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 -- 172.21.15.16:0/619878322 shutdown_connections 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.694+0000 7f69d5b78740 1 -- 172.21.15.16:0/619878322 wait complete. 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 -- start start 2024-09-12T01:11:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 Processor -- start 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e4654890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4654dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ac9a0 con 0x55f3e464c780 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1ac2d0 con 0x55f3e463d5d0 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc1a9550 con 0x55f3e463eb40 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d5b78740 10 client.0 _ll_register_callbacks cb 0x7f69c800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d0d3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46224/0 (socket says 172.21.15.16:46224) 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.695+0000 7f69d0d3a640 1 -- 172.21.15.16:0/3517689991 learned_addr learned my addr 172.21.15.16:0/3517689991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e4654890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d153b640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4654dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 -- 172.21.15.16:0/3517689991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 msgr2=0x55f3e4654dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4654dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 -- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 msgr2=0x55f3e4654890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e4654890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 -- 172.21.15.16:0/3517689991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e463a960 con 0x55f3e464c780 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d0d3a640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c010 tx=0x7f69bc003690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682b26fa316ce55c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0038b0 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f69bc003a50 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69bc005700 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f69bc003c50 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d5b78740 10 client.20115 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.696+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3e46c7850 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc005700 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f69bc033020 con 0x55f3e464c780 2024-09-12T01:11:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 10 client.20115 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 10 client.20115 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 20 client.20115 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e46c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 4 client.20115 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3e45aad70 con 0x55f3e465c410 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.697+0000 7f69d153b640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e46c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d153b640 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e46c9c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=11368e026470a788 server_cookie=42d96d2a15b7574f in_seq=0 out_seq=0 2024-09-12T01:11:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69b4ff9640 10 client.20115 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69b4ff9640 1 -- 172.21.15.16:0/3517689991 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3e45aad70 con 0x55f3e465c410 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69b4ff9640 10 client.20115 handle_command_reply: tid=0 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 client.20115 shutdown 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 msgr2=0x55f3e4651860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c010 tx=0x7f69bc003690 comp rx=0 tx=0).stop 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 shutdown_connections 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55f3e465c410 0x55f3e46c9c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f3e464c780 0x55f3e4651860 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3e463eb40 0x55f3e4654dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 --2- 172.21.15.16:0/3517689991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3e463d5d0 0x55f3e4654890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 >> 172.21.15.16:0/3517689991 conn(0x55f3e4606310 msgr2=0x55f3e464aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69b4ff9640 0 client.20115 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 shutdown_connections 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 1 -- 172.21.15.16:0/3517689991 wait complete. 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.698+0000 7f69d5b78740 20 client.20115 trim_cache size 0 max 16384 2024-09-12T01:11:54.716 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:54.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 msgr2=0x7f69b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 0x7f69b80a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f69c40eda10 tx=0x7f69c40b1bc0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 msgr2=0x7f69cc19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:54.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f69c402ae70 tx=0x7f69c402f5b0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 shutdown_connections 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f69b80a2d70 0x7f69b80a5230 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69cc10c170 0x7f69cc19cc40 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69cc103660 0x7f69cc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 --2- 172.21.15.16:0/4061084926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69cc102d40 0x7f69cc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 >> 172.21.15.16:0/4061084926 conn(0x7f69cc0fc7a0 msgr2=0x7f69cc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 shutdown_connections 2024-09-12T01:11:54.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:54.699+0000 7f69d379f640 1 -- 172.21.15.16:0/4061084926 wait complete. 2024-09-12T01:11:55.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:55 smithi099 ceph-mon[35654]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:55.582 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:55 smithi016 ceph-mon[30972]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:55.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:55 smithi159 ceph-mon[35566]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-12T01:11:55.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:11:55.882 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 -- 172.21.15.16:0/2332789681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 msgr2=0x7f65941006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 --2- 172.21.15.16:0/2332789681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941006d0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f6584009930 tx=0x7f658402a420 comp rx=0 tx=0).stop 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 -- 172.21.15.16:0/2332789681 shutdown_connections 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 --2- 172.21.15.16:0/2332789681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594106f50 0x7f659410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 --2- 172.21.15.16:0/2332789681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594100ca0 0x7f6594106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 --2- 172.21.15.16:0/2332789681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941006d0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 -- 172.21.15.16:0/2332789681 >> 172.21.15.16:0/2332789681 conn(0x7f6594077210 msgr2=0x7f65940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 -- 172.21.15.16:0/2332789681 shutdown_connections 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.380+0000 7f659a242640 1 -- 172.21.15.16:0/2332789681 wait complete. 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 Processor -- start 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 -- start start 2024-09-12T01:11:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594100ca0 0x7f6594198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594106f50 0x7f659419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6594112000 con 0x7f65941002f0 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6594111e80 con 0x7f6594100ca0 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f659a242640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6594112180 con 0x7f6594106f50 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.381+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46244/0 (socket says 172.21.15.16:46244) 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 -- 172.21.15.16:0/2654325280 learned_addr learned my addr 172.21.15.16:0/2654325280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f6593fff640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594106f50 0x7f659419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f6592ffd640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594100ca0 0x7f6594198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 -- 172.21.15.16:0/2654325280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594106f50 msgr2=0x7f659419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594106f50 0x7f659419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 -- 172.21.15.16:0/2654325280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594100ca0 msgr2=0x7f6594198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594100ca0 0x7f6594198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 -- 172.21.15.16:0/2654325280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659419d490 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f65937fe640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6584009a50 tx=0x7f6584004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f6590ff9640 1 -- 172.21.15.16:0/2654325280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658403f030 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f6590ff9640 1 -- 172.21.15.16:0/2654325280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6584003ce0 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659419d720 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f6590ff9640 1 -- 172.21.15.16:0/2654325280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584035070 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.382+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a5060 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f6590ff9640 1 -- 172.21.15.16:0/2654325280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6584042030 con 0x7f65941002f0 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 Processor -- start 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 -- start start 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f6590ff9640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 0x7f657c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0e5d0 0x55dbffe0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbffd742f0 con 0x55dbffe0fb40 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbffd1bcf0 con 0x55dbffe19370 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.385+0000 7f659c61b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dbffd702b0 con 0x55dbffe0e5d0 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46256/0 (socket says 172.21.15.16:46256) 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 -- 172.21.15.16:0/1335855319 learned_addr learned my addr 172.21.15.16:0/1335855319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6593fff640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 0x7f657c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6590ff9640 1 -- 172.21.15.16:0/2654325280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f65840efa20 con 0x7f65941002f0 2024-09-12T01:11:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6593fff640 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6592ffd640 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0e5d0 0x55dbffe0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 -- 172.21.15.16:0/1335855319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0e5d0 msgr2=0x55dbffe0ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0e5d0 0x55dbffe0ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 -- 172.21.15.16:0/1335855319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 msgr2=0x55dbffe1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 -- 172.21.15.16:0/1335855319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbffd003a0 con 0x55dbffe0fb40 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f65937fe640 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f65840b83a0 tx=0x7f658402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ccae1ade43e7ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6593fff640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 0x7f657c0a5230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55dbffe0f070 tx=0x7f658c007410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6579ffb640 1 -- 172.21.15.16:0/1335855319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584049070 con 0x55dbffe0fb40 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6579ffb640 1 -- 172.21.15.16:0/1335855319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65840bc640 con 0x55dbffe0fb40 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.386+0000 7f6579ffb640 1 -- 172.21.15.16:0/1335855319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840dd370 con 0x55dbffe0fb40 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- 172.21.15.16:0/1335855319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 msgr2=0x55dbffe0e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f65840b83a0 tx=0x7f658402a420 comp rx=0 tx=0).stop 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- 172.21.15.16:0/1335855319 shutdown_connections 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0e5d0 0x55dbffe0ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 --2- 172.21.15.16:0/1335855319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe0e090 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- 172.21.15.16:0/1335855319 >> 172.21.15.16:0/1335855319 conn(0x55dbffdd7310 msgr2=0x55dbffe0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- 172.21.15.16:0/1335855319 shutdown_connections 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- 172.21.15.16:0/1335855319 wait complete. 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.387+0000 7f659c61b740 1 -- start start 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 Processor -- start 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 0x55dbffe16000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dbffe1be30 con 0x55dbffe0e5d0 2024-09-12T01:11:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbffd2f060 con 0x55dbffe19370 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dbffe1bfb0 con 0x55dbffe0fb40 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f659c61b740 10 client.0 _ll_register_callbacks cb 0x55dbffa828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46270/0 (socket says 172.21.15.16:46270) 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 -- 172.21.15.16:0/2068753831 learned_addr learned my addr 172.21.15.16:0/2068753831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f6592ffd640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 0x55dbffe16000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f6593fff640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 -- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 msgr2=0x55dbffe16000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 0x55dbffe16000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 -- 172.21.15.16:0/2068753831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 msgr2=0x55dbffe165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f65937fe640 1 -- 172.21.15.16:0/2068753831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbffae53c0 con 0x55dbffe0e5d0 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f6592ffd640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 0x55dbffe16000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.388+0000 7f6593fff640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f65937fe640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f65840bbc40 tx=0x7f6584009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d19567c4095c986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840d4070 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f658403b460 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65840bc640 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6584049050 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 10 client.20127 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbffe1be30 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65840bc640 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f65840d9050 con 0x55dbffe0e5d0 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 10 client.20127 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 10 client.20127 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 20 client.20127 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe28a10 0x55dbffe2ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 4 client.20127 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.389+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbffd81860 con 0x55dbffe28a10 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.390+0000 7f6592ffd640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe28a10 0x55dbffe2ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.390+0000 7f6592ffd640 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe28a10 0x55dbffe2ae00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=210018a796a57c11 server_cookie=9455d7e14e4b2f62 in_seq=0 out_seq=0 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.390+0000 7f6572ffd640 10 client.20127 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f6572ffd640 1 -- 172.21.15.16:0/2068753831 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbffd81860 con 0x55dbffe28a10 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f6572ffd640 10 client.20127 handle_command_reply: tid=0 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 client.20127 shutdown 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 msgr2=0x55dbffe18d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f65840bbc40 tx=0x7f6584009560 comp rx=0 tx=0).stop 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 shutdown_connections 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe19370 0x55dbffe165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe0fb40 0x55dbffe16000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe28a10 0x55dbffe2ae00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 --2- 172.21.15.16:0/2068753831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe18d50 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f6572ffd640 0 client.20127 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 >> 172.21.15.16:0/2068753831 conn(0x55dbffdd7310 msgr2=0x55dbffe10990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 shutdown_connections 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 1 -- 172.21.15.16:0/2068753831 wait complete. 2024-09-12T01:11:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.391+0000 7f659c61b740 20 client.20127 trim_cache size 0 max 16384 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.404+0000 7f659c61b740 1 Processor -- start 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.404+0000 7f659c61b740 1 -- start start 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.404+0000 7f659c61b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.404+0000 7f659c61b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 0x55dbffe11660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.404+0000 7f659c61b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe30680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f659c61b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65941ad8c0 con 0x55dbffe0e5d0 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f659c61b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65941af400 con 0x55dbffe0fb40 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f659c61b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65941adf90 con 0x55dbffe19370 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46284/0 (socket says 172.21.15.16:46284) 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 -- 172.21.15.16:0/1033653439 learned_addr learned my addr 172.21.15.16:0/1033653439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 -- 172.21.15.16:0/1033653439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 msgr2=0x55dbffe30680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f6593fff640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe30680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f6592ffd640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 0x55dbffe11660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe30680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 -- 172.21.15.16:0/1033653439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 msgr2=0x55dbffe11660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 0x55dbffe11660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 -- 172.21.15.16:0/1033653439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbffe09310 con 0x55dbffe0e5d0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f6592ffd640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 0x55dbffe11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f6593fff640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe30680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65937fe640 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f65840b8910 tx=0x7f658402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef488a4bc9c0a58d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65797fa640 1 -- 172.21.15.16:0/1033653439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840d4030 con 0x55dbffe0e5d0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65797fa640 1 -- 172.21.15.16:0/1033653439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65840bc640 con 0x55dbffe0e5d0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.405+0000 7f65797fa640 1 -- 172.21.15.16:0/1033653439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658403b460 con 0x55dbffe0e5d0 2024-09-12T01:11:56.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- 172.21.15.16:0/1033653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 msgr2=0x55dbffe11120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f65840b8910 tx=0x7f658402a420 comp rx=0 tx=0).stop 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- 172.21.15.16:0/1033653439 shutdown_connections 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe30680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0fb40 0x55dbffe11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 --2- 172.21.15.16:0/1033653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0e5d0 0x55dbffe11120 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- 172.21.15.16:0/1033653439 >> 172.21.15.16:0/1033653439 conn(0x55dbffdd7310 msgr2=0x55dbffe09d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- 172.21.15.16:0/1033653439 shutdown_connections 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- 172.21.15.16:0/1033653439 wait complete. 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 -- start start 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.406+0000 7f659c61b740 1 Processor -- start 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0e5d0 0x55dbffe218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65941aed30 con 0x55dbffe0fb40 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65941ae660 con 0x55dbffe0e5d0 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65941ab8c0 con 0x55dbffe19370 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f659c61b740 10 client.0 _ll_register_callbacks cb 0x7f658403eeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46286/0 (socket says 172.21.15.16:46286) 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 -- 172.21.15.16:0/504926271 learned_addr learned my addr 172.21.15.16:0/504926271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f65937fe640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0e5d0 0x55dbffe218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6593fff640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 -- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 msgr2=0x55dbffe1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe1d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 -- 172.21.15.16:0/504926271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0e5d0 msgr2=0x55dbffe218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0e5d0 0x55dbffe218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 -- 172.21.15.16:0/504926271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbffe1d980 con 0x55dbffe0fb40 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6593fff640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6592ffd640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x55dbffe0bac0 tx=0x7f658800b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1800fc0573491a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.407+0000 7f6572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588022020 con 0x55dbffe0fb40 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65840bc640 con 0x55dbffe0fb40 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f658801e410 con 0x55dbffe0fb40 2024-09-12T01:11:56.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f658802d050 con 0x55dbffe0fb40 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 10 client.20133 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dbffe979f0 con 0x55dbffe0fb40 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65840bc640 con 0x55dbffe0fb40 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6588015850 con 0x55dbffe0fb40 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 10 client.20133 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 10 client.20133 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 20 client.20133 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe981f0 0x55dbffe9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 4 client.20133 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.408+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbffd7bd70 con 0x55dbffe981f0 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f65937fe640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe981f0 0x55dbffe9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f65937fe640 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe981f0 0x55dbffe9a610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=352e81c0abd7fea4 server_cookie=e6c88d7b5662956a in_seq=0 out_seq=0 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f6572ffd640 10 client.20133 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:56.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f6572ffd640 1 -- 172.21.15.16:0/504926271 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbffd7bd70 con 0x55dbffe981f0 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f6572ffd640 10 client.20133 handle_command_reply: tid=0 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 client.20133 shutdown 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 msgr2=0x55dbffe1cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x55dbffe0bac0 tx=0x7f658800b310 comp rx=0 tx=0).stop 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 shutdown_connections 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbffe19370 0x55dbffe1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x55dbffe981f0 0x55dbffe9a610 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.436 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dbffe0fb40 0x55dbffe1cc10 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f6572ffd640 0 client.20133 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 --2- 172.21.15.16:0/504926271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbffe0e5d0 0x55dbffe218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.409+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 >> 172.21.15.16:0/504926271 conn(0x55dbffdd7310 msgr2=0x55dbffe09310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 shutdown_connections 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659c61b740 1 -- 172.21.15.16:0/504926271 wait complete. 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659c61b740 20 client.20133 trim_cache size 0 max 16384 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 msgr2=0x7f657c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 0x7f657c0a5230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55dbffe0f070 tx=0x7f658c007410 comp rx=0 tx=0).stop 2024-09-12T01:11:56.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 msgr2=0x7f65941984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6584009a50 tx=0x7f6584004270 comp rx=0 tx=0).stop 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 shutdown_connections 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f657c0a2d70 0x7f657c0a5230 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6594106f50 0x7f659419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6594100ca0 0x7f6594198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 --2- 172.21.15.16:0/2654325280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65941002f0 0x7f65941984e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 >> 172.21.15.16:0/2654325280 conn(0x7f6594077210 msgr2=0x7f6594105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 shutdown_connections 2024-09-12T01:11:56.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:56.410+0000 7f659a242640 1 -- 172.21.15.16:0/2654325280 wait complete. 2024-09-12T01:11:57.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:11:57.512 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:57.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:57 smithi099 ceph-mon[35654]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:57.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:57 smithi159 ceph-mon[35566]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:57.632 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:57 smithi016 ceph-mon[30972]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 101 KiB/s wr, 19 op/s 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 -- 172.21.15.16:0/273252927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc100a80 msgr2=0x7fe5dc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/273252927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc100a80 0x7fe5dc100ee0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009930 tx=0x7fe5d002a420 comp rx=0 tx=0).stop 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 -- 172.21.15.16:0/273252927 shutdown_connections 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/273252927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc101420 0x7fe5dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/273252927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc100a80 0x7fe5dc100ee0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/273252927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 0x7fe5dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 -- 172.21.15.16:0/273252927 >> 172.21.15.16:0/273252927 conn(0x7fe5dc0fc7a0 msgr2=0x7fe5dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 -- 172.21.15.16:0/273252927 shutdown_connections 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.012+0000 7fe5e1d90640 1 -- 172.21.15.16:0/273252927 wait complete. 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 Processor -- start 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 -- start start 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 0x7fe5dc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 0x7fe5dc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110e80 con 0x7fe5dc101420 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc110d00 con 0x7fe5dc100a80 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5e1d90640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc111000 con 0x7fe5dc106a80 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46290/0 (socket says 172.21.15.16:46290) 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5daffd640 1 -- 172.21.15.16:0/2413131039 learned_addr learned my addr 172.21.15.16:0/2413131039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 0x7fe5dc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.013+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 0x7fe5dc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 -- 172.21.15.16:0/2413131039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 msgr2=0x7fe5dc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 0x7fe5dc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 -- 172.21.15.16:0/2413131039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 msgr2=0x7fe5dc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 0x7fe5dc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 -- 172.21.15.16:0/2413131039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc19d430 con 0x7fe5dc101420 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 0x7fe5dc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5daffd640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009230 tx=0x7fe5d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5d8ff9640 1 -- 172.21.15.16:0/2413131039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d003f030 con 0x7fe5dc101420 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5d8ff9640 1 -- 172.21.15.16:0/2413131039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d0003ad0 con 0x7fe5dc101420 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc19d6c0 con 0x7fe5dc101420 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5d8ff9640 1 -- 172.21.15.16:0/2413131039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0035070 con 0x7fe5dc101420 2024-09-12T01:11:58.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.014+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a5000 con 0x7fe5dc101420 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5d8ff9640 1 -- 172.21.15.16:0/2413131039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5d0042030 con 0x7fe5dc101420 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 Processor -- start 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 -- start start 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961582ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x55961583cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5596157912f0 con 0x55961582cb40 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559615738cf0 con 0x55961582b5d0 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5e4169740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55961578d2b0 con 0x55961583a780 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46292/0 (socket says 172.21.15.16:46292) 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.017+0000 7fe5dbfff640 1 -- 172.21.15.16:0/2068004258 learned_addr learned my addr 172.21.15.16:0/2068004258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:58.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5d8ff9640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 0x7fe5c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5daffd640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x55961583cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961582ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5d8ff9640 1 -- 172.21.15.16:0/2413131039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe5d00ef940 con 0x7fe5dc101420 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 -- 172.21.15.16:0/2068004258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 msgr2=0x55961583cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x55961583cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 -- 172.21.15.16:0/2068004258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 msgr2=0x55961582ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961582ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 -- 172.21.15.16:0/2068004258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961571d3a0 con 0x55961582cb40 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961582ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 0x7fe5c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4012510 tx=0x7fe5d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d94b71d26b37436 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.018+0000 7fe5c1ffb640 1 -- 172.21.15.16:0/2068004258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4023070 con 0x55961582cb40 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5c1ffb640 1 -- 172.21.15.16:0/2068004258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d401dae0 con 0x55961582cb40 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5c1ffb640 1 -- 172.21.15.16:0/2068004258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d401c950 con 0x55961582cb40 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 -- 172.21.15.16:0/2068004258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 msgr2=0x55961582b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 0x7fe5c40a51b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4020000 tx=0x7fe5d401b000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4012510 tx=0x7fe5d4009f90 comp rx=0 tx=0).stop 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 -- 172.21.15.16:0/2068004258 shutdown_connections 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x55961583cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961582ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2068004258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x55961582b090 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 -- 172.21.15.16:0/2068004258 >> 172.21.15.16:0/2068004258 conn(0x5596157f4310 msgr2=0x559615827c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 -- 172.21.15.16:0/2068004258 shutdown_connections 2024-09-12T01:11:58.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.019+0000 7fe5e4169740 1 -- 172.21.15.16:0/2068004258 wait complete. 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 -- start start 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 Processor -- start 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 0x55961583a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961583a780 0x559615837a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55961583d480 con 0x55961582cb40 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55961574c060 con 0x55961583a780 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55961583d600 con 0x55961582b5d0 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5e4169740 10 client.0 _ll_register_callbacks cb 0x55961549f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46296/0 (socket says 172.21.15.16:46296) 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5daffd640 1 -- 172.21.15.16:0/714490989 learned_addr learned my addr 172.21.15.16:0/714490989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.020+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 0x55961583a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961583a780 0x559615837a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 -- 172.21.15.16:0/714490989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 msgr2=0x55961583a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 0x55961583a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 -- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961583a780 msgr2=0x559615837a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961583a780 0x559615837a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 -- 172.21.15.16:0/714490989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596155023c0 con 0x55961582cb40 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 0x55961583a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5daffd640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009a50 tx=0x7fe5d0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f2c5670934922a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0049040 con 0x55961582cb40 2024-09-12T01:11:58.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe5d00ed310 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d401dae0 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5d00bbe60 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 10 client.20145 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559615848ef0 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d401dae0 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d00dbd00 con 0x55961582cb40 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 10 client.20145 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 10 client.20145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 20 client.20145 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.021+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55961584a330 0x55961584c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.022+0000 7fe5e4169740 4 client.20145 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.022+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961579e860 con 0x55961584a330 2024-09-12T01:11:58.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.022+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55961584a330 0x55961584c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.022+0000 7fe5baffd640 10 client.20145 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.022+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55961584a330 0x55961584c740 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca5d7fd5d774d5ed server_cookie=358f6db26098c756 in_seq=0 out_seq=0 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5baffd640 1 -- 172.21.15.16:0/714490989 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55961579e860 con 0x55961584a330 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5baffd640 10 client.20145 handle_command_reply: tid=0 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 client.20145 shutdown 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 msgr2=0x559615837410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009a50 tx=0x7fe5d0005e50 comp rx=0 tx=0).stop 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 shutdown_connections 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55961584a330 0x55961584c740 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961583a780 0x559615837a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582cb40 0x559615837410 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 --2- 172.21.15.16:0/714490989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582b5d0 0x55961583a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5baffd640 0 client.20145 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 >> 172.21.15.16:0/714490989 conn(0x5596157f4310 msgr2=0x55961582d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 shutdown_connections 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 1 -- 172.21.15.16:0/714490989 wait complete. 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.023+0000 7fe5e4169740 20 client.20145 trim_cache size 0 max 16384 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.036+0000 7fe5e4169740 1 Processor -- start 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.036+0000 7fe5e4169740 1 -- start start 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582cb40 0x559615852050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x559615852590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ab5a0 con 0x55961582b5d0 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ad0e0 con 0x55961582cb40 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5e4169740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1abc70 con 0x55961583a780 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46308/0 (socket says 172.21.15.16:46308) 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 -- 172.21.15.16:0/2942736427 learned_addr learned my addr 172.21.15.16:0/2942736427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x559615852590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 -- 172.21.15.16:0/2942736427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 msgr2=0x559615852590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x559615852590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 -- 172.21.15.16:0/2942736427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582cb40 msgr2=0x559615852050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582cb40 0x559615852050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 -- 172.21.15.16:0/2942736427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559615837d70 con 0x55961582b5d0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x559615852590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x55961584cfc0 tx=0x7fe5cc013a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75dab4eafc05f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5c17fa640 1 -- 172.21.15.16:0/2942736427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc005020 con 0x55961582b5d0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5c17fa640 1 -- 172.21.15.16:0/2942736427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc0047a0 con 0x55961582b5d0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.037+0000 7fe5c17fa640 1 -- 172.21.15.16:0/2942736427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc01c6e0 con 0x55961582b5d0 2024-09-12T01:11:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- 172.21.15.16:0/2942736427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 msgr2=0x559615846720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x55961584cfc0 tx=0x7fe5cc013a30 comp rx=0 tx=0).stop 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- 172.21.15.16:0/2942736427 shutdown_connections 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961583a780 0x559615852590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582cb40 0x559615852050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 --2- 172.21.15.16:0/2942736427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961582b5d0 0x559615846720 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- 172.21.15.16:0/2942736427 >> 172.21.15.16:0/2942736427 conn(0x5596157f4310 msgr2=0x559615832720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- 172.21.15.16:0/2942736427 shutdown_connections 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- 172.21.15.16:0/2942736427 wait complete. 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 -- start start 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.038+0000 7fe5e4169740 1 Processor -- start 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961583e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582cb40 0x55961583eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1aca10 con 0x55961583a780 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ac340 con 0x55961582b5d0 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1a95a0 con 0x55961582cb40 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5e4169740 10 client.0 _ll_register_callbacks cb 0x7fe5d0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46320/0 (socket says 172.21.15.16:46320) 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 -- 172.21.15.16:0/3274405640 learned_addr learned my addr 172.21.15.16:0/3274405640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961583e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5daffd640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582cb40 0x55961583eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 -- 172.21.15.16:0/3274405640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582cb40 msgr2=0x55961583eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582cb40 0x55961583eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 -- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 msgr2=0x55961583e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961583e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 -- 172.21.15.16:0/3274405640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596158b4270 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5dbfff640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005f40 tx=0x7fe5d4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657cbf22abdf97d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.039+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4023030 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7fe5d4003b80 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5cc0047a0 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5d40135a0 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 10 client.20151 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596158b54e0 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc0047a0 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5d400f040 con 0x55961583a780 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 10 client.20151 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 10 client.20151 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:11:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 20 client.20151 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5596158b5b20 0x5596158b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 4 client.20151 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559615798d70 con 0x5596158b5b20 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.040+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5596158b5b20 0x5596158b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5baffd640 10 client.20151 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5db7fe640 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5596158b5b20 0x5596158b7f40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a83367956ce02b70 server_cookie=a81184cb8c073ee9 in_seq=0 out_seq=0 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5baffd640 1 -- 172.21.15.16:0/3274405640 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559615798d70 con 0x5596158b5b20 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5baffd640 10 client.20151 handle_command_reply: tid=0 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 client.20151 shutdown 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 msgr2=0x55961583f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005f40 tx=0x7fe5d4005f90 comp rx=0 tx=0).stop 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 shutdown_connections 2024-09-12T01:11:58.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x5596158b5b20 0x5596158b7f40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55961583a780 0x55961583f2e0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55961582cb40 0x55961583eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5baffd640 0 client.20151 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 --2- 172.21.15.16:0/3274405640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55961582b5d0 0x55961583e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 >> 172.21.15.16:0/3274405640 conn(0x5596157f4310 msgr2=0x559615838c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 shutdown_connections 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 1 -- 172.21.15.16:0/3274405640 wait complete. 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.041+0000 7fe5e4169740 20 client.20151 trim_cache size 0 max 16384 2024-09-12T01:11:58.063 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 msgr2=0x7fe5c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 0x7fe5c40a51b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4020000 tx=0x7fe5d401b000 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 msgr2=0x7fe5dc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009230 tx=0x7fe5d0004270 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 shutdown_connections 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5c40a2cf0 0x7fe5c40a51b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5dc106a80 0x7fe5dc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5dc101420 0x7fe5dc1989c0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 --2- 172.21.15.16:0/2413131039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5dc100a80 0x7fe5dc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 >> 172.21.15.16:0/2413131039 conn(0x7fe5dc0fc7a0 msgr2=0x7fe5dc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:58.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 shutdown_connections 2024-09-12T01:11:58.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:58.042+0000 7fe5e1d90640 1 -- 172.21.15.16:0/2413131039 wait complete. 2024-09-12T01:11:58.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:11:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:11:59.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:11:59 smithi016 ceph-mon[30972]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 13 op/s 2024-09-12T01:11:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:11:59 smithi099 ceph-mon[35654]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 13 op/s 2024-09-12T01:11:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 -- 172.21.15.16:0/3573497533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244106a80 msgr2=0x7f2244106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 --2- 172.21.15.16:0/3573497533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244106a80 0x7f2244106e60 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f2234009930 tx=0x7f223402a420 comp rx=0 tx=0).stop 2024-09-12T01:11:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 -- 172.21.15.16:0/3573497533 shutdown_connections 2024-09-12T01:11:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 --2- 172.21.15.16:0/3573497533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244101420 0x7f224410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 --2- 172.21.15.16:0/3573497533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244100a80 0x7f2244100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 --2- 172.21.15.16:0/3573497533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244106a80 0x7f2244106e60 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 -- 172.21.15.16:0/3573497533 >> 172.21.15.16:0/3573497533 conn(0x7f22440fc7a0 msgr2=0x7f22440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 -- 172.21.15.16:0/3573497533 shutdown_connections 2024-09-12T01:11:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.571+0000 7f224a7d7640 1 -- 172.21.15.16:0/3573497533 wait complete. 2024-09-12T01:11:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 Processor -- start 2024-09-12T01:11:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 -- start start 2024-09-12T01:11:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 0x7f2244196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244106a80 0x7f224419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f224410fd70 con 0x7f2244100a80 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f224410fbf0 con 0x7f2244101420 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f224a7d7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f224410fef0 con 0x7f2244106a80 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44914/0 (socket says 172.21.15.16:44914) 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 -- 172.21.15.16:0/969678103 learned_addr learned my addr 172.21.15.16:0/969678103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f223bfff640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 0x7f2244196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2248d74640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244106a80 0x7f224419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 -- 172.21.15.16:0/969678103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244106a80 msgr2=0x7f224419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244106a80 0x7f224419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 -- 172.21.15.16:0/969678103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 msgr2=0x7f2244196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 0x7f2244196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f2243fff640 1 -- 172.21.15.16:0/969678103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224419b0e0 con 0x7f2244100a80 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.572+0000 7f223bfff640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 0x7f2244196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f2243fff640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2234009a50 tx=0x7f2234004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f2241ffb640 1 -- 172.21.15.16:0/969678103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223403f030 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f2241ffb640 1 -- 172.21.15.16:0/969678103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2234003ce0 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f2241ffb640 1 -- 172.21.15.16:0/969678103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234035070 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224419b370 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.573+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224419b800 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.575+0000 7f2241ffb640 1 -- 172.21.15.16:0/969678103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2234042030 con 0x7f2244100a80 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.575+0000 7f224cbb0740 1 Processor -- start 2024-09-12T01:11:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.575+0000 7f224cbb0740 1 -- start start 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2b202f0 con 0x55a5d2bba5d0 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2ac7cf0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f224cbb0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2b1c2b0 con 0x55a5d2bc5370 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2241ffb640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 0x7f22300a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44922/0 (socket says 172.21.15.16:44922) 2024-09-12T01:11:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 -- 172.21.15.16:0/4262182073 learned_addr learned my addr 172.21.15.16:0/4262182073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2248d74640 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2248d74640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 0x7f22300a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f223bfff640 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2241ffb640 1 -- 172.21.15.16:0/969678103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f22340efaf0 con 0x7f2244100a80 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 -- 172.21.15.16:0/4262182073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 msgr2=0x55a5d2bc7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 -- 172.21.15.16:0/4262182073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 msgr2=0x55a5d2bba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 -- 172.21.15.16:0/4262182073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d2aac3a0 con 0x55a5d2bba5d0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.576+0000 7f2243fff640 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f22340bbcb0 tx=0x7f22340314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bf81b41cbd63c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f2248d74640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 0x7f22300a73c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f223c00bfd0 tx=0x7f223c004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f223a7fc640 1 -- 172.21.15.16:0/4262182073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234049070 con 0x55a5d2bba5d0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f223a7fc640 1 -- 172.21.15.16:0/4262182073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22340bbfb0 con 0x55a5d2bba5d0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f223a7fc640 1 -- 172.21.15.16:0/4262182073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223403b460 con 0x55a5d2bba5d0 2024-09-12T01:11:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 -- 172.21.15.16:0/4262182073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 msgr2=0x55a5d2bbaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f22340bbcb0 tx=0x7f22340314b0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 -- 172.21.15.16:0/4262182073 shutdown_connections 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbaa10 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 --2- 172.21.15.16:0/4262182073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 -- 172.21.15.16:0/4262182073 >> 172.21.15.16:0/4262182073 conn(0x55a5d2b83310 msgr2=0x55a5d2bb6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 -- 172.21.15.16:0/4262182073 shutdown_connections 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.577+0000 7f224cbb0740 1 -- 172.21.15.16:0/4262182073 wait complete. 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 -- start start 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 Processor -- start 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bc4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2bc7e30 con 0x55a5d2bbbb40 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2adb060 con 0x55a5d2bba5d0 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5d2bc7fb0 con 0x55a5d2bc5370 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f224cbb0740 10 client.0 _ll_register_callbacks cb 0x55a5d282e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f223bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f223bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44934/0 (socket says 172.21.15.16:44934) 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.578+0000 7f223bfff640 1 -- 172.21.15.16:0/453980377 learned_addr learned my addr 172.21.15.16:0/453980377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f2243fff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bc4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f2248d74640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 -- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 msgr2=0x55a5d2bc27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 -- 172.21.15.16:0/453980377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 msgr2=0x55a5d2bc4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bc4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 -- 172.21.15.16:0/453980377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d28913c0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f223bfff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f222c011800 tx=0x7f222c011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf21aa761a01c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f22237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00f9a0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f222c00fb40 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22340bbfb0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f222c00fd80 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f224cbb0740 10 client.20163 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.579+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5d2bd38b0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22340bbfb0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f222c018030 con 0x55a5d2bbbb40 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 10 client.20163 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 10 client.20163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 20 client.20163 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2bd4d70 0x55a5d2bd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 4 client.20163 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d2b2d860 con 0x55a5d2bd4d70 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.580+0000 7f2243fff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2bd4d70 0x55a5d2bd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.581+0000 7f22237fe640 10 client.20163 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.581+0000 7f2243fff640 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2bd4d70 0x55a5d2bd71c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f49fc31e770de33 server_cookie=7c7dccc1dc263e05 in_seq=0 out_seq=0 2024-09-12T01:11:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.581+0000 7f22237fe640 1 -- 172.21.15.16:0/453980377 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5d2b2d860 con 0x55a5d2bd4d70 2024-09-12T01:11:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.581+0000 7f22237fe640 10 client.20163 handle_command_reply: tid=0 2024-09-12T01:11:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.581+0000 7f224cbb0740 1 client.20163 shutdown 2024-09-12T01:11:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 msgr2=0x55a5d2bc2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f222c011800 tx=0x7f222c011c00 comp rx=0 tx=0).stop 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 shutdown_connections 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bc27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc2090 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2bd4d70 0x55a5d2bd71c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 --2- 172.21.15.16:0/453980377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bc4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f22237fe640 0 client.20163 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 >> 172.21.15.16:0/453980377 conn(0x55a5d2b83310 msgr2=0x55a5d2bbc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 shutdown_connections 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 1 -- 172.21.15.16:0/453980377 wait complete. 2024-09-12T01:11:59.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.582+0000 7f224cbb0740 20 client.20163 trim_cache size 0 max 16384 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 Processor -- start 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 -- start start 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bd0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22441ab610 con 0x55a5d2bbbb40 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22441ad150 con 0x55a5d2bba5d0 2024-09-12T01:11:59.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f224cbb0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22441abce0 con 0x55a5d2bc5370 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f223bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f223bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44946/0 (socket says 172.21.15.16:44946) 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f223bfff640 1 -- 172.21.15.16:0/3781413453 learned_addr learned my addr 172.21.15.16:0/3781413453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f2248d74640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bd0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f2243fff640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f223bfff640 1 -- 172.21.15.16:0/3781413453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 msgr2=0x55a5d2bd0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.595+0000 7f223bfff640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bd0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f223bfff640 1 -- 172.21.15.16:0/3781413453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 msgr2=0x55a5d2bbb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f223bfff640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f223bfff640 1 -- 172.21.15.16:0/3781413453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d2bb64f0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f2248d74640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bd0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f2243fff640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f223bfff640 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f222c006010 tx=0x7f222c008840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa2aea1e464c986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f2239ffb640 1 -- 172.21.15.16:0/3781413453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c026040 con 0x55a5d2bbbb40 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f2239ffb640 1 -- 172.21.15.16:0/3781413453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c02f070 con 0x55a5d2bbbb40 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f2239ffb640 1 -- 172.21.15.16:0/3781413453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c003920 con 0x55a5d2bbbb40 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f224cbb0740 1 -- 172.21.15.16:0/3781413453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 msgr2=0x55a5d2bc1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.596+0000 7f224cbb0740 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f222c006010 tx=0x7f222c008840 comp rx=0 tx=0).stop 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- 172.21.15.16:0/3781413453 shutdown_connections 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bd0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bc1950 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- 172.21.15.16:0/3781413453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bbb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- 172.21.15.16:0/3781413453 >> 172.21.15.16:0/3781413453 conn(0x55a5d2b83310 msgr2=0x55a5d2bc0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- 172.21.15.16:0/3781413453 shutdown_connections 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- 172.21.15.16:0/3781413453 wait complete. 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- start start 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 Processor -- start 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bcd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bbe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f22441aca80 con 0x55a5d2bba5d0 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22441ac3b0 con 0x55a5d2bbbb40 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22441a9610 con 0x55a5d2bc5370 2024-09-12T01:11:59.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.597+0000 7f224cbb0740 10 client.0 _ll_register_callbacks cb 0x7f222c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44952/0 (socket says 172.21.15.16:44952) 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 -- 172.21.15.16:0/895528560 learned_addr learned my addr 172.21.15.16:0/895528560 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2248d74640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bbe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f223bfff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bcd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 -- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 msgr2=0x55a5d2bbe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bbe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 -- 172.21.15.16:0/895528560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 msgr2=0x55a5d2bcd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bcd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 -- 172.21.15.16:0/895528560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d2bbf100 con 0x55a5d2bba5d0 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2248d74640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bbe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f223bfff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bcd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:11:59.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f2243fff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f223403d040 tx=0x7f223404bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab032d940f510ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f22237fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2234049040 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f223403bd00 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222c02f070 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22340bbfb0 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f224cbb0740 10 client.20169 fetch_fsmap learned FSMap version 28 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.598+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5d2c3f510 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c02f070 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22340b8620 con 0x55a5d2bba5d0 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 10 client.20169 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 10 client.20169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 20 client.20169 populate_metadata read hostname 'smithi016' 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2c3faf0 0x55a5d2c41ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:11:59.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 4 client.20169 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5d2b27d70 con 0x55a5d2c3faf0 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.599+0000 7f223bfff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2c3faf0 0x55a5d2c41ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f22237fe640 10 client.20169 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f223bfff640 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2c3faf0 0x55a5d2c41ee0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdbb77b24674c780 server_cookie=4b3222b1e440712f in_seq=0 out_seq=0 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f22237fe640 1 -- 172.21.15.16:0/895528560 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5d2b27d70 con 0x55a5d2c3faf0 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f22237fe640 10 client.20169 handle_command_reply: tid=0 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 client.20169 shutdown 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 msgr2=0x55a5d2bcd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f223403d040 tx=0x7f223404bfb0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 shutdown_connections 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5d2bc5370 0x55a5d2bbe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55a5d2c3faf0 0x55a5d2c41ee0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5d2bbbb40 0x55a5d2bcd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 --2- 172.21.15.16:0/895528560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a5d2bba5d0 0x55a5d2bcd370 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 >> 172.21.15.16:0/895528560 conn(0x55a5d2b83310 msgr2=0x55a5d2bb5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f22237fe640 0 client.20169 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 shutdown_connections 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 1 -- 172.21.15.16:0/895528560 wait complete. 2024-09-12T01:11:59.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.600+0000 7f224cbb0740 20 client.20169 trim_cache size 0 max 16384 2024-09-12T01:11:59.608 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 msgr2=0x7f22300a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 0x7f22300a73c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f223c00bfd0 tx=0x7f223c004a40 comp rx=0 tx=0).stop 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 msgr2=0x7f2244196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2234009a50 tx=0x7f2234004270 comp rx=0 tx=0).stop 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 shutdown_connections 2024-09-12T01:11:59.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f22300a4f00 0x7f22300a73c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2244106a80 0x7f224419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2244101420 0x7f2244196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 --2- 172.21.15.16:0/969678103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2244100a80 0x7f2244196130 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 >> 172.21.15.16:0/969678103 conn(0x7f22440fc7a0 msgr2=0x7f22440fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 shutdown_connections 2024-09-12T01:11:59.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:11:59.601+0000 7f224a7d7640 1 -- 172.21.15.16:0/969678103 wait complete. 2024-09-12T01:11:59.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:11:59 smithi159 ceph-mon[35566]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 13 op/s 2024-09-12T01:12:00.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:12:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.283+0000 7f7f30323640 1 -- 172.21.15.16:0/2434426795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 msgr2=0x7f7f28101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.283+0000 7f7f30323640 1 --2- 172.21.15.16:0/2434426795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28101200 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009900 tx=0x7f7f2402a480 comp rx=0 tx=0).stop 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 -- 172.21.15.16:0/2434426795 shutdown_connections 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 --2- 172.21.15.16:0/2434426795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 0x7f7f2810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 --2- 172.21.15.16:0/2434426795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 --2- 172.21.15.16:0/2434426795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28101200 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 -- 172.21.15.16:0/2434426795 >> 172.21.15.16:0/2434426795 conn(0x7f7f280fc7d0 msgr2=0x7f7f280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 -- 172.21.15.16:0/2434426795 shutdown_connections 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 -- 172.21.15.16:0/2434426795 wait complete. 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 Processor -- start 2024-09-12T01:12:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 -- start start 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.284+0000 7f7f30323640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f28199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f30323640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 0x7f7f281a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f30323640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28110f80 con 0x7f7f28100e20 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f30323640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28110e00 con 0x7f7f28102570 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f30323640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28111100 con 0x7f7f28102ef0 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44978/0 (socket says 172.21.15.16:44978) 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/515705070 learned_addr learned my addr 172.21.15.16:0/515705070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f28199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 0x7f7f281a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/515705070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 msgr2=0x7f7f281a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 0x7f7f281a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/515705070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 msgr2=0x7f7f28199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f28199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/515705070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f281a4900 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f28199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.285+0000 7f7f1f7fe640 1 -- 172.21.15.16:0/515705070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2403f070 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.286+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281a4ad0 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.286+0000 7f7f1f7fe640 1 -- 172.21.15.16:0/515705070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24031c30 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.286+0000 7f7f1f7fe640 1 -- 172.21.15.16:0/515705070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24035030 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.286+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a4f80 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.288+0000 7f7f1f7fe640 1 -- 172.21.15.16:0/515705070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f24042070 con 0x7f7f28100e20 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.288+0000 7f7f326fc740 1 Processor -- start 2024-09-12T01:12:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.288+0000 7f7f326fc740 1 -- start start 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55922e52a2f0 con 0x55922e5c5b40 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55922e4d1cf0 con 0x55922e5c45d0 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f326fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55922e5262b0 con 0x55922e5cf370 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e0bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44986/0 (socket says 172.21.15.16:44986) 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e8c0640 1 -- 172.21.15.16:0/150546235 learned_addr learned my addr 172.21.15.16:0/150546235 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f1f7fe640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 0x7f7f140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/150546235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 msgr2=0x55922e5d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/150546235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 msgr2=0x55922e5c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.289+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/150546235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922e4b63a0 con 0x55922e5c45d0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 0x7f7f140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f1f7fe640 1 -- 172.21.15.16:0/515705070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f7f240efa80 con 0x7f7f28100e20 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5c4a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ed1d0 tx=0x7f7f2404b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9cedd4531244fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 0x7f7f140a51b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f200107d0 tx=0x7f7f20004a40 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f1cdea640 1 -- 172.21.15.16:0/150546235 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240f7040 con 0x55922e5c45d0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f1cdea640 1 -- 172.21.15.16:0/150546235 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24031de0 con 0x55922e5c45d0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.290+0000 7f7f1cdea640 1 -- 172.21.15.16:0/150546235 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240a6d10 con 0x55922e5c45d0 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- 172.21.15.16:0/150546235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 msgr2=0x55922e5c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5c4a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ed1d0 tx=0x7f7f2404b000 comp rx=0 tx=0).stop 2024-09-12T01:12:01.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- 172.21.15.16:0/150546235 shutdown_connections 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5c4a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 --2- 172.21.15.16:0/150546235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- 172.21.15.16:0/150546235 >> 172.21.15.16:0/150546235 conn(0x55922e58d310 msgr2=0x55922e5c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- 172.21.15.16:0/150546235 shutdown_connections 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- 172.21.15.16:0/150546235 wait complete. 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 -- start start 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.291+0000 7f7f326fc740 1 Processor -- start 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5cc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55922e5d1e30 con 0x55922e5c5b40 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55922e4e5060 con 0x55922e5c45d0 2024-09-12T01:12:01.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55922e5d1fb0 con 0x55922e5cf370 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f326fc740 10 client.0 _ll_register_callbacks cb 0x55922e2388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44990/0 (socket says 172.21.15.16:44990) 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 -- 172.21.15.16:0/1340883588 learned_addr learned my addr 172.21.15.16:0/1340883588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 -- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 msgr2=0x55922e5cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 -- 172.21.15.16:0/1340883588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 msgr2=0x55922e5cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 -- 172.21.15.16:0/1340883588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922e29b3c0 con 0x55922e5c5b40 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f7f18012d00 tx=0x7f7f1800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ad757c6745c519 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.292+0000 7f7f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18009d90 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f24031de0 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f7f18012110 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f18004050 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 10 client.20181 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55922e5dd840 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24031de0 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f18010430 con 0x55922e5c5b40 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 10 client.20181 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 10 client.20181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 20 client.20181 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e5df0c0 0x55922e5e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 4 client.20181 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:01.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.293+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55922e537860 con 0x55922e5df0c0 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.294+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e5df0c0 0x55922e5e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.294+0000 7f7f11ffb640 10 client.20181 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.294+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e5df0c0 0x55922e5e1520 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af75e6530cf3ce29 server_cookie=7df03622532516e6 in_seq=0 out_seq=0 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f11ffb640 1 -- 172.21.15.16:0/1340883588 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55922e537860 con 0x55922e5df0c0 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f11ffb640 10 client.20181 handle_command_reply: tid=0 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 client.20181 shutdown 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 msgr2=0x55922e5cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f7f18012d00 tx=0x7f7f1800f790 comp rx=0 tx=0).stop 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 shutdown_connections 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e5df0c0 0x55922e5e1520 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c5b40 0x55922e5cbfa0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 --2- 172.21.15.16:0/1340883588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c45d0 0x55922e5cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 >> 172.21.15.16:0/1340883588 conn(0x55922e58d310 msgr2=0x55922e5c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f11ffb640 0 client.20181 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 shutdown_connections 2024-09-12T01:12:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 1 -- 172.21.15.16:0/1340883588 wait complete. 2024-09-12T01:12:01.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.295+0000 7f7f326fc740 20 client.20181 trim_cache size 0 max 16384 2024-09-12T01:12:01.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 Processor -- start 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 -- start start 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281afb10 con 0x55922e5c45d0 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281b1650 con 0x55922e5c5b40 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f326fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281b01e0 con 0x55922e5cf370 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f2e0bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f2e0bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45002/0 (socket says 172.21.15.16:45002) 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/3875226218 learned_addr learned my addr 172.21.15.16:0/3875226218 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:01.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.309+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/3875226218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 msgr2=0x55922e5c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/3875226218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 msgr2=0x55922e5c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 -- 172.21.15.16:0/3875226218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922e5c7ba0 con 0x55922e5c45d0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b8a20 tx=0x7f7f240f35d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3903c6b469c68b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f13fff640 1 -- 172.21.15.16:0/3875226218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240c4e60 con 0x55922e5c45d0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f13fff640 1 -- 172.21.15.16:0/3875226218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24031de0 con 0x55922e5c45d0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.310+0000 7f7f13fff640 1 -- 172.21.15.16:0/3875226218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240a6d10 con 0x55922e5c45d0 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- 172.21.15.16:0/3875226218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 msgr2=0x55922e5daca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b8a20 tx=0x7f7f240f35d0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- 172.21.15.16:0/3875226218 shutdown_connections 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5cf370 0x55922e5c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 --2- 172.21.15.16:0/3875226218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5c45d0 0x55922e5daca0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- 172.21.15.16:0/3875226218 >> 172.21.15.16:0/3875226218 conn(0x55922e58d310 msgr2=0x55922e5cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- 172.21.15.16:0/3875226218 shutdown_connections 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- 172.21.15.16:0/3875226218 wait complete. 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 -- start start 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.311+0000 7f7f326fc740 1 Processor -- start 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5c45d0 0x55922e5d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281b0f80 con 0x55922e5cf370 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281b08b0 con 0x55922e5c5b40 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281adb10 con 0x55922e5c45d0 2024-09-12T01:12:01.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f326fc740 10 client.0 _ll_register_callbacks cb 0x7f7f18009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45018/0 (socket says 172.21.15.16:45018) 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 -- 172.21.15.16:0/2400589918 learned_addr learned my addr 172.21.15.16:0/2400589918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5c45d0 0x55922e5d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 -- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5c45d0 msgr2=0x55922e5d73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5c45d0 0x55922e5d73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 -- 172.21.15.16:0/2400589918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 msgr2=0x55922e5d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 -- 172.21.15.16:0/2400589918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922e5d3300 con 0x55922e5cf370 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2d8be640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f2e8c0640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000ff90 tx=0x7f7f20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cd060e4d5f3dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.312+0000 7f7f11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:01.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004b20 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f7f20004cc0 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f24031de0 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f20032460 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 10 client.20187 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55922e649cb0 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f24031de0 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f20028070 con 0x55922e5cf370 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 10 client.20187 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 10 client.20187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 20 client.20187 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e64a450 0x55922e64c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 4 client.20187 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.313+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55922e531d70 con 0x55922e64a450 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e64a450 0x55922e64c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:01.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f2e0bf640 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e64a450 0x55922e64c870 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e6ade3edaa42e5 server_cookie=989aa1723d52aaad in_seq=0 out_seq=0 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f11ffb640 10 client.20187 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f11ffb640 1 -- 172.21.15.16:0/2400589918 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55922e531d70 con 0x55922e64a450 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f11ffb640 10 client.20187 handle_command_reply: tid=0 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 client.20187 shutdown 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 msgr2=0x55922e5d2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000ff90 tx=0x7f7f20004ab0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 shutdown_connections 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55922e64a450 0x55922e64c870 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55922e5cf370 0x55922e5d2c60 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55922e5c5b40 0x55922e5d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f11ffb640 0 client.20187 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 --2- 172.21.15.16:0/2400589918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55922e5c45d0 0x55922e5d73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 >> 172.21.15.16:0/2400589918 conn(0x55922e58d310 msgr2=0x55922e5bf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 shutdown_connections 2024-09-12T01:12:01.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.314+0000 7f7f326fc740 1 -- 172.21.15.16:0/2400589918 wait complete. 2024-09-12T01:12:01.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f326fc740 20 client.20187 trim_cache size 0 max 16384 2024-09-12T01:12:01.326 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 msgr2=0x7f7f140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 0x7f7f140a51b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f200107d0 tx=0x7f7f20004a40 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 msgr2=0x7f7f28199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f24003c10 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 shutdown_connections 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7f140a2cf0 0x7f7f140a51b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f28102ef0 0x7f7f281a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f28102570 0x7f7f28199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 --2- 172.21.15.16:0/515705070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f28100e20 0x7f7f28199340 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 >> 172.21.15.16:0/515705070 conn(0x7f7f280fc7d0 msgr2=0x7f7f280fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:01.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 shutdown_connections 2024-09-12T01:12:01.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:01.315+0000 7f7f30323640 1 -- 172.21.15.16:0/515705070 wait complete. 2024-09-12T01:12:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:01 smithi099 ceph-mon[35654]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 19 op/s 2024-09-12T01:12:01.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:01 smithi159 ceph-mon[35566]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 19 op/s 2024-09-12T01:12:01.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:01 smithi016 ceph-mon[30972]: pgmap v497: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 95 KiB/s wr, 19 op/s 2024-09-12T01:12:03.160 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:12:03.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:12:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:03.555 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:03 smithi016 ceph-mon[30972]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:03 smithi159 ceph-mon[35566]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:03.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:03 smithi099 ceph-mon[35654]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 -- 172.21.15.16:0/410875626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102570 msgr2=0x7f16b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 --2- 172.21.15.16:0/410875626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102570 0x7f16b01029b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009900 tx=0x7f16a402a480 comp rx=0 tx=0).stop 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 -- 172.21.15.16:0/410875626 shutdown_connections 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 --2- 172.21.15.16:0/410875626 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102ef0 0x7f16b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 --2- 172.21.15.16:0/410875626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102570 0x7f16b01029b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 --2- 172.21.15.16:0/410875626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 0x7f16b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 -- 172.21.15.16:0/410875626 >> 172.21.15.16:0/410875626 conn(0x7f16b00fc7d0 msgr2=0x7f16b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:03.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 -- 172.21.15.16:0/410875626 shutdown_connections 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.961+0000 7f16b4964640 1 -- 172.21.15.16:0/410875626 wait complete. 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 Processor -- start 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 -- start start 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 0x7f16b01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 0x7f16b0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0110f50 con 0x7f16b0102ef0 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0110dd0 con 0x7f16b0102570 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16b4964640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01110d0 con 0x7f16b0100e20 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45028/0 (socket says 172.21.15.16:45028) 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 -- 172.21.15.16:0/2073112158 learned_addr learned my addr 172.21.15.16:0/2073112158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16add9b640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 0x7f16b0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16ae59c640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 0x7f16b01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 -- 172.21.15.16:0/2073112158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 msgr2=0x7f16b01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 0x7f16b01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 -- 172.21.15.16:0/2073112158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 msgr2=0x7f16b0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 0x7f16b0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16aed9d640 1 -- 172.21.15.16:0/2073112158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b019d470 con 0x7f16b0102ef0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.962+0000 7f16add9b640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 0x7f16b0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f16aed9d640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d750 tx=0x7f16a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f169f7fe640 1 -- 172.21.15.16:0/2073112158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8017070 con 0x7f16b0102ef0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b019d760 con 0x7f16b0102ef0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f169f7fe640 1 -- 172.21.15.16:0/2073112158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a8005500 con 0x7f16b0102ef0 2024-09-12T01:12:03.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f169f7fe640 1 -- 172.21.15.16:0/2073112158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8004d90 con 0x7f16b0102ef0 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.963+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b01a5040 con 0x7f16b0102ef0 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.965+0000 7f169f7fe640 1 -- 172.21.15.16:0/2073112158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f16a8004f30 con 0x7f16b0102ef0 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.965+0000 7f16b6d3d740 1 Processor -- start 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.965+0000 7f16b6d3d740 1 -- start start 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846888e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846897c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5648467ee280 con 0x564846898150 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564846795b10 con 0x5648468893a0 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5648467ea240 con 0x564846888a80 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f169f7fe640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 0x7f16980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45032/0 (socket says 172.21.15.16:45032) 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16add9b640 1 -- 172.21.15.16:0/841566822 learned_addr learned my addr 172.21.15.16:0/841566822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846897c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16aed9d640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 0x7f16980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16add9b640 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846888e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f169f7fe640 1 -- 172.21.15.16:0/2073112158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f16a80ceed0 con 0x7f16b0102ef0 2024-09-12T01:12:03.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 -- 172.21.15.16:0/841566822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 msgr2=0x564846888e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846888e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 -- 172.21.15.16:0/841566822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 msgr2=0x56484689a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16ae59c640 1 -- 172.21.15.16:0/841566822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648467a21f0 con 0x5648468893a0 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.966+0000 7f16add9b640 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.967+0000 7f16aed9d640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 0x7f16980a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f16a809b090 tx=0x7f16a8004830 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.967+0000 7f16ae59c640 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846897c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007e10 tx=0x7f16a000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4586ec6aa3fc2433 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:03.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.967+0000 7f169cff9640 1 -- 172.21.15.16:0/841566822 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0014070 con 0x5648468893a0 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.967+0000 7f169cff9640 1 -- 172.21.15.16:0/841566822 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a0003e50 con 0x5648468893a0 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.967+0000 7f169cff9640 1 -- 172.21.15.16:0/841566822 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0010df0 con 0x5648468893a0 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- 172.21.15.16:0/841566822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 msgr2=0x564846897c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846897c10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007e10 tx=0x7f16a000e720 comp rx=0 tx=0).stop 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- 172.21.15.16:0/841566822 shutdown_connections 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x56484689a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846897c10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/841566822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846888e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- 172.21.15.16:0/841566822 >> 172.21.15.16:0/841566822 conn(0x5648468850f0 msgr2=0x5648468854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:03.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- 172.21.15.16:0/841566822 shutdown_connections 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- 172.21.15.16:0/841566822 wait complete. 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 -- start start 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.968+0000 7f16b6d3d740 1 Processor -- start 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846895030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846895570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56484689acc0 con 0x564846898150 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648467a8f10 con 0x5648468893a0 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56484689ae40 con 0x564846888a80 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 10 client.0 _ll_register_callbacks cb 0x564846771a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:03.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45040/0 (socket says 172.21.15.16:45040) 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 -- 172.21.15.16:0/3815312990 learned_addr learned my addr 172.21.15.16:0/3815312990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16ae59c640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846895570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846895030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 -- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 msgr2=0x564846895030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846895030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 -- 172.21.15.16:0/3815312990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 msgr2=0x564846895570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846895570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 -- 172.21.15.16:0/3815312990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484655f6a0 con 0x564846898150 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16ae59c640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846895570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16add9b640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006010 tx=0x7f16a4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7080e2e2795d389e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f1695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4035040 con 0x564846898150 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f16a4031a10 con 0x564846898150 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.969+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a0003e50 con 0x564846898150 2024-09-12T01:12:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16a403d440 con 0x564846898150 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 10 client.20199 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648468a68e0 con 0x564846898150 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a0003e50 con 0x564846898150 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f16a403a040 con 0x564846898150 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 10 client.20199 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 10 client.20199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 20 client.20199 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5648468a7cf0 0x5648468aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 4 client.20199 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648467fb7f0 con 0x5648468a7cf0 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.970+0000 7f16ae59c640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5648468a7cf0 0x5648468aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.971+0000 7f16ae59c640 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5648468a7cf0 0x5648468aa0e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96ff3c9a42045eb server_cookie=9e37193feddb7a55 in_seq=0 out_seq=0 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.971+0000 7f1695ffb640 10 client.20199 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f1695ffb640 1 -- 172.21.15.16:0/3815312990 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648467fb7f0 con 0x5648468a7cf0 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f1695ffb640 10 client.20199 handle_command_reply: tid=0 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 client.20199 shutdown 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 msgr2=0x564846895ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006010 tx=0x7f16a4004040 comp rx=0 tx=0).stop 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 shutdown_connections 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5648468a7cf0 0x5648468aa0e0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846895ab0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648468893a0 0x564846895570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f1695ffb640 0 client.20199 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/3815312990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846888a80 0x564846895030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 >> 172.21.15.16:0/3815312990 conn(0x5648468850f0 msgr2=0x564846884270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 shutdown_connections 2024-09-12T01:12:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 1 -- 172.21.15.16:0/3815312990 wait complete. 2024-09-12T01:12:03.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.972+0000 7f16b6d3d740 20 client.20199 trim_cache size 0 max 16384 2024-09-12T01:12:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 Processor -- start 2024-09-12T01:12:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 -- start start 2024-09-12T01:12:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5648468893a0 0x5648468afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ab6a0 con 0x564846898150 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ad1e0 con 0x564846888a80 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01abd70 con 0x5648468893a0 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45056/0 (socket says 172.21.15.16:45056) 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 -- 172.21.15.16:0/2183092619 learned_addr learned my addr 172.21.15.16:0/2183092619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16ae59c640 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5648468893a0 0x5648468afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 -- 172.21.15.16:0/2183092619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5648468893a0 msgr2=0x5648468afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5648468893a0 0x5648468afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 -- 172.21.15.16:0/2183092619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 msgr2=0x5648468af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.990+0000 7f16add9b640 1 -- 172.21.15.16:0/2183092619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648468891d0 con 0x564846898150 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16add9b640 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f16a4003bc0 tx=0x7f16a40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be87cdf891ad562 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f1697fff640 1 -- 172.21.15.16:0/2183092619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4035040 con 0x564846898150 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f1697fff640 1 -- 172.21.15.16:0/2183092619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4004170 con 0x564846898150 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f1697fff640 1 -- 172.21.15.16:0/2183092619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4031a70 con 0x564846898150 2024-09-12T01:12:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2183092619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 msgr2=0x564846896780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f16a4003bc0 tx=0x7f16a40058b0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2183092619 shutdown_connections 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564846898150 0x564846896780 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5648468893a0 0x5648468afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2183092619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.991+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2183092619 >> 172.21.15.16:0/2183092619 conn(0x5648468850f0 msgr2=0x564846894120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2183092619 shutdown_connections 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2183092619 wait complete. 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- start start 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 Processor -- start 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 0x56484689be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01acb10 con 0x5648468893a0 2024-09-12T01:12:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01ac440 con 0x564846888a80 2024-09-12T01:12:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01a96a0 con 0x564846898150 2024-09-12T01:12:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16b6d3d740 10 client.0 _ll_register_callbacks cb 0x7f16a4003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45064/0 (socket says 172.21.15.16:45064) 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.992+0000 7f16ae59c640 1 -- 172.21.15.16:0/2684103898 learned_addr learned my addr 172.21.15.16:0/2684103898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16add9b640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 0x56484689be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16add9b640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 -- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 msgr2=0x56484689be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 0x56484689be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 -- 172.21.15.16:0/2684103898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 msgr2=0x5648468a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 -- 172.21.15.16:0/2684103898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484689c400 con 0x5648468893a0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16add9b640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 0x56484689be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16ae59c640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f16a0016000 tx=0x7f16a0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8ae1f1efc4ea6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f1695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a002a020 con 0x5648468893a0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8521+0+0 (secure 0 0 0) 0x7f16a0010b60 con 0x5648468893a0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a4004170 con 0x5648468893a0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16a0014040 con 0x5648468893a0 2024-09-12T01:12:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16b6d3d740 10 client.20205 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.993+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564846912750 con 0x5648468893a0 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4004170 con 0x5648468893a0 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f16a0025400 con 0x5648468893a0 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 10 client.20205 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 10 client.20205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 20 client.20205 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564846912c50 0x564846915070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 4 client.20205 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648467f5d00 con 0x564846912c50 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.994+0000 7f16add9b640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564846912c50 0x564846915070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.995+0000 7f1695ffb640 10 client.20205 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.995+0000 7f16add9b640 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564846912c50 0x564846915070 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752ec396cd9c8f18 server_cookie=787a9f94131ca09b in_seq=0 out_seq=0 2024-09-12T01:12:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f1695ffb640 1 -- 172.21.15.16:0/2684103898 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648467f5d00 con 0x564846912c50 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f1695ffb640 10 client.20205 handle_command_reply: tid=0 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 client.20205 shutdown 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 msgr2=0x56484689b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f16a0016000 tx=0x7f16a0007710 comp rx=0 tx=0).stop 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 shutdown_connections 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x564846912c50 0x564846915070 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564846898150 0x56484689be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f1695ffb640 0 client.20205 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5648468893a0 0x56484689b900 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 --2- 172.21.15.16:0/2684103898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564846888a80 0x5648468a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 >> 172.21.15.16:0/2684103898 conn(0x5648468850f0 msgr2=0x564846894ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 shutdown_connections 2024-09-12T01:12:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 1 -- 172.21.15.16:0/2684103898 wait complete. 2024-09-12T01:12:04.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.996+0000 7f16b6d3d740 20 client.20205 trim_cache size 0 max 16384 2024-09-12T01:12:04.012 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:04.013 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:12:04.013 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "01e430bf-8ceb-43a3-80eb-6ada3ad4e736", 2024-09-12T01:12:04.013 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:12:04.013 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 msgr2=0x7f16980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:04.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 0x7f16980a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f16a809b090 tx=0x7f16a8004830 comp rx=0 tx=0).stop 2024-09-12T01:12:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 msgr2=0x7f16b019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d750 tx=0x7f16a800dc10 comp rx=0 tx=0).stop 2024-09-12T01:12:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 shutdown_connections 2024-09-12T01:12:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f16980a2cf0 0x7f16980a51b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b0102ef0 0x7f16b019cd90 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16b0102570 0x7f16b0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 --2- 172.21.15.16:0/2073112158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16b0100e20 0x7f16b01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 >> 172.21.15.16:0/2073112158 conn(0x7f16b00fc7d0 msgr2=0x7f16b006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 shutdown_connections 2024-09-12T01:12:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:03.997+0000 7f16b4964640 1 -- 172.21.15.16:0/2073112158 wait complete. 2024-09-12T01:12:04.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:12:05.063 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:05.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: pgmap v499: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:05.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:12:05.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.482 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:12:05.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:05 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.566+0000 7f3b03185640 1 -- 172.21.15.16:0/2032942841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc10c170 msgr2=0x7f3afc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.566+0000 7f3b03185640 1 --2- 172.21.15.16:0/2032942841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc10c170 0x7f3afc10e610 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009ca0 tx=0x7f3af402a950 comp rx=0 tx=0).stop 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.566+0000 7f3b03185640 1 -- 172.21.15.16:0/2032942841 shutdown_connections 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.566+0000 7f3b03185640 1 --2- 172.21.15.16:0/2032942841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc10c170 0x7f3afc10e610 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.566+0000 7f3b03185640 1 --2- 172.21.15.16:0/2032942841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc103660 0x7f3afc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 --2- 172.21.15.16:0/2032942841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc102d40 0x7f3afc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- 172.21.15.16:0/2032942841 >> 172.21.15.16:0/2032942841 conn(0x7f3afc0fc7a0 msgr2=0x7f3afc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- 172.21.15.16:0/2032942841 shutdown_connections 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- 172.21.15.16:0/2032942841 wait complete. 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 Processor -- start 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- start start 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 0x7f3afc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc10c170 0x7f3afc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111f50 con 0x7f3afc102d40 2024-09-12T01:12:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc111dd0 con 0x7f3afc103660 2024-09-12T01:12:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.567+0000 7f3b03185640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1120d0 con 0x7f3afc10c170 2024-09-12T01:12:05.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45082/0 (socket says 172.21.15.16:45082) 2024-09-12T01:12:05.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 -- 172.21.15.16:0/1813032994 learned_addr learned my addr 172.21.15.16:0/1813032994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:05.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3af3fff640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 0x7f3afc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 -- 172.21.15.16:0/1813032994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc10c170 msgr2=0x7f3afc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b01722640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc10c170 0x7f3afc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc10c170 0x7f3afc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 -- 172.21.15.16:0/1813032994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 msgr2=0x7f3afc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 0x7f3afc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 -- 172.21.15.16:0/1813032994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc19d410 con 0x7f3afc102d40 2024-09-12T01:12:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3af3fff640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 0x7f3afc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b00f21640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3af800d950 tx=0x7f3af800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3af1ffb640 1 -- 172.21.15.16:0/1813032994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017070 con 0x7f3afc102d40 2024-09-12T01:12:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3af1ffb640 1 -- 172.21.15.16:0/1813032994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8009c30 con 0x7f3afc102d40 2024-09-12T01:12:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc19d700 con 0x7f3afc102d40 2024-09-12T01:12:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.568+0000 7f3af1ffb640 1 -- 172.21.15.16:0/1813032994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af800f6b0 con 0x7f3afc102d40 2024-09-12T01:12:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.569+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc1a4fe0 con 0x7f3afc102d40 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.570+0000 7f3af1ffb640 1 -- 172.21.15.16:0/1813032994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3af80109d0 con 0x7f3afc102d40 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 Processor -- start 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 -- start start 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3af1ffb640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 0x7f3ae40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31afde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be31a522e0 con 0x55be31aecca0 2024-09-12T01:12:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be319f9ce0 con 0x55be31afba90 2024-09-12T01:12:05.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b0555e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be31a4e2a0 con 0x55be31aec380 2024-09-12T01:12:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.571+0000 7f3b01722640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 0x7f3ae40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af1ffb640 1 -- 172.21.15.16:0/1813032994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f3af80cf220 con 0x7f3afc102d40 2024-09-12T01:12:05.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45094/0 (socket says 172.21.15.16:45094) 2024-09-12T01:12:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 -- 172.21.15.16:0/4121629484 learned_addr learned my addr 172.21.15.16:0/4121629484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3b00f21640 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3b01722640 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31afde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 -- 172.21.15.16:0/4121629484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 msgr2=0x55be31aec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 -- 172.21.15.16:0/4121629484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 msgr2=0x55be31afde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31afde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 -- 172.21.15.16:0/4121629484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be319de440 con 0x55be31aecca0 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3af3fff640 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00bc60 tx=0x7f3aec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2ecb8c01b2394d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3b01722640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 0x7f3ae40a51b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009430 tx=0x7f3af4009970 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3ae2ffd640 1 -- 172.21.15.16:0/4121629484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec005020 con 0x55be31aecca0 2024-09-12T01:12:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3ae2ffd640 1 -- 172.21.15.16:0/4121629484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec0047a0 con 0x55be31aecca0 2024-09-12T01:12:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.572+0000 7f3ae2ffd640 1 -- 172.21.15.16:0/4121629484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec0116a0 con 0x55be31aecca0 2024-09-12T01:12:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- 172.21.15.16:0/4121629484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 msgr2=0x55be31afb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00bc60 tx=0x7f3aec00f730 comp rx=0 tx=0).stop 2024-09-12T01:12:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- 172.21.15.16:0/4121629484 shutdown_connections 2024-09-12T01:12:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31afde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31afb550 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4121629484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- 172.21.15.16:0/4121629484 >> 172.21.15.16:0/4121629484 conn(0x55be31ae8a30 msgr2=0x55be31ae8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- 172.21.15.16:0/4121629484 shutdown_connections 2024-09-12T01:12:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- 172.21.15.16:0/4121629484 wait complete. 2024-09-12T01:12:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 -- start start 2024-09-12T01:12:05.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.573+0000 7f3b0555e740 1 Processor -- start 2024-09-12T01:12:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 0x55be31afb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aecca0 0x55be31af8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55be31afe660 con 0x55be31afba90 2024-09-12T01:12:05.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55be31a0d050 con 0x55be31aec380 2024-09-12T01:12:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55be31afe7e0 con 0x55be31aecca0 2024-09-12T01:12:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b0555e740 10 client.0 _ll_register_callbacks cb 0x55be31a4d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45110/0 (socket says 172.21.15.16:45110) 2024-09-12T01:12:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 -- 172.21.15.16:0/4208107130 learned_addr learned my addr 172.21.15.16:0/4208107130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3af3fff640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aecca0 0x55be31af8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b00f21640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 0x55be31afb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 -- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aecca0 msgr2=0x55be31af8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aecca0 0x55be31af8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 -- 172.21.15.16:0/4208107130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 msgr2=0x55be31afb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 0x55be31afb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 -- 172.21.15.16:0/4208107130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be317c33c0 con 0x55be31afba90 2024-09-12T01:12:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b00f21640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 0x55be31afb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3b01722640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3af4035000 tx=0x7f3af402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7a57b656401d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.574+0000 7f3adbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4050060 con 0x55be31afba90 2024-09-12T01:12:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f3af4004030 con 0x55be31afba90 2024-09-12T01:12:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec0047a0 con 0x55be31afba90 2024-09-12T01:12:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3af4034050 con 0x55be31afba90 2024-09-12T01:12:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 10 client.20217 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be31b0a070 con 0x55be31afba90 2024-09-12T01:12:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec0047a0 con 0x55be31afba90 2024-09-12T01:12:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3af404b8d0 con 0x55be31afba90 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 10 client.20217 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 10 client.20217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 20 client.20217 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b0b4d0 0x55be31b0d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 4 client.20217 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:05.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.575+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be31a5f850 con 0x55be31b0b4d0 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.576+0000 7f3b00f21640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b0b4d0 0x55be31b0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.576+0000 7f3b00f21640 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b0b4d0 0x55be31b0d950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1a98c2a3dbf5947 server_cookie=62c8be42476472a6 in_seq=0 out_seq=0 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.576+0000 7f3adbfff640 10 client.20217 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3adbfff640 1 -- 172.21.15.16:0/4208107130 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be31a5f850 con 0x55be31b0b4d0 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3adbfff640 10 client.20217 handle_command_reply: tid=0 2024-09-12T01:12:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 client.20217 shutdown 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 msgr2=0x55be31af8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3af4035000 tx=0x7f3af402a950 comp rx=0 tx=0).stop 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 shutdown_connections 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b0b4d0 0x55be31b0d950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31af8bf0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aecca0 0x55be31af8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3adbfff640 0 client.20217 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 --2- 172.21.15.16:0/4208107130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aec380 0x55be31afb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 >> 172.21.15.16:0/4208107130 conn(0x55be31ae8a30 msgr2=0x55be31aee790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 shutdown_connections 2024-09-12T01:12:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 1 -- 172.21.15.16:0/4208107130 wait complete. 2024-09-12T01:12:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.577+0000 7f3b0555e740 20 client.20217 trim_cache size 0 max 16384 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: pgmap v499: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.619 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:05 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 Processor -- start 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 -- start start 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31b07900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31b137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ab770 con 0x55be31aecca0 2024-09-12T01:12:05.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ad2b0 con 0x55be31afba90 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.594+0000 7f3b0555e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1abe40 con 0x55be31aec380 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45114/0 (socket says 172.21.15.16:45114) 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 -- 172.21.15.16:0/3233314355 learned_addr learned my addr 172.21.15.16:0/3233314355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3b00f21640 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31b07900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3b01722640 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31b137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 -- 172.21.15.16:0/3233314355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 msgr2=0x55be31b07900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31b07900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 -- 172.21.15.16:0/3233314355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 msgr2=0x55be31b137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31b137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 -- 172.21.15.16:0/3233314355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be31aed910 con 0x55be31aecca0 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3af3fff640 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b7e0 tx=0x7f3aec00bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf38eb6967205f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3ae27fc640 1 -- 172.21.15.16:0/3233314355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec02b070 con 0x55be31aecca0 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3ae27fc640 1 -- 172.21.15.16:0/3233314355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005520 con 0x55be31aecca0 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.595+0000 7f3ae27fc640 1 -- 172.21.15.16:0/3233314355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aec004dc0 con 0x55be31aecca0 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- 172.21.15.16:0/3233314355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 msgr2=0x55be31b13260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00b7e0 tx=0x7f3aec00bb60 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- 172.21.15.16:0/3233314355 shutdown_connections 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31afba90 0x55be31b137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31aecca0 0x55be31b13260 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3233314355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31b07900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- 172.21.15.16:0/3233314355 >> 172.21.15.16:0/3233314355 conn(0x55be31ae8a30 msgr2=0x55be31af3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- 172.21.15.16:0/3233314355 shutdown_connections 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- 172.21.15.16:0/3233314355 wait complete. 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 -- start start 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.596+0000 7f3b0555e740 1 Processor -- start 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aecca0 0x55be31affc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1acbe0 con 0x55be31afba90 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1ac510 con 0x55be31aecca0 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc1a9770 con 0x55be31aec380 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b0555e740 10 client.0 _ll_register_callbacks cb 0x7f3af4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45128/0 (socket says 172.21.15.16:45128) 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 -- 172.21.15.16:0/3507960097 learned_addr learned my addr 172.21.15.16:0/3507960097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b00f21640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3af3fff640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aecca0 0x55be31affc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 -- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 msgr2=0x55be31aff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 -- 172.21.15.16:0/3507960097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aecca0 msgr2=0x55be31affc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aecca0 0x55be31affc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.597+0000 7f3b01722640 1 -- 172.21.15.16:0/3507960097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be31b75230 con 0x55be31afba90 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b01722640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009bf0 tx=0x7f3af404b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cca68cb0645fed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4061060 con 0x55be31afba90 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f3af4004230 con 0x55be31afba90 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aec005520 con 0x55be31afba90 2024-09-12T01:12:05.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3af402afd0 con 0x55be31afba90 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 10 client.20223 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be31b76480 con 0x55be31afba90 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aec005520 con 0x55be31afba90 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3af403ca20 con 0x55be31afba90 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 10 client.20223 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 10 client.20223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 20 client.20223 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b76b40 0x55be31b78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 4 client.20223 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.598+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be31a59d60 con 0x55be31b76b40 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b00f21640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b76b40 0x55be31b78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b00f21640 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b76b40 0x55be31b78f30 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e472adbcb82b0f5 server_cookie=999b1d24c6c2c4cd in_seq=0 out_seq=0 2024-09-12T01:12:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3adbfff640 10 client.20223 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3adbfff640 1 -- 172.21.15.16:0/3507960097 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55be31a59d60 con 0x55be31b76b40 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3adbfff640 10 client.20223 handle_command_reply: tid=0 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 client.20223 shutdown 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 msgr2=0x55be31b001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009bf0 tx=0x7f3af404b6f0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 shutdown_connections 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55be31b76b40 0x55be31b78f30 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be31afba90 0x55be31b001b0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55be31aecca0 0x55be31affc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 --2- 172.21.15.16:0/3507960097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55be31aec380 0x55be31aff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3adbfff640 0 client.20223 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 >> 172.21.15.16:0/3507960097 conn(0x55be31ae8a30 msgr2=0x55be31af9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.599+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 shutdown_connections 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b0555e740 1 -- 172.21.15.16:0/3507960097 wait complete. 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b0555e740 20 client.20223 trim_cache size 0 max 16384 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:12:05.636 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 msgr2=0x7f3ae40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 0x7f3ae40a51b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009430 tx=0x7f3af4009970 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 msgr2=0x7f3afc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3af800d950 tx=0x7f3af800de10 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 shutdown_connections 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f3ae40a2cf0 0x7f3ae40a51b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3afc10c170 0x7f3afc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3afc103660 0x7f3afc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 --2- 172.21.15.16:0/1813032994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc102d40 0x7f3afc198460 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:05.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 >> 172.21.15.16:0/1813032994 conn(0x7f3afc0fc7a0 msgr2=0x7f3afc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 shutdown_connections 2024-09-12T01:12:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:05.600+0000 7f3b03185640 1 -- 172.21.15.16:0/1813032994 wait complete. 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: pgmap v499: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 12 op/s 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:05.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:05 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:12:06.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:12:06.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:12:06.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:06 smithi159 ceph-mon[35566]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:12:06.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:06 smithi016 ceph-mon[30972]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:12:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:06.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:06 smithi099 ceph-mon[35654]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T01:12:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- 172.21.15.16:0/1963634743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641036e0 msgr2=0x7fc864107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 --2- 172.21.15.16:0/1963634743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641036e0 0x7fc864107b90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc858009930 tx=0x7fc85802a420 comp rx=0 tx=0).stop 2024-09-12T01:12:07.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- 172.21.15.16:0/1963634743 shutdown_connections 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 --2- 172.21.15.16:0/1963634743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641082a0 0x7fc86410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 --2- 172.21.15.16:0/1963634743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641036e0 0x7fc864107b90 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 --2- 172.21.15.16:0/1963634743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 0x7fc864103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- 172.21.15.16:0/1963634743 >> 172.21.15.16:0/1963634743 conn(0x7fc8640fc7d0 msgr2=0x7fc8640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- 172.21.15.16:0/1963634743 shutdown_connections 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- 172.21.15.16:0/1963634743 wait complete. 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 Processor -- start 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.225+0000 7fc86b560640 1 -- start start 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 0x7fc864198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641036e0 0x7fc864198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc864112140 con 0x7fc8641082a0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc864111fc0 con 0x7fc8641036e0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc86b560640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8641122c0 con 0x7fc864102d30 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45140/0 (socket says 172.21.15.16:45140) 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 -- 172.21.15.16:0/266812676 learned_addr learned my addr 172.21.15.16:0/266812676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc868afb640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641036e0 0x7fc864198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc8692fc640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 0x7fc864198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 -- 172.21.15.16:0/266812676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 msgr2=0x7fc864198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 0x7fc864198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 -- 172.21.15.16:0/266812676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641036e0 msgr2=0x7fc864198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641036e0 0x7fc864198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 -- 172.21.15.16:0/266812676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86419d6b0 con 0x7fc8641082a0 2024-09-12T01:12:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc869afd640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc85400b970 tx=0x7fc85400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.226+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854019070 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.227+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85400cb40 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.227+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86419d9a0 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.227+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854007650 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.227+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8641a5280 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.227+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc864103190 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.229+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc85401c070 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.229+0000 7fc8627fc640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 0x7fc84c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.229+0000 7fc8692fc640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 0x7fc84c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.229+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fc8540cee10 con 0x7fc8641082a0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.230+0000 7fc8692fc640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 0x7fc84c0a51b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009890 tx=0x7fc85c008040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:07.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.233+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc85409b4c0 con 0x7fc8641082a0 2024-09-12T01:12:07.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:07 smithi016 ceph-mon[30972]: pgmap v501: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:07.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.445+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc86410a2b0 con 0x7fc8641082a0 2024-09-12T01:12:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.446+0000 7fc8627fc640 1 -- 172.21.15.16:0/266812676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7fc85409f7c0 con 0x7fc8641082a0 2024-09-12T01:12:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:12:07.446 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:12:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:12:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 msgr2=0x7fc84c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 0x7fc84c0a51b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009890 tx=0x7fc85c008040 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 msgr2=0x7fc86419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc85400b970 tx=0x7fc85400be30 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 shutdown_connections 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc84c0a2cf0 0x7fc84c0a51b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8641082a0 0x7fc86419cfd0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8641036e0 0x7fc864198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 --2- 172.21.15.16:0/266812676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc864102d30 0x7fc864198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.448+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 >> 172.21.15.16:0/266812676 conn(0x7fc8640fc7d0 msgr2=0x7fc864105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.449+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 shutdown_connections 2024-09-12T01:12:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:07.449+0000 7fc86b560640 1 -- 172.21.15.16:0/266812676 wait complete. 2024-09-12T01:12:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:07 smithi159 ceph-mon[35566]: pgmap v501: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:07.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:07 smithi099 ceph-mon[35654]: pgmap v501: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:08.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:12:08.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:08 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/266812676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:08.429 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:08.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:08 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/266812676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:08.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:08 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/266812676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 -- 172.21.15.16:0/1477804718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0106a80 msgr2=0x7fb6f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/1477804718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0106a80 0x7fb6f0106e60 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009cd0 tx=0x7fb6e002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 -- 172.21.15.16:0/1477804718 shutdown_connections 2024-09-12T01:12:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/1477804718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0101420 0x7fb6f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/1477804718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/1477804718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0106a80 0x7fb6f0106e60 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 -- 172.21.15.16:0/1477804718 >> 172.21.15.16:0/1477804718 conn(0x7fb6f00fc7a0 msgr2=0x7fb6f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 -- 172.21.15.16:0/1477804718 shutdown_connections 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.918+0000 7fb6f51b2640 1 -- 172.21.15.16:0/1477804718 wait complete. 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 Processor -- start 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 -- start start 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 0x7fb6f01a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0110e80 con 0x7fb6f0101420 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0110d00 con 0x7fb6f0106a80 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6f51b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0111000 con 0x7fb6f0100a80 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.919+0000 7fb6ee59c640 1 -- 172.21.15.16:0/3236806988 learned_addr learned my addr 172.21.15.16:0/3236806988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 0x7fb6f01a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 -- 172.21.15.16:0/3236806988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 msgr2=0x7fb6f0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 -- 172.21.15.16:0/3236806988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 msgr2=0x7fb6f01a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 0x7fb6f01a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 -- 172.21.15.16:0/3236806988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a48f0 con 0x7fb6f0101420 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 0x7fb6f01a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400e9f0 tx=0x7fb6e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6d7fff640 1 -- 172.21.15.16:0/3236806988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400cd80 con 0x7fb6f0101420 2024-09-12T01:12:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a4ac0 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6d7fff640 1 -- 172.21.15.16:0/3236806988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e4005500 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6d7fff640 1 -- 172.21.15.16:0/3236806988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4010690 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.920+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a4fd0 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.922+0000 7fb6d7fff640 1 -- 172.21.15.16:0/3236806988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb6e40046c0 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 Processor -- start 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 -- start start 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6d7fff640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 0x7fb6d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3c4370 0x5598fa3c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6d7fff640 1 -- 172.21.15.16:0/3236806988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fb6e40cf9e0 con 0x7fb6f0101420 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 0x7fb6d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598fa31f2f0 con 0x5598fa3bab40 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5598fa2c6cf0 con 0x5598fa3c4370 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6f758b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5598fa31b2b0 con 0x5598fa3b95d0 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58580/0 (socket says 172.21.15.16:58580) 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.923+0000 7fb6eed9d640 1 -- 172.21.15.16:0/2480777095 learned_addr learned my addr 172.21.15.16:0/2480777095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 -- 172.21.15.16:0/2480777095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 msgr2=0x5598fa3b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 -- 172.21.15.16:0/2480777095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3c4370 msgr2=0x5598fa3c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3c4370 0x5598fa3c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 -- 172.21.15.16:0/2480777095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fa2ab3a0 con 0x5598fa3bab40 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 0x7fb6d80a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009b20 tx=0x7fb6e8005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f80 tx=0x7fb6e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2bb9cd6eb99621 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6d57fa640 1 -- 172.21.15.16:0/2480777095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00318c0 con 0x5598fa3bab40 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6d57fa640 1 -- 172.21.15.16:0/2480777095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0036040 con 0x5598fa3bab40 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.924+0000 7fb6d57fa640 1 -- 172.21.15.16:0/2480777095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00044b0 con 0x5598fa3bab40 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- 172.21.15.16:0/2480777095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 msgr2=0x5598fa3b9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f80 tx=0x7fb6e0005f90 comp rx=0 tx=0).stop 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- 172.21.15.16:0/2480777095 shutdown_connections 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3c4370 0x5598fa3c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2480777095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3bab40 0x5598fa3b9090 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- 172.21.15.16:0/2480777095 >> 172.21.15.16:0/2480777095 conn(0x5598fa382310 msgr2=0x5598fa3b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- 172.21.15.16:0/2480777095 shutdown_connections 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- 172.21.15.16:0/2480777095 wait complete. 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 -- start start 2024-09-12T01:12:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 Processor -- start 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.925+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 0x5598fa3c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5598fa3c7010 con 0x5598fa3b95d0 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5598fa2da060 con 0x5598fa3bab40 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5598fa3c7190 con 0x5598fa3c4370 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 10 client.0 _ll_register_callbacks cb 0x5598fa02d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58594/0 (socket says 172.21.15.16:58594) 2024-09-12T01:12:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 -- 172.21.15.16:0/1348408475 learned_addr learned my addr 172.21.15.16:0/1348408475 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 0x5598fa3c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 -- 172.21.15.16:0/1348408475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 msgr2=0x5598fa3c14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 0x5598fa3c14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 -- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 msgr2=0x5598fa3c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 -- 172.21.15.16:0/1348408475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fa0903c0 con 0x5598fa3b95d0 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 0x5598fa3c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009430 tx=0x7fb6e0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1765f2d7f19f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0054060 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fb6e002e040 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.926+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e0036040 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6e0031670 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 10 client.20241 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598fa3d2a50 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0036040 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e0030070 con 0x5598fa3b95d0 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 10 client.20241 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 10 client.20241 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 20 client.20241 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa3d3da0 0x5598fa3d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 4 client.20241 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598fa32c860 con 0x5598fa3d3da0 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.927+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa3d3da0 0x5598fa3d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.928+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa3d3da0 0x5598fa3d6190 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=677e7b2e2399d7b3 server_cookie=d87d5d633b0a57d1 in_seq=0 out_seq=0 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.928+0000 7fb6d2ff5640 10 client.20241 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/1348408475 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598fa32c860 con 0x5598fa3d3da0 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6d2ff5640 10 client.20241 handle_command_reply: tid=0 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 client.20241 shutdown 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 msgr2=0x5598fa3c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009430 tx=0x7fb6e0009880 comp rx=0 tx=0).stop 2024-09-12T01:12:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 shutdown_connections 2024-09-12T01:12:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa3d3da0 0x5598fa3d6190 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3c4370 0x5598fa3c14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6d2ff5640 0 client.20241 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 --2- 172.21.15.16:0/1348408475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3b95d0 0x5598fa3c3ce0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 >> 172.21.15.16:0/1348408475 conn(0x5598fa382310 msgr2=0x5598fa3bb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 shutdown_connections 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 1 -- 172.21.15.16:0/1348408475 wait complete. 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.929+0000 7fb6f758b740 20 client.20241 trim_cache size 0 max 16384 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 Processor -- start 2024-09-12T01:12:08.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 -- start start 2024-09-12T01:12:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ad8f0 con 0x5598fa3c4370 2024-09-12T01:12:08.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01af430 con 0x5598fa3bab40 2024-09-12T01:12:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6f758b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01adfc0 con 0x5598fa3b95d0 2024-09-12T01:12:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.942+0000 7fb6ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6eed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58608/0 (socket says 172.21.15.16:58608) 2024-09-12T01:12:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 -- 172.21.15.16:0/4013235862 learned_addr learned my addr 172.21.15.16:0/4013235862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 -- 172.21.15.16:0/4013235862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 msgr2=0x5598fa3c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3c0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 -- 172.21.15.16:0/4013235862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 msgr2=0x5598fa3bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 -- 172.21.15.16:0/4013235862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598fa3bcba0 con 0x5598fa3c4370 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8010d60 tx=0x7fb6e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2c13429c9b994 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6d4ff9640 1 -- 172.21.15.16:0/4013235862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8020430 con 0x5598fa3c4370 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6d4ff9640 1 -- 172.21.15.16:0/4013235862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8020ab0 con 0x5598fa3c4370 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.943+0000 7fb6d4ff9640 1 -- 172.21.15.16:0/4013235862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8028aa0 con 0x5598fa3c4370 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- 172.21.15.16:0/4013235862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 msgr2=0x5598fa3bc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8010d60 tx=0x7fb6e8004730 comp rx=0 tx=0).stop 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- 172.21.15.16:0/4013235862 shutdown_connections 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa3bc660 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3bab40 0x5598fa3bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 --2- 172.21.15.16:0/4013235862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3b95d0 0x5598fa3c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- 172.21.15.16:0/4013235862 >> 172.21.15.16:0/4013235862 conn(0x5598fa382310 msgr2=0x5598fa3c0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- 172.21.15.16:0/4013235862 shutdown_connections 2024-09-12T01:12:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- 172.21.15.16:0/4013235862 wait complete. 2024-09-12T01:12:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 -- start start 2024-09-12T01:12:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.944+0000 7fb6f758b740 1 Processor -- start 2024-09-12T01:12:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3b95d0 0x5598fa3d6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 0x5598fa3d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01aed60 con 0x5598fa3c4370 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ae690 con 0x5598fa3b95d0 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01ab8f0 con 0x5598fa3bab40 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6f758b740 10 client.0 _ll_register_callbacks cb 0x7fb6e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58618/0 (socket says 172.21.15.16:58618) 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 -- 172.21.15.16:0/2661834682 learned_addr learned my addr 172.21.15.16:0/2661834682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 0x5598fa3d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3b95d0 0x5598fa3d6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 -- 172.21.15.16:0/2661834682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 msgr2=0x5598fa3d6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 0x5598fa3d6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 -- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3b95d0 msgr2=0x5598fa3d6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3b95d0 0x5598fa3d6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 -- 172.21.15.16:0/2661834682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598fa436d40 con 0x5598fa3c4370 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ee59c640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 0x5598fa3d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6ef59e640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005b60 tx=0x7fb6e8010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3306e6fa4831ab8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.945+0000 7fb6d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8037060 con 0x5598fa3c4370 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fb6e8020640 con 0x5598fa3c4370 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e8020ab0 con 0x5598fa3c4370 2024-09-12T01:12:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb6e8020840 con 0x5598fa3c4370 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 10 client.20247 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598fa43e110 con 0x5598fa3c4370 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e8020ab0 con 0x5598fa3c4370 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb6e800f040 con 0x5598fa3c4370 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 10 client.20247 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 10 client.20247 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 20 client.20247 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa43e720 0x5598fa440b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 4 client.20247 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.946+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598fa326d70 con 0x5598fa43e720 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa43e720 0x5598fa440b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6eed9d640 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa43e720 0x5598fa440b40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30152fdbbd9ddc91 server_cookie=d64e9130270535d6 in_seq=0 out_seq=0 2024-09-12T01:12:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6d2ff5640 10 client.20247 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6d2ff5640 1 -- 172.21.15.16:0/2661834682 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598fa326d70 con 0x5598fa43e720 2024-09-12T01:12:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6d2ff5640 10 client.20247 handle_command_reply: tid=0 2024-09-12T01:12:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 client.20247 shutdown 2024-09-12T01:12:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 msgr2=0x5598fa4366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8005b60 tx=0x7fb6e8010fc0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 shutdown_connections 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x5598fa43e720 0x5598fa440b40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598fa3c4370 0x5598fa4366a0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5598fa3bab40 0x5598fa3d6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 --2- 172.21.15.16:0/2661834682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5598fa3b95d0 0x5598fa3d6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6d2ff5640 0 client.20247 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 >> 172.21.15.16:0/2661834682 conn(0x5598fa382310 msgr2=0x5598fa3c0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 shutdown_connections 2024-09-12T01:12:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.947+0000 7fb6f758b740 1 -- 172.21.15.16:0/2661834682 wait complete. 2024-09-12T01:12:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f758b740 20 client.20247 trim_cache size 0 max 16384 2024-09-12T01:12:08.971 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:08.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 msgr2=0x7fb6d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 0x7fb6d80a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009b20 tx=0x7fb6e8005f90 comp rx=0 tx=0).stop 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 msgr2=0x7fb6f0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400e9f0 tx=0x7fb6e400edf0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 shutdown_connections 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb6d80a2cf0 0x7fb6d80a51b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6f0106a80 0x7fb6f01a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6f0101420 0x7fb6f0199870 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 --2- 172.21.15.16:0/3236806988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6f0100a80 0x7fb6f0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.948+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 >> 172.21.15.16:0/3236806988 conn(0x7fb6f00fc7a0 msgr2=0x7fb6f00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.949+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 shutdown_connections 2024-09-12T01:12:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:08.949+0000 7fb6f51b2640 1 -- 172.21.15.16:0/3236806988 wait complete. 2024-09-12T01:12:09.584 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:09 smithi016 ceph-mon[30972]: pgmap v502: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:09 smithi159 ceph-mon[35566]: pgmap v502: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:09.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:09 smithi099 ceph-mon[35654]: pgmap v502: 97 pgs: 97 active+clean; 29 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 136 KiB/s wr, 36 op/s 2024-09-12T01:12:09.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:12:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:10.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.527+0000 7fd55c883640 1 -- 172.21.15.16:0/889347743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558103660 msgr2=0x7fd55810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.527+0000 7fd55c883640 1 --2- 172.21.15.16:0/889347743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558103660 0x7fd55810bb50 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009900 tx=0x7fd54c02a480 comp rx=0 tx=0).stop 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 -- 172.21.15.16:0/889347743 shutdown_connections 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 --2- 172.21.15.16:0/889347743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd55810c170 0x7fd55810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 --2- 172.21.15.16:0/889347743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558103660 0x7fd55810bb50 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 --2- 172.21.15.16:0/889347743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd558102d40 0x7fd558103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 -- 172.21.15.16:0/889347743 >> 172.21.15.16:0/889347743 conn(0x7fd5580fc7a0 msgr2=0x7fd5580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 -- 172.21.15.16:0/889347743 shutdown_connections 2024-09-12T01:12:10.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.528+0000 7fd55c883640 1 -- 172.21.15.16:0/889347743 wait complete. 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 Processor -- start 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 -- start start 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 0x7fd558100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 0x7fd558101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111fd0 con 0x7fd558102d40 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111e50 con 0x7fd558103660 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55c883640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd558112150 con 0x7fd55810c170 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58632/0 (socket says 172.21.15.16:58632) 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 -- 172.21.15.16:0/2870943810 learned_addr learned my addr 172.21.15.16:0/2870943810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd556d9d640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 0x7fd558101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd555d9b640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 0x7fd558100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 -- 172.21.15.16:0/2870943810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 msgr2=0x7fd558101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 0x7fd558101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 -- 172.21.15.16:0/2870943810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 msgr2=0x7fd558100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 0x7fd558100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd55659c640 1 -- 172.21.15.16:0/2870943810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5581a6960 con 0x7fd558102d40 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd555d9b640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 0x7fd558100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.529+0000 7fd556d9d640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 0x7fd558101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd55659c640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd54800ea80 tx=0x7fd54800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd5477fe640 1 -- 172.21.15.16:0/2870943810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54800ce40 con 0x7fd558102d40 2024-09-12T01:12:10.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5581a6c50 con 0x7fd558102d40 2024-09-12T01:12:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd5477fe640 1 -- 172.21.15.16:0/2870943810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd548005020 con 0x7fd558102d40 2024-09-12T01:12:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd5477fe640 1 -- 172.21.15.16:0/2870943810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548011760 con 0x7fd558102d40 2024-09-12T01:12:10.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.530+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5581a7140 con 0x7fd558102d40 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.532+0000 7fd5477fe640 1 -- 172.21.15.16:0/2870943810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd5480051d0 con 0x7fd558102d40 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.532+0000 7fd55ec5c740 1 Processor -- start 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 -- start start 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd5477fe640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 0x7fd5400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 0x5636a8da7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636a8d002f0 con 0x5636a8d9bb40 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636a8ca7cf0 con 0x5636a8da5370 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636a8cfc2b0 con 0x5636a8d9a5d0 2024-09-12T01:12:10.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd556d9d640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 0x7fd5400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58646/0 (socket says 172.21.15.16:58646) 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd5477fe640 1 -- 172.21.15.16:0/2870943810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fd5480cfbb0 con 0x7fd558102d40 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd55659c640 1 -- 172.21.15.16:0/3292948180 learned_addr learned my addr 172.21.15.16:0/3292948180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd555d9b640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.533+0000 7fd556d9d640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 0x5636a8da7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 -- 172.21.15.16:0/3292948180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 msgr2=0x5636a8d9aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 -- 172.21.15.16:0/3292948180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 msgr2=0x5636a8da7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 0x5636a8da7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 -- 172.21.15.16:0/3292948180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a8c8c3a0 con 0x5636a8d9bb40 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd556d9d640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 0x5636a8da7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd556d9d640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 0x7fd5400a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d70 tx=0x7fd550014660 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd55659c640 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd54800efd0 tx=0x7fd5480914d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe11f8f227413 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd544ff9640 1 -- 172.21.15.16:0/3292948180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809c740 con 0x5636a8d9bb40 2024-09-12T01:12:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd544ff9640 1 -- 172.21.15.16:0/3292948180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54809c040 con 0x5636a8d9bb40 2024-09-12T01:12:10.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.534+0000 7fd544ff9640 1 -- 172.21.15.16:0/3292948180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480b5600 con 0x5636a8d9bb40 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- 172.21.15.16:0/3292948180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 msgr2=0x5636a8d9a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd54800efd0 tx=0x7fd5480914d0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- 172.21.15.16:0/3292948180 shutdown_connections 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8da5370 0x5636a8da7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/3292948180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9bb40 0x5636a8d9a090 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- 172.21.15.16:0/3292948180 >> 172.21.15.16:0/3292948180 conn(0x5636a8d63310 msgr2=0x5636a8d96c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- 172.21.15.16:0/3292948180 shutdown_connections 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- 172.21.15.16:0/3292948180 wait complete. 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 -- start start 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.535+0000 7fd55ec5c740 1 Processor -- start 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8da2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8da5370 0x5636a8da27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5636a8da8070 con 0x5636a8d9a5d0 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5636a8cbb060 con 0x5636a8d9bb40 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636a8da81f0 con 0x5636a8da5370 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55ec5c740 10 client.0 _ll_register_callbacks cb 0x5636a8a0e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2024-09-12T01:12:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 -- 172.21.15.16:0/433209372 learned_addr learned my addr 172.21.15.16:0/433209372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd555d9b640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8da2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 -- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8da5370 msgr2=0x5636a8da27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8da5370 0x5636a8da27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 -- 172.21.15.16:0/433209372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 msgr2=0x5636a8da2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8da2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 -- 172.21.15.16:0/433209372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a8a713c0 con 0x5636a8d9a5d0 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd555d9b640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8da2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd55659c640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd5480a4780 tx=0x7fd54800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75de5cb4dbefcba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd53dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.536+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548005540 con 0x5636a8d9a5d0 2024-09-12T01:12:10.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fd5480cd4f0 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54809c040 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd548096050 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 10 client.20259 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636a8db3b50 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54809c040 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd54800f040 con 0x5636a8d9a5d0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 10 client.20259 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 10 client.20259 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 20 client.20259 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8db4ea0 0x5636a8db7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 4 client.20259 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.537+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636a8d0d860 con 0x5636a8db4ea0 2024-09-12T01:12:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.538+0000 7fd555d9b640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8db4ea0 0x5636a8db7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.538+0000 7fd53dffb640 10 client.20259 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.538+0000 7fd555d9b640 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8db4ea0 0x5636a8db7290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1f882851ca28d30 server_cookie=a44f5c681b3ca3cb in_seq=0 out_seq=0 2024-09-12T01:12:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd53dffb640 1 -- 172.21.15.16:0/433209372 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636a8d0d860 con 0x5636a8db4ea0 2024-09-12T01:12:10.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd53dffb640 10 client.20259 handle_command_reply: tid=0 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 client.20259 shutdown 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 msgr2=0x5636a8da4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd5480a4780 tx=0x7fd54800c8a0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 shutdown_connections 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8da5370 0x5636a8da27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8da2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8db4ea0 0x5636a8db7290 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/433209372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8d9a5d0 0x5636a8da4da0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 >> 172.21.15.16:0/433209372 conn(0x5636a8d63310 msgr2=0x5636a8d9c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd53dffb640 0 client.20259 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 shutdown_connections 2024-09-12T01:12:10.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 1 -- 172.21.15.16:0/433209372 wait complete. 2024-09-12T01:12:10.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.539+0000 7fd55ec5c740 20 client.20259 trim_cache size 0 max 16384 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 Processor -- start 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 -- start start 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 0x5636a8d9d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581afc00 con 0x5636a8da5370 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b1740 con 0x5636a8d9a5d0 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.552+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b02d0 con 0x5636a8d9bb40 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58652/0 (socket says 172.21.15.16:58652) 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 -- 172.21.15.16:0/1806360263 learned_addr learned my addr 172.21.15.16:0/1806360263 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd555d9b640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 0x5636a8d9d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd55659c640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 -- 172.21.15.16:0/1806360263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 msgr2=0x5636a8d9d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 0x5636a8d9d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 -- 172.21.15.16:0/1806360263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9a5d0 msgr2=0x5636a8d9d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 -- 172.21.15.16:0/1806360263 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a8d95310 con 0x5636a8da5370 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd555d9b640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 0x5636a8d9d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd556d9d640 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd550009ea0 tx=0x7fd550017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd411e23a9ef37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd53ffff640 1 -- 172.21.15.16:0/1806360263 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550022410 con 0x5636a8da5370 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd53ffff640 1 -- 172.21.15.16:0/1806360263 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550027070 con 0x5636a8da5370 2024-09-12T01:12:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.553+0000 7fd53ffff640 1 -- 172.21.15.16:0/1806360263 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550016870 con 0x5636a8da5370 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- 172.21.15.16:0/1806360263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 msgr2=0x5636a8db9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd550009ea0 tx=0x7fd550017000 comp rx=0 tx=0).stop 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- 172.21.15.16:0/1806360263 shutdown_connections 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db9ed0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9bb40 0x5636a8d9d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/1806360263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9a5d0 0x5636a8d9d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- 172.21.15.16:0/1806360263 >> 172.21.15.16:0/1806360263 conn(0x5636a8d63310 msgr2=0x5636a8da1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- 172.21.15.16:0/1806360263 shutdown_connections 2024-09-12T01:12:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- 172.21.15.16:0/1806360263 wait complete. 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 -- start start 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.554+0000 7fd55ec5c740 1 Processor -- start 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8dba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8dba7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b1070 con 0x5636a8da5370 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b09a0 con 0x5636a8d9bb40 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581adc00 con 0x5636a8d9a5d0 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55ec5c740 10 client.0 _ll_register_callbacks cb 0x7fd55806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58656/0 (socket says 172.21.15.16:58656) 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 -- 172.21.15.16:0/4178886032 learned_addr learned my addr 172.21.15.16:0/4178886032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd555d9b640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8dba7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd55659c640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8dba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 -- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 msgr2=0x5636a8dba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8dba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 -- 172.21.15.16:0/4178886032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 msgr2=0x5636a8dba7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8dba7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 -- 172.21.15.16:0/4178886032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a8db8040 con 0x5636a8da5370 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd556d9d640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd550014b60 tx=0x7fd550022c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f54349c054567a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.555+0000 7fd53dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500059c0 con 0x5636a8da5370 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7fd550005b60 con 0x5636a8da5370 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550027070 con 0x5636a8da5370 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd550005d60 con 0x5636a8da5370 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 10 client.20265 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636a8e1f3b0 con 0x5636a8da5370 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550027070 con 0x5636a8da5370 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd55003a070 con 0x5636a8da5370 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 10 client.20265 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 10 client.20265 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 20 client.20265 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8e1f9d0 0x5636a8e21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 4 client.20265 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.556+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636a8d07d70 con 0x5636a8e1f9d0 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55659c640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8e1f9d0 0x5636a8e21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55659c640 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8e1f9d0 0x5636a8e21df0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fc6db2a31a2de81 server_cookie=26b93433f7405e0d in_seq=0 out_seq=0 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd53dffb640 10 client.20265 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd53dffb640 1 -- 172.21.15.16:0/4178886032 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636a8d07d70 con 0x5636a8e1f9d0 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd53dffb640 10 client.20265 handle_command_reply: tid=0 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 client.20265 shutdown 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 msgr2=0x5636a8db7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd550014b60 tx=0x7fd550022c50 comp rx=0 tx=0).stop 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 shutdown_connections 2024-09-12T01:12:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x5636a8e1f9d0 0x5636a8e21df0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636a8da5370 0x5636a8db7810 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5636a8d9bb40 0x5636a8dba7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 --2- 172.21.15.16:0/4178886032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a8d9a5d0 0x5636a8dba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 >> 172.21.15.16:0/4178886032 conn(0x5636a8d63310 msgr2=0x5636a8db0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.557+0000 7fd53dffb640 0 client.20265 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 shutdown_connections 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55ec5c740 1 -- 172.21.15.16:0/4178886032 wait complete. 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55ec5c740 20 client.20265 trim_cache size 0 max 16384 2024-09-12T01:12:10.563 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 msgr2=0x7fd5400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 0x7fd5400a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d70 tx=0x7fd550014660 comp rx=0 tx=0).stop 2024-09-12T01:12:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 msgr2=0x7fd5581009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd54800ea80 tx=0x7fd54800ef40 comp rx=0 tx=0).stop 2024-09-12T01:12:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 shutdown_connections 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fd5400a2cf0 0x7fd5400a51b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd55810c170 0x7fd558101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd558103660 0x7fd558100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 --2- 172.21.15.16:0/2870943810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd558102d40 0x7fd5581009a0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 >> 172.21.15.16:0/2870943810 conn(0x7fd5580fc7a0 msgr2=0x7fd5580fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 shutdown_connections 2024-09-12T01:12:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:10.558+0000 7fd55c883640 1 -- 172.21.15.16:0/2870943810 wait complete. 2024-09-12T01:12:11.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:12:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:11 smithi159 ceph-mon[35566]: pgmap v503: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:11.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:11 smithi016 ceph-mon[30972]: pgmap v503: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:11.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:11 smithi099 ceph-mon[35654]: pgmap v503: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- 172.21.15.16:0/3686495464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 msgr2=0x7f636810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 --2- 172.21.15.16:0/3686495464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f636810d950 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6360009cd0 tx=0x7f636002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- 172.21.15.16:0/3686495464 shutdown_connections 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 --2- 172.21.15.16:0/3686495464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f636810d950 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 --2- 172.21.15.16:0/3686495464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368100a80 0x7f6368100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 --2- 172.21.15.16:0/3686495464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368106a80 0x7f6368106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- 172.21.15.16:0/3686495464 >> 172.21.15.16:0/3686495464 conn(0x7f63680fc7a0 msgr2=0x7f63680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- 172.21.15.16:0/3686495464 shutdown_connections 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- 172.21.15.16:0/3686495464 wait complete. 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 Processor -- start 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.045+0000 7f636d3f3640 1 -- start start 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368100a80 0x7f6368196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 0x7f63681979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6368110e80 con 0x7f6368101420 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6368110d00 con 0x7f6368100a80 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f636d3f3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6368111000 con 0x7f6368106a80 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58670/0 (socket says 172.21.15.16:58670) 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 -- 172.21.15.16:0/769663013 learned_addr learned my addr 172.21.15.16:0/769663013 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f6366ffd640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368100a80 0x7f6368196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63677fe640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 0x7f63681979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 -- 172.21.15.16:0/769663013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 msgr2=0x7f63681979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 0x7f63681979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 -- 172.21.15.16:0/769663013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368100a80 msgr2=0x7f6368196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368100a80 0x7f6368196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 -- 172.21.15.16:0/769663013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681a67e0 con 0x7f6368101420 2024-09-12T01:12:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63677fe640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 0x7f63681979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.046+0000 7f63667fc640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f635000c890 tx=0x7f635000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.047+0000 7f635ffff640 1 -- 172.21.15.16:0/769663013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350004220 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.047+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681a6ad0 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.047+0000 7f635ffff640 1 -- 172.21.15.16:0/769663013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63500043c0 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.047+0000 7f635ffff640 1 -- 172.21.15.16:0/769663013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350004a00 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.047+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a7060 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.049+0000 7f635ffff640 1 -- 172.21.15.16:0/769663013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63500079b0 con 0x7f6368101420 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.049+0000 7f636f7cc740 1 Processor -- start 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.049+0000 7f636f7cc740 1 -- start start 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847caaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55847ca122f0 con 0x55847caadb40 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55847c9b9cf0 con 0x55847cab7370 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f636f7cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847ca0e2b0 con 0x55847caac5d0 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58684/0 (socket says 172.21.15.16:58684) 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 -- 172.21.15.16:0/2756212585 learned_addr learned my addr 172.21.15.16:0/2756212585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f6366ffd640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847caaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63667fc640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f635ffff640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 0x7f63540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 -- 172.21.15.16:0/2756212585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 msgr2=0x55847caaca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847caaca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 -- 172.21.15.16:0/2756212585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 msgr2=0x55847cab9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 -- 172.21.15.16:0/2756212585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847c99e3a0 con 0x55847caadb40 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f6366ffd640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847caaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.051+0000 7f635ffff640 1 -- 172.21.15.16:0/769663013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f63500cf150 con 0x7f6368101420 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.050+0000 7f63677fe640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 0x7f63540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.051+0000 7f63677fe640 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6360031650 tx=0x7f6360031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2aa653ed72d6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.051+0000 7f635d7fa640 1 -- 172.21.15.16:0/2756212585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360003e40 con 0x55847caadb40 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.051+0000 7f635d7fa640 1 -- 172.21.15.16:0/2756212585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360036040 con 0x55847caadb40 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.051+0000 7f635d7fa640 1 -- 172.21.15.16:0/2756212585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360030920 con 0x55847caadb40 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- 172.21.15.16:0/2756212585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 msgr2=0x55847caac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f63677fe640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 0x7f63540a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f636004b000 tx=0x7f6360048000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f6360031650 tx=0x7f6360031680 comp rx=0 tx=0).stop 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- 172.21.15.16:0/2756212585 shutdown_connections 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847caaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2756212585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847caac090 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- 172.21.15.16:0/2756212585 >> 172.21.15.16:0/2756212585 conn(0x55847ca75310 msgr2=0x55847caa8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- 172.21.15.16:0/2756212585 shutdown_connections 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- 172.21.15.16:0/2756212585 wait complete. 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- start start 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 Processor -- start 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cab3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55847caba070 con 0x55847caac5d0 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55847c9cd060 con 0x55847cab7370 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.052+0000 7f636f7cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847caba1f0 con 0x55847caadb40 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f636f7cc740 10 client.0 _ll_register_callbacks cb 0x55847c7208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58692/0 (socket says 172.21.15.16:58692) 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 -- 172.21.15.16:0/1751744853 learned_addr learned my addr 172.21.15.16:0/1751744853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f63677fe640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f63667fc640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cab3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 -- 172.21.15.16:0/1751744853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 msgr2=0x55847cab3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cab3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 -- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 msgr2=0x55847cab45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 -- 172.21.15.16:0/1751744853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847c7833c0 con 0x55847caac5d0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f6358010a50 tx=0x7f6358010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8636132d777428dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f634e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635801e060 con 0x55847caac5d0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f635800fb10 con 0x55847caac5d0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360036040 con 0x55847caac5d0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f635800fd10 con 0x55847caac5d0 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f636f7cc740 10 client.20277 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.053+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55847cac5b30 con 0x55847caac5d0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360036040 con 0x55847caac5d0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f635800e570 con 0x55847caac5d0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 10 client.20277 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 10 client.20277 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 20 client.20277 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cac6ff0 0x55847cac93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 4 client.20277 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847ca1f860 con 0x55847cac6ff0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.054+0000 7f63667fc640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cac6ff0 0x55847cac93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f634e7fc640 10 client.20277 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f63667fc640 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cac6ff0 0x55847cac93e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a603f68a0450995f server_cookie=3b67cd7eca2a23f2 in_seq=0 out_seq=0 2024-09-12T01:12:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f634e7fc640 1 -- 172.21.15.16:0/1751744853 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55847ca1f860 con 0x55847cac6ff0 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f634e7fc640 10 client.20277 handle_command_reply: tid=0 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 client.20277 shutdown 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 msgr2=0x55847cab6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f6358010a50 tx=0x7f6358010f10 comp rx=0 tx=0).stop 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 shutdown_connections 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cac6ff0 0x55847cac93e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cab45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cab3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f634e7fc640 0 client.20277 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:12.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1751744853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847cab6d30 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.055+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 >> 172.21.15.16:0/1751744853 conn(0x55847ca75310 msgr2=0x55847caae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.056+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 shutdown_connections 2024-09-12T01:12:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.056+0000 7f636f7cc740 1 -- 172.21.15.16:0/1751744853 wait complete. 2024-09-12T01:12:12.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.056+0000 7f636f7cc740 20 client.20277 trim_cache size 0 max 16384 2024-09-12T01:12:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 Processor -- start 2024-09-12T01:12:12.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 -- start start 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cac3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cacec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63681afb20 con 0x55847caac5d0 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b1660 con 0x55847cab7370 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f636f7cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b01f0 con 0x55847caadb40 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58700/0 (socket says 172.21.15.16:58700) 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 -- 172.21.15.16:0/1118676178 learned_addr learned my addr 172.21.15.16:0/1118676178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f63677fe640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cacec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 -- 172.21.15.16:0/1118676178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 msgr2=0x55847cac3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f63667fc640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cac3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cac3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 -- 172.21.15.16:0/1118676178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 msgr2=0x55847cacec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cacec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f6366ffd640 1 -- 172.21.15.16:0/1118676178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847cab4940 con 0x55847caac5d0 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.069+0000 7f63677fe640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cacec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f6366ffd640 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f635800f010 tx=0x7f6358005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131fbc68c496869f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f635cff9640 1 -- 172.21.15.16:0/1118676178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635800fb10 con 0x55847caac5d0 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f635cff9640 1 -- 172.21.15.16:0/1118676178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635800ec00 con 0x55847caac5d0 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f635cff9640 1 -- 172.21.15.16:0/1118676178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63580048c0 con 0x55847caac5d0 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f636f7cc740 1 -- 172.21.15.16:0/1118676178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 msgr2=0x55847caa8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.070+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f635800f010 tx=0x7f6358005ec0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- 172.21.15.16:0/1118676178 shutdown_connections 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cacec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caadb40 0x55847cac3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- 172.21.15.16:0/1118676178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caac5d0 0x55847caa8e10 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- 172.21.15.16:0/1118676178 >> 172.21.15.16:0/1118676178 conn(0x55847ca75310 msgr2=0x55847caa8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- 172.21.15.16:0/1118676178 shutdown_connections 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- 172.21.15.16:0/1118676178 wait complete. 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- start start 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 Processor -- start 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847cabb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cabbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b0f90 con 0x55847caadb40 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63681b08c0 con 0x55847cab7370 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63681adb20 con 0x55847caac5d0 2024-09-12T01:12:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f636f7cc740 10 client.0 _ll_register_callbacks cb 0x7f635800a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f63667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f63667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58712/0 (socket says 172.21.15.16:58712) 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.071+0000 7f63667fc640 1 -- 172.21.15.16:0/2906716020 learned_addr learned my addr 172.21.15.16:0/2906716020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f6366ffd640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847cabb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63677fe640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cabbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 -- 172.21.15.16:0/2906716020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 msgr2=0x55847cabb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847cabb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 -- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 msgr2=0x55847cabbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cabbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 -- 172.21.15.16:0/2906716020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847cb30c60 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f63667fc640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f63500cc8d0 tx=0x7f63500977c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bb4cccca732e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f634e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635001d040 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8565+0+0 (secure 0 0 0) 0x7f63500ccaf0 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f635800ec00 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f635009b180 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f636f7cc740 10 client.20283 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55847cb31ee0 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.072+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f635800ec00 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6350097cb0 con 0x55847caadb40 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 10 client.20283 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 10 client.20283 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 20 client.20283 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cb325c0 0x55847cb349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 4 client.20283 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55847ca19d70 con 0x55847cb325c0 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f6366ffd640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cb325c0 0x55847cb349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f6366ffd640 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cb325c0 0x55847cb349e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=165b55ee33603b07 server_cookie=2c724a3ed508641a in_seq=0 out_seq=0 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f634e7fc640 10 client.20283 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f634e7fc640 1 -- 172.21.15.16:0/2906716020 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55847ca19d70 con 0x55847cb325c0 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f634e7fc640 10 client.20283 handle_command_reply: tid=0 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 1 client.20283 shutdown 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.073+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 msgr2=0x55847cabb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f63500cc8d0 tx=0x7f63500977c0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 shutdown_connections 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x55847cb325c0 0x55847cb349e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55847cab7370 0x55847cabbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55847caadb40 0x55847cabb700 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f634e7fc640 0 client.20283 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 --2- 172.21.15.16:0/2906716020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847caac5d0 0x55847cabb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 >> 172.21.15.16:0/2906716020 conn(0x55847ca75310 msgr2=0x55847caae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 shutdown_connections 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 1 -- 172.21.15.16:0/2906716020 wait complete. 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636f7cc740 20 client.20283 trim_cache size 0 max 16384 2024-09-12T01:12:12.083 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 msgr2=0x7f63540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 0x7f63540a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f636004b000 tx=0x7f6360048000 comp rx=0 tx=0).stop 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 msgr2=0x7f63681974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f635000c890 tx=0x7f635000cc90 comp rx=0 tx=0).stop 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 shutdown_connections 2024-09-12T01:12:12.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f63540a2cf0 0x7f63540a51b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6368106a80 0x7f63681979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6368101420 0x7f63681974b0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 --2- 172.21.15.16:0/769663013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6368100a80 0x7f6368196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.074+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 >> 172.21.15.16:0/769663013 conn(0x7f63680fc7a0 msgr2=0x7f63680fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.075+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 shutdown_connections 2024-09-12T01:12:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:12.075+0000 7f636d3f3640 1 -- 172.21.15.16:0/769663013 wait complete. 2024-09-12T01:12:12.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:12:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:13 smithi159 ceph-mon[35566]: pgmap v504: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:13 smithi159 ceph-mon[35566]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T01:12:13.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:13 smithi016 ceph-mon[30972]: pgmap v504: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:13.629 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:13 smithi016 ceph-mon[30972]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- 172.21.15.16:0/3008607300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4101df0 msgr2=0x7f9cd4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/3008607300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4101df0 0x7f9cd4105920 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009900 tx=0x7f9cc802a410 comp rx=0 tx=0).stop 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- 172.21.15.16:0/3008607300 shutdown_connections 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/3008607300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4106050 0x7f9cd410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/3008607300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4101df0 0x7f9cd4105920 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/3008607300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101440 0x7f9cd4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- 172.21.15.16:0/3008607300 >> 172.21.15.16:0/3008607300 conn(0x7f9cd4077210 msgr2=0x7f9cd40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- 172.21.15.16:0/3008607300 shutdown_connections 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- 172.21.15.16:0/3008607300 wait complete. 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 Processor -- start 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- start start 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4101440 0x7f9cd4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101df0 0x7f9cd4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41121d0 con 0x7f9cd4106050 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4112050 con 0x7f9cd4101df0 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.712+0000 7f9cdadf4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4112350 con 0x7f9cd4101440 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58734/0 (socket says 172.21.15.16:58734) 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 -- 172.21.15.16:0/1495409865 learned_addr learned my addr 172.21.15.16:0/1495409865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4101440 0x7f9cd4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101df0 0x7f9cd4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 -- 172.21.15.16:0/1495409865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4101440 msgr2=0x7f9cd4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4101440 0x7f9cd4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 -- 172.21.15.16:0/1495409865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101df0 msgr2=0x7f9cd4198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101df0 0x7f9cd4198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 -- 172.21.15.16:0/1495409865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd419d620 con 0x7f9cd4106050 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd9391640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00eaa0 tx=0x7f9ccc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd1ffb640 1 -- 172.21.15.16:0/1495409865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00cdb0 con 0x7f9cd4106050 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd1ffb640 1 -- 172.21.15.16:0/1495409865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc005020 con 0x7f9cd4106050 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd419d910 con 0x7f9cd4106050 2024-09-12T01:12:13.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.713+0000 7f9cd1ffb640 1 -- 172.21.15.16:0/1495409865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0116b0 con 0x7f9cd4106050 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.714+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a51f0 con 0x7f9cd4106050 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.716+0000 7f9cd1ffb640 1 -- 172.21.15.16:0/1495409865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9ccc005560 con 0x7f9cd4106050 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.716+0000 7f9cdd1cd740 1 Processor -- start 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.716+0000 7f9cdd1cd740 1 -- start start 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.716+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f809882090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 0x55f809893b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f8097e82f0 con 0x55f809891780 2024-09-12T01:12:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f80978fcf0 con 0x55f8098825d0 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd1ffb640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 0x7f9cbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f8097e42b0 con 0x55f809883b40 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 0x55f809893b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd9391640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 0x7f9cbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:43862/0 (socket says 172.21.15.16:43862) 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 -- 172.21.15.16:0/3132561764 learned_addr learned my addr 172.21.15.16:0/3132561764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 0x55f809893b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58740/0 (socket says 172.21.15.16:58740) 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd1ffb640 1 -- 172.21.15.16:0/1495409865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f9ccc0cf580 con 0x7f9cd4106050 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f809882090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 -- 172.21.15.16:0/3132561764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 msgr2=0x55f809882090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f809882090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 -- 172.21.15.16:0/3132561764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 msgr2=0x55f809893b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 0x55f809893b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 -- 172.21.15.16:0/3132561764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8097743a0 con 0x55f8098825d0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd9391640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 0x7f9cbc0a51b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0ccd80 tx=0x7f9ccc066c20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.717+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400bd10 tx=0x7f9cc40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e44e70a824a350 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cbaffd640 1 -- 172.21.15.16:0/3132561764 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4004030 con 0x55f8098825d0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cbaffd640 1 -- 172.21.15.16:0/3132561764 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc40041d0 con 0x55f8098825d0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cbaffd640 1 -- 172.21.15.16:0/3132561764 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4011840 con 0x55f8098825d0 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/3132561764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 msgr2=0x55f809882a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400bd10 tx=0x7f9cc40107a0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/3132561764 shutdown_connections 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809891780 0x55f809893b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809882a10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/3132561764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f809882090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/3132561764 >> 172.21.15.16:0/3132561764 conn(0x55f80984b310 msgr2=0x55f80987ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/3132561764 shutdown_connections 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.718+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/3132561764 wait complete. 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 -- start start 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 Processor -- start 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809891160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f80988e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f809894480 con 0x55f809883b40 2024-09-12T01:12:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f8097a3060 con 0x55f8098825d0 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f809894600 con 0x55f809891780 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cdd1cd740 10 client.0 _ll_register_callbacks cb 0x55f8094f68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cd8b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cd8b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58756/0 (socket says 172.21.15.16:58756) 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cd8b90640 1 -- 172.21.15.16:0/1966660741 learned_addr learned my addr 172.21.15.16:0/1966660741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.719+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f80988e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809891160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 -- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 msgr2=0x55f80988e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f80988e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 -- 172.21.15.16:0/1966660741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 msgr2=0x55f809891160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809891160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 -- 172.21.15.16:0/1966660741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8095593c0 con 0x55f809883b40 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809891160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4003cf0 tx=0x7f9cc4003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acd85f24af5e7df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4029060 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc40041d0 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f9cc4004380 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9cc40035f0 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cdd1cd740 10 client.20295 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.720+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f80989ff00 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc40041d0 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9cc4004880 con 0x55f809883b40 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 10 client.20295 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 10 client.20295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 20 client.20295 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f8098a12b0 0x55f8098a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 4 client.20295 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8097f5860 con 0x55f8098a12b0 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.721+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f8098a12b0 0x55f8098a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.722+0000 7f9cb3fff640 10 client.20295 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.722+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f8098a12b0 0x55f8098a36a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2ca288d8133dc2b server_cookie=f44ddbcc75fefb58 in_seq=0 out_seq=0 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.722+0000 7f9cb3fff640 1 -- 172.21.15.16:0/1966660741 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8097f5860 con 0x55f8098a12b0 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.722+0000 7f9cb3fff640 10 client.20295 handle_command_reply: tid=0 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 client.20295 shutdown 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 msgr2=0x55f80988e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4003cf0 tx=0x7f9cc4003d20 comp rx=0 tx=0).stop 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 shutdown_connections 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f80988e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f80988e410 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f8098a12b0 0x55f8098a36a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/1966660741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098825d0 0x55f809891160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 >> 172.21.15.16:0/1966660741 conn(0x55f80984b310 msgr2=0x55f809884960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cb3fff640 0 client.20295 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 shutdown_connections 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/1966660741 wait complete. 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.723+0000 7f9cdd1cd740 20 client.20295 trim_cache size 0 max 16384 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 Processor -- start 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 -- start start 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f8098a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f809891780 0x55f809892b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41ab7b0 con 0x55f8098825d0 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41ad2f0 con 0x55f809891780 2024-09-12T01:12:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41abe80 con 0x55f809883b40 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58772/0 (socket says 172.21.15.16:58772) 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 -- 172.21.15.16:0/2878523918 learned_addr learned my addr 172.21.15.16:0/2878523918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f8098a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 -- 172.21.15.16:0/2878523918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 msgr2=0x55f8098a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f8098a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 -- 172.21.15.16:0/2878523918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f809891780 msgr2=0x55f809892b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f809891780 0x55f809892b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.736+0000 7f9cd3fff640 1 -- 172.21.15.16:0/2878523918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f809885120 con 0x55f8098825d0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.737+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009230 tx=0x7f9cc8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27760f7dd20e5e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.737+0000 7f9cba7fc640 1 -- 172.21.15.16:0/2878523918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8035040 con 0x55f8098825d0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.737+0000 7f9cba7fc640 1 -- 172.21.15.16:0/2878523918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc80043d0 con 0x55f8098825d0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.737+0000 7f9cba7fc640 1 -- 172.21.15.16:0/2878523918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc803d6b0 con 0x55f8098825d0 2024-09-12T01:12:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/2878523918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 msgr2=0x55f80989d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009230 tx=0x7f9cc8041040 comp rx=0 tx=0).stop 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/2878523918 shutdown_connections 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f809891780 0x55f809892b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809883b40 0x55f8098a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/2878523918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8098825d0 0x55f80989d720 secure :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009230 tx=0x7f9cc8041040 comp rx=0 tx=0).stop 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/2878523918 >> 172.21.15.16:0/2878523918 conn(0x55f80984b310 msgr2=0x55f809889720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/2878523918 shutdown_connections 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/2878523918 wait complete. 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 -- start start 2024-09-12T01:12:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.738+0000 7f9cdd1cd740 1 Processor -- start 2024-09-12T01:12:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f8098a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098a8910 0x55f8098a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41acc20 con 0x55f809883b40 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41ac550 con 0x55f8098a8910 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41a97b0 con 0x55f809891780 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cdd1cd740 10 client.0 _ll_register_callbacks cb 0x7f9cc80095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58776/0 (socket says 172.21.15.16:58776) 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 -- 172.21.15.16:0/418417568 learned_addr learned my addr 172.21.15.16:0/418417568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f8098a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098a8910 0x55f8098a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 -- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 msgr2=0x55f8098a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f8098a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 -- 172.21.15.16:0/418417568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098a8910 msgr2=0x55f8098a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098a8910 0x55f8098a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 -- 172.21.15.16:0/418417568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f809897a00 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f8098a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cd3fff640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9cc802a930 tx=0x7f9cc8032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5908c43038eee972 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.739+0000 7f9cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8035030 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc80043d0 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f9cc8031dd0 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9cc8005030 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 10 client.20301 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f80990bf40 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cc80043d0 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9cc80035f0 con 0x55f809883b40 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 10 client.20301 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 10 client.20301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 20 client.20301 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f80990c590 0x55f80990e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 4 client.20301 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.740+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8098a3a90 con 0x55f80990c590 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f80990c590 0x55f80990e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cd8b90640 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f80990c590 0x55f80990e9b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10c91afa42f04ec8 server_cookie=81676313cf97ee6a in_seq=0 out_seq=0 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cb3fff640 10 client.20301 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cb3fff640 1 -- 172.21.15.16:0/418417568 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8098a3a90 con 0x55f80990c590 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cb3fff640 10 client.20301 handle_command_reply: tid=0 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 client.20301 shutdown 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 msgr2=0x55f809899ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9cc802a930 tx=0x7f9cc8032d70 comp rx=0 tx=0).stop 2024-09-12T01:12:13.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 shutdown_connections 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x55f80990c590 0x55f80990e9b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f8098a8910 0x55f8098a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f809891780 0x55f8098a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cb3fff640 0 client.20301 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 --2- 172.21.15.16:0/418417568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f809883b40 0x55f809899ed0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.741+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 >> 172.21.15.16:0/418417568 conn(0x55f80984b310 msgr2=0x55f80987d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 shutdown_connections 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdd1cd740 1 -- 172.21.15.16:0/418417568 wait complete. 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdd1cd740 20 client.20301 trim_cache size 0 max 16384 2024-09-12T01:12:13.758 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:13.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 msgr2=0x7f9cbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 0x7f9cbc0a51b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc0ccd80 tx=0x7f9ccc066c20 comp rx=0 tx=0).stop 2024-09-12T01:12:13.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 msgr2=0x7f9cd419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00eaa0 tx=0x7f9ccc00ef60 comp rx=0 tx=0).stop 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 shutdown_connections 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f9cbc0a2cf0 0x7f9cbc0a51b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cd4106050 0x7f9cd419cf40 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cd4101df0 0x7f9cd4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 --2- 172.21.15.16:0/1495409865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cd4101440 0x7f9cd4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 >> 172.21.15.16:0/1495409865 conn(0x7f9cd4077210 msgr2=0x7f9cd4104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 shutdown_connections 2024-09-12T01:12:13.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:13.742+0000 7f9cdadf4640 1 -- 172.21.15.16:0/1495409865 wait complete. 2024-09-12T01:12:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:13 smithi099 ceph-mon[35654]: pgmap v504: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 137 KiB/s wr, 36 op/s 2024-09-12T01:12:13.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:13 smithi099 ceph-mon[35654]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T01:12:14.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:12:14.821 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 -- 172.21.15.16:0/84352480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1036e0 msgr2=0x7f567c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 --2- 172.21.15.16:0/84352480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1036e0 0x7f567c107b90 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5664009930 tx=0x7f566402a420 comp rx=0 tx=0).stop 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 -- 172.21.15.16:0/84352480 shutdown_connections 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 --2- 172.21.15.16:0/84352480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1082a0 0x7f567c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 --2- 172.21.15.16:0/84352480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1036e0 0x7f567c107b90 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 --2- 172.21.15.16:0/84352480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 0x7f567c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 -- 172.21.15.16:0/84352480 >> 172.21.15.16:0/84352480 conn(0x7f567c0fc7d0 msgr2=0x7f567c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 -- 172.21.15.16:0/84352480 shutdown_connections 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 -- 172.21.15.16:0/84352480 wait complete. 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.300+0000 7f56812e0640 1 Processor -- start 2024-09-12T01:12:15.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 -- start start 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 0x7f567c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1036e0 0x7f567c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1120c0 con 0x7f567c1082a0 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f567c111f40 con 0x7f567c1036e0 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f56812e0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f567c112240 con 0x7f567c102d30 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58790/0 (socket says 172.21.15.16:58790) 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567affd640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 0x7f567c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 -- 172.21.15.16:0/536603748 learned_addr learned my addr 172.21.15.16:0/536603748 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567a7fc640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1036e0 0x7f567c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 -- 172.21.15.16:0/536603748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 msgr2=0x7f567c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 0x7f567c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 -- 172.21.15.16:0/536603748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1036e0 msgr2=0x7f567c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1036e0 0x7f567c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.301+0000 7f567b7fe640 1 -- 172.21.15.16:0/536603748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c1a6ae0 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f567b7fe640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f567400b970 tx=0x7f567400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f5673fff640 1 -- 172.21.15.16:0/536603748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674019070 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c1a6dd0 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f5673fff640 1 -- 172.21.15.16:0/536603748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567400cb40 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f5673fff640 1 -- 172.21.15.16:0/536603748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674007650 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.302+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c1a7290 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.304+0000 7f5673fff640 1 -- 172.21.15.16:0/536603748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f567401c070 con 0x7f567c1082a0 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f56836b9740 1 Processor -- start 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f56836b9740 1 -- start start 2024-09-12T01:12:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f5673fff640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 0x7f56680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f567a7fc640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 0x7f56680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f5673fff640 1 -- 172.21.15.16:0/536603748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f56740ce8e0 con 0x7f567c1082a0 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f56836b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971313090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f56836b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971313a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.305+0000 7f56836b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56836b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d9712792f0 con 0x55d97131e370 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56836b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d971220cf0 con 0x55d971314b40 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56836b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9712752b0 con 0x55d9713135d0 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58806/0 (socket says 172.21.15.16:58806) 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 -- 172.21.15.16:0/2377196566 learned_addr learned my addr 172.21.15.16:0/2377196566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567affd640 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971313090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567a7fc640 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971313a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567a7fc640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 0x7f56680a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f5664032d10 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 -- 172.21.15.16:0/2377196566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 msgr2=0x55d971313a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971313a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 -- 172.21.15.16:0/2377196566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 msgr2=0x55d971313090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971313090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 -- 172.21.15.16:0/2377196566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9712053a0 con 0x55d97131e370 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f567b7fe640 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f5674097990 tx=0x7f567409aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbb40b3963e9732 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56717fa640 1 -- 172.21.15.16:0/2377196566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740a3b80 con 0x55d97131e370 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56717fa640 1 -- 172.21.15.16:0/2377196566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567400ccf0 con 0x55d97131e370 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.306+0000 7f56717fa640 1 -- 172.21.15.16:0/2377196566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740a3950 con 0x55d97131e370 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 -- 172.21.15.16:0/2377196566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 msgr2=0x55d971320760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f5674097990 tx=0x7f567409aec0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 -- 172.21.15.16:0/2377196566 shutdown_connections 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971320760 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971313a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 --2- 172.21.15.16:0/2377196566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971313090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 -- 172.21.15.16:0/2377196566 >> 172.21.15.16:0/2377196566 conn(0x55d9712dc310 msgr2=0x55d97130fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 -- 172.21.15.16:0/2377196566 shutdown_connections 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.307+0000 7f56836b9740 1 -- 172.21.15.16:0/2377196566 wait complete. 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 -- start start 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 Processor -- start 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d97131ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d97131e370 0x55d97131b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d971321070 con 0x55d971314b40 2024-09-12T01:12:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d971234060 con 0x55d97131e370 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9713211f0 con 0x55d9713135d0 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f56836b9740 10 client.0 _ll_register_callbacks cb 0x55d970f878c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f567affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d97131ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58810/0 (socket says 172.21.15.16:58810) 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f567a7fc640 1 -- 172.21.15.16:0/1946781135 learned_addr learned my addr 172.21.15.16:0/1946781135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.308+0000 7f567b7fe640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d97131e370 0x55d97131b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 -- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 msgr2=0x55d97131ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d97131ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 -- 172.21.15.16:0/1946781135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d97131e370 msgr2=0x55d97131b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d97131e370 0x55d97131b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 -- 172.21.15.16:0/1946781135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d970fea3c0 con 0x55d971314b40 2024-09-12T01:12:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567affd640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d97131ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f567a7fc640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5664005ec0 tx=0x7f566402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b466d664c571eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.309+0000 7f56627fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664035050 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f56640046c0 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567400ccf0 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56640048c0 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 10 client.20313 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d97132cb80 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567400ccf0 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5664003c60 con 0x55d971314b40 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 10 client.20313 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 10 client.20313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.312+0000 7f56836b9740 20 client.20313 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d97132df90 0x55d9713303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f56836b9740 4 client.20313 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d971286860 con 0x55d97132df90 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f567affd640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d97132df90 0x55d9713303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f567affd640 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d97132df90 0x55d9713303c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf8d86924ac8af86 server_cookie=b33532780638d0f in_seq=0 out_seq=0 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.313+0000 7f56627fc640 10 client.20313 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56627fc640 1 -- 172.21.15.16:0/1946781135 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d971286860 con 0x55d97132df90 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56627fc640 10 client.20313 handle_command_reply: tid=0 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 client.20313 shutdown 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 msgr2=0x55d97131b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f5664005ec0 tx=0x7f566402a420 comp rx=0 tx=0).stop 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 shutdown_connections 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d97132df90 0x55d9713303c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d97131e370 0x55d97131b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d97131b070 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 --2- 172.21.15.16:0/1946781135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d97131ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56627fc640 0 client.20313 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 >> 172.21.15.16:0/1946781135 conn(0x55d9712dc310 msgr2=0x55d971315990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 shutdown_connections 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 1 -- 172.21.15.16:0/1946781135 wait complete. 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.314+0000 7f56836b9740 20 client.20313 trim_cache size 0 max 16384 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 Processor -- start 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 -- start start 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971320760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971319940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1afb60 con 0x55d97131e370 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1b16a0 con 0x55d971314b40 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f56836b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1b0230 con 0x55d9713135d0 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58816/0 (socket says 172.21.15.16:58816) 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 -- 172.21.15.16:0/3610641439 learned_addr learned my addr 172.21.15.16:0/3610641439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567affd640 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971320760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567a7fc640 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971319940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 -- 172.21.15.16:0/3610641439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 msgr2=0x55d971320760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971320760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 -- 172.21.15.16:0/3610641439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 msgr2=0x55d971319940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971319940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.328+0000 7f567b7fe640 1 -- 172.21.15.16:0/3610641439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97131bb70 con 0x55d97131e370 2024-09-12T01:12:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.329+0000 7f567b7fe640 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5674097420 tx=0x7f5674014d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933d4ecdef0e941f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.329+0000 7f5670ff9640 1 -- 172.21.15.16:0/3610641439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567407e070 con 0x55d97131e370 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.329+0000 7f5670ff9640 1 -- 172.21.15.16:0/3610641439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56740a3b80 con 0x55d97131e370 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.329+0000 7f5670ff9640 1 -- 172.21.15.16:0/3610641439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740a3950 con 0x55d97131e370 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- 172.21.15.16:0/3610641439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 msgr2=0x55d971319e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f5674097420 tx=0x7f5674014d00 comp rx=0 tx=0).stop 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- 172.21.15.16:0/3610641439 shutdown_connections 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d97131e370 0x55d971319e80 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d971314b40 0x55d971319940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- 172.21.15.16:0/3610641439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9713135d0 0x55d971320760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- 172.21.15.16:0/3610641439 >> 172.21.15.16:0/3610641439 conn(0x55d9712dc310 msgr2=0x55d97130aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- 172.21.15.16:0/3610641439 shutdown_connections 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- 172.21.15.16:0/3610641439 wait complete. 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 -- start start 2024-09-12T01:12:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 Processor -- start 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9713135d0 0x55d9713269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.330+0000 7f56836b9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d97131e370 0x55d971322260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1b0fd0 con 0x55d971314b40 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1b0900 con 0x55d9713135d0 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f567c1adb60 con 0x55d97131e370 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 10 client.0 _ll_register_callbacks cb 0x7f5664009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58824/0 (socket says 172.21.15.16:58824) 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 -- 172.21.15.16:0/3247539838 learned_addr learned my addr 172.21.15.16:0/3247539838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567affd640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9713135d0 0x55d9713269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567b7fe640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d97131e370 0x55d971322260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 -- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d97131e370 msgr2=0x55d971322260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d97131e370 0x55d971322260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 -- 172.21.15.16:0/3247539838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9713135d0 msgr2=0x55d9713269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9713135d0 0x55d9713269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 -- 172.21.15.16:0/3247539838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d971322820 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f567a7fc640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f5664009930 tx=0x7f5664009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110dc394790fd992 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56627fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664035070 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f5664004c70 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.331+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56740a3b80 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.332+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f566406a050 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.332+0000 7f56836b9740 10 client.20319 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.332+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d971398a30 con 0x55d971314b40 2024-09-12T01:12:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56740a3b80 con 0x55d971314b40 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5664062020 con 0x55d971314b40 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 10 client.20319 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 10 client.20319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 20 client.20319 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d971399150 0x55d97139b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 4 client.20319 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d971280d70 con 0x55d971399150 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.333+0000 7f567affd640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d971399150 0x55d97139b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56627fc640 10 client.20319 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f567affd640 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d971399150 0x55d97139b570 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b52b52838d1cbd server_cookie=f46eedbf6bbd84b8 in_seq=0 out_seq=0 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56627fc640 1 -- 172.21.15.16:0/3247539838 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d971280d70 con 0x55d971399150 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56627fc640 10 client.20319 handle_command_reply: tid=0 2024-09-12T01:12:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 client.20319 shutdown 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 msgr2=0x55d971321d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f5664009930 tx=0x7f5664009230 comp rx=0 tx=0).stop 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 shutdown_connections 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55d971399150 0x55d97139b570 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d97131e370 0x55d971322260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d971314b40 0x55d971321d20 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 --2- 172.21.15.16:0/3247539838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9713135d0 0x55d9713269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56627fc640 0 client.20319 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.334+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 >> 172.21.15.16:0/3247539838 conn(0x55d9712dc310 msgr2=0x55d9713197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 shutdown_connections 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56836b9740 1 -- 172.21.15.16:0/3247539838 wait complete. 2024-09-12T01:12:15.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56836b9740 20 client.20319 trim_cache size 0 max 16384 2024-09-12T01:12:15.351 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:15.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 msgr2=0x7f56680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 0x7f56680a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f5664032d10 comp rx=0 tx=0).stop 2024-09-12T01:12:15.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 msgr2=0x7f567c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f567400b970 tx=0x7f567400be30 comp rx=0 tx=0).stop 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 shutdown_connections 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f56680a2cf0 0x7f56680a51b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f567c1082a0 0x7f567c101410 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f567c1036e0 0x7f567c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 --2- 172.21.15.16:0/536603748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f567c102d30 0x7f567c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.335+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 >> 172.21.15.16:0/536603748 conn(0x7f567c0fc7d0 msgr2=0x7f567c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.336+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 shutdown_connections 2024-09-12T01:12:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:15.336+0000 7f56812e0640 1 -- 172.21.15.16:0/536603748 wait complete. 2024-09-12T01:12:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:15 smithi159 ceph-mon[35566]: pgmap v506: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-09-12T01:12:15.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:15 smithi016 ceph-mon[30972]: pgmap v506: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-09-12T01:12:15.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:15 smithi099 ceph-mon[35654]: pgmap v506: 97 pgs: 97 active+clean; 29 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 43 KiB/s wr, 9 op/s 2024-09-12T01:12:17.061 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T01:12:17.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T01:12:17.332 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:17.395 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:17 smithi016 ceph-mon[30972]: pgmap v507: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:17 smithi159 ceph-mon[35566]: pgmap v507: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:17.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:17 smithi099 ceph-mon[35654]: pgmap v507: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 -- 172.21.15.16:0/1047035819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 msgr2=0x7f95d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 --2- 172.21.15.16:0/1047035819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d410ea90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009ca0 tx=0x7f95cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 -- 172.21.15.16:0/1047035819 shutdown_connections 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 --2- 172.21.15.16:0/1047035819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d410ea90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 --2- 172.21.15.16:0/1047035819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 0x7f95d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 --2- 172.21.15.16:0/1047035819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 -- 172.21.15.16:0/1047035819 >> 172.21.15.16:0/1047035819 conn(0x7f95d40fd8c0 msgr2=0x7f95d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 -- 172.21.15.16:0/1047035819 shutdown_connections 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.818+0000 7f95d9a67640 1 -- 172.21.15.16:0/1047035819 wait complete. 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 Processor -- start 2024-09-12T01:12:17.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 -- start start 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 0x7f95d41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95d4111f70 con 0x7f95d4106f10 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d4111df0 con 0x7f95d4101de0 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d9a67640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41120f0 con 0x7f95d4102790 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58842/0 (socket says 172.21.15.16:58842) 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d3fff640 1 -- 172.21.15.16:0/206978463 learned_addr learned my addr 172.21.15.16:0/206978463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:17.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95caffd640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 0x7f95d41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.819+0000 7f95d37fe640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 -- 172.21.15.16:0/206978463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 msgr2=0x7f95d41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 0x7f95d41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 -- 172.21.15.16:0/206978463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 msgr2=0x7f95d41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 -- 172.21.15.16:0/206978463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d419d360 con 0x7f95d4106f10 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d37fe640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d3fff640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f95cc031a20 tx=0x7f95cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d17fa640 1 -- 172.21.15.16:0/206978463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc003c30 con 0x7f95d4106f10 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d419d5f0 con 0x7f95d4106f10 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d17fa640 1 -- 172.21.15.16:0/206978463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc003dd0 con 0x7f95d4106f10 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d17fa640 1 -- 172.21.15.16:0/206978463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc02fcf0 con 0x7f95d4106f10 2024-09-12T01:12:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.820+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d41a4f30 con 0x7f95d4106f10 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.822+0000 7f95d17fa640 1 -- 172.21.15.16:0/206978463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f95cc004050 con 0x7f95d4106f10 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 Processor -- start 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 -- start start 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec104f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652ec06a280 con 0x5652ec10fd40 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5652ec011b10 con 0x5652ec104b60 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95dbe40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5652ec066240 con 0x5652ec105480 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58856/0 (socket says 172.21.15.16:58856) 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95caffd640 1 -- 172.21.15.16:0/1966068168 learned_addr learned my addr 172.21.15.16:0/1966068168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95d37fe640 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.823+0000 7f95caffd640 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 -- 172.21.15.16:0/1966068168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 msgr2=0x5652ec10f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 -- 172.21.15.16:0/1966068168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 msgr2=0x5652ec104f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec104f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95d17fa640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 0x7f95c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 -- 172.21.15.16:0/1966068168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ec01e1f0 con 0x5652ec10fd40 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95caffd640 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f95bc012540 tx=0x7f95bc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c0a6d8533c210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95d3fff640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 0x7f95c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95d17fa640 1 -- 172.21.15.16:0/206978463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f95cc0f0690 con 0x7f95d4106f10 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95c9ffb640 1 -- 172.21.15.16:0/1966068168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc01e640 con 0x5652ec10fd40 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95c9ffb640 1 -- 172.21.15.16:0/1966068168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc010910 con 0x5652ec10fd40 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.824+0000 7f95c9ffb640 1 -- 172.21.15.16:0/1966068168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc01d940 con 0x5652ec10fd40 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- 172.21.15.16:0/1966068168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 msgr2=0x5652ec112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95d3fff640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 0x7f95c00a51b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0bcb70 tx=0x7f95cc03d040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f95bc012540 tx=0x7f95bc00b7b0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- 172.21.15.16:0/1966068168 shutdown_connections 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec112130 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1966068168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- 172.21.15.16:0/1966068168 >> 172.21.15.16:0/1966068168 conn(0x5652ec1010f0 msgr2=0x5652ec1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- 172.21.15.16:0/1966068168 shutdown_connections 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- 172.21.15.16:0/1966068168 wait complete. 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 -- start start 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.825+0000 7f95dbe40740 1 Processor -- start 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec10fd40 0x5652ec10d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5652ec112910 con 0x5652ec104b60 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5652ec024f10 con 0x5652ec10fd40 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5652ec112a90 con 0x5652ec105480 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95dbe40740 10 client.0 _ll_register_callbacks cb 0x5652ebfeda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:17.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58872/0 (socket says 172.21.15.16:58872) 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 -- 172.21.15.16:0/2732077553 learned_addr learned my addr 172.21.15.16:0/2732077553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95d37fe640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 -- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 msgr2=0x5652ec10d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 -- 172.21.15.16:0/2732077553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec10fd40 msgr2=0x5652ec10d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec10fd40 0x5652ec10d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.826+0000 7f95caffd640 1 -- 172.21.15.16:0/2732077553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ebddb6a0 con 0x5652ec104b60 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95caffd640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f95bc013fd0 tx=0x7f95bc010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee79c0e2bce96fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc023040 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f95bc010ac0 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95bc010910 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95bc036050 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 10 client.20331 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652ec11e540 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc010910 con 0x5652ec104b60 2024-09-12T01:12:17.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f95bc031030 con 0x5652ec104b60 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 10 client.20331 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 10 client.20331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.827+0000 7f95dbe40740 20 client.20331 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec11f9a0 0x5652ec121d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95dbe40740 4 client.20331 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652ec0777f0 con 0x5652ec11f9a0 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95d37fe640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec11f9a0 0x5652ec121d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95b2ffd640 10 client.20331 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.828+0000 7f95d37fe640 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec11f9a0 0x5652ec121d90 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2019f3e2901f8e server_cookie=ca4cdaf74e471065 in_seq=0 out_seq=0 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95b2ffd640 1 -- 172.21.15.16:0/2732077553 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652ec0777f0 con 0x5652ec11f9a0 2024-09-12T01:12:17.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95b2ffd640 10 client.20331 handle_command_reply: tid=0 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 client.20331 shutdown 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 msgr2=0x5652ec10cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f95bc013fd0 tx=0x7f95bc010660 comp rx=0 tx=0).stop 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 shutdown_connections 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec11f9a0 0x5652ec121d90 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec10fd40 0x5652ec10d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec10d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 --2- 172.21.15.16:0/2732077553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec104b60 0x5652ec10cc40 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95b2ffd640 0 client.20331 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:17.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 >> 172.21.15.16:0/2732077553 conn(0x5652ec1010f0 msgr2=0x5652ec106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.829+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 shutdown_connections 2024-09-12T01:12:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.830+0000 7f95dbe40740 1 -- 172.21.15.16:0/2732077553 wait complete. 2024-09-12T01:12:17.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.830+0000 7f95dbe40740 20 client.20331 trim_cache size 0 max 16384 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 Processor -- start 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 -- start start 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec10baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec1275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41ad8a0 con 0x5652ec10fd40 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41af3e0 con 0x5652ec104b60 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95dbe40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41adf70 con 0x5652ec105480 2024-09-12T01:12:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58878/0 (socket says 172.21.15.16:58878) 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95caffd640 1 -- 172.21.15.16:0/1447321696 learned_addr learned my addr 172.21.15.16:0/1447321696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.847+0000 7f95d37fe640 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec1275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 -- 172.21.15.16:0/1447321696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 msgr2=0x5652ec1275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec1275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 -- 172.21.15.16:0/1447321696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 msgr2=0x5652ec10baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec10baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 -- 172.21.15.16:0/1447321696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ec1052b0 con 0x5652ec10fd40 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95caffd640 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00bf50 tx=0x7f95bc010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed296b1cbfcc5124 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95c97fa640 1 -- 172.21.15.16:0/1447321696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc03e070 con 0x5652ec10fd40 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95c97fa640 1 -- 172.21.15.16:0/1447321696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc01dd30 con 0x5652ec10fd40 2024-09-12T01:12:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.848+0000 7f95c97fa640 1 -- 172.21.15.16:0/1447321696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc010a70 con 0x5652ec10fd40 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- 172.21.15.16:0/1447321696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 msgr2=0x5652ec127b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00bf50 tx=0x7f95bc010f50 comp rx=0 tx=0).stop 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- 172.21.15.16:0/1447321696 shutdown_connections 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec10fd40 0x5652ec127b00 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec105480 0x5652ec1275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- 172.21.15.16:0/1447321696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- 172.21.15.16:0/1447321696 >> 172.21.15.16:0/1447321696 conn(0x5652ec1010f0 msgr2=0x5652ec10b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- 172.21.15.16:0/1447321696 shutdown_connections 2024-09-12T01:12:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- 172.21.15.16:0/1447321696 wait complete. 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 -- start start 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 Processor -- start 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec113960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.849+0000 7f95dbe40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec10fd40 0x5652ec1143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95dbe40740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41aed10 con 0x5652ec105480 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95dbe40740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41ae640 con 0x5652ec104b60 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95dbe40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d41ab8a0 con 0x5652ec10fd40 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95dbe40740 10 client.0 _ll_register_callbacks cb 0x7f95c400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58894/0 (socket says 172.21.15.16:58894) 2024-09-12T01:12:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 -- 172.21.15.16:0/245421718 learned_addr learned my addr 172.21.15.16:0/245421718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95caffd640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95caffd640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec10fd40 0x5652ec1143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 -- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec10fd40 msgr2=0x5652ec1143e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec10fd40 0x5652ec1143e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 -- 172.21.15.16:0/245421718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 msgr2=0x5652ec113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 -- 172.21.15.16:0/245421718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ec189590 con 0x5652ec105480 2024-09-12T01:12:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95caffd640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95d37fe640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f95c400bfd0 tx=0x7f95c4013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48742c02644f5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004230 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f95c40043d0 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.850+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95bc01dd30 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95c402d050 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 10 client.20337 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652ec18a860 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95bc01dd30 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f95c401d7b0 con 0x5652ec105480 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 10 client.20337 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 10 client.20337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 20 client.20337 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec18b010 0x5652ec18d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 4 client.20337 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5652ec071d00 con 0x5652ec18b010 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.851+0000 7f95caffd640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec18b010 0x5652ec18d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.852+0000 7f95caffd640 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec18b010 0x5652ec18d430 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae2b00f65a18f75 server_cookie=146dee9228ded50 in_seq=0 out_seq=0 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.852+0000 7f95b2ffd640 10 client.20337 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95b2ffd640 1 -- 172.21.15.16:0/245421718 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5652ec071d00 con 0x5652ec18b010 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95b2ffd640 10 client.20337 handle_command_reply: tid=0 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 client.20337 shutdown 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 msgr2=0x5652ec113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f95c400bfd0 tx=0x7f95c4013a50 comp rx=0 tx=0).stop 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 shutdown_connections 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x5652ec18b010 0x5652ec18d430 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5652ec10fd40 0x5652ec1143e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652ec105480 0x5652ec113ea0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 --2- 172.21.15.16:0/245421718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5652ec104b60 0x5652ec113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 >> 172.21.15.16:0/245421718 conn(0x5652ec1010f0 msgr2=0x5652ec100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95b2ffd640 0 client.20337 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 shutdown_connections 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 1 -- 172.21.15.16:0/245421718 wait complete. 2024-09-12T01:12:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.853+0000 7f95dbe40740 20 client.20337 trim_cache size 0 max 16384 2024-09-12T01:12:17.860 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:17.860 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-12T01:12:17.860 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d26c74c2-23b8-4f33-b21e-d70d6c4e7f0c", 2024-09-12T01:12:17.860 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-12T01:12:17.860 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 msgr2=0x7f95c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 0x7f95c00a51b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0bcb70 tx=0x7f95cc03d040 comp rx=0 tx=0).stop 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 msgr2=0x7f95d419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f95cc031a20 tx=0x7f95cc031a50 comp rx=0 tx=0).stop 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 shutdown_connections 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f95c00a2cf0 0x7f95c00a51b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95d4106f10 0x7f95d419cc80 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d4102790 0x7f95d41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 --2- 172.21.15.16:0/206978463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95d4101de0 0x7f95d41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 >> 172.21.15.16:0/206978463 conn(0x7f95d40fd8c0 msgr2=0x7f95d40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 shutdown_connections 2024-09-12T01:12:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:17.854+0000 7f95d9a67640 1 -- 172.21.15.16:0/206978463 wait complete. 2024-09-12T01:12:18.444 INFO:tasks.workunit.client.0.smithi016.stdout:561/600: open fd = 3 2024-09-12T01:12:18.444 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:562/600: open fd = 3 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:563/600: open fd = 3 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:564/600: open fd = 3 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.445 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:565/600: open fd = 3 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:566/600: open fd = 3 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:567/600: open fd = 3 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:568/600: open fd = 3 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.446 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:569/600: open fd = 3 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:570/600: open fd = 3 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:571/600: open fd = 3 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.447 INFO:tasks.workunit.client.0.smithi016.stdout:572/600: open fd = 3 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:573/600: open fd = 3 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:574/600: open fd = 3 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:575/600: open fd = 3 2024-09-12T01:12:18.448 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:576/600: open fd = 3 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:577/600: open fd = 3 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:578/600: open fd = 3 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.449 INFO:tasks.workunit.client.0.smithi016.stdout:579/600: open fd = 3 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:580/600: open fd = 3 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:581/600: open fd = 3 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:582/600: open fd = 3 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.450 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:583/600: open fd = 3 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:584/600: open fd = 3 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:585/600: open fd = 3 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.451 INFO:tasks.workunit.client.0.smithi016.stdout:586/600: open fd = 3 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:587/600: open fd = 3 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:588/600: open fd = 3 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:589/600: open fd = 3 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.452 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:590/600: open fd = 3 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:591/600: open fd = 3 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:592/600: open fd = 3 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:593/600: open fd = 3 2024-09-12T01:12:18.453 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:594/600: open fd = 3 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:595/600: open fd = 3 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:596/600: open fd = 3 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.454 INFO:tasks.workunit.client.0.smithi016.stdout:597/600: open fd = 3 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:598/600: open fd = 3 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:599/600: open fd = 3 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:600/600: open fd = 3 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.455 INFO:tasks.workunit.client.0.smithi016.stdout:write ret = 32 2024-09-12T01:12:18.456 INFO:tasks.workunit.client.0.smithi016.stdout:pread ret = 64 2024-09-12T01:12:18.456 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-12T01:12:18.457 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-12T01:12:18.457 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T01:12:18.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T01:12:18.498 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-09-12T01:12:18.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T01:12:18.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:12:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:18.860 DEBUG:teuthology.parallel:result is None 2024-09-12T01:12:18.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T01:12:18.899 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T01:12:18.899 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T01:12:18.910 INFO:tasks.quiescer:joining Quiescers 2024-09-12T01:12:18.910 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T01:12:18.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.364+0000 7fa2d8f40640 1 -- 172.21.15.16:0/4155688430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4100a80 msgr2=0x7fa2d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.364+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/4155688430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4100a80 0x7fa2d4100ee0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009930 tx=0x7fa2c802a420 comp rx=0 tx=0).stop 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 -- 172.21.15.16:0/4155688430 shutdown_connections 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/4155688430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 0x7fa2d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/4155688430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4100a80 0x7fa2d4100ee0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/4155688430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4106a80 0x7fa2d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 -- 172.21.15.16:0/4155688430 >> 172.21.15.16:0/4155688430 conn(0x7fa2d40fc7a0 msgr2=0x7fa2d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 -- 172.21.15.16:0/4155688430 shutdown_connections 2024-09-12T01:12:19.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.365+0000 7fa2d8f40640 1 -- 172.21.15.16:0/4155688430 wait complete. 2024-09-12T01:12:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 Processor -- start 2024-09-12T01:12:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 -- start start 2024-09-12T01:12:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 0x7fa2d4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4110f10 con 0x7fa2d4106a80 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4110d90 con 0x7fa2d4101420 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d8f40640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4111090 con 0x7fa2d4100a80 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.366+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:51928/0 (socket says 172.21.15.16:51928) 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53470/0 (socket says 172.21.15.16:53470) 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 -- 172.21.15.16:0/1628312197 learned_addr learned my addr 172.21.15.16:0/1628312197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 0x7fa2d4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 -- 172.21.15.16:0/1628312197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 msgr2=0x7fa2d4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 0x7fa2d4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 -- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 msgr2=0x7fa2d419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 -- 172.21.15.16:0/1628312197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d419d4b0 con 0x7fa2d4100a80 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2d259c640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d770 tx=0x7fa2c400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.367+0000 7fa2c37fe640 1 -- 172.21.15.16:0/1628312197 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4017070 con 0x7fa2d4100a80 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.368+0000 7fa2c37fe640 1 -- 172.21.15.16:0/1628312197 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c4005500 con 0x7fa2d4100a80 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.368+0000 7fa2c37fe640 1 -- 172.21.15.16:0/1628312197 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4004e00 con 0x7fa2d4100a80 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.368+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d419d7a0 con 0x7fa2d4100a80 2024-09-12T01:12:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.368+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a5080 con 0x7fa2d4100a80 2024-09-12T01:12:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.370+0000 7fa2c37fe640 1 -- 172.21.15.16:0/1628312197 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2c4004fa0 con 0x7fa2d4100a80 2024-09-12T01:12:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 Processor -- start 2024-09-12T01:12:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2c37fe640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 0x7fa2bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 0x7fa2bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 -- start start 2024-09-12T01:12:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2c37fe640 1 -- 172.21.15.16:0/1628312197 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa2c40cee80 con 0x7fa2d4100a80 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 0x558e6d4b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558e6d41e2e0 con 0x558e6d4b8880 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558e6d3c5ce0 con 0x558e6d4b9a20 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2db319740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e6d41a2a0 con 0x558e6d4cbd20 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53480/0 (socket says 172.21.15.16:53480) 2024-09-12T01:12:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 0x558e6d4b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/540179040 learned_addr learned my addr 172.21.15.16:0/540179040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.371+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 0x7fa2bc0a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00a730 tx=0x7fa2cc00d040 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 -- 172.21.15.16:0/540179040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 msgr2=0x558e6d4ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 -- 172.21.15.16:0/540179040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 msgr2=0x558e6d4b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 -- 172.21.15.16:0/540179040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6d3aa440 con 0x558e6d4b9a20 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2d259c640 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 0x558e6d4b8340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cc540 tx=0x7fa2c409ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49149766a9a5ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2c0ff9640 1 -- 172.21.15.16:0/540179040 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4020050 con 0x558e6d4b9a20 2024-09-12T01:12:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2c0ff9640 1 -- 172.21.15.16:0/540179040 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c40a3fb0 con 0x558e6d4b9a20 2024-09-12T01:12:19.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.372+0000 7fa2c0ff9640 1 -- 172.21.15.16:0/540179040 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c409a840 con 0x558e6d4b9a20 2024-09-12T01:12:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- 172.21.15.16:0/540179040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 msgr2=0x558e6d4b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 0x558e6d4b8340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40cc540 tx=0x7fa2c409ae70 comp rx=0 tx=0).stop 2024-09-12T01:12:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- 172.21.15.16:0/540179040 shutdown_connections 2024-09-12T01:12:19.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 --2- 172.21.15.16:0/540179040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b9a20 0x558e6d4b8340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- 172.21.15.16:0/540179040 >> 172.21.15.16:0/540179040 conn(0x558e6d4b4a30 msgr2=0x558e6d4b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- 172.21.15.16:0/540179040 shutdown_connections 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- 172.21.15.16:0/540179040 wait complete. 2024-09-12T01:12:19.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.373+0000 7fa2db319740 1 -- start start 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 Processor -- start 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4b8880 0x558e6d4bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 0x558e6d4bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558e6d4ce990 con 0x558e6d4b9a20 2024-09-12T01:12:19.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558e6d3d9050 con 0x558e6d4cbd20 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e6d4ceb10 con 0x558e6d4b8880 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2db319740 10 client.0 _ll_register_callbacks cb 0x558e6d419160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53486/0 (socket says 172.21.15.16:53486) 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/3988412058 learned_addr learned my addr 172.21.15.16:0/3988412058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:19.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 0x558e6d4bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.374+0000 7fa2d259c640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4b8880 0x558e6d4bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4b8880 msgr2=0x558e6d4bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4b8880 0x558e6d4bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/3988412058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 msgr2=0x558e6d4bc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 0x558e6d4bc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/3988412058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6d18f3c0 con 0x558e6d4b9a20 2024-09-12T01:12:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 0x558e6d4bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006010 tx=0x7fa2c80045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb073756a1a57d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8035040 con 0x558e6d4b9a20 2024-09-12T01:12:19.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c40a3fb0 con 0x558e6d4b9a20 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa2c8005550 con 0x558e6d4b9a20 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c80046c0 con 0x558e6d4b9a20 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2db319740 10 client.20349 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e6d4da420 con 0x558e6d4b9a20 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.375+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c40a3fb0 con 0x558e6d4b9a20 2024-09-12T01:12:19.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c8003ca0 con 0x558e6d4b9a20 2024-09-12T01:12:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 10 client.20349 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 10 client.20349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 20 client.20349 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d4dbbf0 0x558e6d4de030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 4 client.20349 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6d42b850 con 0x558e6d4dbbf0 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2d259c640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d4dbbf0 0x558e6d4de030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2b9ffb640 10 client.20349 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.376+0000 7fa2d259c640 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d4dbbf0 0x558e6d4de030 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37059909c4ef16a1 server_cookie=dcb95221a29c96ba in_seq=0 out_seq=0 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/3988412058 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e6d42b850 con 0x558e6d4dbbf0 2024-09-12T01:12:19.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2b9ffb640 10 client.20349 handle_command_reply: tid=0 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 client.20349 shutdown 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 msgr2=0x558e6d4bbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8006010 tx=0x7fa2c80045c0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 shutdown_connections 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d4dbbf0 0x558e6d4de030 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4cbd20 0x558e6d4bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b9a20 0x558e6d4bbe70 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 --2- 172.21.15.16:0/3988412058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4b8880 0x558e6d4bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.377+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 >> 172.21.15.16:0/3988412058 conn(0x558e6d4b4a30 msgr2=0x558e6d4ba750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.378+0000 7fa2b9ffb640 0 client.20349 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:19.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.378+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 shutdown_connections 2024-09-12T01:12:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.378+0000 7fa2db319740 1 -- 172.21.15.16:0/3988412058 wait complete. 2024-09-12T01:12:19.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.378+0000 7fa2db319740 20 client.20349 trim_cache size 0 max 16384 2024-09-12T01:12:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:19 smithi016 ceph-mon[30972]: pgmap v508: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:19 smithi016 ceph-mon[30972]: scrub queued for path: / 2024-09-12T01:12:19.393 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:19 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:19.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:19 smithi016 ceph-mon[30972]: scrub summary: active paths [/] 2024-09-12T01:12:19.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:19 smithi016 ceph-mon[30972]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:19.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 Processor -- start 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 -- start start 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b8880 0x558e6d4d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 0x558e6d4bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ab6d0 con 0x558e6d4cbd20 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ad210 con 0x558e6d4b8880 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2db319740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41abda0 con 0x558e6d4c8410 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53500/0 (socket says 172.21.15.16:53500) 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/1857207400 learned_addr learned my addr 172.21.15.16:0/1857207400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 0x558e6d4bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.397+0000 7fa2d259c640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b8880 0x558e6d4d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/1857207400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 msgr2=0x558e6d4bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 0x558e6d4bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/1857207400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b8880 msgr2=0x558e6d4d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b8880 0x558e6d4d7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 -- 172.21.15.16:0/1857207400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6d4cb020 con 0x558e6d4cbd20 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 0x558e6d4bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005ec0 tx=0x7fa2c8031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd732f2754730d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2bbfff640 1 -- 172.21.15.16:0/1857207400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8035030 con 0x558e6d4cbd20 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2bbfff640 1 -- 172.21.15.16:0/1857207400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8041070 con 0x558e6d4cbd20 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.398+0000 7fa2bbfff640 1 -- 172.21.15.16:0/1857207400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8003870 con 0x558e6d4cbd20 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- 172.21.15.16:0/1857207400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 msgr2=0x558e6d4bb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005ec0 tx=0x7fa2c8031a90 comp rx=0 tx=0).stop 2024-09-12T01:12:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- 172.21.15.16:0/1857207400 shutdown_connections 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4c8410 0x558e6d4bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4cbd20 0x558e6d4bb730 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 --2- 172.21.15.16:0/1857207400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4b8880 0x558e6d4d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- 172.21.15.16:0/1857207400 >> 172.21.15.16:0/1857207400 conn(0x558e6d4b4a30 msgr2=0x558e6d4c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- 172.21.15.16:0/1857207400 shutdown_connections 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- 172.21.15.16:0/1857207400 wait complete. 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.399+0000 7fa2db319740 1 -- start start 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 Processor -- start 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4c8410 0x558e6d4d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41acb40 con 0x558e6d4b8880 2024-09-12T01:12:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41ac470 con 0x558e6d4c8410 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41a96b0 con 0x558e6d4cbd20 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2db319740 10 client.0 _ll_register_callbacks cb 0x7fa2c8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53504/0 (socket says 172.21.15.16:53504) 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 -- 172.21.15.16:0/2800204289 learned_addr learned my addr 172.21.15.16:0/2800204289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4c8410 0x558e6d4d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 -- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 msgr2=0x558e6d4d42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4d42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 -- 172.21.15.16:0/2800204289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4c8410 msgr2=0x558e6d4d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4c8410 0x558e6d4d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d259c640 1 -- 172.21.15.16:0/2800204289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6d4c9130 con 0x558e6d4b8880 2024-09-12T01:12:19.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.400+0000 7fa2d2d9d640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.401+0000 7fa2d259c640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d740 tx=0x7fa2c4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820730c7440f11a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.401+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.401+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40b3070 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.401+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa2c40996f0 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.401+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c8041070 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c4020040 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 10 client.20355 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e6d546ac0 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8041070 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa2c409a840 con 0x558e6d4b8880 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 10 client.20355 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 10 client.20355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 20 client.20355 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d546e00 0x558e6d5491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 4 client.20355 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:12:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e6d425d60 con 0x558e6d546e00 2024-09-12T01:12:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.402+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d546e00 0x558e6d5491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2d1d9b640 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d546e00 0x558e6d5491f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea253cc73ce128d5 server_cookie=1cf18faf7a5ab692 in_seq=0 out_seq=0 2024-09-12T01:12:19.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2b9ffb640 10 client.20355 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2b9ffb640 1 -- 172.21.15.16:0/2800204289 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e6d425d60 con 0x558e6d546e00 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2b9ffb640 10 client.20355 handle_command_reply: tid=0 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2db319740 1 client.20355 shutdown 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 msgr2=0x558e6d4ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d740 tx=0x7fa2c4009c30 comp rx=0 tx=0).stop 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 shutdown_connections 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.403+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x558e6d546e00 0x558e6d5491f0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6d4cbd20 0x558e6d4d42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558e6d4c8410 0x558e6d4d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 --2- 172.21.15.16:0/2800204289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558e6d4b8880 0x558e6d4ca5c0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 >> 172.21.15.16:0/2800204289 conn(0x558e6d4b4a30 msgr2=0x558e6d4bb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2b9ffb640 0 client.20355 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 shutdown_connections 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 1 -- 172.21.15.16:0/2800204289 wait complete. 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.404+0000 7fa2db319740 20 client.20355 trim_cache size 0 max 16384 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout: "d26c74c2-23b8-4f33-b21e-d70d6c4e7f0c": { 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-12T01:12:19.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d26c74c2-23b8-4f33-b21e-d70d6c4e7f0c", 2024-09-12T01:12:19.411 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-12T01:12:19.411 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:12:19.411 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:12:19.411 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:19.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 msgr2=0x7fa2bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 0x7fa2bc0a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00a730 tx=0x7fa2cc00d040 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 msgr2=0x7fa2d4198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400d770 tx=0x7fa2c400dc30 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 shutdown_connections 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa2bc0a2cf0 0x7fa2bc0a51b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d4106a80 0x7fa2d419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2d4101420 0x7fa2d4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 --2- 172.21.15.16:0/1628312197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d4100a80 0x7fa2d4198500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 >> 172.21.15.16:0/1628312197 conn(0x7fa2d40fc7a0 msgr2=0x7fa2d406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:19.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 shutdown_connections 2024-09-12T01:12:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:19.405+0000 7fa2d8f40640 1 -- 172.21.15.16:0/1628312197 wait complete. 2024-09-12T01:12:19.413 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:19 smithi159 ceph-mon[35566]: pgmap v508: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:19 smithi159 ceph-mon[35566]: scrub queued for path: / 2024-09-12T01:12:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:19 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:19 smithi159 ceph-mon[35566]: scrub summary: active paths [/] 2024-09-12T01:12:19.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:19 smithi159 ceph-mon[35566]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:19 smithi099 ceph-mon[35654]: pgmap v508: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-09-12T01:12:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:19 smithi099 ceph-mon[35654]: scrub queued for path: / 2024-09-12T01:12:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:19 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:19 smithi099 ceph-mon[35654]: scrub summary: active paths [/] 2024-09-12T01:12:19.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:19 smithi099 ceph-mon[35654]: scrub summary: idle+waiting paths [/] 2024-09-12T01:12:20.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:20 smithi016 ceph-mon[30972]: scrub summary: idle 2024-09-12T01:12:20.440 INFO:tasks.cephfs.filesystem:scrub status for tag:d26c74c2-23b8-4f33-b21e-d70d6c4e7f0c - {'path': '/', 'tag': 'd26c74c2-23b8-4f33-b21e-d70d6c4e7f0c', 'options': 'recursive,force'} 2024-09-12T01:12:20.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:20 smithi159 ceph-mon[35566]: scrub summary: idle 2024-09-12T01:12:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 -- 172.21.15.16:0/3571024989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 msgr2=0x7f346c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 --2- 172.21.15.16:0/3571024989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c10d950 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f3464009cd0 tx=0x7f346402a8e0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 -- 172.21.15.16:0/3571024989 shutdown_connections 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 --2- 172.21.15.16:0/3571024989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c10d950 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 --2- 172.21.15.16:0/3571024989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 0x7f346c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 --2- 172.21.15.16:0/3571024989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 0x7f346c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 -- 172.21.15.16:0/3571024989 >> 172.21.15.16:0/3571024989 conn(0x7f346c0fc7a0 msgr2=0x7f346c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 -- 172.21.15.16:0/3571024989 shutdown_connections 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.645+0000 7f3472798640 1 -- 172.21.15.16:0/3571024989 wait complete. 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 Processor -- start 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 -- start start 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 0x7f346c0fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 0x7f346c0fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f346c110ea0 con 0x7f346c101420 2024-09-12T01:12:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f346c110d20 con 0x7f346c100a80 2024-09-12T01:12:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3472798640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111020 con 0x7f346c106a80 2024-09-12T01:12:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53510/0 (socket says 172.21.15.16:53510) 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 -- 172.21.15.16:0/623145711 learned_addr learned my addr 172.21.15.16:0/623145711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f346bfff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 0x7f346c0fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3470d35640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 0x7f346c0fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 -- 172.21.15.16:0/623145711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 msgr2=0x7f346c0fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 0x7f346c0fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 -- 172.21.15.16:0/623145711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 msgr2=0x7f346c0fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:20.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 0x7f346c0fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.646+0000 7f3463fff640 1 -- 172.21.15.16:0/623145711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a68b0 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3463fff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f345400c920 tx=0x7f345400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454007bb0 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a6ba0 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3454007d50 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454018780 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.647+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a7130 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.648+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c100ee0 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.649+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f345400f450 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.650+0000 7f3469ffb640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.650+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f34540cf5e0 con 0x7f346c101420 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.650+0000 7f346bfff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.650+0000 7f346bfff640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 0x7f34580a51b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f345c009990 tx=0x7f345c009110 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.654+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3454014030 con 0x7f346c101420 2024-09-12T01:12:20.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:20 smithi099 ceph-mon[35654]: scrub summary: idle 2024-09-12T01:12:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.865+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346c109a90 con 0x7f346c101420 2024-09-12T01:12:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.866+0000 7f3469ffb640 1 -- 172.21.15.16:0/623145711 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f345409c1a0 con 0x7f346c101420 2024-09-12T01:12:20.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:12:20.866 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:12:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 msgr2=0x7f34580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 0x7f34580a51b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f345c009990 tx=0x7f345c009110 comp rx=0 tx=0).stop 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 msgr2=0x7f346c0fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f345400c920 tx=0x7f345400cd20 comp rx=0 tx=0).stop 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 shutdown_connections 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f34580a2cf0 0x7f34580a51b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f346c106a80 0x7f346c0fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c101420 0x7f346c0fd230 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 --2- 172.21.15.16:0/623145711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346c100a80 0x7f346c0fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 >> 172.21.15.16:0/623145711 conn(0x7f346c0fc7a0 msgr2=0x7f346c06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 shutdown_connections 2024-09-12T01:12:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:20.868+0000 7f3472798640 1 -- 172.21.15.16:0/623145711 wait complete. 2024-09-12T01:12:21.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:21 smithi159 ceph-mon[35566]: pgmap v509: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:21.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:21 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/623145711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:21.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:21 smithi016 ceph-mon[30972]: pgmap v509: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:21.634 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:21 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/623145711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:21.664 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24505 quiesce db --cancel --all' 2024-09-12T01:12:21.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24505 quiesce db --cancel --all 2024-09-12T01:12:21.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:21 smithi099 ceph-mon[35654]: pgmap v509: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:21.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:21 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/623145711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 -- 172.21.15.16:0/3780694198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4106f10 msgr2=0x7f2ec410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/3780694198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4106f10 0x7f2ec410ea90 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009ca0 tx=0x7f2ebc02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 -- 172.21.15.16:0/3780694198 shutdown_connections 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/3780694198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4106f10 0x7f2ec410ea90 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/3780694198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4102790 0x7f2ec41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/3780694198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4101de0 0x7f2ec41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 -- 172.21.15.16:0/3780694198 >> 172.21.15.16:0/3780694198 conn(0x7f2ec40fd8c0 msgr2=0x7f2ec40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 -- 172.21.15.16:0/3780694198 shutdown_connections 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.528+0000 7f2ec93b7640 1 -- 172.21.15.16:0/3780694198 wait complete. 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 Processor -- start 2024-09-12T01:12:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 -- start start 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4101de0 0x7f2ec40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4106f10 0x7f2ec4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4111f70 con 0x7f2ec4102790 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4111df0 con 0x7f2ec4106f10 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec93b7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41120f0 con 0x7f2ec4101de0 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.529+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53528/0 (socket says 172.21.15.16:53528) 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 -- 172.21.15.16:0/620674754 learned_addr learned my addr 172.21.15.16:0/620674754 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4106f10 0x7f2ec4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4101de0 0x7f2ec40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 -- 172.21.15.16:0/620674754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4101de0 msgr2=0x7f2ec40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4101de0 0x7f2ec40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 -- 172.21.15.16:0/620674754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4106f10 msgr2=0x7f2ec4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4106f10 0x7f2ec4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 -- 172.21.15.16:0/620674754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec41a68b0 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00d950 tx=0x7f2eac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ebbfff640 1 -- 172.21.15.16:0/620674754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac017030 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ebbfff640 1 -- 172.21.15.16:0/620674754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac004020 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec41a6ba0 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.530+0000 7f2ebbfff640 1 -- 172.21.15.16:0/620674754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0047a0 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.531+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec41a7130 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.533+0000 7f2ebbfff640 1 -- 172.21.15.16:0/620674754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2eac0041d0 con 0x7f2ec4102790 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.533+0000 7f2ecb790740 1 Processor -- start 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.533+0000 7f2ecb790740 1 -- start start 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbaede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf8fb13200 con 0x55bf8fbb9be0 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf8fab9dc0 con 0x55bf8fbaea00 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ecb790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf8fb0f1c0 con 0x55bf8fbaf320 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbaede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.16:51994/0 (socket says 172.21.15.16:51994) 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/3128587858 learned_addr learned my addr 172.21.15.16:0/3128587858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ebbfff640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 0x7f2eb00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 -- 172.21.15.16:0/3128587858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 msgr2=0x55bf8fbb96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 -- 172.21.15.16:0/3128587858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbb9be0 msgr2=0x55bf8fbbbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbbbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ebbfff640 1 -- 172.21.15.16:0/620674754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2eac0ced20 con 0x7f2ec4102790 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 -- 172.21.15.16:0/3128587858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8f884620 con 0x55bf8fbaea00 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 0x7f2eb00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 28, 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24505, 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 56, 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-09-12T01:12:22.626 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.534+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbaede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463512a1f3e19771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 0x7f2eb00a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc049000 tx=0x7f2ebc04e000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2eb97fa640 1 -- 172.21.15.16:0/3128587858 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc042040 con 0x55bf8fbaea00 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2eb97fa640 1 -- 172.21.15.16:0/3128587858 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ebc036040 con 0x55bf8fbaea00 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2eb97fa640 1 -- 172.21.15.16:0/3128587858 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc02fa30 con 0x55bf8fbaea00 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2ecb790740 1 -- 172.21.15.16:0/3128587858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 msgr2=0x55bf8fbaede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.535+0000 7f2ecb790740 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbaede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc006010 tx=0x7f2ebc005fa0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 -- 172.21.15.16:0/3128587858 shutdown_connections 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbbbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 --2- 172.21.15.16:0/3128587858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbaede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 -- 172.21.15.16:0/3128587858 >> 172.21.15.16:0/3128587858 conn(0x55bf8fbaaf90 msgr2=0x55bf8fbab370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 -- 172.21.15.16:0/3128587858 shutdown_connections 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 -- 172.21.15.16:0/3128587858 wait complete. 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 -- start start 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.536+0000 7f2ecb790740 1 Processor -- start 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbb6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf8fbbc7b0 con 0x55bf8fbaea00 2024-09-12T01:12:22.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf8faadc50 con 0x55bf8fbaf320 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf8fbbc930 con 0x55bf8fbb9be0 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ecb790740 10 client.0 _ll_register_callbacks cb 0x55bf8fabfba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53546/0 (socket says 172.21.15.16:53546) 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/413912227 learned_addr learned my addr 172.21.15.16:0/413912227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbb6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 msgr2=0x55bf8fbb6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbb6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/413912227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 msgr2=0x55bf8fbb67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/413912227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8fbb74f0 con 0x55bf8fbaea00 2024-09-12T01:12:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40126d0 tx=0x7f2eb4012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04e078ac46dc5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2eaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004320 con 0x55bf8fbaea00 2024-09-12T01:12:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.537+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2eb40035f0 con 0x55bf8fbaea00 2024-09-12T01:12:22.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ebc036040 con 0x55bf8fbaea00 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2eb40044c0 con 0x55bf8fbaea00 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 10 client.20373 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf8fbd0b70 con 0x55bf8fbaea00 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ebc036040 con 0x55bf8fbaea00 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb4011d00 con 0x55bf8fbaea00 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 10 client.20373 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 10 client.20373 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 20 client.20373 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fbd1f80 0x55bf8fbd4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 4 client.20373 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.538+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf8fb20770 con 0x55bf8fbd1f80 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.539+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fbd1f80 0x55bf8fbd4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.539+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fbd1f80 0x55bf8fbd4390 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482cca33486bdc5f server_cookie=a1fbc06026396f18 in_seq=0 out_seq=0 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.539+0000 7f2eaa7fc640 10 client.20373 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/413912227 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf8fb20770 con 0x55bf8fbd1f80 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2eaa7fc640 10 client.20373 handle_command_reply: tid=0 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 client.20373 shutdown 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 msgr2=0x55bf8fbb9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40126d0 tx=0x7f2eb4012ad0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 shutdown_connections 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fbd1f80 0x55bf8fbd4390 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbb6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbb67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2eaa7fc640 0 client.20373 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 --2- 172.21.15.16:0/413912227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbb9500 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 >> 172.21.15.16:0/413912227 conn(0x55bf8fbaaf90 msgr2=0x55bf8fbaa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 shutdown_connections 2024-09-12T01:12:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 1 -- 172.21.15.16:0/413912227 wait complete. 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.540+0000 7f2ecb790740 20 client.20373 trim_cache size 0 max 16384 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.557+0000 7f2ecb790740 1 Processor -- start 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.557+0000 7f2ecb790740 1 -- start start 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41ab6a0 con 0x55bf8fbaf320 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41ad1e0 con 0x55bf8fbaea00 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ecb790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41abd70 con 0x55bf8fbb9be0 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53550/0 (socket says 172.21.15.16:53550) 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 -- 172.21.15.16:0/186284572 learned_addr learned my addr 172.21.15.16:0/186284572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 -- 172.21.15.16:0/186284572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 msgr2=0x55bf8fbca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 -- 172.21.15.16:0/186284572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 msgr2=0x55bf8fbce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.558+0000 7f2ec27fc640 1 -- 172.21.15.16:0/186284572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8fbaaa30 con 0x55bf8fbaf320 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0976a0 tx=0x7f2eac0d1230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff4bb2f588ad008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2eb8ff9640 1 -- 172.21.15.16:0/186284572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac09aba0 con 0x55bf8fbaf320 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2eb8ff9640 1 -- 172.21.15.16:0/186284572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac09b940 con 0x55bf8fbaf320 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2eb8ff9640 1 -- 172.21.15.16:0/186284572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0978b0 con 0x55bf8fbaf320 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2ecb790740 1 -- 172.21.15.16:0/186284572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 msgr2=0x55bf8fbaa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.559+0000 7f2ecb790740 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0976a0 tx=0x7f2eac0d1230 comp rx=0 tx=0).stop 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- 172.21.15.16:0/186284572 shutdown_connections 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaf320 0x55bf8fbaa0c0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- 172.21.15.16:0/186284572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaea00 0x55bf8fbce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- 172.21.15.16:0/186284572 >> 172.21.15.16:0/186284572 conn(0x55bf8fbaaf90 msgr2=0x55bf8fba99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- 172.21.15.16:0/186284572 shutdown_connections 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- 172.21.15.16:0/186284572 wait complete. 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- start start 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 Processor -- start 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbc89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbc8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41acb10 con 0x55bf8fbaea00 2024-09-12T01:12:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41ac440 con 0x55bf8fbaf320 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec41a96c0 con 0x55bf8fbb9be0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.560+0000 7f2ecb790740 10 client.0 _ll_register_callbacks cb 0x7f2eb4009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53552/0 (socket says 172.21.15.16:53552) 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/2674854032 learned_addr learned my addr 172.21.15.16:0/2674854032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbc89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec37fe640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbc8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 msgr2=0x55bf8fbc8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbc8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/2674854032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 msgr2=0x55bf8fbc89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbc89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 -- 172.21.15.16:0/2674854032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8fc3bf10 con 0x55bf8fbaea00 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ec2ffd640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40126d0 tx=0x7f2eb4026db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dddfeb8db84fed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2eaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb402b040 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2eb4005040 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.561+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eac09b940 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2eb4004320 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 10 client.20379 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf8fc3d230 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac09b940 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2eb4037030 con 0x55bf8fbaea00 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 10 client.20379 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 10 client.20379 resolve_mds: validated gid 24505 aka daemon mds.i 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 20 client.20379 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fc3d940 0x55bf8fc3fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 4 client.20379 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T01:12:22.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55bf8fb1ac80 con 0x55bf8fc3d940 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.562+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fc3d940 0x55bf8fc3fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ec27fc640 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fc3d940 0x55bf8fc3fd30 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d60a1e110c59fe2 server_cookie=f58a85b25c78cb19 in_seq=0 out_seq=0 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2eaa7fc640 10 client.20379 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2eaa7fc640 1 -- 172.21.15.16:0/2674854032 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55bf8fb1ac80 con 0x55bf8fc3d940 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2eaa7fc640 10 client.20379 handle_command_reply: tid=0 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 client.20379 shutdown 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 msgr2=0x55bf8fbc8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40126d0 tx=0x7f2eb4026db0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 shutdown_connections 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55bf8fc3d940 0x55bf8fc3fd30 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8fbb9be0 0x55bf8fbc8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf8fbaf320 0x55bf8fbc89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 --2- 172.21.15.16:0/2674854032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf8fbaea00 0x55bf8fbc8470 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 >> 172.21.15.16:0/2674854032 conn(0x55bf8fbaaf90 msgr2=0x55bf8fbaa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2eaa7fc640 0 client.20379 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 shutdown_connections 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 1 -- 172.21.15.16:0/2674854032 wait complete. 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.563+0000 7f2ecb790740 20 client.20379 trim_cache size 0 max 16384 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 msgr2=0x7f2eb00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 0x7f2eb00a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc049000 tx=0x7f2ebc04e000 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 msgr2=0x7f2ec4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00d950 tx=0x7f2eac00de10 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 shutdown_connections 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2eb00a2d70 0x7f2eb00a5230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec4106f10 0x7f2ec4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ec4102790 0x7f2ec4074a30 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 --2- 172.21.15.16:0/620674754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ec4101de0 0x7f2ec40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 >> 172.21.15.16:0/620674754 conn(0x7f2ec40fd8c0 msgr2=0x7f2ec40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 shutdown_connections 2024-09-12T01:12:22.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:22.564+0000 7f2ec93b7640 1 -- 172.21.15.16:0/620674754 wait complete. 2024-09-12T01:12:23.405 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T01:12:23.405 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T01:12:23.417 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T01:12:23.417 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T01:12:23.417 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:12:23.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T01:12:23.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T01:12:23.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T01:12:23.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:23 smithi159 ceph-mon[35566]: pgmap v510: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:23.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:23 smithi016 ceph-mon[30972]: pgmap v510: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 -- 172.21.15.16:0/2133527179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc1009b0 msgr2=0x7f09cc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 --2- 172.21.15.16:0/2133527179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc1009b0 0x7f09cc100e10 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009930 tx=0x7f09c002a420 comp rx=0 tx=0).stop 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 -- 172.21.15.16:0/2133527179 shutdown_connections 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 --2- 172.21.15.16:0/2133527179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc101350 0x7f09cc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 --2- 172.21.15.16:0/2133527179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc1009b0 0x7f09cc100e10 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 --2- 172.21.15.16:0/2133527179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1069c0 0x7f09cc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 -- 172.21.15.16:0/2133527179 >> 172.21.15.16:0/2133527179 conn(0x7f09cc077990 msgr2=0x7f09cc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 -- 172.21.15.16:0/2133527179 shutdown_connections 2024-09-12T01:12:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.663+0000 7f09d358d640 1 -- 172.21.15.16:0/2133527179 wait complete. 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 Processor -- start 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 -- start start 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1009b0 0x7f09cc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc1069c0 0x7f09cc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc110e50 con 0x7f09cc101350 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc110cd0 con 0x7f09cc1069c0 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d358d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc110fd0 con 0x7f09cc1009b0 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53564/0 (socket says 172.21.15.16:53564) 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 -- 172.21.15.16:0/3523735768 learned_addr learned my addr 172.21.15.16:0/3523735768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d1329640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1009b0 0x7f09cc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 -- 172.21.15.16:0/3523735768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1009b0 msgr2=0x7f09cc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1009b0 0x7f09cc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 -- 172.21.15.16:0/3523735768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc1069c0 msgr2=0x7f09cc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d1b2a640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc1069c0 0x7f09cc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc1069c0 0x7f09cc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.664+0000 7f09d0b28640 1 -- 172.21.15.16:0/3523735768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc19d370 con 0x7f09cc101350 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09d0b28640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f09c002a940 tx=0x7f09c0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c003f070 con 0x7f09cc101350 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc19d600 con 0x7f09cc101350 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c003b460 con 0x7f09cc101350 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0035030 con 0x7f09cc101350 2024-09-12T01:12:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.665+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc1a4ef0 con 0x7f09cc101350 2024-09-12T01:12:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.666+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09cc100e10 con 0x7f09cc101350 2024-09-12T01:12:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.667+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09c0042070 con 0x7f09cc101350 2024-09-12T01:12:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.668+0000 7f09be7fc640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 0x7f09b80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.668+0000 7f09d1329640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 0x7f09b80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.668+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f09c00f00f0 con 0x7f09cc101350 2024-09-12T01:12:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.668+0000 7f09d1329640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 0x7f09b80a5260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005d90 tx=0x7f09c8005d20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.671+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09c00bcdb0 con 0x7f09cc101350 2024-09-12T01:12:23.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:23 smithi099 ceph-mon[35654]: pgmap v510: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-09-12T01:12:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.869+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f09cc062630 con 0x7f09b80a2da0 2024-09-12T01:12:23.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.877+0000 7f09be7fc640 1 -- 172.21.15.16:0/3523735768 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f09cc062630 con 0x7f09b80a2da0 2024-09-12T01:12:23.877 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T00:56:13", "first": "2024-09-12T00:57:00", "last": "2024-09-12T01:12:00", "last_pruned": "2024-09-12T01:12:00", "created_count": 16, "pruned_count": 9, "active": true} 2024-09-12T01:12:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 msgr2=0x7f09b80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 0x7f09b80a5260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005d90 tx=0x7f09c8005d20 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 msgr2=0x7f09cc1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f09c002a940 tx=0x7f09c0003c00 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 shutdown_connections 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f09b80a2da0 0x7f09b80a5260 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09cc1069c0 0x7f09cc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc101350 0x7f09cc1988c0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 --2- 172.21.15.16:0/3523735768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09cc1009b0 0x7f09cc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 >> 172.21.15.16:0/3523735768 conn(0x7f09cc077990 msgr2=0x7f09cc069a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 shutdown_connections 2024-09-12T01:12:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.880+0000 7f09d358d640 1 -- 172.21.15.16:0/3523735768 wait complete. 2024-09-12T01:12:23.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T01:12:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 -- 172.21.15.16:0/1595924793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41080f0 msgr2=0x7f53f410ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 --2- 172.21.15.16:0/1595924793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41080f0 0x7f53f410ea10 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009ca0 tx=0x7f53ec02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 -- 172.21.15.16:0/1595924793 shutdown_connections 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 --2- 172.21.15.16:0/1595924793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41080f0 0x7f53f410ea10 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 --2- 172.21.15.16:0/1595924793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4101250 0x7f53f4107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 --2- 172.21.15.16:0/1595924793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41008a0 0x7f53f4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 -- 172.21.15.16:0/1595924793 >> 172.21.15.16:0/1595924793 conn(0x7f53f4075ef0 msgr2=0x7f53f4076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 -- 172.21.15.16:0/1595924793 shutdown_connections 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.996+0000 7f53f968b640 1 -- 172.21.15.16:0/1595924793 wait complete. 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 Processor -- start 2024-09-12T01:12:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 -- start start 2024-09-12T01:12:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41080f0 0x7f53f4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 0x7f53f41a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4111f20 con 0x7f53f41008a0 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53f4111da0 con 0x7f53f4102290 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f968b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53f41120a0 con 0x7f53f41080f0 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.997+0000 7f53f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53582/0 (socket says 172.21.15.16:53582) 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 -- 172.21.15.16:0/1938310290 learned_addr learned my addr 172.21.15.16:0/1938310290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f27fc640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41080f0 0x7f53f4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f37fe640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 0x7f53f41a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 -- 172.21.15.16:0/1938310290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41080f0 msgr2=0x7f53f4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41080f0 0x7f53f4101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 -- 172.21.15.16:0/1938310290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 msgr2=0x7f53f41a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 0x7f53f41a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 -- 172.21.15.16:0/1938310290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f41a6b80 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f37fe640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 0x7f53f41a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f2ffd640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e9f0 tx=0x7f53e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e400cd80 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e4005500 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f41a6e70 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4010690 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.998+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f41a72e0 con 0x7f53f41008a0 2024-09-12T01:12:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:23.999+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f4100c80 con 0x7f53f41008a0 2024-09-12T01:12:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.000+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f53e40046c0 con 0x7f53f41008a0 2024-09-12T01:12:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.001+0000 7f53dbfff640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 0x7f53dc0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.001+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f53e40cf890 con 0x7f53f41008a0 2024-09-12T01:12:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.001+0000 7f53f27fc640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 0x7f53dc0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.002+0000 7f53f27fc640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 0x7f53dc0a5290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f53f41032a0 tx=0x7f53e800a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.005+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53e409c4b0 con 0x7f53f41008a0 2024-09-12T01:12:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.203+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 --> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f53f4101390 con 0x7f53dc0a2dd0 2024-09-12T01:12:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.211+0000 7f53dbfff640 1 -- 172.21.15.16:0/1938310290 <== mgr.15213 v2:172.21.15.16:6800/1052477540 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f53f4101390 con 0x7f53dc0a2dd0 2024-09-12T01:12:24.210 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-12T01:12:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 msgr2=0x7f53dc0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 0x7f53dc0a5290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f53f41032a0 tx=0x7f53e800a2d0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 msgr2=0x7f53f4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f53e400e9f0 tx=0x7f53e400edf0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 shutdown_connections 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f53dc0a2dd0 0x7f53dc0a5290 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53f4102290 0x7f53f41a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53f41080f0 0x7f53f4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 --2- 172.21.15.16:0/1938310290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f41008a0 0x7f53f4101810 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 >> 172.21.15.16:0/1938310290 conn(0x7f53f4075ef0 msgr2=0x7f53f40ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 shutdown_connections 2024-09-12T01:12:24.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:24.213+0000 7f53f968b640 1 -- 172.21.15.16:0/1938310290 wait complete. 2024-09-12T01:12:24.223 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T01:12:24.233 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T01:12:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:25 smithi159 ceph-mon[35566]: pgmap v511: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:12:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:25 smithi159 ceph-mon[35566]: from='client.20385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:25 smithi159 ceph-mon[35566]: from='client.20391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:25 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:12:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:25 smithi016 ceph-mon[30972]: pgmap v511: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:12:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:25 smithi016 ceph-mon[30972]: from='client.20385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:25 smithi016 ceph-mon[30972]: from='client.20391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:25 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:12:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:25 smithi099 ceph-mon[35654]: pgmap v511: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 69 KiB/s wr, 15 op/s 2024-09-12T01:12:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:25 smithi099 ceph-mon[35654]: from='client.20385 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:25 smithi099 ceph-mon[35654]: from='client.20391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T01:12:25.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:25 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:12:26.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:12:26.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:12:26.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:12:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:12:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:12:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:12:26.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:12:26.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:12:26.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:12:27.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:27 smithi159 ceph-mon[35566]: pgmap v512: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 70 KiB/s wr, 21 op/s 2024-09-12T01:12:27.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:27 smithi016 ceph-mon[30972]: pgmap v512: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 70 KiB/s wr, 21 op/s 2024-09-12T01:12:27.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:27 smithi099 ceph-mon[35654]: pgmap v512: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 70 KiB/s wr, 21 op/s 2024-09-12T01:12:29.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:29 smithi159 ceph-mon[35566]: pgmap v513: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-12T01:12:29.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:29 smithi016 ceph-mon[30972]: pgmap v513: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-12T01:12:29.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:29 smithi099 ceph-mon[35654]: pgmap v513: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-09-12T01:12:31.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:31 smithi159 ceph-mon[35566]: pgmap v514: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:12:31.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:31 smithi016 ceph-mon[30972]: pgmap v514: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:12:31.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:31 smithi099 ceph-mon[35654]: pgmap v514: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-12T01:12:33.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:33 smithi159 ceph-mon[35566]: pgmap v515: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:33.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:33 smithi016 ceph-mon[30972]: pgmap v515: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:33.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:33 smithi099 ceph-mon[35654]: pgmap v515: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:34.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:34 smithi099 ceph-mon[35654]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:34.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:34 smithi159 ceph-mon[35566]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:34.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:34 smithi016 ceph-mon[30972]: pgmap v516: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-12T01:12:37.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:36 smithi099 ceph-mon[35654]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-12T01:12:37.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:36 smithi159 ceph-mon[35566]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-12T01:12:37.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:36 smithi016 ceph-mon[30972]: pgmap v517: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-09-12T01:12:38.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:38 smithi099 ceph-mon[35654]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T01:12:38.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:38 smithi159 ceph-mon[35566]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T01:12:38.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:38 smithi016 ceph-mon[30972]: pgmap v518: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-12T01:12:41.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:40 smithi099 ceph-mon[35654]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-12T01:12:41.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:40 smithi159 ceph-mon[35566]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-12T01:12:41.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:40 smithi016 ceph-mon[30972]: pgmap v519: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-09-12T01:12:43.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:42 smithi099 ceph-mon[35654]: pgmap v520: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:43.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:42 smithi159 ceph-mon[35566]: pgmap v520: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:43.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:42 smithi016 ceph-mon[30972]: pgmap v520: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:45.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:44 smithi099 ceph-mon[35654]: pgmap v521: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:45.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:44 smithi159 ceph-mon[35566]: pgmap v521: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:45.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:44 smithi016 ceph-mon[30972]: pgmap v521: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-09-12T01:12:47.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:46 smithi099 ceph-mon[35654]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T01:12:47.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:46 smithi159 ceph-mon[35566]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T01:12:47.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:46 smithi016 ceph-mon[30972]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T01:12:49.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:48 smithi099 ceph-mon[35654]: pgmap v523: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8 op/s 2024-09-12T01:12:49.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:48 smithi159 ceph-mon[35566]: pgmap v523: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8 op/s 2024-09-12T01:12:49.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:48 smithi016 ceph-mon[30972]: pgmap v523: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8 op/s 2024-09-12T01:12:50.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T01:12:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:50.833 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:50 smithi016 ceph-mon[30972]: pgmap v524: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 op/s 2024-09-12T01:12:51.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:50 smithi099 ceph-mon[35654]: pgmap v524: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 op/s 2024-09-12T01:12:51.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:50 smithi159 ceph-mon[35566]: pgmap v524: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 12 op/s 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.384+0000 7f268edf1640 1 -- 172.21.15.16:0/445236322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268810c2f0 msgr2=0x7f268810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.384+0000 7f268edf1640 1 --2- 172.21.15.16:0/445236322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268810c2f0 0x7f268810e790 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2670009900 tx=0x7f267002a410 comp rx=0 tx=0).stop 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 -- 172.21.15.16:0/445236322 shutdown_connections 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 --2- 172.21.15.16:0/445236322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268810c2f0 0x7f268810e790 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 --2- 172.21.15.16:0/445236322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26881037a0 0x7f268810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 --2- 172.21.15.16:0/445236322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2688102df0 0x7f26881031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 -- 172.21.15.16:0/445236322 >> 172.21.15.16:0/445236322 conn(0x7f26880fc7d0 msgr2=0x7f26880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 -- 172.21.15.16:0/445236322 shutdown_connections 2024-09-12T01:12:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 -- 172.21.15.16:0/445236322 wait complete. 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 Processor -- start 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.385+0000 7f268edf1640 1 -- start start 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 0x7f26881a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f268810c2f0 0x7f268819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2688112110 con 0x7f2688102df0 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2688111f90 con 0x7f268810c2f0 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268edf1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2688112290 con 0x7f26881037a0 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35038/0 (socket says 172.21.15.16:35038) 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 -- 172.21.15.16:0/2729291946 learned_addr learned my addr 172.21.15.16:0/2729291946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f2687fff640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 0x7f26881a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268d38e640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f268810c2f0 0x7f268819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 -- 172.21.15.16:0/2729291946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 msgr2=0x7f26881a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 0x7f26881a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 -- 172.21.15.16:0/2729291946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f268810c2f0 msgr2=0x7f268819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f268810c2f0 0x7f268819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f268cb8d640 1 -- 172.21.15.16:0/2729291946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268819ad30 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.386+0000 7f2687fff640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 0x7f26881a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f268000ea80 tx=0x7f268000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f2685ffb640 1 -- 172.21.15.16:0/2729291946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000ce40 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268819b020 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f2685ffb640 1 -- 172.21.15.16:0/2729291946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2680005020 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f2685ffb640 1 -- 172.21.15.16:0/2729291946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680011760 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.387+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881a7310 con 0x7f2688102df0 2024-09-12T01:12:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.389+0000 7f2685ffb640 1 -- 172.21.15.16:0/2729291946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26800051d0 con 0x7f2688102df0 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 Processor -- start 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 -- start start 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de08b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556de081d2e0 con 0x556de08b7dc0 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556de07c4ce0 con 0x556de08c2680 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f26911ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556de08192a0 con 0x556de08b74a0 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f2685ffb640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 0x7f26740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35040/0 (socket says 172.21.15.16:35040) 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f268cb8d640 1 -- 172.21.15.16:0/2210415236 learned_addr learned my addr 172.21.15.16:0/2210415236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.390+0000 7f268d38e640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de08b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268d38e640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 0x7f26740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f2687fff640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f2685ffb640 1 -- 172.21.15.16:0/2729291946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f26800cfa30 con 0x7f2688102df0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 -- 172.21.15.16:0/2210415236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 msgr2=0x556de08b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de08b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 -- 172.21.15.16:0/2210415236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 msgr2=0x556de08c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 -- 172.21.15.16:0/2210415236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de07a9440 con 0x556de08b7dc0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268d38e640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de08b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268cb8d640 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f26800a4690 tx=0x7f26800a6710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204753525441b01c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f268d38e640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 0x7f26740a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2670031550 tx=0x7f2670009260 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f267effd640 1 -- 172.21.15.16:0/2210415236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268001f040 con 0x556de08b7dc0 2024-09-12T01:12:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f267effd640 1 -- 172.21.15.16:0/2210415236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268009c000 con 0x556de08b7dc0 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.391+0000 7f267effd640 1 -- 172.21.15.16:0/2210415236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680098670 con 0x556de08b7dc0 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- 172.21.15.16:0/2210415236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 msgr2=0x556de08c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f26800a4690 tx=0x7f26800a6710 comp rx=0 tx=0).stop 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- 172.21.15.16:0/2210415236 shutdown_connections 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de08c2140 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 --2- 172.21.15.16:0/2210415236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de08b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- 172.21.15.16:0/2210415236 >> 172.21.15.16:0/2210415236 conn(0x556de08b3a30 msgr2=0x556de08b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- 172.21.15.16:0/2210415236 shutdown_connections 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- 172.21.15.16:0/2210415236 wait complete. 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.392+0000 7f26911ca740 1 -- start start 2024-09-12T01:12:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 Processor -- start 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556de08c5250 con 0x556de08b74a0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556de07d8050 con 0x556de08c2680 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556de08c53d0 con 0x556de08b7dc0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f26911ca740 10 client.0 _ll_register_callbacks cb 0x556de0818160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f268cb8d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35052/0 (socket says 172.21.15.16:35052) 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f268cb8d640 1 -- 172.21.15.16:0/56988185 learned_addr learned my addr 172.21.15.16:0/56988185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f2687fff640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.393+0000 7f268d38e640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 -- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 msgr2=0x556de08bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 -- 172.21.15.16:0/56988185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 msgr2=0x556de08bf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08bf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 -- 172.21.15.16:0/56988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de058e3c0 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268d38e640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2687fff640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f268cb8d640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f268009bf90 tx=0x7f268009a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b11b8fb24fbf33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268001f070 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f268009c260 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268009c000 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f268009c4e0 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f26911ca740 10 client.20403 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556de08d0c70 con 0x556de08b74a0 2024-09-12T01:12:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.394+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f268009c000 con 0x556de08b74a0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f268000f040 con 0x556de08b74a0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 10 client.20403 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 10 client.20403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 20 client.20403 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de08d1fc0 0x556de08d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 4 client.20403 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556de082a850 con 0x556de08d1fc0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.395+0000 7f2687fff640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de08d1fc0 0x556de08d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.396+0000 7f2667fff640 10 client.20403 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.396+0000 7f2687fff640 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de08d1fc0 0x556de08d43b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f940d7d63b4876a server_cookie=c45334dad41c7087 in_seq=0 out_seq=0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.396+0000 7f2667fff640 1 -- 172.21.15.16:0/56988185 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556de082a850 con 0x556de08d1fc0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.396+0000 7f2667fff640 10 client.20403 handle_command_reply: tid=0 2024-09-12T01:12:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.396+0000 7f26911ca740 1 client.20403 shutdown 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 msgr2=0x556de08c2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f268009bf90 tx=0x7f268009a510 comp rx=0 tx=0).stop 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 shutdown_connections 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de08d1fc0 0x556de08d43b0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de08bf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 --2- 172.21.15.16:0/56988185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b74a0 0x556de08c2040 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 >> 172.21.15.16:0/56988185 conn(0x556de08b3a30 msgr2=0x556de08b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f2667fff640 0 client.20403 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 shutdown_connections 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 1 -- 172.21.15.16:0/56988185 wait complete. 2024-09-12T01:12:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.397+0000 7f26911ca740 20 client.20403 trim_cache size 0 max 16384 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 Processor -- start 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 -- start start 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 0x556de08d73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26881afbb0 con 0x556de08c2680 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b16f0 con 0x556de08b74a0 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f26911ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b0280 con 0x556de08b7dc0 2024-09-12T01:12:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f268d38e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f268d38e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35066/0 (socket says 172.21.15.16:35066) 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f268d38e640 1 -- 172.21.15.16:0/713355282 learned_addr learned my addr 172.21.15.16:0/713355282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f268cb8d640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 0x556de08d73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.415+0000 7f2687fff640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08d7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 -- 172.21.15.16:0/713355282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 msgr2=0x556de08d7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08d7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 -- 172.21.15.16:0/713355282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 msgr2=0x556de08d73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 0x556de08d73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 -- 172.21.15.16:0/713355282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de08b32e0 con 0x556de08c2680 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268cb8d640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 0x556de08d73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f268d38e640 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f2670033000 tx=0x7f267003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205f2b598eb1f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f267e7fc640 1 -- 172.21.15.16:0/713355282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670035070 con 0x556de08c2680 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f267e7fc640 1 -- 172.21.15.16:0/713355282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2670051de0 con 0x556de08c2680 2024-09-12T01:12:51.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.416+0000 7f267e7fc640 1 -- 172.21.15.16:0/713355282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670004a00 con 0x556de08c2680 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 -- 172.21.15.16:0/713355282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 msgr2=0x556de08bc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f2670033000 tx=0x7f267003a000 comp rx=0 tx=0).stop 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 -- 172.21.15.16:0/713355282 shutdown_connections 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08c2680 0x556de08bc550 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b7dc0 0x556de08d7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 --2- 172.21.15.16:0/713355282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08b74a0 0x556de08d73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 -- 172.21.15.16:0/713355282 >> 172.21.15.16:0/713355282 conn(0x556de08b3a30 msgr2=0x556de08b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.417+0000 7f26911ca740 1 -- 172.21.15.16:0/713355282 shutdown_connections 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 -- 172.21.15.16:0/713355282 wait complete. 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 -- start start 2024-09-12T01:12:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 Processor -- start 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de0930890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de093b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b1020 con 0x556de08b7dc0 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26881b0950 con 0x556de08c2680 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26881adbb0 con 0x556de08b74a0 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.418+0000 7f26911ca740 10 client.0 _ll_register_callbacks cb 0x7f267800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35080/0 (socket says 172.21.15.16:35080) 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 -- 172.21.15.16:0/3067299903 learned_addr learned my addr 172.21.15.16:0/3067299903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f268cb8d640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de0930890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f268d38e640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de093b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 -- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 msgr2=0x556de0930890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de0930890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 -- 172.21.15.16:0/3067299903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 msgr2=0x556de093b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de093b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 -- 172.21.15.16:0/3067299903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de08c23b0 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2687fff640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f2678011570 tx=0x7f2678012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82d8045c5ce5478 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678010e30 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2678005020 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.419+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2670051de0 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2678005220 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 10 client.20409 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556de093cec0 con 0x556de08b7dc0 2024-09-12T01:12:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2670051de0 con 0x556de08b7dc0 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2678004b40 con 0x556de08b7dc0 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 10 client.20409 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 10 client.20409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 20 client.20409 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de093d670 0x556de093fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 4 client.20409 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556de0824d60 con 0x556de093d670 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.420+0000 7f268cb8d640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de093d670 0x556de093fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f2667fff640 10 client.20409 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f268cb8d640 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de093d670 0x556de093fa90 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd7e4754e50ded7 server_cookie=c22a9b286db9a937 in_seq=0 out_seq=0 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f2667fff640 1 -- 172.21.15.16:0/3067299903 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556de0824d60 con 0x556de093d670 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f2667fff640 10 client.20409 handle_command_reply: tid=0 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 client.20409 shutdown 2024-09-12T01:12:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 msgr2=0x556de0930dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f2678011570 tx=0x7f2678012bd0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 shutdown_connections 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x556de093d670 0x556de093fa90 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556de08c2680 0x556de093b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556de08b7dc0 0x556de0930dd0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 --2- 172.21.15.16:0/3067299903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556de08b74a0 0x556de0930890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 >> 172.21.15.16:0/3067299903 conn(0x556de08b3a30 msgr2=0x556de08c0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f2667fff640 0 client.20409 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 shutdown_connections 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.421+0000 7f26911ca740 1 -- 172.21.15.16:0/3067299903 wait complete. 2024-09-12T01:12:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f26911ca740 20 client.20409 trim_cache size 0 max 16384 2024-09-12T01:12:51.429 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:12:51.429 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-12T01:12:51.429 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-12T01:12:51.429 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 msgr2=0x7f26740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 0x7f26740a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2670031550 tx=0x7f2670009260 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 msgr2=0x7f26881a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f268000ea80 tx=0x7f268000ef40 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 shutdown_connections 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f26740a2cf0 0x7f26740a51b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f268810c2f0 0x7f268819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26881037a0 0x7f26881a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 --2- 172.21.15.16:0/2729291946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2688102df0 0x7f26881a0440 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 >> 172.21.15.16:0/2729291946 conn(0x7f26880fc7d0 msgr2=0x7f2688101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 shutdown_connections 2024-09-12T01:12:51.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:51.422+0000 7f268edf1640 1 -- 172.21.15.16:0/2729291946 wait complete. 2024-09-12T01:12:52.220 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T01:12:52.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:12:52.480 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:52.871 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:52 smithi016 ceph-mon[30972]: pgmap v525: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 -- 172.21.15.16:0/2366038549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904102dd0 msgr2=0x7f49041031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 --2- 172.21.15.16:0/2366038549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904102dd0 0x7f49041031b0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009cd0 tx=0x7f48f402a8d0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 -- 172.21.15.16:0/2366038549 shutdown_connections 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 --2- 172.21.15.16:0/2366038549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904108300 0x7f490410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 --2- 172.21.15.16:0/2366038549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 0x7f4904107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 --2- 172.21.15.16:0/2366038549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904102dd0 0x7f49041031b0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 -- 172.21.15.16:0/2366038549 >> 172.21.15.16:0/2366038549 conn(0x7f49040fc7d0 msgr2=0x7f49040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 -- 172.21.15.16:0/2366038549 shutdown_connections 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.956+0000 7f49099f5640 1 -- 172.21.15.16:0/2366038549 wait complete. 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 Processor -- start 2024-09-12T01:12:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 -- start start 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904102dd0 0x7f4904198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 0x7f4904198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4904112110 con 0x7f4904108300 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4904111f90 con 0x7f4904103780 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49099f5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4904112290 con 0x7f4904102dd0 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35100/0 (socket says 172.21.15.16:35100) 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49037fe640 1 -- 172.21.15.16:0/1260488490 learned_addr learned my addr 172.21.15.16:0/1260488490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.957+0000 7f49027fc640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 0x7f4904198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f4902ffd640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904102dd0 0x7f4904198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 -- 172.21.15.16:0/1260488490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904102dd0 msgr2=0x7f4904198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:52.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904102dd0 0x7f4904198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 -- 172.21.15.16:0/1260488490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 msgr2=0x7f4904198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 0x7f4904198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 -- 172.21.15.16:0/1260488490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490419d5c0 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49037fe640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009fd0 tx=0x7f48fc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc011830 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc016070 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490419d8b0 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00f2e0 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.958+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49041a5190 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.959+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490406a7b0 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.960+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f48fc0119d0 con 0x7f4904108300 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.961+0000 7f49089f3640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 0x7f48ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.961+0000 7f4902ffd640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 0x7f48ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:52.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.961+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f48fc0cf0e0 con 0x7f4904108300 2024-09-12T01:12:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.962+0000 7f4902ffd640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 0x7f48ec0a51b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000f80 tx=0x7f48f4005ef0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:52.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:52.964+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48fc09bdb0 con 0x7f4904108300 2024-09-12T01:12:53.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:52 smithi099 ceph-mon[35654]: pgmap v525: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:53.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:52 smithi159 ceph-mon[35566]: pgmap v525: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.174+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49041031b0 con 0x7f4904108300 2024-09-12T01:12:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.175+0000 7f49089f3640 1 -- 172.21.15.16:0/1260488490 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f49041031b0 con 0x7f4904108300 2024-09-12T01:12:53.174 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:12:53.174 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:12:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 msgr2=0x7f48ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 0x7f48ec0a51b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f48f4000f80 tx=0x7f48f4005ef0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 msgr2=0x7f490419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009fd0 tx=0x7f48fc00b8e0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 shutdown_connections 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f48ec0a2cf0 0x7f48ec0a51b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4904108300 0x7f490419cee0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4904103780 0x7f4904198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 --2- 172.21.15.16:0/1260488490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4904102dd0 0x7f4904198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 >> 172.21.15.16:0/1260488490 conn(0x7f49040fc7d0 msgr2=0x7f4904105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 shutdown_connections 2024-09-12T01:12:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:53.177+0000 7f49099f5640 1 -- 172.21.15.16:0/1260488490 wait complete. 2024-09-12T01:12:53.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:53 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1260488490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:53.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T01:12:54.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:53 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1260488490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:54.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:53 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1260488490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:12:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 -- 172.21.15.16:0/2328307546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3898102b90 msgr2=0x7f3898102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 --2- 172.21.15.16:0/2328307546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3898102b90 0x7f3898102f70 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3888009900 tx=0x7f388802a480 comp rx=0 tx=0).stop 2024-09-12T01:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 -- 172.21.15.16:0/2328307546 shutdown_connections 2024-09-12T01:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 --2- 172.21.15.16:0/2328307546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f389810bfc0 0x7f389810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 --2- 172.21.15.16:0/2328307546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38981034b0 0x7f389810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 --2- 172.21.15.16:0/2328307546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3898102b90 0x7f3898102f70 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 -- 172.21.15.16:0/2328307546 >> 172.21.15.16:0/2328307546 conn(0x7f38980fc630 msgr2=0x7f38980fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 -- 172.21.15.16:0/2328307546 shutdown_connections 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.730+0000 7f389759e640 1 -- 172.21.15.16:0/2328307546 wait complete. 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.731+0000 7f389759e640 1 Processor -- start 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.731+0000 7f389759e640 1 -- start start 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3898102b90 0x7f389819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f389810bfc0 0x7f389819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3898111dc0 con 0x7f38981034b0 2024-09-12T01:12:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3898111c40 con 0x7f389810bfc0 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3898111f40 con 0x7f3898102b90 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35116/0 (socket says 172.21.15.16:35116) 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 -- 172.21.15.16:0/3048599078 learned_addr learned my addr 172.21.15.16:0/3048599078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f3896d9d640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f389810bfc0 0x7f389819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389659c640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3898102b90 0x7f389819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 -- 172.21.15.16:0/3048599078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3898102b90 msgr2=0x7f389819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3898102b90 0x7f389819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 -- 172.21.15.16:0/3048599078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f389810bfc0 msgr2=0x7f389819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f389810bfc0 0x7f389819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 -- 172.21.15.16:0/3048599078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389819f4d0 con 0x7f38981034b0 2024-09-12T01:12:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f388000d950 tx=0x7f388000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388ffff640 1 -- 172.21.15.16:0/3048599078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880017030 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f388ffff640 1 -- 172.21.15.16:0/3048599078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880004020 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.732+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389819f7c0 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.733+0000 7f388ffff640 1 -- 172.21.15.16:0/3048599078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800047a0 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.733+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981a70a0 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.735+0000 7f388ffff640 1 -- 172.21.15.16:0/3048599078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f38800041d0 con 0x7f38981034b0 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.735+0000 7f389efdb740 1 Processor -- start 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.735+0000 7f389efdb740 1 -- start start 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcbebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bccb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e62bc242f0 con 0x55e62bcbfd20 2024-09-12T01:12:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e62bbcbcf0 con 0x55e62bcc9550 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389efdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e62bc202b0 con 0x55e62bcbe7b0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f388ffff640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 0x7f38840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35130/0 (socket says 172.21.15.16:35130) 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 -- 172.21.15.16:0/2641154033 learned_addr learned my addr 172.21.15.16:0/2641154033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 0x7f38840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f388dd9b640 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bccb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f389659c640 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcbebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f388ffff640 1 -- 172.21.15.16:0/3048599078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f38800cedd0 con 0x7f38981034b0 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 -- 172.21.15.16:0/2641154033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 msgr2=0x55e62bcbebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcbebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 -- 172.21.15.16:0/2641154033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 msgr2=0x55e62bccb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bccb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 -- 172.21.15.16:0/2641154033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62bbb03a0 con 0x55e62bcbfd20 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.736+0000 7f3896d9d640 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3890005cc0 tx=0x7f38900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a06d695a5ca3cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f3896d9d640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 0x7f38840a51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3890011cb0 tx=0x7f3890016000 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f388cd99640 1 -- 172.21.15.16:0/2641154033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890024070 con 0x55e62bcbfd20 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f388cd99640 1 -- 172.21.15.16:0/2641154033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389000ac90 con 0x55e62bcbfd20 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f388cd99640 1 -- 172.21.15.16:0/2641154033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890020550 con 0x55e62bcbfd20 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 -- 172.21.15.16:0/2641154033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 msgr2=0x55e62bcbe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f3890005cc0 tx=0x7f38900117b0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 -- 172.21.15.16:0/2641154033 shutdown_connections 2024-09-12T01:12:54.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bccb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcbebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 --2- 172.21.15.16:0/2641154033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcbe270 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.737+0000 7f389efdb740 1 -- 172.21.15.16:0/2641154033 >> 172.21.15.16:0/2641154033 conn(0x55e62bc87310 msgr2=0x55e62bcbae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- 172.21.15.16:0/2641154033 shutdown_connections 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- 172.21.15.16:0/2641154033 wait complete. 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- start start 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 Processor -- start 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bcc68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e62bccc1f0 con 0x55e62bcbfd20 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e62bbdf060 con 0x55e62bcc9550 2024-09-12T01:12:54.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e62bccc370 con 0x55e62bcbe7b0 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389efdb740 10 client.0 _ll_register_callbacks cb 0x55e62b9328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f388dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35140/0 (socket says 172.21.15.16:35140) 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f388dd9b640 1 -- 172.21.15.16:0/3731460272 learned_addr learned my addr 172.21.15.16:0/3731460272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.738+0000 7f389659c640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcc8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f3896d9d640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bcc68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 -- 172.21.15.16:0/3731460272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 msgr2=0x55e62bcc8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcc8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 -- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 msgr2=0x55e62bcc68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bcc68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 -- 172.21.15.16:0/3731460272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62b9953c0 con 0x55e62bcbfd20 2024-09-12T01:12:54.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f38800cc5d0 tx=0x7f388001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33253fdad8a587d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f387dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880095e60 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389000ac90 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f3880097960 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f388009ac50 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f389efdb740 10 client.20427 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e62bcd7d80 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.739+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389000ac90 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38800b4140 con 0x55e62bcbfd20 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 10 client.20427 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 10 client.20427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 20 client.20427 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bcd91e0 0x55e62bcdb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 4 client.20427 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e62bc31860 con 0x55e62bcd91e0 2024-09-12T01:12:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389659c640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bcd91e0 0x55e62bcdb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f387dffb640 10 client.20427 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.740+0000 7f389659c640 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bcd91e0 0x55e62bcdb5d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a83615d54cd1084d server_cookie=b155faf5c2493034 in_seq=0 out_seq=0 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f387dffb640 1 -- 172.21.15.16:0/3731460272 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e62bc31860 con 0x55e62bcd91e0 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f387dffb640 10 client.20427 handle_command_reply: tid=0 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f389efdb740 1 client.20427 shutdown 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 msgr2=0x55e62bcc6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f38800cc5d0 tx=0x7f388001af90 comp rx=0 tx=0).stop 2024-09-12T01:12:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 shutdown_connections 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.741+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bcd91e0 0x55e62bcdb5d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcc9550 0x55e62bcc68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbfd20 0x55e62bcc6240 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 --2- 172.21.15.16:0/3731460272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcbe7b0 0x55e62bcc8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 >> 172.21.15.16:0/3731460272 conn(0x55e62bc87310 msgr2=0x55e62bcc0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f387dffb640 0 client.20427 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 shutdown_connections 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 1 -- 172.21.15.16:0/3731460272 wait complete. 2024-09-12T01:12:54.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.742+0000 7f389efdb740 20 client.20427 trim_cache size 0 max 16384 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 Processor -- start 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 -- start start 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bcb5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bcc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38981af7d0 con 0x55e62bcbe7b0 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b1310 con 0x55e62bcbfd20 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389efdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38981afea0 con 0x55e62bcc9550 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35144/0 (socket says 172.21.15.16:35144) 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 -- 172.21.15.16:0/3672568065 learned_addr learned my addr 172.21.15.16:0/3672568065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f388dd9b640 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bcb5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f3896d9d640 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bcc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 -- 172.21.15.16:0/3672568065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 msgr2=0x55e62bcc5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bcc5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 -- 172.21.15.16:0/3672568065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 msgr2=0x55e62bcb5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bcb5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.755+0000 7f389659c640 1 -- 172.21.15.16:0/3672568065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62bcc92b0 con 0x55e62bcbe7b0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389659c640 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3888032fd0 tx=0x7f38880058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc6b25afeb47e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f387ffff640 1 -- 172.21.15.16:0/3672568065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888049070 con 0x55e62bcbe7b0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f387ffff640 1 -- 172.21.15.16:0/3672568065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3888005520 con 0x55e62bcbe7b0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f387ffff640 1 -- 172.21.15.16:0/3672568065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888035040 con 0x55e62bcbe7b0 2024-09-12T01:12:54.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 -- 172.21.15.16:0/3672568065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 msgr2=0x55e62bcd5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f3888032fd0 tx=0x7f38880058b0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 -- 172.21.15.16:0/3672568065 shutdown_connections 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bcc5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bcb5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 --2- 172.21.15.16:0/3672568065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd5490 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 -- 172.21.15.16:0/3672568065 >> 172.21.15.16:0/3672568065 conn(0x55e62bc87310 msgr2=0x55e62bcc4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 -- 172.21.15.16:0/3672568065 shutdown_connections 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.756+0000 7f389efdb740 1 -- 172.21.15.16:0/3672568065 wait complete. 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 -- start start 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 Processor -- start 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bccce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bccd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b0c40 con 0x55e62bcbe7b0 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38981b0570 con 0x55e62bcbfd20 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38981ad7d0 con 0x55e62bcc9550 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389efdb740 10 client.0 _ll_register_callbacks cb 0x7f38800046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35154/0 (socket says 172.21.15.16:35154) 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 -- 172.21.15.16:0/1510614948 learned_addr learned my addr 172.21.15.16:0/1510614948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f388dd9b640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bccce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 -- 172.21.15.16:0/1510614948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 msgr2=0x55e62bccd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bccd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 -- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 msgr2=0x55e62bccce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bccce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f389659c640 1 -- 172.21.15.16:0/1510614948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62bccd940 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.757+0000 7f388dd9b640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bccce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389659c640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f388802a9a0 tx=0x7f3888009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58f43f8ab358cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888049040 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f38880048c0 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3888005520 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3888035040 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 10 client.20433 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e62bd43c40 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3888005520 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3888055030 con 0x55e62bcbe7b0 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 10 client.20433 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 10 client.20433 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 20 client.20433 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bd44300 0x55e62bd46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.758+0000 7f389efdb740 4 client.20433 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 --> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e62bc2bd70 con 0x55e62bd44300 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f388dd9b640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bd44300 0x55e62bd46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f388dd9b640 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bd44300 0x55e62bd46760 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97623ca78771a5be server_cookie=c536aa84fb12b43a in_seq=0 out_seq=0 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f387dffb640 10 client.20433 ms_handle_connect on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f387dffb640 1 -- 172.21.15.16:0/1510614948 <== mds.4 v2:172.21.15.99:6836/1045510613 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e62bc2bd70 con 0x55e62bd44300 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f387dffb640 10 client.20433 handle_command_reply: tid=0 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 client.20433 shutdown 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 msgr2=0x55e62bcd1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f388802a9a0 tx=0x7f3888009520 comp rx=0 tx=0).stop 2024-09-12T01:12:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 shutdown_connections 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:6836/1045510613,v1:172.21.15.99:6837/1045510613] conn(0x55e62bd44300 0x55e62bd46760 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f387dffb640 0 client.20433 ms_handle_reset on v2:172.21.15.99:6836/1045510613 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.759+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e62bcc9550 0x55e62bccd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e62bcbfd20 0x55e62bccce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 1 --2- 172.21.15.16:0/1510614948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e62bcbe7b0 0x55e62bcd1b10 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 >> 172.21.15.16:0/1510614948 conn(0x55e62bc87310 msgr2=0x55e62bcb9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 shutdown_connections 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 1 -- 172.21.15.16:0/1510614948 wait complete. 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389efdb740 20 client.20433 trim_cache size 0 max 16384 2024-09-12T01:12:54.767 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 msgr2=0x7f38840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 0x7f38840a51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3890011cb0 tx=0x7f3890016000 comp rx=0 tx=0).stop 2024-09-12T01:12:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 msgr2=0x7f389819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f388000d950 tx=0x7f388000de10 comp rx=0 tx=0).stop 2024-09-12T01:12:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 shutdown_connections 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f38840a2cf0 0x7f38840a51b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f389810bfc0 0x7f389819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38981034b0 0x7f389819aa60 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 --2- 172.21.15.16:0/3048599078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3898102b90 0x7f389819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.760+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 >> 172.21.15.16:0/3048599078 conn(0x7f38980fc630 msgr2=0x7f3898100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.761+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 shutdown_connections 2024-09-12T01:12:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:54.761+0000 7f389759e640 1 -- 172.21.15.16:0/3048599078 wait complete. 2024-09-12T01:12:55.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:54 smithi099 ceph-mon[35654]: pgmap v526: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:55.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:54 smithi016 ceph-mon[30972]: pgmap v526: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:55.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:54 smithi159 ceph-mon[35566]: pgmap v526: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T01:12:55.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T01:12:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 -- 172.21.15.16:0/513075614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b410e480 msgr2=0x7fa6b4110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/513075614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b410e480 0x7fa6b4110870 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009ca0 tx=0x7fa6ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 -- 172.21.15.16:0/513075614 shutdown_connections 2024-09-12T01:12:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/513075614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b410e480 0x7fa6b4110870 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/513075614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b4101280 0x7fa6b410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/513075614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b41008d0 0x7fa6b4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 -- 172.21.15.16:0/513075614 >> 172.21.15.16:0/513075614 conn(0x7fa6b40fc7a0 msgr2=0x7fa6b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.383+0000 7fa6bc06c640 1 -- 172.21.15.16:0/513075614 shutdown_connections 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 -- 172.21.15.16:0/513075614 wait complete. 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 Processor -- start 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 -- start start 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4101280 0x7fa6b419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b410e480 0x7fa6b419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4114160 con 0x7fa6b41008d0 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4113fe0 con 0x7fa6b4101280 2024-09-12T01:12:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.384+0000 7fa6bc06c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41142e0 con 0x7fa6b410e480 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35164/0 (socket says 172.21.15.16:35164) 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 -- 172.21.15.16:0/1774759093 learned_addr learned my addr 172.21.15.16:0/1774759093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9607640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4101280 0x7fa6b419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6ba609640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b410e480 0x7fa6b419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 -- 172.21.15.16:0/1774759093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b410e480 msgr2=0x7fa6b419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b410e480 0x7fa6b419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 -- 172.21.15.16:0/1774759093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4101280 msgr2=0x7fa6b419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4101280 0x7fa6b419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 -- 172.21.15.16:0/1774759093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b419f670 con 0x7fa6b41008d0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000d950 tx=0x7fa6b000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6aaffd640 1 -- 172.21.15.16:0/1774759093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0017070 con 0x7fa6b41008d0 2024-09-12T01:12:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6aaffd640 1 -- 172.21.15.16:0/1774759093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b000de80 con 0x7fa6b41008d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b419f960 con 0x7fa6b41008d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6aaffd640 1 -- 172.21.15.16:0/1774759093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000f620 con 0x7fa6b41008d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.385+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a7240 con 0x7fa6b41008d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6aaffd640 1 -- 172.21.15.16:0/1774759093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6b0009c30 con 0x7fa6b41008d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 Processor -- start 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 -- start start 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6aaffd640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 0x7fa6a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff808da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 0x558ff809a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff7ff32f0 con 0x558ff8098370 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff7f9acf0 con 0x558ff808eb40 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.388+0000 7fa6be445740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ff7fef2b0 con 0x558ff808d5d0 2024-09-12T01:12:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6ba609640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 0x7fa6a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6aaffd640 1 -- 172.21.15.16:0/1774759093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa6b00cf080 con 0x7fa6b41008d0 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6ba609640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 0x558ff809a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:38048/0 (socket says 172.21.15.16:38048) 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 -- 172.21.15.16:0/2866875591 learned_addr learned my addr 172.21.15.16:0/2866875591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9607640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff808da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 -- 172.21.15.16:0/2866875591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 msgr2=0x558ff808da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff808da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 -- 172.21.15.16:0/2866875591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 msgr2=0x558ff809a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 0x558ff809a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 -- 172.21.15.16:0/2866875591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff7f7f3a0 con 0x558ff808eb40 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6ba609640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 0x558ff809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9607640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff808da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6ba609640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 0x7fa6a00a5230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006010 tx=0x7fa6ac02f410 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.389+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6b009b530 tx=0x7fa6b00a10e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6dff1d8b58160a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa69ffff640 1 -- 172.21.15.16:0/2866875591 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0020040 con 0x558ff808eb40 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa69ffff640 1 -- 172.21.15.16:0/2866875591 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b009b5a0 con 0x558ff808eb40 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa69ffff640 1 -- 172.21.15.16:0/2866875591 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a12c0 con 0x558ff808eb40 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 -- 172.21.15.16:0/2866875591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 msgr2=0x558ff808d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6b009b530 tx=0x7fa6b00a10e0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 -- 172.21.15.16:0/2866875591 shutdown_connections 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff8098370 0x558ff809a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff808da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 --2- 172.21.15.16:0/2866875591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff808eb40 0x558ff808d090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.390+0000 7fa6be445740 1 -- 172.21.15.16:0/2866875591 >> 172.21.15.16:0/2866875591 conn(0x558ff8056310 msgr2=0x558ff8089c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- 172.21.15.16:0/2866875591 shutdown_connections 2024-09-12T01:12:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- 172.21.15.16:0/2866875591 wait complete. 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- start start 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 Processor -- start 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff8097da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff80957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ff809b010 con 0x558ff808eb40 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558ff7fae060 con 0x558ff8098370 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ff809b190 con 0x558ff808d5d0 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6be445740 10 client.0 _ll_register_callbacks cb 0x558ff7d018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.391+0000 7fa6b9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 -- 172.21.15.16:0/3059834608 learned_addr learned my addr 172.21.15.16:0/3059834608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6ba609640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff80957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff8097da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 -- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 msgr2=0x558ff8097da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff8097da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 -- 172.21.15.16:0/3059834608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 msgr2=0x558ff80957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff80957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 -- 172.21.15.16:0/3059834608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff7d643c0 con 0x558ff808eb40 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4010840 tx=0x7fa6a4010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e36588e4db2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa69d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400fb10 con 0x558ff808eb40 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa6a400fcb0 con 0x558ff808eb40 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b009b5a0 con 0x558ff808eb40 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6a4005020 con 0x558ff808eb40 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6be445740 10 client.20445 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:56.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.392+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ff80a6b50 con 0x558ff808eb40 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b009b5a0 con 0x558ff808eb40 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6a4019410 con 0x558ff808eb40 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 10 client.20445 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 10 client.20445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 20 client.20445 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff80a80b0 0x558ff80aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 4 client.20445 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff8000860 con 0x558ff80a80b0 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.393+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff80a80b0 0x558ff80aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.394+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff80a80b0 0x558ff80aa4d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f52c8016246c15fb server_cookie=9136552347482742 in_seq=0 out_seq=0 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.394+0000 7fa69d7fa640 10 client.20445 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.394+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3059834608 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ff8000860 con 0x558ff80a80b0 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.394+0000 7fa69d7fa640 10 client.20445 handle_command_reply: tid=0 2024-09-12T01:12:56.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.394+0000 7fa6be445740 1 client.20445 shutdown 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 msgr2=0x558ff8095050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4010840 tx=0x7fa6a4010d00 comp rx=0 tx=0).stop 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 shutdown_connections 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff80957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff80a80b0 0x558ff80aa4d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808eb40 0x558ff8095050 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 --2- 172.21.15.16:0/3059834608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808d5d0 0x558ff8097da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa69d7fa640 0 client.20445 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 >> 172.21.15.16:0/3059834608 conn(0x558ff8056310 msgr2=0x558ff808f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 shutdown_connections 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 1 -- 172.21.15.16:0/3059834608 wait complete. 2024-09-12T01:12:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.395+0000 7fa6be445740 20 client.20445 trim_cache size 0 max 16384 2024-09-12T01:12:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 Processor -- start 2024-09-12T01:12:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 -- start start 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff8099760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8099ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.408+0000 7fa6be445740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41afb70 con 0x558ff808d5d0 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6be445740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41b16b0 con 0x558ff8098370 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6be445740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41b0240 con 0x558ff808eb40 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35204/0 (socket says 172.21.15.16:35204) 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 -- 172.21.15.16:0/660826382 learned_addr learned my addr 172.21.15.16:0/660826382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9607640 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff8099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6ba609640 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8099ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 -- 172.21.15.16:0/660826382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 msgr2=0x558ff8099760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff8099760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 -- 172.21.15.16:0/660826382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 msgr2=0x558ff8099ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8099ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 -- 172.21.15.16:0/660826382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff808a960 con 0x558ff808d5d0 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00cc880 tx=0x7fa6b0097970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82f5da25a9db088 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa69f7fe640 1 -- 172.21.15.16:0/660826382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0020040 con 0x558ff808d5d0 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa69f7fe640 1 -- 172.21.15.16:0/660826382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b00a4210 con 0x558ff808d5d0 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.409+0000 7fa69f7fe640 1 -- 172.21.15.16:0/660826382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00cca50 con 0x558ff808d5d0 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- 172.21.15.16:0/660826382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 msgr2=0x558ff8094090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00cc880 tx=0x7fa6b0097970 comp rx=0 tx=0).stop 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- 172.21.15.16:0/660826382 shutdown_connections 2024-09-12T01:12:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8099ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff8099760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 --2- 172.21.15.16:0/660826382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff8094090 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- 172.21.15.16:0/660826382 >> 172.21.15.16:0/660826382 conn(0x558ff8056310 msgr2=0x558ff8093980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- 172.21.15.16:0/660826382 shutdown_connections 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- 172.21.15.16:0/660826382 wait complete. 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 -- start start 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.410+0000 7fa6be445740 1 Processor -- start 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff80a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8091320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41b0fe0 con 0x558ff808d5d0 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41b0910 con 0x558ff8098370 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41adb70 con 0x558ff808eb40 2024-09-12T01:12:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6be445740 10 client.0 _ll_register_callbacks cb 0x7fa6a4010450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35210/0 (socket says 172.21.15.16:35210) 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 -- 172.21.15.16:0/3763370975 learned_addr learned my addr 172.21.15.16:0/3763370975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6ba609640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8091320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff80a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 -- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 msgr2=0x558ff80a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff80a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 -- 172.21.15.16:0/3763370975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 msgr2=0x558ff8091320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8091320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9e08640 1 -- 172.21.15.16:0/3763370975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff8091860 con 0x558ff808d5d0 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff80a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.411+0000 7fa6ba609640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8091320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6b9e08640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00971a0 tx=0x7fa6b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b977e3d6f641ac95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b8020 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fa6b00a0d20 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b00a4210 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6b0020070 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 10 client.20451 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ff8092b60 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b00a4210 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6b009aad0 con 0x558ff808d5d0 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 10 client.20451 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 10 client.20451 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 20 client.20451 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:56.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff8092ea0 0x558ff8115d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 4 client.20451 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.412+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 --> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff7ffad70 con 0x558ff8092ea0 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.413+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff8092ea0 0x558ff8115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.413+0000 7fa6b9607640 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff8092ea0 0x558ff8115d90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90698f08c2b75347 server_cookie=ca4a63008a2d3fa2 in_seq=0 out_seq=0 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.413+0000 7fa69d7fa640 10 client.20451 ms_handle_connect on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa69d7fa640 1 -- 172.21.15.16:0/3763370975 <== mds.3 v2:172.21.15.159:6834/1983976622 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff7ffad70 con 0x558ff8092ea0 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa69d7fa640 10 client.20451 handle_command_reply: tid=0 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 client.20451 shutdown 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 msgr2=0x558ff80a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00971a0 tx=0x7fa6b0000bc0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 shutdown_connections 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558ff8098370 0x558ff8091320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ff808eb40 0x558ff80a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.159:6834/1983976622,v1:172.21.15.159:6835/1983976622] conn(0x558ff8092ea0 0x558ff8115d90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 --2- 172.21.15.16:0/3763370975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ff808d5d0 0x558ff80a0420 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa69d7fa640 0 client.20451 ms_handle_reset on v2:172.21.15.159:6834/1983976622 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 >> 172.21.15.16:0/3763370975 conn(0x558ff8056310 msgr2=0x558ff80941a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 shutdown_connections 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 1 -- 172.21.15.16:0/3763370975 wait complete. 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6be445740 20 client.20451 trim_cache size 0 max 16384 2024-09-12T01:12:56.427 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 msgr2=0x7fa6a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 0x7fa6a00a5230 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac006010 tx=0x7fa6ac02f410 comp rx=0 tx=0).stop 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 msgr2=0x7fa6b419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.414+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000d950 tx=0x7fa6b000dd50 comp rx=0 tx=0).stop 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 shutdown_connections 2024-09-12T01:12:56.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa6a00a2d70 0x7fa6a00a5230 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6b410e480 0x7fa6b419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b4101280 0x7fa6b419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 --2- 172.21.15.16:0/1774759093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6b41008d0 0x7fa6b419a6c0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 >> 172.21.15.16:0/1774759093 conn(0x7fa6b40fc7a0 msgr2=0x7fa6b40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 shutdown_connections 2024-09-12T01:12:56.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:56.415+0000 7fa6bc06c640 1 -- 172.21.15.16:0/1774759093 wait complete. 2024-09-12T01:12:57.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:56 smithi099 ceph-mon[35654]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:12:57.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:56 smithi159 ceph-mon[35566]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:12:57.118 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:56 smithi016 ceph-mon[30972]: pgmap v527: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:12:57.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T01:12:57.411 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 -- 172.21.15.16:0/222120736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348101de0 msgr2=0x7f83481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 --2- 172.21.15.16:0/222120736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348101de0 0x7f83481021c0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8330009900 tx=0x7f833002a410 comp rx=0 tx=0).stop 2024-09-12T01:12:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 -- 172.21.15.16:0/222120736 shutdown_connections 2024-09-12T01:12:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 --2- 172.21.15.16:0/222120736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348106f10 0x7f834810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 --2- 172.21.15.16:0/222120736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f83481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 --2- 172.21.15.16:0/222120736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348101de0 0x7f83481021c0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 -- 172.21.15.16:0/222120736 >> 172.21.15.16:0/222120736 conn(0x7f83480fd8c0 msgr2=0x7f83480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 -- 172.21.15.16:0/222120736 shutdown_connections 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.900+0000 7f834f216640 1 -- 172.21.15.16:0/222120736 wait complete. 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 Processor -- start 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 -- start start 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348101de0 0x7f83481984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f8348198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8348111f70 con 0x7f8348106f10 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8348111df0 con 0x7f8348102790 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834f216640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83481120f0 con 0x7f8348101de0 2024-09-12T01:12:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834d7b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834d7b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35232/0 (socket says 172.21.15.16:35232) 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834d7b3640 1 -- 172.21.15.16:0/3774945508 learned_addr learned my addr 172.21.15.16:0/3774945508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f834cfb2640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348101de0 0x7f83481984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.901+0000 7f8347fff640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f8348198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 -- 172.21.15.16:0/3774945508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348101de0 msgr2=0x7f83481984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348101de0 0x7f83481984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 -- 172.21.15.16:0/3774945508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 msgr2=0x7f8348198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f8348198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 -- 172.21.15.16:0/3774945508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834819d480 con 0x7f8348106f10 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f8347fff640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f8348198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834d7b3640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f834000eaa0 tx=0x7f834000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f8345ffb640 1 -- 172.21.15.16:0/3774945508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834000cdb0 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f8345ffb640 1 -- 172.21.15.16:0/3774945508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8340005020 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834819d770 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f8345ffb640 1 -- 172.21.15.16:0/3774945508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400116b0 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.902+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a5050 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.904+0000 7f8345ffb640 1 -- 172.21.15.16:0/3774945508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8340005560 con 0x7f8348106f10 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 Processor -- start 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 -- start start 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d46af62f0 con 0x558d46b91b40 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d46a9dcf0 con 0x558d46b9b370 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f83515ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d46af22b0 con 0x558d46b905d0 2024-09-12T01:12:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f8345ffb640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 0x7f83340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f8347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f8347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35240/0 (socket says 172.21.15.16:35240) 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f8347fff640 1 -- 172.21.15.16:0/2680081205 learned_addr learned my addr 172.21.15.16:0/2680081205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f834cfb2640 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f834d7b3640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 0x7f83340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.905+0000 7f8345ffb640 1 -- 172.21.15.16:0/3774945508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f83400cf5f0 con 0x7f8348106f10 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 -- 172.21.15.16:0/2680081205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 msgr2=0x558d46b90a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b90a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 -- 172.21.15.16:0/2680081205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 msgr2=0x558d46b9d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 -- 172.21.15.16:0/2680081205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d46a823a0 con 0x558d46b91b40 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f834d7b3640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 0x7f83340a5230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f834009b7b0 tx=0x7f834006a9e0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f8347fff640 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8338009460 tx=0x7f8338012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737ffef9627b1f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f833effd640 1 -- 172.21.15.16:0/2680081205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338004030 con 0x558d46b91b40 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f833effd640 1 -- 172.21.15.16:0/2680081205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83380041d0 con 0x558d46b91b40 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.906+0000 7f833effd640 1 -- 172.21.15.16:0/2680081205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338013780 con 0x558d46b91b40 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- 172.21.15.16:0/2680081205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 msgr2=0x558d46b90090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8338009460 tx=0x7f8338012760 comp rx=0 tx=0).stop 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- 172.21.15.16:0/2680081205 shutdown_connections 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b9d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b90a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 --2- 172.21.15.16:0/2680081205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b90090 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- 172.21.15.16:0/2680081205 >> 172.21.15.16:0/2680081205 conn(0x558d46b59310 msgr2=0x558d46b8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- 172.21.15.16:0/2680081205 shutdown_connections 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- 172.21.15.16:0/2680081205 wait complete. 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 -- start start 2024-09-12T01:12:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.907+0000 7f83515ef740 1 Processor -- start 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 0x558d46b97fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b98520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d46b9e070 con 0x558d46b905d0 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d46ab1060 con 0x558d46b9b370 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d46b9e1f0 con 0x558d46b91b40 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f83515ef740 10 client.0 _ll_register_callbacks cb 0x558d468048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35242/0 (socket says 172.21.15.16:35242) 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 -- 172.21.15.16:0/645222622 learned_addr learned my addr 172.21.15.16:0/645222622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f834cfb2640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 0x558d46b97fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b98520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 -- 172.21.15.16:0/645222622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 msgr2=0x558d46b97fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 0x558d46b97fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 -- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 msgr2=0x558d46b98520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b98520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 -- 172.21.15.16:0/645222622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d468673c0 con 0x558d46b905d0 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f834cfb2640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 0x558d46b97fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8347fff640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8338013460 tx=0x7f8338009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e53fc0cbe43f0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.908+0000 7f8327fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338021070 con 0x558d46b905d0 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f8338013500 con 0x558d46b905d0 2024-09-12T01:12:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83380041d0 con 0x558d46b905d0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8338024040 con 0x558d46b905d0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 10 client.20463 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d46ba9aa0 con 0x558d46b905d0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83380041d0 con 0x558d46b905d0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f833800f040 con 0x558d46b905d0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 10 client.20463 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 10 client.20463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 20 client.20463 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46baae50 0x558d46bad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 4 client.20463 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.909+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d46b03860 con 0x558d46baae50 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.910+0000 7f834cfb2640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46baae50 0x558d46bad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.910+0000 7f834cfb2640 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46baae50 0x558d46bad240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a3ff71851ef5329 server_cookie=6c196f584355ee90 in_seq=0 out_seq=0 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.910+0000 7f8327fff640 10 client.20463 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f8327fff640 1 -- 172.21.15.16:0/645222622 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d46b03860 con 0x558d46baae50 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f8327fff640 10 client.20463 handle_command_reply: tid=0 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 client.20463 shutdown 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 msgr2=0x558d46b9acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8338013460 tx=0x7f8338009570 comp rx=0 tx=0).stop 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 shutdown_connections 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46baae50 0x558d46bad240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46b98520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b91b40 0x558d46b97fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 --2- 172.21.15.16:0/645222622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b905d0 0x558d46b9acf0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f8327fff640 0 client.20463 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 >> 172.21.15.16:0/645222622 conn(0x558d46b59310 msgr2=0x558d46b92960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 shutdown_connections 2024-09-12T01:12:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 1 -- 172.21.15.16:0/645222622 wait complete. 2024-09-12T01:12:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.911+0000 7f83515ef740 20 client.20463 trim_cache size 0 max 16384 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 Processor -- start 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 -- start start 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b93120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46bb2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83481ad9d0 con 0x558d46b91b40 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83481af510 con 0x558d46b9b370 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.924+0000 7f83515ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83481ae0a0 con 0x558d46b905d0 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35244/0 (socket says 172.21.15.16:35244) 2024-09-12T01:12:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 -- 172.21.15.16:0/1348149253 learned_addr learned my addr 172.21.15.16:0/1348149253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f8347fff640 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b93120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 -- 172.21.15.16:0/1348149253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 msgr2=0x558d46b93120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b93120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 -- 172.21.15.16:0/1348149253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 msgr2=0x558d46bb2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46bb2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 -- 172.21.15.16:0/1348149253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d46b8d960 con 0x558d46b91b40 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f8347fff640 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b93120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f834cfb2640 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f833002a930 tx=0x7f83300318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9bb727c9c0f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f833e7fc640 1 -- 172.21.15.16:0/1348149253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833004d030 con 0x558d46b91b40 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f833e7fc640 1 -- 172.21.15.16:0/1348149253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8330031a20 con 0x558d46b91b40 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.925+0000 7f833e7fc640 1 -- 172.21.15.16:0/1348149253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330035040 con 0x558d46b91b40 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- 172.21.15.16:0/1348149253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 msgr2=0x558d46b93660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f833002a930 tx=0x7f83300318f0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- 172.21.15.16:0/1348149253 shutdown_connections 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b9b370 0x558d46bb2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b93660 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 --2- 172.21.15.16:0/1348149253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b905d0 0x558d46b93120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- 172.21.15.16:0/1348149253 >> 172.21.15.16:0/1348149253 conn(0x558d46b59310 msgr2=0x558d46b8bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- 172.21.15.16:0/1348149253 shutdown_connections 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- 172.21.15.16:0/1348149253 wait complete. 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 -- start start 2024-09-12T01:12:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.926+0000 7f83515ef740 1 Processor -- start 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b905d0 0x558d46b9f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 0x558d46b9fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83481aee40 con 0x558d46b91b40 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83481ae770 con 0x558d46b905d0 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83481ab9d0 con 0x558d46b9b370 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f83515ef740 10 client.0 _ll_register_callbacks cb 0x7f8330009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35260/0 (socket says 172.21.15.16:35260) 2024-09-12T01:12:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 -- 172.21.15.16:0/2413782679 learned_addr learned my addr 172.21.15.16:0/2413782679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f8347fff640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b905d0 0x558d46b9f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f8347fff640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 0x558d46b9fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 -- 172.21.15.16:0/2413782679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 msgr2=0x558d46b9fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 0x558d46b9fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 -- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b905d0 msgr2=0x558d46b9f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b905d0 0x558d46b9f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f834cfb2640 1 -- 172.21.15.16:0/2413782679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d46b93ba0 con 0x558d46b91b40 2024-09-12T01:12:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.927+0000 7f8347fff640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 0x558d46b9fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f834cfb2640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f833002ac00 tx=0x7f833002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e55d0788ea7100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330056070 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f8330004a10 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8330031a20 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8330062050 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 10 client.20469 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d46c15980 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8330031a20 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8330052470 con 0x558d46b91b40 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 10 client.20469 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 10 client.20469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 20 client.20469 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46c16080 0x558d46c184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 4 client.20469 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.928+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 --> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d46afdd70 con 0x558d46c16080 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f8347fff640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46c16080 0x558d46c184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f8327fff640 10 client.20469 ms_handle_connect on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f8347fff640 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46c16080 0x558d46c184a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f50d39149a2fb1c server_cookie=53bfe38ee6eb50d7 in_seq=0 out_seq=0 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f8327fff640 1 -- 172.21.15.16:0/2413782679 <== mds.2 v2:172.21.15.99:6838/850851643 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d46afdd70 con 0x558d46c16080 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f8327fff640 10 client.20469 handle_command_reply: tid=0 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f83515ef740 1 client.20469 shutdown 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 msgr2=0x558d46b9f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.929+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f833002ac00 tx=0x7f833002a410 comp rx=0 tx=0).stop 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 shutdown_connections 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:6838/850851643,v1:172.21.15.99:6839/850851643] conn(0x558d46c16080 0x558d46c184a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d46b9b370 0x558d46b9fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d46b91b40 0x558d46b9f850 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f8327fff640 0 client.20469 ms_handle_reset on v2:172.21.15.99:6838/850851643 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 --2- 172.21.15.16:0/2413782679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d46b905d0 0x558d46b9f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 >> 172.21.15.16:0/2413782679 conn(0x558d46b59310 msgr2=0x558d46b99800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 shutdown_connections 2024-09-12T01:12:57.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 1 -- 172.21.15.16:0/2413782679 wait complete. 2024-09-12T01:12:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f83515ef740 20 client.20469 trim_cache size 0 max 16384 2024-09-12T01:12:57.945 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:57.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 msgr2=0x7f83340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 0x7f83340a5230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f834009b7b0 tx=0x7f834006a9e0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 msgr2=0x7f834819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f834000eaa0 tx=0x7f834000ef60 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 shutdown_connections 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f83340a2d70 0x7f83340a5230 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348106f10 0x7f834819cda0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8348102790 0x7f8348198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 --2- 172.21.15.16:0/3774945508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8348101de0 0x7f83481984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:57.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.930+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 >> 172.21.15.16:0/3774945508 conn(0x7f83480fd8c0 msgr2=0x7f83480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.931+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 shutdown_connections 2024-09-12T01:12:57.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:57.931+0000 7f834f216640 1 -- 172.21.15.16:0/3774945508 wait complete. 2024-09-12T01:12:58.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T01:12:59.035 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:12:59.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:12:59 smithi016 ceph-mon[30972]: pgmap v528: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:12:59.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:12:59 smithi099 ceph-mon[35654]: pgmap v528: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- 172.21.15.16:0/222899202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728102790 msgr2=0x7fc7281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 --2- 172.21.15.16:0/222899202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728102790 0x7fc7281067e0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402a410 comp rx=0 tx=0).stop 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- 172.21.15.16:0/222899202 shutdown_connections 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 --2- 172.21.15.16:0/222899202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 0x7fc72810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 --2- 172.21.15.16:0/222899202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728102790 0x7fc7281067e0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 --2- 172.21.15.16:0/222899202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728101de0 0x7fc7281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- 172.21.15.16:0/222899202 >> 172.21.15.16:0/222899202 conn(0x7fc7280fd8c0 msgr2=0x7fc7280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- 172.21.15.16:0/222899202 shutdown_connections 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- 172.21.15.16:0/222899202 wait complete. 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 Processor -- start 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.500+0000 7fc72ece5640 1 -- start start 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 0x7fc728074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 0x7fc728074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc728111f30 con 0x7fc728101de0 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc728111db0 con 0x7fc728102790 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281120b0 con 0x7fc728106f10 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52350/0 (socket says 172.21.15.16:52350) 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 -- 172.21.15.16:0/3411536489 learned_addr learned my addr 172.21.15.16:0/3411536489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc727fff640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 0x7fc728074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72d282640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 0x7fc728074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 -- 172.21.15.16:0/3411536489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 msgr2=0x7fc728074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 0x7fc728074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 -- 172.21.15.16:0/3411536489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 msgr2=0x7fc728074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 0x7fc728074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 -- 172.21.15.16:0/3411536489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7281a68f0 con 0x7fc728101de0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc727fff640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 0x7fc728074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ca81640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00d950 tx=0x7fc71c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc725ffb640 1 -- 172.21.15.16:0/3411536489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc728101de0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc725ffb640 1 -- 172.21.15.16:0/3411536489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c009c30 con 0x7fc728101de0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281a6be0 con 0x7fc728101de0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.501+0000 7fc725ffb640 1 -- 172.21.15.16:0/3411536489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00f6b0 con 0x7fc728101de0 2024-09-12T01:12:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.502+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a7120 con 0x7fc728101de0 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc725ffb640 1 -- 172.21.15.16:0/3411536489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc71c0109d0 con 0x7fc728101de0 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 Processor -- start 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 -- start start 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632a8c922f0 con 0x5632a8d2c5d0 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632a8c39cf0 con 0x5632a8d2db40 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc7310be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5632a8c8e2b0 con 0x5632a8d37370 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.504+0000 7fc725ffb640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 0x7fc7180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52354/0 (socket says 172.21.15.16:52354) 2024-09-12T01:12:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72d282640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 0x7fc7180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 -- 172.21.15.16:0/864578249 learned_addr learned my addr 172.21.15.16:0/864578249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72d282640 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc725ffb640 1 -- 172.21.15.16:0/3411536489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fc71c0cf150 con 0x7fc728101de0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc727fff640 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 -- 172.21.15.16:0/864578249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 msgr2=0x5632a8d39760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d39760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 -- 172.21.15.16:0/864578249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 msgr2=0x5632a8d2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 -- 172.21.15.16:0/864578249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a8c1e3a0 con 0x5632a8d2c5d0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72d282640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 0x7fc7180a51b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc720005c40 tx=0x7fc720009c90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc72ca81640 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc71c09b600 tx=0x7fc71c06e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54ddd47e0c0e80a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc712ffd640 1 -- 172.21.15.16:0/864578249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c020070 con 0x5632a8d2c5d0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc712ffd640 1 -- 172.21.15.16:0/864578249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c09bd70 con 0x5632a8d2c5d0 2024-09-12T01:12:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.505+0000 7fc712ffd640 1 -- 172.21.15.16:0/864578249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0a42e0 con 0x5632a8d2c5d0 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- 172.21.15.16:0/864578249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 msgr2=0x5632a8d2ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc71c09b600 tx=0x7fc71c06e6e0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- 172.21.15.16:0/864578249 shutdown_connections 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d39760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2c5d0 0x5632a8d2ca10 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 --2- 172.21.15.16:0/864578249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- 172.21.15.16:0/864578249 >> 172.21.15.16:0/864578249 conn(0x5632a8cf5310 msgr2=0x5632a8d28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- 172.21.15.16:0/864578249 shutdown_connections 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- 172.21.15.16:0/864578249 wait complete. 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.506+0000 7fc7310be740 1 -- start start 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 Processor -- start 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d37370 0x5632a8d34520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632a8d39fb0 con 0x5632a8d2db40 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5632a8c4d060 con 0x5632a8d37370 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5632a8d3a130 con 0x5632a8d2c5d0 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc7310be740 10 client.0 _ll_register_callbacks cb 0x5632a89a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52360/0 (socket says 172.21.15.16:52360) 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 -- 172.21.15.16:0/1905359206 learned_addr learned my addr 172.21.15.16:0/1905359206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc72d282640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d37370 0x5632a8d34520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc72ca81640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 -- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 msgr2=0x5632a8d36cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d36cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 -- 172.21.15.16:0/1905359206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d37370 msgr2=0x5632a8d34520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d37370 0x5632a8d34520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.507+0000 7fc727fff640 1 -- 172.21.15.16:0/1905359206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a8a033c0 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc727fff640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc7140318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4768955228a56f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714035040 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fc714004240 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc71c09bd70 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc714004440 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc7310be740 10 client.20481 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632a8d459e0 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc71c09bd70 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.508+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc714004bb0 con 0x5632a8d2db40 2024-09-12T01:12:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 10 client.20481 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 10 client.20481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 20 client.20481 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8d46ef0 0x5632a8d49310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 4 client.20481 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632a8c9f860 con 0x5632a8d46ef0 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc72ca81640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8d46ef0 0x5632a8d49310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc72ca81640 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8d46ef0 0x5632a8d49310 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec46a208f1593619 server_cookie=7e6ead5b2eb5b8e2 in_seq=0 out_seq=0 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.509+0000 7fc707fff640 10 client.20481 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc707fff640 1 -- 172.21.15.16:0/1905359206 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632a8c9f860 con 0x5632a8d46ef0 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc707fff640 10 client.20481 handle_command_reply: tid=0 2024-09-12T01:12:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 client.20481 shutdown 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 msgr2=0x5632a8d33fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc7140318d0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 shutdown_connections 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8d46ef0 0x5632a8d49310 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d37370 0x5632a8d34520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc707fff640 0 client.20481 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d33fa0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 --2- 172.21.15.16:0/1905359206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d36cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.510+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 >> 172.21.15.16:0/1905359206 conn(0x5632a8cf5310 msgr2=0x5632a8d2e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.511+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 shutdown_connections 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.511+0000 7fc7310be740 1 -- 172.21.15.16:0/1905359206 wait complete. 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.511+0000 7fc7310be740 20 client.20481 trim_cache size 0 max 16384 2024-09-12T01:12:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 Processor -- start 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 -- start start 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2c5d0 0x5632a8d33f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d4ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281afb60 con 0x5632a8d2db40 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281b16a0 con 0x5632a8d2c5d0 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc7310be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281b0230 con 0x5632a8d37370 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52364/0 (socket says 172.21.15.16:52364) 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 -- 172.21.15.16:0/3178462039 learned_addr learned my addr 172.21.15.16:0/3178462039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc72ca81640 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2c5d0 0x5632a8d33f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 -- 172.21.15.16:0/3178462039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 msgr2=0x5632a8d4ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d4ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 -- 172.21.15.16:0/3178462039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2c5d0 msgr2=0x5632a8d33f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2c5d0 0x5632a8d33f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 -- 172.21.15.16:0/3178462039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a8d324f0 con 0x5632a8d2db40 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.524+0000 7fc727fff640 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc714009850 tx=0x7fc71402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5878e53c70419a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7127fc640 1 -- 172.21.15.16:0/3178462039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714000e60 con 0x5632a8d2db40 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7127fc640 1 -- 172.21.15.16:0/3178462039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc714003960 con 0x5632a8d2db40 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7127fc640 1 -- 172.21.15.16:0/3178462039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714052070 con 0x5632a8d2db40 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 -- 172.21.15.16:0/3178462039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 msgr2=0x5632a8d43250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fc714009850 tx=0x7fc71402a410 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 -- 172.21.15.16:0/3178462039 shutdown_connections 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d37370 0x5632a8d4ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d2db40 0x5632a8d43250 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 --2- 172.21.15.16:0/3178462039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2c5d0 0x5632a8d33f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 -- 172.21.15.16:0/3178462039 >> 172.21.15.16:0/3178462039 conn(0x5632a8cf5310 msgr2=0x5632a8d33860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 -- 172.21.15.16:0/3178462039 shutdown_connections 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.525+0000 7fc7310be740 1 -- 172.21.15.16:0/3178462039 wait complete. 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 -- start start 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 Processor -- start 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d3b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d3bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281b0fd0 con 0x5632a8d37370 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281b0900 con 0x5632a8d2db40 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7281adb60 con 0x5632a8d2c5d0 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc7310be740 10 client.0 _ll_register_callbacks cb 0x7fc714005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc72d282640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc72d282640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52374/0 (socket says 172.21.15.16:52374) 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc72d282640 1 -- 172.21.15.16:0/4030914544 learned_addr learned my addr 172.21.15.16:0/4030914544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc72ca81640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d3b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.526+0000 7fc727fff640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d3bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 -- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 msgr2=0x5632a8d3b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d3b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 -- 172.21.15.16:0/4030914544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 msgr2=0x5632a8d3bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d3bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 -- 172.21.15.16:0/4030914544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a8db11a0 con 0x5632a8d37370 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc727fff640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d3bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc72d282640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc720010c60 tx=0x7fc720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61b5a9b03483961 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc707fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72001ea00 con 0x5632a8d37370 2024-09-12T01:12:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fc72001eba0 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc714003960 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc72001eda0 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc7310be740 10 client.20487 fetch_fsmap learned FSMap version 28 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632a8db2440 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.527+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc714003960 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc72001f070 con 0x5632a8d37370 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 10 client.20487 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 10 client.20487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 20 client.20487 populate_metadata read hostname 'smithi016' 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8db2bc0 0x5632a8db4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 4 client.20487 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 --> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632a8c99d70 con 0x5632a8db2bc0 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc72ca81640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8db2bc0 0x5632a8db4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:12:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc72ca81640 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8db2bc0 0x5632a8db4fe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53420eeddebd76cc server_cookie=3c7c237602086373 in_seq=0 out_seq=0 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc707fff640 10 client.20487 ms_handle_connect on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc707fff640 1 -- 172.21.15.16:0/4030914544 <== mds.1 v2:172.21.15.159:6836/247659815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632a8c99d70 con 0x5632a8db2bc0 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc707fff640 10 client.20487 handle_command_reply: tid=0 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.528+0000 7fc7310be740 1 client.20487 shutdown 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 msgr2=0x5632a8db0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fc720010c60 tx=0x7fc720004a40 comp rx=0 tx=0).stop 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:12:59.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 shutdown_connections 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632a8d37370 0x5632a8db0bb0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:6836/247659815,v1:172.21.15.159:6837/247659815] conn(0x5632a8db2bc0 0x5632a8db4fe0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc707fff640 0 client.20487 ms_handle_reset on v2:172.21.15.159:6836/247659815 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5632a8d2db40 0x5632a8d3bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 --2- 172.21.15.16:0/4030914544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5632a8d2c5d0 0x5632a8d3b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 >> 172.21.15.16:0/4030914544 conn(0x5632a8cf5310 msgr2=0x5632a8d35800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 shutdown_connections 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 1 -- 172.21.15.16:0/4030914544 wait complete. 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc7310be740 20 client.20487 trim_cache size 0 max 16384 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 msgr2=0x7fc7180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 0x7fc7180a51b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc720005c40 tx=0x7fc720009c90 comp rx=0 tx=0).stop 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 msgr2=0x7fc7280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:12:59.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.529+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00d950 tx=0x7fc71c00de10 comp rx=0 tx=0).stop 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 shutdown_connections 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc7180a2cf0 0x7fc7180a51b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc728106f10 0x7fc728074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc728102790 0x7fc728074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 --2- 172.21.15.16:0/3411536489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc728101de0 0x7fc7280744f0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 >> 172.21.15.16:0/3411536489 conn(0x7fc7280fd8c0 msgr2=0x7fc7280fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 shutdown_connections 2024-09-12T01:12:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:12:59.530+0000 7fc72ece5640 1 -- 172.21.15.16:0/3411536489 wait complete. 2024-09-12T01:12:59.617 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:12:59 smithi159 ceph-mon[35566]: pgmap v528: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:00.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T01:13:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.198+0000 7f2d256e9640 1 -- 172.21.15.16:0/4270624726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d20102d30 msgr2=0x7f2d20103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.198+0000 7f2d256e9640 1 --2- 172.21.15.16:0/4270624726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d20102d30 0x7f2d20103110 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009cd0 tx=0x7f2d1002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 -- 172.21.15.16:0/4270624726 shutdown_connections 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 --2- 172.21.15.16:0/4270624726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 0x7f2d2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 --2- 172.21.15.16:0/4270624726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d201036e0 0x7f2d20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 --2- 172.21.15.16:0/4270624726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d20102d30 0x7f2d20103110 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 -- 172.21.15.16:0/4270624726 >> 172.21.15.16:0/4270624726 conn(0x7f2d200fc7d0 msgr2=0x7f2d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 -- 172.21.15.16:0/4270624726 shutdown_connections 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 -- 172.21.15.16:0/4270624726 wait complete. 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 Processor -- start 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.199+0000 7f2d256e9640 1 -- start start 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d20102d30 0x7f2d20100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 0x7f2d20101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201120a0 con 0x7f2d201036e0 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20111f20 con 0x7f2d201082a0 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112220 con 0x7f2d20102d30 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52392/0 (socket says 172.21.15.16:52392) 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2138983115 learned_addr learned my addr 172.21.15.16:0/2138983115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 0x7f2d20101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d20102d30 0x7f2d20100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2138983115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d20102d30 msgr2=0x7f2d20100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d20102d30 0x7f2d20100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2138983115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 msgr2=0x7f2d20101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 0x7f2d20101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2138983115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a6aa0 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d750 tx=0x7f2d1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d07fff640 1 -- 172.21.15.16:0/2138983115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14017070 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d07fff640 1 -- 172.21.15.16:0/2138983115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14005500 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a6d90 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.200+0000 7f2d07fff640 1 -- 172.21.15.16:0/2138983115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004e00 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.201+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a7270 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d07fff640 1 -- 172.21.15.16:0/2138983115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d14004fa0 con 0x7f2d201036e0 2024-09-12T01:13:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 Processor -- start 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 -- start start 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 0x55b1762ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1762522f0 con 0x55b1762f7370 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b1761f9cf0 con 0x55b1762edb40 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b17624e2b0 con 0x55b1762ec5d0 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.203+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52408/0 (socket says 172.21.15.16:52408) 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2695184784 learned_addr learned my addr 172.21.15.16:0/2695184784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d07fff640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 0x7f2d080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 0x55b1762ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 0x7f2d080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 -- 172.21.15.16:0/2695184784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 msgr2=0x55b1762ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 0x55b1762ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 -- 172.21.15.16:0/2695184784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 msgr2=0x55b1762f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 -- 172.21.15.16:0/2695184784 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1761de3a0 con 0x55b1762ec5d0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 0x55b1762ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d07fff640 1 -- 172.21.15.16:0/2138983115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2d140cedc0 con 0x7f2d201036e0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762eca10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d10006010 tx=0x7f2d10031580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781037c08139628b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 0x7f2d080a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005fd0 tx=0x7f2d18005f60 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d057fa640 1 -- 172.21.15.16:0/2695184784 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10031ac0 con 0x55b1762ec5d0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d057fa640 1 -- 172.21.15.16:0/2695184784 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d10036040 con 0x55b1762ec5d0 2024-09-12T01:13:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.204+0000 7f2d057fa640 1 -- 172.21.15.16:0/2695184784 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d10004770 con 0x55b1762ec5d0 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2695184784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 msgr2=0x55b1762eca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762eca10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d10006010 tx=0x7f2d10031580 comp rx=0 tx=0).stop 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2695184784 shutdown_connections 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762f7370 0x55b1762f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762eca10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2695184784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762edb40 0x55b1762ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2695184784 >> 172.21.15.16:0/2695184784 conn(0x55b1762b5310 msgr2=0x55b1762e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2695184784 shutdown_connections 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2695184784 wait complete. 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.205+0000 7f2d27ac2740 1 -- start start 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 Processor -- start 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762edb40 0x55b1762f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762f4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b1762fa010 con 0x55b1762ec5d0 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b17620d060 con 0x55b1762f7370 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1762fa190 con 0x55b1762edb40 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d27ac2740 10 client.0 _ll_register_callbacks cb 0x55b175f608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52414/0 (socket says 172.21.15.16:52414) 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 -- 172.21.15.16:0/1622190258 learned_addr learned my addr 172.21.15.16:0/1622190258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762edb40 0x55b1762f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 -- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762edb40 msgr2=0x55b1762f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762edb40 0x55b1762f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 -- 172.21.15.16:0/1622190258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 msgr2=0x55b1762f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 -- 172.21.15.16:0/1622190258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b175fc33c0 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d1effd640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005cc0 tx=0x7f2d10031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190398841215fd77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.206+0000 7f2d02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1004e070 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2d10003b80 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d10036040 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d1005a050 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 10 client.20499 fetch_fsmap learned FSMap version 28 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b176305a60 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d10036040 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d10030070 con 0x55b1762ec5d0 2024-09-12T01:13:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 10 client.20499 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 10 client.20499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 20 client.20499 populate_metadata read hostname 'smithi016' 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b176306db0 0x55b1763091a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 4 client.20499 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.207+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b17625f860 con 0x55b176306db0 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.208+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b176306db0 0x55b1763091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.208+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b176306db0 0x55b1763091a0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb9f4c50c01c500f server_cookie=564158a15f3fa242 in_seq=0 out_seq=0 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.208+0000 7f2d02ff5640 10 client.20499 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d02ff5640 1 -- 172.21.15.16:0/1622190258 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b17625f860 con 0x55b176306db0 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d02ff5640 10 client.20499 handle_command_reply: tid=0 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 client.20499 shutdown 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 msgr2=0x55b1762f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f2d10005cc0 tx=0x7f2d10031b70 comp rx=0 tx=0).stop 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 shutdown_connections 2024-09-12T01:13:01.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b176306db0 0x55b1763091a0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d02ff5640 0 client.20499 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762edb40 0x55b1762f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/1622190258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762ec5d0 0x55b1762f6d10 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 >> 172.21.15.16:0/1622190258 conn(0x55b1762b5310 msgr2=0x55b1762ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 shutdown_connections 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 1 -- 172.21.15.16:0/1622190258 wait complete. 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.209+0000 7f2d27ac2740 20 client.20499 trim_cache size 0 max 16384 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 Processor -- start 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 -- start start 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201afa30 con 0x55b1762edb40 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b1570 con 0x55b1762f7370 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b0100 con 0x55b1762ec5d0 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52430/0 (socket says 172.21.15.16:52430) 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/51758973 learned_addr learned my addr 172.21.15.16:0/51758973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1effd640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/51758973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 msgr2=0x55b1762f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/51758973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 msgr2=0x55b1762ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/51758973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1762f2510 con 0x55b1762edb40 2024-09-12T01:13:01.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1effd640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.223+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2d140ccb10 tx=0x7f2d140ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7247a3d5b1c3b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d04ff9640 1 -- 172.21.15.16:0/51758973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14020040 con 0x55b1762edb40 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d04ff9640 1 -- 172.21.15.16:0/51758973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14095e50 con 0x55b1762edb40 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d04ff9640 1 -- 172.21.15.16:0/51758973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14009cc0 con 0x55b1762edb40 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 -- 172.21.15.16:0/51758973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 msgr2=0x55b1762ef120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2d140ccb10 tx=0x7f2d140ce040 comp rx=0 tx=0).stop 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 -- 172.21.15.16:0/51758973 shutdown_connections 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ef120 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/51758973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 -- 172.21.15.16:0/51758973 >> 172.21.15.16:0/51758973 conn(0x55b1762b5310 msgr2=0x55b1762e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.224+0000 7f2d27ac2740 1 -- 172.21.15.16:0/51758973 shutdown_connections 2024-09-12T01:13:01.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 -- 172.21.15.16:0/51758973 wait complete. 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 -- start start 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 Processor -- start 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b0ea0 con 0x55b1762edb40 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b07d0 con 0x55b1762f7370 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ada30 con 0x55b1762ec5d0 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d27ac2740 10 client.0 _ll_register_callbacks cb 0x7f2d1409b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52442/0 (socket says 172.21.15.16:52442) 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2427788187 learned_addr learned my addr 172.21.15.16:0/2427788187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:01.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.225+0000 7f2d1f7fe640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 msgr2=0x55b1762ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2427788187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 msgr2=0x55b1762fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 -- 172.21.15.16:0/2427788187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1762fc930 con 0x55b1762edb40 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d1e7fc640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d720 tx=0x7f2d1409b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7746a51fa0ec9c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14020050 con 0x55b1762edb40 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7f2d140cc790 con 0x55b1762edb40 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14095e50 con 0x55b1762edb40 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d140a3f50 con 0x55b1762edb40 2024-09-12T01:13:01.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d27ac2740 10 client.20505 fetch_fsmap learned FSMap version 28 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b176371d40 con 0x55b1762edb40 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.226+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14095e50 con 0x55b1762edb40 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d140bb0b0 con 0x55b1762edb40 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 10 client.20505 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 10 client.20505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 20 client.20505 populate_metadata read hostname 'smithi016' 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1763724a0 0x55b1763748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 4 client.20505 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 --> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b176259d70 con 0x55b1763724a0 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1763724a0 0x55b1763748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d1effd640 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1763724a0 0x55b1763748c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46fa8cb4baaba31 server_cookie=b2ff44f3de158f54 in_seq=0 out_seq=0 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d02ff5640 10 client.20505 ms_handle_connect on v2:172.21.15.159:6838/1957754994 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d02ff5640 1 -- 172.21.15.16:0/2427788187 <== mds.0 v2:172.21.15.159:6838/1957754994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b176259d70 con 0x55b1763724a0 2024-09-12T01:13:01.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d02ff5640 10 client.20505 handle_command_reply: tid=0 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.227+0000 7f2d27ac2740 1 client.20505 shutdown 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 msgr2=0x55b1762ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d720 tx=0x7f2d1409b460 comp rx=0 tx=0).stop 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 shutdown_connections 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:6838/1957754994,v1:172.21.15.159:6839/1957754994] conn(0x55b1763724a0 0x55b1763748c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b1762f7370 0x55b1762fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1762edb40 0x55b1762ff960 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 --2- 172.21.15.16:0/2427788187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1762ec5d0 0x55b1762ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d02ff5640 0 client.20505 ms_handle_reset on v2:172.21.15.159:6838/1957754994 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 >> 172.21.15.16:0/2427788187 conn(0x55b1762b5310 msgr2=0x55b1762f5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 shutdown_connections 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 1 -- 172.21.15.16:0/2427788187 wait complete. 2024-09-12T01:13:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.228+0000 7f2d27ac2740 20 client.20505 trim_cache size 0 max 16384 2024-09-12T01:13:01.249 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-12T01:13:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 msgr2=0x7f2d080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 0x7f2d080a5230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005fd0 tx=0x7f2d18005f60 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 msgr2=0x7f2d20100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d750 tx=0x7f2d1400dc10 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 shutdown_connections 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f2d080a2d70 0x7f2d080a5230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d201082a0 0x7f2d20101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d201036e0 0x7f2d20100ed0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 --2- 172.21.15.16:0/2138983115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d20102d30 0x7f2d20100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 >> 172.21.15.16:0/2138983115 conn(0x7f2d200fc7d0 msgr2=0x7f2d20105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 shutdown_connections 2024-09-12T01:13:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:01.229+0000 7f2d256e9640 1 -- 172.21.15.16:0/2138983115 wait complete. 2024-09-12T01:13:01.394 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:01 smithi016 ceph-mon[30972]: pgmap v529: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:13:01.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:01 smithi099 ceph-mon[35654]: pgmap v529: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:13:01.620 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:01 smithi159 ceph-mon[35566]: pgmap v529: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T01:13:01.930 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T01:13:01.931 INFO:tasks.fwd_scrub:done joining 2024-09-12T01:13:01.931 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T01:13:03.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.178+0000 7fd0122f1640 1 -- 172.21.15.16:0/2793406610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c10c100 msgr2=0x7fd00c10e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2793406610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c10c100 0x7fd00c10e570 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fd004009ca0 tx=0x7fd00402a950 comp rx=0 tx=0).stop 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 -- 172.21.15.16:0/2793406610 shutdown_connections 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2793406610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c10c100 0x7fd00c10e570 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2793406610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c103650 0x7fd00c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2793406610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c102d30 0x7fd00c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 -- 172.21.15.16:0/2793406610 >> 172.21.15.16:0/2793406610 conn(0x7fd00c0fc6a0 msgr2=0x7fd00c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 -- 172.21.15.16:0/2793406610 shutdown_connections 2024-09-12T01:13:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 -- 172.21.15.16:0/2793406610 wait complete. 2024-09-12T01:13:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 Processor -- start 2024-09-12T01:13:03.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.179+0000 7fd0122f1640 1 -- start start 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c102d30 0x7fd00c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 0x7fd00c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c111ed0 con 0x7fd00c103650 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c111d50 con 0x7fd00c102d30 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd0122f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c112050 con 0x7fd00c10c100 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52446/0 (socket says 172.21.15.16:52446) 2024-09-12T01:13:03.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 -- 172.21.15.16:0/2655805254 learned_addr learned my addr 172.21.15.16:0/2655805254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00bfff640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c102d30 0x7fd00c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd01088e640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 0x7fd00c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 -- 172.21.15.16:0/2655805254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 msgr2=0x7fd00c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 0x7fd00c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 -- 172.21.15.16:0/2655805254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c102d30 msgr2=0x7fd00c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c102d30 0x7fd00c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 -- 172.21.15.16:0/2655805254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c19d3d0 con 0x7fd00c103650 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd01088e640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 0x7fd00c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.180+0000 7fd00b7fe640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fd00000d950 tx=0x7fd00000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.181+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd000017030 con 0x7fd00c103650 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.181+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c19d6c0 con 0x7fd00c103650 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.181+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd000004020 con 0x7fd00c103650 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.181+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0000047a0 con 0x7fd00c103650 2024-09-12T01:13:03.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.181+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c1a4f50 con 0x7fd00c103650 2024-09-12T01:13:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.182+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c103110 con 0x7fd00c103650 2024-09-12T01:13:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.183+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd0000041d0 con 0x7fd00c103650 2024-09-12T01:13:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.183+0000 7fd0097fa640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 0x7fcff40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.184+0000 7fd00bfff640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 0x7fcff40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.184+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fd0000cede0 con 0x7fd00c103650 2024-09-12T01:13:03.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.184+0000 7fd00bfff640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 0x7fcff40a51c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005fd0 tx=0x7fcffc00b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.188+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd00009ba00 con 0x7fd00c103650 2024-09-12T01:13:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.365+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd00c106c20 con 0x7fd00c103650 2024-09-12T01:13:03.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.367+0000 7fd0097fa640 1 -- 172.21.15.16:0/2655805254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+23111 (secure 0 0 0) 0x7fd00009b300 con 0x7fd00c103650 2024-09-12T01:13:03.366 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:13:03.366 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":103,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T01:12:12.457558+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2952840401":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2290747564":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/90830535":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1572487203":"2024-09-13T00:56:03.339943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":13,"length":1},{"begin":26,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 msgr2=0x7fcff40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 0x7fcff40a51c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005fd0 tx=0x7fcffc00b390 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 msgr2=0x7fd00c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fd00000d950 tx=0x7fd00000de10 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 shutdown_connections 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fcff40a2d00 0x7fcff40a51c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd00c10c100 0x7fd00c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd00c103650 0x7fd00c198920 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 --2- 172.21.15.16:0/2655805254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd00c102d30 0x7fd00c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 >> 172.21.15.16:0/2655805254 conn(0x7fd00c0fc6a0 msgr2=0x7fd00c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 shutdown_connections 2024-09-12T01:13:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.369+0000 7fd0122f1640 1 -- 172.21.15.16:0/2655805254 wait complete. 2024-09-12T01:13:03.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:03 smithi016 ceph-mon[30972]: pgmap v530: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:03.379 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T00:49:22.631453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T00:54:00.136499+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T00:54:02.731797+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T01:13:03.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T01:13:03.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 -- 172.21.15.16:0/1713665308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c107a90 msgr2=0x7fba0c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 --2- 172.21.15.16:0/1713665308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c107a90 0x7fba0c107e70 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009930 tx=0x7fb9fc02a420 comp rx=0 tx=0).stop 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 -- 172.21.15.16:0/1713665308 shutdown_connections 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 --2- 172.21.15.16:0/1713665308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c102420 0x7fba0c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 --2- 172.21.15.16:0/1713665308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c101a80 0x7fba0c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 --2- 172.21.15.16:0/1713665308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c107a90 0x7fba0c107e70 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 -- 172.21.15.16:0/1713665308 >> 172.21.15.16:0/1713665308 conn(0x7fba0c0fd7c0 msgr2=0x7fba0c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 -- 172.21.15.16:0/1713665308 shutdown_connections 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.489+0000 7fba1165e640 1 -- 172.21.15.16:0/1713665308 wait complete. 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 Processor -- start 2024-09-12T01:13:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 -- start start 2024-09-12T01:13:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c101a80 0x7fba0c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 0x7fba0c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c111ed0 con 0x7fba0c102420 2024-09-12T01:13:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c111d50 con 0x7fba0c101a80 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba1165e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba0c112050 con 0x7fba0c107a90 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52460/0 (socket says 172.21.15.16:52460) 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba03fff640 1 -- 172.21.15.16:0/466966847 learned_addr learned my addr 172.21.15.16:0/466966847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.490+0000 7fba0affd640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c101a80 0x7fba0c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba0b7fe640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 0x7fba0c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 -- 172.21.15.16:0/466966847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 msgr2=0x7fba0c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 0x7fba0c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 -- 172.21.15.16:0/466966847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c101a80 msgr2=0x7fba0c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c101a80 0x7fba0c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 -- 172.21.15.16:0/466966847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0c19d460 con 0x7fba0c102420 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba0b7fe640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 0x7fba0c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:13:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba03fff640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400d770 tx=0x7fb9f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4017070 con 0x7fba0c102420 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f4005500 con 0x7fba0c102420 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0c19d750 con 0x7fba0c102420 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4004e00 con 0x7fba0c102420 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.491+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0c1a4fe0 con 0x7fba0c102420 2024-09-12T01:13:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.492+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba0c101ee0 con 0x7fba0c102420 2024-09-12T01:13:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.494+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb9f4004fa0 con 0x7fba0c102420 2024-09-12T01:13:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.494+0000 7fba08ff9640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 0x7fb9f80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.495+0000 7fba0affd640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 0x7fb9f80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.495+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fb9f40cef20 con 0x7fba0c102420 2024-09-12T01:13:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.495+0000 7fba0affd640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 0x7fb9f80a5240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.499+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9f409bae0 con 0x7fba0c102420 2024-09-12T01:13:03.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:03 smithi099 ceph-mon[35654]: pgmap v530: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:03.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:03 smithi159 ceph-mon[35566]: pgmap v530: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.685+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba0c02d7c0 con 0x7fba0c102420 2024-09-12T01:13:03.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.685+0000 7fba08ff9640 1 -- 172.21.15.16:0/466966847 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fb9f409b3e0 con 0x7fba0c102420 2024-09-12T01:13:03.684 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-12T01:13:03.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 msgr2=0x7fb9f80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 0x7fb9f80a5240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000c00 tx=0x7fb9fc005a20 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 msgr2=0x7fba0c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400d770 tx=0x7fb9f400dc30 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 shutdown_connections 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fb9f80a2d80 0x7fb9f80a5240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba0c107a90 0x7fba0c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0c102420 0x7fba0c1989b0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 --2- 172.21.15.16:0/466966847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba0c101a80 0x7fba0c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.687+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 >> 172.21.15.16:0/466966847 conn(0x7fba0c0fd7c0 msgr2=0x7fba0c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.688+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 shutdown_connections 2024-09-12T01:13:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.688+0000 7fba1165e640 1 -- 172.21.15.16:0/466966847 wait complete. 2024-09-12T01:13:03.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T01:13:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 -- 172.21.15.16:0/965738153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c1069c0 msgr2=0x7fc38c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 --2- 172.21.15.16:0/965738153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c1069c0 0x7fc38c106da0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009930 tx=0x7fc37c02a420 comp rx=0 tx=0).stop 2024-09-12T01:13:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 -- 172.21.15.16:0/965738153 shutdown_connections 2024-09-12T01:13:03.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 --2- 172.21.15.16:0/965738153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c101350 0x7fc38c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 --2- 172.21.15.16:0/965738153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1009b0 0x7fc38c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 --2- 172.21.15.16:0/965738153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c1069c0 0x7fc38c106da0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 -- 172.21.15.16:0/965738153 >> 172.21.15.16:0/965738153 conn(0x7fc38c0fc690 msgr2=0x7fc38c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 -- 172.21.15.16:0/965738153 shutdown_connections 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.805+0000 7fc39196d640 1 -- 172.21.15.16:0/965738153 wait complete. 2024-09-12T01:13:03.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 Processor -- start 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 -- start start 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c1009b0 0x7fc38c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1069c0 0x7fc38c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c110dc0 con 0x7fc38c101350 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c110c40 con 0x7fc38c1009b0 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc39196d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c110f40 con 0x7fc38c1069c0 2024-09-12T01:13:03.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc38a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc38a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52472/0 (socket says 172.21.15.16:52472) 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc38a7fc640 1 -- 172.21.15.16:0/2097879348 learned_addr learned my addr 172.21.15.16:0/2097879348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.806+0000 7fc38b7fe640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1069c0 0x7fc38c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38affd640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c1009b0 0x7fc38c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 -- 172.21.15.16:0/2097879348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1069c0 msgr2=0x7fc38c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1069c0 0x7fc38c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 -- 172.21.15.16:0/2097879348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c1009b0 msgr2=0x7fc38c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c1009b0 0x7fc38c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 -- 172.21.15.16:0/2097879348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c19d430 con 0x7fc38c101350 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc38a7fc640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc38000d950 tx=0x7fc38000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380017070 con 0x7fc38c101350 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c19d720 con 0x7fc38c101350 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380009c30 con 0x7fc38c101350 2024-09-12T01:13:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38000f6b0 con 0x7fc38c101350 2024-09-12T01:13:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.807+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c1a4fb0 con 0x7fc38c101350 2024-09-12T01:13:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.808+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c100e10 con 0x7fc38c101350 2024-09-12T01:13:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.809+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc3800109d0 con 0x7fc38c101350 2024-09-12T01:13:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.810+0000 7fc39096b640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 0x7fc3740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:03.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.810+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7fc3800cf110 con 0x7fc38c101350 2024-09-12T01:13:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.810+0000 7fc38affd640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 0x7fc3740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.810+0000 7fc38affd640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 0x7fc3740a51c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc37c000c00 tx=0x7fc37c005a20 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.814+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc380009de0 con 0x7fc38c101350 2024-09-12T01:13:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.990+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc38c02d7c0 con 0x7fc38c101350 2024-09-12T01:13:03.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.990+0000 7fc39096b640 1 -- 172.21.15.16:0/2097879348 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) ==== 104+0+11 (secure 0 0 0) 0x7fc38009bd30 con 0x7fc38c101350 2024-09-12T01:13:03.990 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-12T01:13:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 msgr2=0x7fc3740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 0x7fc3740a51c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc37c000c00 tx=0x7fc37c005a20 comp rx=0 tx=0).stop 2024-09-12T01:13:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 msgr2=0x7fc38c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:03.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc38000d950 tx=0x7fc38000de10 comp rx=0 tx=0).stop 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 shutdown_connections 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fc3740a2d00 0x7fc3740a51c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc38c1069c0 0x7fc38c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc38c101350 0x7fc38c198980 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 --2- 172.21.15.16:0/2097879348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc38c1009b0 0x7fc38c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 >> 172.21.15.16:0/2097879348 conn(0x7fc38c0fc690 msgr2=0x7fc38c06a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 shutdown_connections 2024-09-12T01:13:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:03.992+0000 7fc39196d640 1 -- 172.21.15.16:0/2097879348 wait complete. 2024-09-12T01:13:04.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T01:13:04.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 -- 172.21.15.16:0/1402918428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001023a0 msgr2=0x7ff400106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 --2- 172.21.15.16:0/1402918428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001023a0 0x7ff400106840 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009900 tx=0x7ff3ec02a410 comp rx=0 tx=0).stop 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 -- 172.21.15.16:0/1402918428 shutdown_connections 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 --2- 172.21.15.16:0/1402918428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff40010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 --2- 172.21.15.16:0/1402918428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001023a0 0x7ff400106840 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 --2- 172.21.15.16:0/1402918428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4001019f0 0x7ff400101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 -- 172.21.15.16:0/1402918428 >> 172.21.15.16:0/1402918428 conn(0x7ff400075ea0 msgr2=0x7ff4000762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 -- 172.21.15.16:0/1402918428 shutdown_connections 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.110+0000 7ff406ac9640 1 -- 172.21.15.16:0/1402918428 wait complete. 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 Processor -- start 2024-09-12T01:13:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 -- start start 2024-09-12T01:13:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff400102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff400103350 0x7ff4001a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff400110e60 con 0x7ff4001019f0 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff400110ce0 con 0x7ff400106fa0 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff406ac9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff400110fe0 con 0x7ff400103350 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff404865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff404865640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52478/0 (socket says 172.21.15.16:52478) 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff404865640 1 -- 172.21.15.16:0/3440014370 learned_addr learned my addr 172.21.15.16:0/3440014370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.111+0000 7ff405066640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff400103350 0x7ff4001a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 -- 172.21.15.16:0/3440014370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff400103350 msgr2=0x7ff4001a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff400103350 0x7ff4001a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 -- 172.21.15.16:0/3440014370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 msgr2=0x7ff400102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff3fffff640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff400102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff400102e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 -- 172.21.15.16:0/3440014370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4001a6dd0 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff3fffff640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff400102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff404865640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001a70c0 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4009c30 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400f6b0 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.112+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a7610 con 0x7ff4001019f0 2024-09-12T01:13:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.113+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff400101dd0 con 0x7ff4001019f0 2024-09-12T01:13:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.114+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3f40109d0 con 0x7ff4001019f0 2024-09-12T01:13:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.115+0000 7ff3fdffb640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 0x7ff3f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.115+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7ff3f40cf120 con 0x7ff4001019f0 2024-09-12T01:13:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.115+0000 7ff3fffff640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 0x7ff3f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.115+0000 7ff3fffff640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 0x7ff3f00a51f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec032c70 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.119+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3f409bd40 con 0x7ff4001019f0 2024-09-12T01:13:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.298+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff40010a020 con 0x7ff4001019f0 2024-09-12T01:13:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.298+0000 7ff3fdffb640 1 -- 172.21.15.16:0/3440014370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) ==== 100+0+11 (secure 0 0 0) 0x7ff3f4097030 con 0x7ff4001019f0 2024-09-12T01:13:04.299 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-12T01:13:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 msgr2=0x7ff3f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 0x7ff3f00a51f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec000c00 tx=0x7ff3ec032c70 comp rx=0 tx=0).stop 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 msgr2=0x7ff4001028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).stop 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 shutdown_connections 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ff3f00a2d30 0x7ff3f00a51f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff400103350 0x7ff4001a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff400106fa0 0x7ff400102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 --2- 172.21.15.16:0/3440014370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4001019f0 0x7ff4001028d0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 >> 172.21.15.16:0/3440014370 conn(0x7ff400075ea0 msgr2=0x7ff4000ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 shutdown_connections 2024-09-12T01:13:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.300+0000 7ff406ac9640 1 -- 172.21.15.16:0/3440014370 wait complete. 2024-09-12T01:13:04.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T01:13:04.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 -- 172.21.15.16:0/827773000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf8101360 msgr2=0x7ffaf810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/827773000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf8101360 0x7ffaf810d890 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009cd0 tx=0x7ffaf002a8e0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 -- 172.21.15.16:0/827773000 shutdown_connections 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/827773000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf8101360 0x7ffaf810d890 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/827773000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf81009c0 0x7ffaf8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/827773000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81069d0 0x7ffaf8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 -- 172.21.15.16:0/827773000 >> 172.21.15.16:0/827773000 conn(0x7ffaf80fc6a0 msgr2=0x7ffaf80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 -- 172.21.15.16:0/827773000 shutdown_connections 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.416+0000 7ffafe7fd640 1 -- 172.21.15.16:0/827773000 wait complete. 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 Processor -- start 2024-09-12T01:13:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 -- start start 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81009c0 0x7ffaf81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf8101360 0x7ffaf8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8111f00 con 0x7ffaf81069d0 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8111d80 con 0x7ffaf81009c0 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.417+0000 7ffafe7fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8112080 con 0x7ffaf8101360 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52496/0 (socket says 172.21.15.16:52496) 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 -- 172.21.15.16:0/3369189987 learned_addr learned my addr 172.21.15.16:0/3369189987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffaf7fff640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81009c0 0x7ffaf81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffaf77fe640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf8101360 0x7ffaf8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 -- 172.21.15.16:0/3369189987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf8101360 msgr2=0x7ffaf8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf8101360 0x7ffaf8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 -- 172.21.15.16:0/3369189987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81009c0 msgr2=0x7ffaf81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81009c0 0x7ffaf81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 -- 172.21.15.16:0/3369189987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf819d3d0 con 0x7ffaf81069d0 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafcd9a640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005cc0 tx=0x7ffaf0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0004040 con 0x7ffaf81069d0 2024-09-12T01:13:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf819d660 con 0x7ffaf81069d0 2024-09-12T01:13:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf00046c0 con 0x7ffaf81069d0 2024-09-12T01:13:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0046480 con 0x7ffaf81069d0 2024-09-12T01:13:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.418+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81a4f50 con 0x7ffaf81069d0 2024-09-12T01:13:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.419+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf8100e20 con 0x7ffaf81069d0 2024-09-12T01:13:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.421+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffaf00305a0 con 0x7ffaf81069d0 2024-09-12T01:13:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.421+0000 7ffaf57fa640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 0x7ffae40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.422+0000 7ffaf7fff640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 0x7ffae40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.422+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8521+0+0 (secure 0 0 0) 0x7ffaf002e030 con 0x7ffaf81069d0 2024-09-12T01:13:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.422+0000 7ffaf7fff640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 0x7ffae40a51c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffae800bb60 tx=0x7ffae800b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.425+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaf004b060 con 0x7ffaf81069d0 2024-09-12T01:13:04.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2655805254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:04.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/466966847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:04 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2097879348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2655805254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:04.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/466966847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:04 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2097879348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2655805254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:04.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/466966847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:04 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2097879348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T01:13:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.651+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ffaf8199730 con 0x7ffaf81069d0 2024-09-12T01:13:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.659+0000 7ffaf57fa640 1 -- 172.21.15.16:0/3369189987 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740902 (secure 0 0 0) 0x7ffaf8199730 con 0x7ffaf81069d0 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 25, 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 15213, 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1052477540 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1052477540 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/1052477540", 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-12T00:56:03.340271+0000", 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-12T01:13:04.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24857, 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.677 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T01:13:04.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T01:13:04.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T01:13:04.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.681 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.682 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T01:13:04.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T01:13:04.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:13:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.698 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.699 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.700 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T01:13:04.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T01:13:04.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.721 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:13:04.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:13:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T01:13:04.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T01:13:04.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T01:13:04.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.739 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:13:04.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:13:04.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.746 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:13:04.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:13:04.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T01:13:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.755 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T01:13:04.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T01:13:04.766 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T01:13:04.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:13:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.784 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.788 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:04.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.844 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T01:13:04.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:04.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T01:13:04.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.857 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:13:04.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.872 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T01:13:04.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T01:13:04.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:04.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T01:13:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.884 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.885 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:04.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:13:04.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:13:04.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.894 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.913 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:13:04.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.926 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.929 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.937 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:04.939 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:13:04.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.944 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.946 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T01:13:04.947 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.948 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.966 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.974 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:13:04.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.984 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:04.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:04.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T01:13:04.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:13:04.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.991 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:04.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:04.998 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:04.999 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.020 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.026 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.029 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.030 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T01:13:05.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.047 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.050 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.058 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.060 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.061 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T01:13:05.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.068 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.069 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.073 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T01:13:05.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T01:13:05.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.088 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T01:13:05.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T01:13:05.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.099 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.101 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.102 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:13:05.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:13:05.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T01:13:05.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.115 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T01:13:05.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T01:13:05.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T01:13:05.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.122 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24880, 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.125 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:13:05.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T01:13:05.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T01:13:05.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.140 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.141 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.142 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.143 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.144 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.145 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T01:13:05.153 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.158 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:13:05.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:13:05.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.165 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:13:05.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:13:05.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T01:13:05.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T01:13:05.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T01:13:05.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T01:13:05.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.180 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.181 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.182 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.183 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.186 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:13:05.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:13:05.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:13:05.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:13:05.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.199 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.203 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T01:13:05.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T01:13:05.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T01:13:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T01:13:05.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.214 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.215 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:13:05.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T01:13:05.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.232 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.233 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:13:05.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:13:05.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:13:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:13:05.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.250 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.251 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.252 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:13:05.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T01:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T01:13:05.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T01:13:05.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.281 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.282 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.283 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.284 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:05.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.300 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:13:05.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.313 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T01:13:05.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.326 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.340 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.346 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T01:13:05.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T01:13:05.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.351 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T01:13:05.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:13:05.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.371 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.374 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.389 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.396 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.399 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.405 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.417 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T01:13:05.421 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:13:05.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.424 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.427 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.432 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.438 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T01:13:05.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:05.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.449 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.459 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.464 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.465 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.468 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.499 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.503 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T01:13:05.505 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.511 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.513 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:13:05.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:13:05.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T01:13:05.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.530 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T01:13:05.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T01:13:05.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T01:13:05.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.535 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.539 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.542 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.544 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.545 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T01:13:05.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T01:13:05.551 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.555 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.558 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T01:13:05.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:13:05.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-12T01:13:05.563 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.566 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.579 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T01:13:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-12T01:13:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.591 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T01:13:05.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-12T01:13:05.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-12T01:13:05.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-12T01:13:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-12T01:13:05.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T01:13:05.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-12T01:13:05.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-12T01:13:05.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-12T01:13:05.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T01:13:05.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.643 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-12T01:13:05.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.647 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-12T01:13:05.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-12T01:13:05.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-12T01:13:05.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.658 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.659 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.660 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.661 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T01:13:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T01:13:05.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.674 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.675 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.676 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T01:13:05.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T01:13:05.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T01:13:05.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T01:13:05.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T01:13:05.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.701 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.703 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.730 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.739 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.740 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.741 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.746 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-12T01:13:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-12T01:13:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.759 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-12T01:13:05.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-12T01:13:05.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.786 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.787 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.791 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-12T01:13:05.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.797 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.815 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.825 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.829 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.831 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.832 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.833 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-12T01:13:05.838 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.839 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.840 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.846 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.849 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.850 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.854 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-12T01:13:05.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.864 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-12T01:13:05.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-12T01:13:05.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.868 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T01:13:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.871 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T01:13:05.884 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.885 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.888 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.893 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.894 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.896 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.906 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.908 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.909 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.912 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.915 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.916 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.919 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-12T01:13:05.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-12T01:13:05.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.936 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.944 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.947 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-12T01:13:05.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.955 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:05.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-12T01:13:05.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-12T01:13:05.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.972 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.974 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.983 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.987 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T01:13:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-12T01:13:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-12T01:13:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-12T01:13:06.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-12T01:13:06.009 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T01:13:06.010 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T01:13:06.011 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T01:13:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-12T01:13:06.013 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 85, 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2789201555 2024-09-12T01:13:06.014 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 279414702 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1582339822 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2804724989 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1793801160 2024-09-12T01:13:06.017 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3883991746 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-12T01:13:06.018 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:05 smithi159 ceph-mon[35566]: pgmap v531: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3440014370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:05 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3369189987' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:05 smithi099 ceph-mon[35654]: pgmap v531: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3440014370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:13:06.020 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:05 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3369189987' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 msgr2=0x7ffae40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 0x7ffae40a51c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffae800bb60 tx=0x7ffae800b390 comp rx=0 tx=0).stop 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 msgr2=0x7ffaf819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005cc0 tx=0x7ffaf0003970 comp rx=0 tx=0).stop 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 shutdown_connections 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.664+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7ffae40a2d00 0x7ffae40a51c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:06.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffaf81069d0 0x7ffaf819cca0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:06.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffaf8101360 0x7ffaf8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:06.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 --2- 172.21.15.16:0/3369189987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf81009c0 0x7ffaf81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:06.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 >> 172.21.15.16:0/3369189987 conn(0x7ffaf80fc6a0 msgr2=0x7ffaf80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:06.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 shutdown_connections 2024-09-12T01:13:06.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:04.665+0000 7ffafe7fd640 1 -- 172.21.15.16:0/3369189987 wait complete. 2024-09-12T01:13:06.089 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T01:13:06.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T01:13:06.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:05 smithi016 ceph-mon[30972]: pgmap v531: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T01:13:06.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3440014370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T01:13:06.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:05 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3369189987' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T01:13:06.359 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:06.873 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:13:06.873 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:06.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1475, 2024-09-12T01:13:06.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1464, 2024-09-12T01:13:06.873 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1886272, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 554190, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.174859300, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.065731474, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.074082891, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.034979692, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1463, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.069338531, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000047394 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:06.874 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1506928, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 195168 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1052, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1286, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 259272, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 915024, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 8, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-09-12T01:13:06.875 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.114835994, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.039450443, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.026254175, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.038004240, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1283, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.043056199, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000033559 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.876 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 581056 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1142, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1142, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 303639, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 362333, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:06.877 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.099041284, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.032508235, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.028337883, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.028744270, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1141, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.040948864, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000035888 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-12T01:13:06.879 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5265, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.143259583, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000027209 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:06.881 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-12T01:13:06.882 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:06.883 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:06.884 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-12T01:13:06.885 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:06.886 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-12T01:13:06.887 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 9, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-12T01:13:06.888 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4196802, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4196661, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 9 2024-09-12T01:13:06.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-12T01:13:06.890 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 49212, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215328 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-12T01:13:06.891 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.892 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.895 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-12T01:13:06.896 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-12T01:13:06.897 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.898 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.899 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.900 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.901 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.902 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-12T01:13:06.904 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 4, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 3, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 3, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-12T01:13:06.905 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 4137, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:06.906 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:06.907 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:06.908 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 190177, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 53, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:06.909 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 180, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T01:13:06.910 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 3317, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 3317, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3317, 2024-09-12T01:13:06.911 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.988967070, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000599628 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3317, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3318 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 3317, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 3317, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 2176, 2024-09-12T01:13:06.912 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1139, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-12T01:13:06.913 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-12T01:13:06.914 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-12T01:13:06.915 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 103, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 21, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-09-12T01:13:06.916 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 7, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 2, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:06.917 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:06.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3665, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2098561, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3665, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3665, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2098561, 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:06.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:06.920 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3317, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2503, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3317, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2503, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-12T01:13:06.921 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3317, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 3317, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3317, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3317, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.922 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:06.923 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55b6de0e6b40": { 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:06.924 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:06.925 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:13:07.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:07 smithi099 ceph-mon[35654]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:07.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:07 smithi159 ceph-mon[35566]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:07.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:07 smithi016 ceph-mon[30972]: pgmap v532: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:07.769 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:07.769 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T01:13:07.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T01:13:08.028 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 324, 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 314, 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1472480, 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 193840, 2024-09-12T01:13:08.598 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.052427494, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021142108, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.041959260, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 313, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017074456, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000054550 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:08.599 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1472480, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 193840 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 241, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 480, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 278720, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:08.600 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.020990110, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014821913, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002885243, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 238, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.022677301, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000095282 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.601 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 278720 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3072, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002322617, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000043217, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002251570, 2024-09-12T01:13:08.602 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004037072, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001345690 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:08.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:08.604 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:08.605 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:08.606 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148771, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 32, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T01:13:08.607 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:08.608 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 327, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1426771, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 327, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 327, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1426771, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:08.609 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:08.610 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:08.610 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T01:13:09.359 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T01:13:09.359 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T01:13:09.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T01:13:09.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:09 smithi099 ceph-mon[35654]: pgmap v533: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T01:13:09.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:09 smithi159 ceph-mon[35566]: pgmap v533: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T01:13:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:09.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:09 smithi016 ceph-mon[30972]: pgmap v533: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1479, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1468, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1887552, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 555784, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.179465706, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.077685986, 2024-09-12T01:13:10.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.068474045, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.031982048, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1467, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.074126783, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000050529 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.161 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1507200, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 195808 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2170, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2404, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 556269, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1273297, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.151445428, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.050283441, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.038299009, 2024-09-12T01:13:10.162 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.050723195, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2402, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.077696249, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000032346 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.163 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 585088 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 6, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1629, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 1284, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002206210, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000064001, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001791481, 2024-09-12T01:13:10.164 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000094425, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002704674, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000676168 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:13:10.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5553, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.130352417, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000023474 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.166 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.167 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 0, 2024-09-12T01:13:10.168 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 1, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:10.169 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-09-12T01:13:10.170 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:10.171 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:10.172 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 8, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-09-12T01:13:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4196661, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4196520, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 8 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 3, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 3, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 1, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 1, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-09-12T01:13:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 47888, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215328 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-09-12T01:13:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-09-12T01:13:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 4, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 4, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 3100, 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:10.191 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:10.192 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 198314, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 59, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:10.193 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 180, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 8512, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 13, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-09-12T01:13:10.194 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 3300, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 3300, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3300, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.648972668, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000499688 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3300, 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3301 2024-09-12T01:13:10.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 3300, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 3300, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 2157, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1141, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-09-12T01:13:10.196 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-09-12T01:13:10.197 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-09-12T01:13:10.198 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 103, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 21, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 8, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 2, 2024-09-12T01:13:10.199 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.200 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3649, 2024-09-12T01:13:10.201 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2096216, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3649, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3649, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2096216, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.202 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3300, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2362, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3300, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2362, 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:10.203 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:10.204 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3300, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 3300, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3300, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3300, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.205 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.206 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x56365b296b40": { 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:10.207 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:10.208 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:11.010 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:11.010 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T01:13:11.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T01:13:11.265 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:11 smithi099 ceph-mon[35654]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:11.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:11 smithi099 ceph-mon[35654]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T01:13:11.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:11 smithi016 ceph-mon[30972]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:11.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:11 smithi016 ceph-mon[30972]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T01:13:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:11 smithi159 ceph-mon[35566]: pgmap v534: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-12T01:13:11.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:11 smithi159 ceph-mon[35566]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3072, 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-12T01:13:11.824 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.003438276, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000042646, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002432495, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004326708, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001442236 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:11.825 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 243, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 480, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 279072, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:11.826 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.021103270, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015521660, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002489651, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 240, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.014932545, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000062218 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 279072 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 325, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 315, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1472752, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 194480, 2024-09-12T01:13:11.828 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.052595691, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.022404391, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.043005717, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 314, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017049461, 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000054297 2024-09-12T01:13:11.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1472752, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 194480 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:11.831 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148754, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-09-12T01:13:11.833 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:11.834 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 328, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1426901, 2024-09-12T01:13:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 328, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 328, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1426901, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-12T01:13:11.836 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-12T01:13:12.591 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T01:13:12.592 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T01:13:12.592 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T01:13:12.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T01:13:12.852 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:13:13.379 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-12T01:13:13.379 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 464, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 445, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1548024, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 988326, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 72, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.089430285, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.028192704, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.067603466, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000299999, 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:13.380 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 445, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.034275462, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000077023 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 71, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000006351, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000089 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1535328, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 201168 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 18, 2024-09-12T01:13:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 252, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4396, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 621091, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 10, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.017393792, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.010093615, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.004073518, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000330404, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 249, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.023729167, 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000095297 2024-09-12T01:13:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000132, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000066 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 584288 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1348, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1117, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 398322, 2024-09-12T01:13:13.383 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 197307, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.115193750, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.044358090, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.071760570, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.007471837, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1115, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.049162104, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000044091 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:13.384 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2397, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000518118, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000216 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:13.385 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 51, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001319495, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000025872 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000014080, 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000007040 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:13.387 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 16, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 16, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 4, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 42, 2024-09-12T01:13:13.388 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 33, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 7, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 1, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:13.389 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 5, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 298, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 7, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:13.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 19, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 14, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 11, 2024-09-12T01:13:13.391 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-12T01:13:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 14, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 83, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 4, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-12T01:13:13.393 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 4, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195426, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-09-12T01:13:13.394 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.005473254, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001824418 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 20, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 22, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 19, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 22, 2024-09-12T01:13:13.395 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 3, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 16, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 20, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 4, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 51076, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 223520 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-12T01:13:13.396 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 4, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.397 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:13.398 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.400 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-12T01:13:13.401 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.402 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.404 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:13.405 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:13.407 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 0, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 0, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-12T01:13:13.408 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 74, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:13.409 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:13.410 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:13.411 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 222807, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 71, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 180, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:13.412 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 81781, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 326, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-12T01:13:13.413 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 38, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 11334, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 28, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 38, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 38, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.204995252, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005394611 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 38, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 60 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.414 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 38, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 38, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 3, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 6, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-12T01:13:13.415 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:13.416 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-12T01:13:13.417 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 104, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 22, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 8, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 10, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 2, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 14, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-09-12T01:13:13.418 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 1 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 97 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:13.419 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 1826, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1763933, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:13.420 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 1826, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1826, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1763933, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:13.421 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 38, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 11334, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 38, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 11334, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 38, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 38, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 38, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 38, 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:13.423 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:13.424 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x556ec6614960": { 2024-09-12T01:13:13.425 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1122, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1122, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:13.426 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:13.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:13.427 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-12T01:13:13.524 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:13 smithi099 ceph-mon[35654]: pgmap v536: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:13.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:13 smithi159 ceph-mon[35566]: pgmap v536: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:13.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:13 smithi016 ceph-mon[30972]: pgmap v536: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=19 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:14.172 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:14.172 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T01:13:14.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T01:13:14.422 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2277, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2279, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 681875, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 724093, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.209265019, 2024-09-12T01:13:14.917 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.075526350, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.052236058, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.062480000, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2277, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.083590258, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000036710 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 5, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 244, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1036, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 674266, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.015041550, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.009939832, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002463403, 2024-09-12T01:13:14.919 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000052894, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 241, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.018311813, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000075982 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 673568 2024-09-12T01:13:14.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 358, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 345, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1512764, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 199950, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.060382857, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.024286496, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.049069100, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000035897, 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:14.921 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 345, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.026212050, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000075976 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1512496, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 199632 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-12T01:13:14.922 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4553, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.116098685, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000025499 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:14.923 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 0, 2024-09-12T01:13:14.925 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-12T01:13:14.926 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:14.927 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-12T01:13:14.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-12T01:13:14.929 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:14.930 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 2, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-12T01:13:14.931 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4194892, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 4194352, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 2 2024-09-12T01:13:14.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 3, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-12T01:13:14.933 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 48932, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 215328 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.934 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.936 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:14.937 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-12T01:13:14.938 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.939 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-12T01:13:14.940 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.941 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.942 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.943 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.944 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 0, 2024-09-12T01:13:14.946 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:14.947 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:14.948 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:14.949 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 182097, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 49, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8256, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 183, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8512, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 13, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 2280, 2024-09-12T01:13:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 2280, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2280, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.090973967, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000478497 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2280, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2281 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 2280, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 2280, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1137, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1141, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-12T01:13:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-12T01:13:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 104, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 22, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 4, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 5, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 1, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-12T01:13:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 2636, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1937257, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 2636, 2024-09-12T01:13:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2636, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1937257, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2280, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 593, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2280, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 593, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2280, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 2280, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2280, 2024-09-12T01:13:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2280, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5636508beb40": { 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:14.965 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-12T01:13:15.564 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:15 smithi099 ceph-mon[35654]: pgmap v537: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:15.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:15 smithi159 ceph-mon[35566]: pgmap v537: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:15.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:15 smithi016 ceph-mon[30972]: pgmap v537: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-12T01:13:15.800 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:15.801 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:15.801 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T01:13:15.801 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T01:13:16.045 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 50, 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 288, 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 9739, 2024-09-12T01:13:16.545 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 582841, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 22, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.035369344, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.013187204, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.013030500, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000496533, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 284, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.031616740, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000111326 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.546 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 91, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000026023, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000285 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 410960 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1749, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1553, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1927686, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 818762, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 13, 2024-09-12T01:13:16.547 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.181304734, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.071863560, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.124407039, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.007762733, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1555, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.082022485, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000052747 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2490, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000602094, 2024-09-12T01:13:16.548 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000241 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1528048, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 222496 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 85, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 146, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 16806, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 888143, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 63, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.030026717, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.004486865, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.022890254, 2024-09-12T01:13:16.549 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000449217, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 143, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.019752407, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000138128 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 81, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000007811, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000096 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 4624, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 270032 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-12T01:13:16.550 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 57, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001983498, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000034798 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:16.551 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000009485, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000004742 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 10, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 7, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001131579, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000161654 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-12T01:13:16.552 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 3, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 16, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 16, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 4, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 2, 2024-09-12T01:13:16.553 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 2, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 53, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 46, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 6, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-12T01:13:16.554 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 5, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 298, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 7, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:16.555 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 5, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 5, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.556 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 18, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 12, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 12, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:16.557 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 14, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 83, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-12T01:13:16.558 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 8, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 8, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-12T01:13:16.559 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4196661, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.012780869, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001825838 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 20, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 22, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2, 2024-09-12T01:13:16.560 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 19, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 23, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 4, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 16, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 20, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 4, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 2, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 5, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 3, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 53820, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 223520 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-12T01:13:16.561 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 10, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 10, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 34, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000703642, 2024-09-12T01:13:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000175910 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-12T01:13:16.563 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000304687, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000152343 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000123250, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000123250 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.564 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.565 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.566 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.567 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.568 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:16.569 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:16.570 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.571 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.572 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-12T01:13:16.573 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-09-12T01:13:16.574 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 5, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 1, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 988, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:16.575 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:16.576 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:16.577 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 263351, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 89, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:16.578 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8256, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 183, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 82605, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 341, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-12T01:13:16.579 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 48, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 13255, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 28, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 48, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 48, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.237994477, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004958218 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-12T01:13:16.580 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 48, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 73 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 48, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 48, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 7, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 9, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-12T01:13:16.581 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-12T01:13:16.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-12T01:13:16.583 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 104, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 22, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 9, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 14, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 5, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-12T01:13:16.584 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 20, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 1 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 97 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.585 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 1880, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1761198, 2024-09-12T01:13:16.586 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 1880, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1880, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1761198, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:16.587 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 48, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 13255, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 48, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 13255, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.588 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 48, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 48, 2024-09-12T01:13:16.589 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 48, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 48, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:16.590 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:16.591 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x55e2c4b8c960": { 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 8, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 2357, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 8, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 7, 2024-09-12T01:13:16.592 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2357, 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:16.593 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=18 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:17.369 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:17.369 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T01:13:17.370 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T01:13:17.616 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T01:13:17.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:17 smithi159 ceph-mon[35566]: pgmap v538: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:13:17.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:17 smithi016 ceph-mon[30972]: pgmap v538: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:13:17.738 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:17 smithi099 ceph-mon[35654]: pgmap v538: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1150, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1150, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 305037, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 364940, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.107519928, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.042747623, 2024-09-12T01:13:18.137 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.027414861, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.026902120, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1149, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.046529267, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000040495 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:18.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1417, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1406, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1771486, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 538365, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.154396990, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.053889614, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.070014460, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.035583435, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1405, 2024-09-12T01:13:18.139 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.055643058, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000039603 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1510704, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 201280 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1142, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1383, 2024-09-12T01:13:18.140 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 383360, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 948040, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.145974134, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.052602046, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.034867748, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.042047556, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1380, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.086453265, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000062647 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:18.141 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 585520 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:18.142 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5492, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.151562925, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000027597 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-12T01:13:18.143 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-12T01:13:18.144 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-12T01:13:18.145 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-12T01:13:18.146 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:18.147 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:18.148 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-12T01:13:18.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 4, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-12T01:13:18.150 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4195426, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 4195285, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 4 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.151 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 3, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 49120, 2024-09-12T01:13:18.152 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 215328 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-12T01:13:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.155 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.156 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-12T01:13:18.157 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.158 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.160 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-12T01:13:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.163 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 0, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 1, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-12T01:13:18.164 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 1224, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:18.165 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:18.166 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 190177, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 53, 2024-09-12T01:13:18.167 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8256, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 183, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-12T01:13:18.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8512, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 13, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3346, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3346, 2024-09-12T01:13:18.169 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3346, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.534963342, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000458745 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3346, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3347 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3346, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3346, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-12T01:13:18.170 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2198, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1146, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-09-12T01:13:18.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-12T01:13:18.172 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 105, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-12T01:13:18.173 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 23, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 5, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 6, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 1, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-12T01:13:18.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:18.175 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 3705, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 2105100, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 3705, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3705, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2105100, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3346, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 1127, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3346, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1127, 2024-09-12T01:13:18.177 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:18.178 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3346, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3346, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3346, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3346, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:18.179 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.180 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x55ddca8dcb40": { 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-12T01:13:18.181 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-12T01:13:18.182 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-12T01:13:18.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:18 smithi099 ceph-mon[35654]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T01:13:18.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:18 smithi099 ceph-mon[35654]: pgmap v540: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-12T01:13:18.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:18 smithi159 ceph-mon[35566]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T01:13:18.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:18 smithi159 ceph-mon[35566]: pgmap v540: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-12T01:13:18.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:18 smithi016 ceph-mon[30972]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T01:13:18.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:18 smithi016 ceph-mon[30972]: pgmap v540: 97 pgs: 97 active+clean; 30 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-09-12T01:13:19.004 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-12T01:13:19.004 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-12T01:13:19.004 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-12T01:13:19.004 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-12T01:13:19.004 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-12T01:13:19.005 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:19.005 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:19.005 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T01:13:19.005 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T01:13:19.265 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:13:19.812 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-12T01:13:19.812 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:19.812 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1363, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1132, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 403250, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 187004, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.150057339, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.066141032, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.078912441, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.006752569, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1131, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.067757570, 2024-09-12T01:13:19.813 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000059909 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2403, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000819869, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000341 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 463, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 444, 2024-09-12T01:13:19.814 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1549444, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1013565, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 74, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.083840277, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.025351302, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.071187219, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000269507, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 443, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.037212711, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000084001 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.815 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 66, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000005181, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000078 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1537120, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 203360 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 11, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 250, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 2707, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 685859, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2024-09-12T01:13:19.816 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.017186812, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.010420523, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.003513829, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000340130, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 247, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.023404791, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000094756 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 676800 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 31, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001615224, 2024-09-12T01:13:19.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000052104 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000012392, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000006196 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-09-12T01:13:19.819 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 16, 2024-09-12T01:13:19.820 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 16, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 4, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 40, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 34, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 6, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:19.821 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 5, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 298, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 7, 2024-09-12T01:13:19.822 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:19.823 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 18, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 12, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 12, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:19.824 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 14, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 83, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:19.825 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 2, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-09-12T01:13:19.826 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4194892, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001252292, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001252292 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 20, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 22, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 2, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 19, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 23, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 4, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 16, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 20, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 4, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-09-12T01:13:19.828 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 51244, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.829 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.830 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-12T01:13:19.831 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.832 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.834 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.835 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.837 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-12T01:13:19.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.839 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.840 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 0, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.841 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:19.842 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:19.843 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:19.844 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 238535, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 73, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 180, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:19.845 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 81781, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 326, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 35, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 10710, 2024-09-12T01:13:19.846 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 28, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 35, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.207996004, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005942742 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 57 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 35, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 0, 2024-09-12T01:13:19.847 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 35, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 5, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 6, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-12T01:13:19.848 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-12T01:13:19.849 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-12T01:13:19.850 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 105, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 9, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 14, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 5, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 14, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 1 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-12T01:13:19.851 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 97 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:19.852 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1833, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1767697, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1833, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1833, 2024-09-12T01:13:19.853 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1767697, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:19.854 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 35, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 10710, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 35, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 10710, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-12T01:13:19.855 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 35, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 35, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 35, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 35, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:19.857 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55fdb7ea2960": { 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 588, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:19.858 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1, 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 588, 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:19.859 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-12T01:13:20.840 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=18 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:20.841 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:20.841 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T01:13:20.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T01:13:21.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:20 smithi099 ceph-mon[35654]: pgmap v541: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 109 B/s wr, 10 op/s 2024-09-12T01:13:21.088 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:13:21.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:20 smithi159 ceph-mon[35566]: pgmap v541: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 109 B/s wr, 10 op/s 2024-09-12T01:13:21.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:20 smithi016 ceph-mon[30972]: pgmap v541: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 109 B/s wr, 10 op/s 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1377, 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1141, 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 405698, 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 454315, 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-09-12T01:13:21.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.147966337, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.067808675, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.075568705, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.006070752, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1141, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.066115202, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000057944 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2408, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000696889, 2024-09-12T01:13:21.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000289 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 465, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 451, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1541910, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 942871, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 68, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.086880175, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.026747908, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.077683293, 2024-09-12T01:13:21.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000226923, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 450, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.046394306, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000103098 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 68, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000006027, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000088 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1529952, 2024-09-12T01:13:21.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 205776 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 45, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 341, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 11959, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 641486, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 15, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-12T01:13:21.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.034016466, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.015961501, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.014300876, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000211639, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 337, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.024387230, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000072365 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 112, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000027868, 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000248 2024-09-12T01:13:21.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 7632, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 594128 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:21.631 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 54, 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001741189, 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000032244 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:21.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000019564, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000009782 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 6, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 4, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4, 2024-09-12T01:13:21.633 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000534699, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000133674 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 2, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-12T01:13:21.634 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 16, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 16, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 4, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 1, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 1, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 48, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 39, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 1, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 7, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:21.635 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 5, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 298, 2024-09-12T01:13:21.636 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 7, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 3, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:21.637 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 3, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 1 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 16, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 11, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 12, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:21.638 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-12T01:13:21.639 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 14, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 83, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.640 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 9, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 9, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-12T01:13:21.641 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4196802, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.012454605, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001556825 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 20, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 22, 2024-09-12T01:13:21.642 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 2, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 19, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 22, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 3, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 16, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 20, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 4, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 1, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 4, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 3, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 53876, 2024-09-12T01:13:21.643 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 7, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 6, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 26, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000287320, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000143660 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.645 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000125528, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000125528 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-12T01:13:21.646 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000121851, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000121851 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-12T01:13:21.648 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.649 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-12T01:13:21.650 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:21.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 4, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 3, 2024-09-12T01:13:21.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 2, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1009, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:21.657 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:21.658 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:21.659 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 239015, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 78, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-12T01:13:21.660 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8256, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 183, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:21.661 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 82205, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 335, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-12T01:13:21.662 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 48, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 13125, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 48, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 48, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.227995605, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004749908 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-12T01:13:21.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 48, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 72 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 48, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 0, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 48, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-09-12T01:13:21.664 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 8, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 9, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 6, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:21.665 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-12T01:13:21.666 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 105, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2024-09-12T01:13:21.667 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 8, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 13, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 5, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 17, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 2 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-12T01:13:21.668 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 97 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:21.669 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1883, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1766473, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1883, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1883, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1766473, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 48, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 13125, 2024-09-12T01:13:21.671 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 48, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 13125, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:21.672 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 48, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 48, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 48, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 48, 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:21.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:21.674 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55d2a6564960": { 2024-09-12T01:13:21.675 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 9, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 2498, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 9, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 8, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2498, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:21.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:21.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.677 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:21.677 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=16 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:22.417 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:22.418 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T01:13:22.418 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T01:13:22.664 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:13:22.964 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:22 smithi159 ceph-mon[35566]: pgmap v542: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:23.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:22 smithi099 ceph-mon[35654]: pgmap v542: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:23.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:22 smithi016 ceph-mon[30972]: pgmap v542: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:23.187 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-12T01:13:23.187 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2666, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2934, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 2020105, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 8819313, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 282, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.402365073, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.138520475, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.234282893, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.027175517, 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:23.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2935, 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.119505348, 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000040717 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2960, 2024-09-12T01:13:23.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000735924, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000248 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1649408, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 737552 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 4737, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 6402, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1604331, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 15865443, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-09-12T01:13:23.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.552954049, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.225427642, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.241871939, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.096589920, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6400, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.187528347, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000029301 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7647, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002005857, 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000262 2024-09-12T01:13:23.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 43120, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 133008 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 5816, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 5263, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1897912, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 14451976, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 22, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.545930768, 2024-09-12T01:13:23.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.182023254, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.245551011, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.127998526, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5259, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.210152753, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000039960 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6386, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001345139, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000210 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 575984, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 180624 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5399, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.543666606, 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000285917 2024-09-12T01:13:23.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 15, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000361722, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000024114 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 874, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 874, 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-12T01:13:23.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 874, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.440966990, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003937033 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 12, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 47, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 2, 2024-09-12T01:13:23.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 45, 2024-09-12T01:13:23.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:23.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 44, 2024-09-12T01:13:23.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-12T01:13:23.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 2, 2024-09-12T01:13:23.197 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 2, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 989, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 975, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-12T01:13:23.198 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-12T01:13:23.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-12T01:13:23.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-12T01:13:23.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:23.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 415, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 7, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 415, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 267, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 30, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 237, 2024-09-12T01:13:23.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 148, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 665, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 667, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 599, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 779, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 719, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 2, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 626 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 23, 2024-09-12T01:13:23.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 47, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 2, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 3, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 1, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:23.202 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 14, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 194, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 30, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 2342, 2024-09-12T01:13:23.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2342, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 9, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 9, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-12T01:13:23.204 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 34546679, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2255, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.488629842, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001990523 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 40, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 71, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 31, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 20, 2024-09-12T01:13:23.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 20, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 0, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 47, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 48, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 1, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 2, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 20, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 18, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 59560, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 231712 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-12T01:13:23.206 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 2152, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 874, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 44, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 18, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.011739312, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000652184 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 182, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.027823440, 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000152876 2024-09-12T01:13:23.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002376709, 2024-09-12T01:13:23.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000396118 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-12T01:13:23.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-12T01:13:23.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17, 2024-09-12T01:13:23.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002356401, 2024-09-12T01:13:23.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000138611 2024-09-12T01:13:23.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.013417929, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001677241 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.083222158, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004895421 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000192323, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000192323 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.004702244, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002351122 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001990596, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000995298 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:23.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.013020055, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.006510027 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.043487325, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004348732 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 600, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.214194302, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005356990 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-09-12T01:13:23.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.019898957, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002487369 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002545239, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002545239 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:23.218 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 6, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 4, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-12T01:13:23.219 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 76, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 38, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1008, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:23.220 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:23.221 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:23.222 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 335574, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 117, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 180, 2024-09-12T01:13:23.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 221021, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 1543, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-12T01:13:23.224 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 3971, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 30375696, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 32, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 3971, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3971, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 21.572587458, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005432532 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-12T01:13:23.225 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3971, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4016 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 3971, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 476, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 3495, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-12T01:13:23.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 11, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 31, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 2256, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 68, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 520, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-12T01:13:23.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 445, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 22, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:23.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 599, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-12T01:13:23.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 105, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-12T01:13:23.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 13, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 1, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 56, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 8 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-12T01:13:23.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 208 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 1, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 1, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 1, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 1, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:23.235 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 13215, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 4232565, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 13215, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 13215, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 4232565, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.236 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3971, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 30375853, 2024-09-12T01:13:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3971, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 30375853, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3971, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 3971, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3971, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3971, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:23.239 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 157, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:23.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 157, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55aec2b86960": { 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2342, 2024-09-12T01:13:23.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 30352375, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2342, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2255, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 30352375, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:23.242 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-12T01:13:23.991 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=23 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=17 2024-09-12T01:13:23.992 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=10 2024-09-12T01:13:23.992 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T01:13:23.992 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T01:13:24.237 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T01:13:24.757 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-12T01:13:24.757 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T01:13:24.757 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2689, 2024-09-12T01:13:24.757 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2677, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 14781694, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 943028, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.306772982, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.094244986, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.127381689, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.077993915, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2676, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.115639476, 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000043213 2024-09-12T01:13:24.758 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1511408, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 202720 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 745, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 989, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 9568721, 2024-09-12T01:13:24.759 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 827236, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 9, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.085699213, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.030010784, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027065287, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.020598728, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 986, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.042254044, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000042854 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:24.760 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 590896 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1043, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1043, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 8651812, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 330857, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-09-12T01:13:24.761 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.096098681, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.032672797, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.027745995, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.027705218, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1042, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.049988046, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000047973 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.762 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:24.763 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6975, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.187736881, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000026915 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.764 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-12T01:13:24.765 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 36, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 36, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 36, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-12T01:13:24.766 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-09-12T01:13:24.767 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 7, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-12T01:13:24.768 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-09-12T01:13:24.769 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-12T01:13:24.770 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-12T01:13:24.771 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2342, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 9, 2024-09-12T01:13:24.772 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 34546679, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 34546567, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 2342 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-12T01:13:24.773 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 31, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 32, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 1, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 11, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 11, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 0, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 36, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 36, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 0, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 50228, 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215328 2024-09-12T01:13:24.774 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-12T01:13:24.775 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.776 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.777 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-12T01:13:24.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.779 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.781 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-12T01:13:24.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.783 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-12T01:13:24.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.785 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.786 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-12T01:13:24.787 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 6, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 4, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 5, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 4167, 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-12T01:13:24.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T01:13:24.789 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T01:13:24.790 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-12T01:13:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 217428, 2024-09-12T01:13:24.792 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 108, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8136, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 180, 2024-09-12T01:13:24.793 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 184427, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 1359, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-12T01:13:24.794 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 4110, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 4, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 4110, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4110, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.201938478, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000779060 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4110, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4111 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 4110, 2024-09-12T01:13:24.795 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 4110, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2286, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1821, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-12T01:13:24.796 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T01:13:24.797 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-12T01:13:24.798 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 105, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 23, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 8, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 10, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 2, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-09-12T01:13:24.799 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-12T01:13:24.800 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4473, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 32578174, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:24.801 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4473, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4473, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 32578174, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:24.802 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4110, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 30352385, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4110, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 30352385, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 4110, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 4110, 2024-09-12T01:13:24.804 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4110, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 4110, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:24.805 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x562501fb2b40": { 2024-09-12T01:13:24.806 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-12T01:13:24.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-12T01:13:24.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.808 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-12T01:13:24.808 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-12T01:13:25.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:24 smithi099 ceph-mon[35654]: pgmap v543: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:25.057 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:24 smithi159 ceph-mon[35566]: pgmap v543: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:25.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:24 smithi016 ceph-mon[30972]: pgmap v543: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-12T01:13:25.480 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T01:13:25.480 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T01:13:25.481 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-12T01:13:25.481 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-12T01:13:25.481 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2024-09-12T01:13:25.481 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-12T01:13:25.481 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T01:13:25.481 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T01:13:25.491 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T01:13:25.492 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:13:25.492 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:13:25.514 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-12T01:13:25.514 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T01:13:25.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T01:13:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2731738402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 msgr2=0x7f0cf410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2731738402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf410ea90 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009ca0 tx=0x7f0cec02a8c0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2731738402 shutdown_connections 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2731738402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf410ea90 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2731738402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4102790 0x7f0cf41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2731738402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4101de0 0x7f0cf41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2731738402 >> 172.21.15.16:0/2731738402 conn(0x7f0cf40fd8c0 msgr2=0x7f0cf40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2731738402 shutdown_connections 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.443+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2731738402 wait complete. 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.444+0000 7f0cf9adc640 1 Processor -- start 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.444+0000 7f0cf9adc640 1 -- start start 2024-09-12T01:13:26.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.444+0000 7f0cf9adc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4101de0 0x7f0cf40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf9adc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 0x7f0cf4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf9adc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf9adc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4111f70 con 0x7f0cf4106f10 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf9adc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf4111df0 con 0x7f0cf4102790 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf9adc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41120f0 con 0x7f0cf4101de0 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40828/0 (socket says 172.21.15.16:40828) 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 -- 172.21.15.16:0/2948638753 learned_addr learned my addr 172.21.15.16:0/2948638753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf2ffd640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 0x7f0cf4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf37fe640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4101de0 0x7f0cf40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 -- 172.21.15.16:0/2948638753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4101de0 msgr2=0x7f0cf40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4101de0 0x7f0cf40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 -- 172.21.15.16:0/2948638753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 msgr2=0x7f0cf4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 0x7f0cf4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 -- 172.21.15.16:0/2948638753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41a6850 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf2ffd640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 0x7f0cf4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf3fff640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031a20 tx=0x7f0cec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.445+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec003b70 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.446+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a6ae0 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.446+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cec003d10 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.446+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec02fc40 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.446+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a7070 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.447+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf41021c0 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.448+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0cec004050 con 0x7f0cf4106f10 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.448+0000 7f0cf0ff9640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.449+0000 7f0cf37fe640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.449+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7f0cec0f0410 con 0x7f0cf4106f10 2024-09-12T01:13:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.449+0000 7f0cf37fe640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40095f0 tx=0x7f0ce4005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:26.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.453+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cec0bd2f0 con 0x7f0cf4106f10 2024-09-12T01:13:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.643+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0cf402d7c0 con 0x7f0cf4106f10 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.643+0000 7f0cf0ff9640 1 -- 172.21.15.16:0/2948638753 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v105) ==== 81+0+938 (secure 0 0 0) 0x7f0cec0bcbf0 con 0x7f0cf4106f10 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1272021982 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1272021982 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2952840401 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/7504959 2024-09-13T00:46:43.483931+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1067528076 2024-09-13T00:46:43.483931+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1286739529 2024-09-13T00:47:08.126530+0000 2024-09-12T01:13:26.643 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3359074253 2024-09-13T00:46:43.483931+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/1538480573 2024-09-13T00:47:08.126530+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2290747564 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2269637621 2024-09-13T00:46:43.483931+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/90830535 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/7504959 2024-09-13T00:46:43.483931+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2087944877 2024-09-13T00:47:08.126530+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/1538480573 2024-09-13T00:47:08.126530+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1789090922 2024-09-13T00:47:08.126530+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1572487203 2024-09-13T00:56:03.339943+0000 2024-09-12T01:13:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:listed 16 entries 2024-09-12T01:13:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 msgr2=0x7f0cdc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40095f0 tx=0x7f0ce4005f90 comp rx=0 tx=0).stop 2024-09-12T01:13:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 msgr2=0x7f0cf4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:26.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031a20 tx=0x7f0cec031a50 comp rx=0 tx=0).stop 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 shutdown_connections 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f0cdc0a2cf0 0x7f0cdc0a51b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4106f10 0x7f0cf4074f70 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cf4102790 0x7f0cf4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 --2- 172.21.15.16:0/2948638753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cf4101de0 0x7f0cf40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 >> 172.21.15.16:0/2948638753 conn(0x7f0cf40fd8c0 msgr2=0x7f0cf40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 shutdown_connections 2024-09-12T01:13:26.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:26.646+0000 7f0cf9adc640 1 -- 172.21.15.16:0/2948638753 wait complete. 2024-09-12T01:13:26.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: pgmap v544: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T01:13:26.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:13:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:13:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:13:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:13:26.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:26 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2948638753' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: pgmap v544: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:13:27.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:26 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2948638753' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:13:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: pgmap v544: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T01:13:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T01:13:27.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T01:13:27.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T01:13:27.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' 2024-09-12T01:13:27.117 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:26 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2948638753' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T01:13:27.456 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T01:13:27.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:13:27.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T01:13:27.503 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:13:27.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:13:27.503 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T01:13:27.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:13:27.542 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-12T01:13:27.598 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T01:13:27.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:13:27.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T01:13:27.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T01:13:27.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-12T01:13:27.763 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T01:13:27.774 INFO:tasks.cephadm:Teardown begin 2024-09-12T01:13:27.774 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T01:13:27.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:13:28.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 -- 172.21.15.16:0/1026804079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 msgr2=0x7f1d1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 --2- 172.21.15.16:0/1026804079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c10da40 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009ca0 tx=0x7f1d1402a950 comp rx=0 tx=0).stop 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 -- 172.21.15.16:0/1026804079 shutdown_connections 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 --2- 172.21.15.16:0/1026804079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c10da40 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 --2- 172.21.15.16:0/1026804079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 --2- 172.21.15.16:0/1026804079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 0x7f1d1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 -- 172.21.15.16:0/1026804079 >> 172.21.15.16:0/1026804079 conn(0x7f1d1c0fc7d0 msgr2=0x7f1d1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 -- 172.21.15.16:0/1026804079 shutdown_connections 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.522+0000 7f1d22f52640 1 -- 172.21.15.16:0/1026804079 wait complete. 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 Processor -- start 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 -- start start 2024-09-12T01:13:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 0x7f1d1c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c110f80 con 0x7f1d1c102ef0 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c110e00 con 0x7f1d1c100e20 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d22f52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c111100 con 0x7f1d1c102570 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60940/0 (socket says 172.21.15.16:60940) 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 -- 172.21.15.16:0/3505360767 learned_addr learned my addr 172.21.15.16:0/3505360767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d1bfff640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d20cee640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 0x7f1d1c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 -- 172.21.15.16:0/3505360767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 msgr2=0x7f1d1c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 -- 172.21.15.16:0/3505360767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 msgr2=0x7f1d1c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 0x7f1d1c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.523+0000 7f1d214ef640 1 -- 172.21.15.16:0/3505360767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c19d440 con 0x7f1d1c102ef0 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d1bfff640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d214ef640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1d1402ae70 tx=0x7f1d1402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14034040 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d14030920 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c19d6d0 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14045480 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.524+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1a5010 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.525+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c1021b0 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.526+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1d14037020 con 0x7f1d1c102ef0 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.527+0000 7f1d19ffb640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 0x7f1d0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.527+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7f1d140f0300 con 0x7f1d1c102ef0 2024-09-12T01:13:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.527+0000 7f1d20cee640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 0x7f1d0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.527+0000 7f1d20cee640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 0x7f1d0c0a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d100058c0 tx=0x7f1d1000a4c0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.531+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d140bd030 con 0x7f1d1c102ef0 2024-09-12T01:13:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.742+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d1c10a0a0 con 0x7f1d1c102ef0 2024-09-12T01:13:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.743+0000 7f1d19ffb640 1 -- 172.21.15.16:0/3505360767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f1d140bc930 con 0x7f1d1c102ef0 2024-09-12T01:13:28.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:13:28.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:13:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 msgr2=0x7f1d0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 0x7f1d0c0a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d100058c0 tx=0x7f1d1000a4c0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 msgr2=0x7f1d1c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f1d1402ae70 tx=0x7f1d1402f5d0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 shutdown_connections 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f1d0c0a2cf0 0x7f1d0c0a51b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d1c102ef0 0x7f1d1c19cd60 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1c102570 0x7f1d1c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 --2- 172.21.15.16:0/3505360767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1c100e20 0x7f1d1c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 >> 172.21.15.16:0/3505360767 conn(0x7f1d1c0fc7d0 msgr2=0x7f1d1c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 shutdown_connections 2024-09-12T01:13:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:28.746+0000 7f1d22f52640 1 -- 172.21.15.16:0/3505360767 wait complete. 2024-09-12T01:13:29.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:28 smithi099 ceph-mon[35654]: pgmap v545: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-12T01:13:29.066 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:28 smithi016 ceph-mon[30972]: pgmap v545: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-12T01:13:29.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:28 smithi159 ceph-mon[35566]: pgmap v545: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-12T01:13:29.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T01:13:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:29.980 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:29 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3505360767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:30.018 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:29 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3505360767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:30.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:29 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3505360767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 -- 172.21.15.16:0/1119267103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820101410 msgr2=0x7f682010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 --2- 172.21.15.16:0/1119267103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820101410 0x7f682010d940 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f6818009cd0 tx=0x7f681802a8e0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 -- 172.21.15.16:0/1119267103 shutdown_connections 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 --2- 172.21.15.16:0/1119267103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820101410 0x7f682010d940 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 --2- 172.21.15.16:0/1119267103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820100a70 0x7f6820100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 --2- 172.21.15.16:0/1119267103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820106a70 0x7f6820106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 -- 172.21.15.16:0/1119267103 >> 172.21.15.16:0/1119267103 conn(0x7f68200fc790 msgr2=0x7f68200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 -- 172.21.15.16:0/1119267103 shutdown_connections 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.178+0000 7f68267e3640 1 -- 172.21.15.16:0/1119267103 wait complete. 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 Processor -- start 2024-09-12T01:13:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 -- start start 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820101410 0x7f6820198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 0x7f682019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6820110e70 con 0x7f6820100a70 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6820110cf0 con 0x7f6820101410 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f68267e3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6820110ff0 con 0x7f6820106a70 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f681ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f681ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60944/0 (socket says 172.21.15.16:60944) 2024-09-12T01:13:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.179+0000 7f681ffff640 1 -- 172.21.15.16:0/1016631220 learned_addr learned my addr 172.21.15.16:0/1016631220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681f7fe640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820101410 0x7f6820198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f6824d80640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 0x7f682019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 -- 172.21.15.16:0/1016631220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 msgr2=0x7f682019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 0x7f682019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 -- 172.21.15.16:0/1016631220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820101410 msgr2=0x7f6820198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820101410 0x7f6820198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 -- 172.21.15.16:0/1016631220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682019d3d0 con 0x7f6820100a70 2024-09-12T01:13:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f6824d80640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 0x7f682019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681ffff640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f681000d950 tx=0x7f681000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810017030 con 0x7f6820100a70 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6810004020 con 0x7f6820100a70 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68100047a0 con 0x7f6820100a70 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682019d6c0 con 0x7f6820100a70 2024-09-12T01:13:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.180+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a4fa0 con 0x7f6820100a70 2024-09-12T01:13:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.182+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6820100ed0 con 0x7f6820100a70 2024-09-12T01:13:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.183+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f68100041d0 con 0x7f6820100a70 2024-09-12T01:13:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.183+0000 7f681d7fa640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 0x7f68080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.184+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7f68100cee00 con 0x7f6820100a70 2024-09-12T01:13:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.184+0000 7f681f7fe640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 0x7f68080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.184+0000 7f681f7fe640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 0x7f68080a51b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6820105180 tx=0x7f681400b390 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:30.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.188+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f681009b9c0 con 0x7f6820100a70 2024-09-12T01:13:30.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.402+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6820199780 con 0x7f6820100a70 2024-09-12T01:13:30.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.403+0000 7f681d7fa640 1 -- 172.21.15.16:0/1016631220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11968 (secure 0 0 0) 0x7f681009b2c0 con 0x7f6820100a70 2024-09-12T01:13:30.403 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:13:30.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-12T00:56:02:686312+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2008522291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2008522291},{"type":"v1","addr":"172.21.15.16:6835","nonce":2008522291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2601027197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2601027197},{"type":"v1","addr":"172.21.15.16:6837","nonce":2601027197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T00:54:04.465326+0000","modified":"2024-09-12T00:56:02.636132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24505,"mds_1":24487,"mds_2":24482,"mds_3":24481,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14589":{"gid":14589,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6839/3482588678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3482588678},{"type":"v1","addr":"172.21.15.16:6839","nonce":3482588678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6841/2826352912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":2826352912},{"type":"v1","addr":"172.21.15.16:6841","nonce":2826352912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/4113169911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":4113169911},{"type":"v1","addr":"172.21.15.99:6833","nonce":4113169911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/3000703552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3000703552},{"type":"v1","addr":"172.21.15.99:6835","nonce":3000703552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/310857906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":310857906},{"type":"v1","addr":"172.21.15.159:6833","nonce":310857906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/1045510613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":1045510613},{"type":"v1","addr":"172.21.15.99:6837","nonce":1045510613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1983976622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1983976622},{"type":"v1","addr":"172.21.15.159:6835","nonce":1983976622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.99:6839/850851643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":850851643},{"type":"v1","addr":"172.21.15.99:6839","nonce":850851643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.159:6837/247659815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":247659815},{"type":"v1","addr":"172.21.15.159:6837","nonce":247659815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6839/1957754994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":1957754994},{"type":"v1","addr":"172.21.15.159:6839","nonce":1957754994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24505,"qdb_cluster":[24452,24481,24482,24487,24505]},"id":1}]} 2024-09-12T01:13:30.403 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-12T01:13:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 msgr2=0x7f68080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 0x7f68080a51b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6820105180 tx=0x7f681400b390 comp rx=0 tx=0).stop 2024-09-12T01:13:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 msgr2=0x7f6820198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f681000d950 tx=0x7f681000de10 comp rx=0 tx=0).stop 2024-09-12T01:13:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 shutdown_connections 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f68080a2cf0 0x7f68080a51b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6820106a70 0x7f682019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6820101410 0x7f6820198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 --2- 172.21.15.16:0/1016631220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6820100a70 0x7f6820198420 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 >> 172.21.15.16:0/1016631220 conn(0x7f68200fc790 msgr2=0x7f682006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 shutdown_connections 2024-09-12T01:13:30.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:30.405+0000 7f68267e3640 1 -- 172.21.15.16:0/1016631220 wait complete. 2024-09-12T01:13:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:30 smithi099 ceph-mon[35654]: pgmap v546: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-12T01:13:31.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:30 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/1016631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:30 smithi159 ceph-mon[35566]: pgmap v546: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-12T01:13:31.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:30 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/1016631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:30 smithi016 ceph-mon[30972]: pgmap v546: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-09-12T01:13:31.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:30 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/1016631220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T01:13:31.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T01:13:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2580465004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 msgr2=0x7fa844106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2580465004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa844106e40 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa840009930 tx=0x7fa84002a420 comp rx=0 tx=0).stop 2024-09-12T01:13:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2580465004 shutdown_connections 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2580465004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa84410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2580465004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 0x7fa844100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2580465004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa844106e40 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2580465004 >> 172.21.15.16:0/2580465004 conn(0x7fa8440fc780 msgr2=0x7fa8440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2580465004 shutdown_connections 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2580465004 wait complete. 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 Processor -- start 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.962+0000 7fa84c2e2640 1 -- start start 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 0x7fa844198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa8441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa844110e60 con 0x7fa844106a60 2024-09-12T01:13:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa844110ce0 con 0x7fa844101400 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa844110fe0 con 0x7fa844100a60 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60968/0 (socket says 172.21.15.16:60968) 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 -- 172.21.15.16:0/2486194002 learned_addr learned my addr 172.21.15.16:0/2486194002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a07e640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 0x7fa844198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84987d640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa8441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 -- 172.21.15.16:0/2486194002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 msgr2=0x7fa844198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 0x7fa844198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 -- 172.21.15.16:0/2486194002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 msgr2=0x7fa8441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa8441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 -- 172.21.15.16:0/2486194002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419d420 con 0x7fa844106a60 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84987d640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa8441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84a87f640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00eaa0 tx=0x7fa83c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00cdb0 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa83c005020 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419d710 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0116b0 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.963+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a4ff0 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.964+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844100ec0 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.966+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa83c005560 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.967+0000 7fa83b7fe640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 0x7fa8300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.967+0000 7fa84a07e640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 0x7fa8300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.967+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7fa83c0cf9b0 con 0x7fa844106a60 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.967+0000 7fa84a07e640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 0x7fa8300a5230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa8400058d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:31.970+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa83c016030 con 0x7fa844106a60 2024-09-12T01:13:32.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.146+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8441a5340 con 0x7fa844106a60 2024-09-12T01:13:32.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.147+0000 7fa83b7fe640 1 -- 172.21.15.16:0/2486194002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+23111 (secure 0 0 0) 0x7fa83c09c5f0 con 0x7fa844106a60 2024-09-12T01:13:32.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-12T01:13:32.148 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"434eab66-70a0-11ef-bcea-c7b262605968","created":"2024-09-12T00:46:20.927223+0000","modified":"2024-09-12T01:13:17.476315+0000","last_up_change":"2024-09-12T00:53:12.392309+0000","last_in_change":"2024-09-12T00:52:49.397546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T00:49:22.631453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T00:54:00.136499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T00:54:02.731797+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":28,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"717d4d96-381a-4246-826b-f3ca949a2f46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6803","nonce":4092481179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6805","nonce":4092481179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6809","nonce":4092481179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4092481179},{"type":"v1","addr":"172.21.15.16:6807","nonce":4092481179}]},"public_addr":"172.21.15.16:6803/4092481179","cluster_addr":"172.21.15.16:6805/4092481179","heartbeat_back_addr":"172.21.15.16:6809/4092481179","heartbeat_front_addr":"172.21.15.16:6807/4092481179","state":["exists","up"]},{"osd":1,"uuid":"5541ef54-29b2-4099-bc7b-3b758b6f0a6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6801","nonce":524186263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6803","nonce":524186263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6807","nonce":524186263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":524186263},{"type":"v1","addr":"172.21.15.99:6805","nonce":524186263}]},"public_addr":"172.21.15.99:6801/524186263","cluster_addr":"172.21.15.99:6803/524186263","heartbeat_back_addr":"172.21.15.99:6807/524186263","heartbeat_front_addr":"172.21.15.99:6805/524186263","state":["exists","up"]},{"osd":2,"uuid":"f46058bc-91f3-4062-ab02-30e93a02b1d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6801","nonce":3066904609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6803","nonce":3066904609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6807","nonce":3066904609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3066904609},{"type":"v1","addr":"172.21.15.159:6805","nonce":3066904609}]},"public_addr":"172.21.15.159:6801/3066904609","cluster_addr":"172.21.15.159:6803/3066904609","heartbeat_back_addr":"172.21.15.159:6807/3066904609","heartbeat_front_addr":"172.21.15.159:6805/3066904609","state":["exists","up"]},{"osd":3,"uuid":"ae799129-d588-4925-a79d-869205ffc0d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6811","nonce":846431474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6813","nonce":846431474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6817","nonce":846431474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":846431474},{"type":"v1","addr":"172.21.15.16:6815","nonce":846431474}]},"public_addr":"172.21.15.16:6811/846431474","cluster_addr":"172.21.15.16:6813/846431474","heartbeat_back_addr":"172.21.15.16:6817/846431474","heartbeat_front_addr":"172.21.15.16:6815/846431474","state":["exists","up"]},{"osd":4,"uuid":"70a24b6e-0414-4620-a0ec-89f48dedbef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6809","nonce":2525858476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6811","nonce":2525858476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6815","nonce":2525858476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2525858476},{"type":"v1","addr":"172.21.15.99:6813","nonce":2525858476}]},"public_addr":"172.21.15.99:6809/2525858476","cluster_addr":"172.21.15.99:6811/2525858476","heartbeat_back_addr":"172.21.15.99:6815/2525858476","heartbeat_front_addr":"172.21.15.99:6813/2525858476","state":["exists","up"]},{"osd":5,"uuid":"fff6ef9d-4e03-4343-8e20-fa384843bc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6809","nonce":516420178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6811","nonce":516420178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6815","nonce":516420178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":516420178},{"type":"v1","addr":"172.21.15.159:6813","nonce":516420178}]},"public_addr":"172.21.15.159:6809/516420178","cluster_addr":"172.21.15.159:6811/516420178","heartbeat_back_addr":"172.21.15.159:6815/516420178","heartbeat_front_addr":"172.21.15.159:6813/516420178","state":["exists","up"]},{"osd":6,"uuid":"9c365b94-3cbc-4aec-b6a6-d503fd51fd0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6819","nonce":4235557256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6821","nonce":4235557256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6825","nonce":4235557256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4235557256},{"type":"v1","addr":"172.21.15.16:6823","nonce":4235557256}]},"public_addr":"172.21.15.16:6819/4235557256","cluster_addr":"172.21.15.16:6821/4235557256","heartbeat_back_addr":"172.21.15.16:6825/4235557256","heartbeat_front_addr":"172.21.15.16:6823/4235557256","state":["exists","up"]},{"osd":7,"uuid":"6e1b90b2-de0e-4bd2-9cab-01595a0ec06e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6817","nonce":777307661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6819","nonce":777307661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6823","nonce":777307661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":777307661},{"type":"v1","addr":"172.21.15.99:6821","nonce":777307661}]},"public_addr":"172.21.15.99:6817/777307661","cluster_addr":"172.21.15.99:6819/777307661","heartbeat_back_addr":"172.21.15.99:6823/777307661","heartbeat_front_addr":"172.21.15.99:6821/777307661","state":["exists","up"]},{"osd":8,"uuid":"2c93a157-0688-48d0-a978-1cf9ef2cd1ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6817","nonce":274647051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6819","nonce":274647051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6823","nonce":274647051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":274647051},{"type":"v1","addr":"172.21.15.159:6821","nonce":274647051}]},"public_addr":"172.21.15.159:6817/274647051","cluster_addr":"172.21.15.159:6819/274647051","heartbeat_back_addr":"172.21.15.159:6823/274647051","heartbeat_front_addr":"172.21.15.159:6821/274647051","state":["exists","up"]},{"osd":9,"uuid":"fd2b63bf-d329-4318-87c5-945c8d0b12fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6827","nonce":45245397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6829","nonce":45245397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6833","nonce":45245397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":45245397},{"type":"v1","addr":"172.21.15.16:6831","nonce":45245397}]},"public_addr":"172.21.15.16:6827/45245397","cluster_addr":"172.21.15.16:6829/45245397","heartbeat_back_addr":"172.21.15.16:6833/45245397","heartbeat_front_addr":"172.21.15.16:6831/45245397","state":["exists","up"]},{"osd":10,"uuid":"df53f8db-e889-404c-a107-c3fde59b80b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6825","nonce":1469352212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6827","nonce":1469352212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6831","nonce":1469352212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":1469352212},{"type":"v1","addr":"172.21.15.99:6829","nonce":1469352212}]},"public_addr":"172.21.15.99:6825/1469352212","cluster_addr":"172.21.15.99:6827/1469352212","heartbeat_back_addr":"172.21.15.99:6831/1469352212","heartbeat_front_addr":"172.21.15.99:6829/1469352212","state":["exists","up"]},{"osd":11,"uuid":"a6545934-6ceb-4f96-aecf-d9ded7ceefde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6825","nonce":4132934366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6827","nonce":4132934366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6831","nonce":4132934366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":4132934366},{"type":"v1","addr":"172.21.15.159:6829","nonce":4132934366}]},"public_addr":"172.21.15.159:6825/4132934366","cluster_addr":"172.21.15.159:6827/4132934366","heartbeat_back_addr":"172.21.15.159:6831/4132934366","heartbeat_front_addr":"172.21.15.159:6829/4132934366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:30.719777+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:48:55.270960+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:19.900282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:49:46.236520+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:13.026428+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:50:37.614922+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:03.400534+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:28.833792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:51:54.065649+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:20.736833+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:52:45.973240+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T00:53:10.888160+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/1272021982":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2952840401":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6801/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1067528076":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/1286739529":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/3359074253":"2024-09-13T00:46:43.483931+0000","172.21.15.16:6800/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/2290747564":"2024-09-13T00:56:03.339943+0000","172.21.15.16:0/2269637621":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/90830535":"2024-09-13T00:56:03.339943+0000","172.21.15.16:6800/7504959":"2024-09-13T00:46:43.483931+0000","172.21.15.16:0/2087944877":"2024-09-13T00:47:08.126530+0000","172.21.15.16:6801/1538480573":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1789090922":"2024-09-13T00:47:08.126530+0000","172.21.15.16:0/1572487203":"2024-09-13T00:56:03.339943+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":15,"length":1},{"begin":28,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T01:13:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.149+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 msgr2=0x7fa8300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:32.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.149+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 0x7fa8300a5230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa8400058d0 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.149+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 msgr2=0x7fa84419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.149+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00eaa0 tx=0x7fa83c00ef60 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 shutdown_connections 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fa8300a2d70 0x7fa8300a5230 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844106a60 0x7fa84419cd40 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa844101400 0x7fa8441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 --2- 172.21.15.16:0/2486194002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa844100a60 0x7fa844198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 >> 172.21.15.16:0/2486194002 conn(0x7fa8440fc780 msgr2=0x7fa84406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 shutdown_connections 2024-09-12T01:13:32.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:32.150+0000 7fa84c2e2640 1 -- 172.21.15.16:0/2486194002 wait complete. 2024-09-12T01:13:32.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:32 smithi016 ceph-mon[30972]: pgmap v547: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:32.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:32 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/2486194002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:32.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T01:13:33.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:32 smithi099 ceph-mon[35654]: pgmap v547: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:33.025 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:32 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/2486194002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:32 smithi159 ceph-mon[35566]: pgmap v547: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:33.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:32 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/2486194002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T01:13:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 -- 172.21.15.16:0/3174682131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598103660 msgr2=0x7fe59810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 --2- 172.21.15.16:0/3174682131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598103660 0x7fe59810bb50 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009900 tx=0x7fe58c02a480 comp rx=0 tx=0).stop 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 -- 172.21.15.16:0/3174682131 shutdown_connections 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 --2- 172.21.15.16:0/3174682131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe59810c170 0x7fe59810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 --2- 172.21.15.16:0/3174682131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598103660 0x7fe59810bb50 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 --2- 172.21.15.16:0/3174682131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.696+0000 7fe59fca1640 1 -- 172.21.15.16:0/3174682131 >> 172.21.15.16:0/3174682131 conn(0x7fe5980fc7a0 msgr2=0x7fe5980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.697+0000 7fe59fca1640 1 -- 172.21.15.16:0/3174682131 shutdown_connections 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.697+0000 7fe59fca1640 1 -- 172.21.15.16:0/3174682131 wait complete. 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.697+0000 7fe59fca1640 1 Processor -- start 2024-09-12T01:13:33.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.697+0000 7fe59fca1640 1 -- start start 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111f70 con 0x7fe59810c170 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111df0 con 0x7fe598102d40 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59fca1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5981120f0 con 0x7fe598103660 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59da3d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60980/0 (socket says 172.21.15.16:60980) 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59da3d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.16:43304/0 (socket says 172.21.15.16:43304) 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 -- 172.21.15.16:0/4187290408 learned_addr learned my addr 172.21.15.16:0/4187290408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59d23c640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:33.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 -- 172.21.15.16:0/4187290408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe598103660 msgr2=0x7fe5981989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 -- 172.21.15.16:0/4187290408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 msgr2=0x7fe598198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 -- 172.21.15.16:0/4187290408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59819d420 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59da3d640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.698+0000 7fe59e23e640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe59000cad0 tx=0x7fe59000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.699+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59000ae20 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.699+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe590005020 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.699+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59819d710 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.699+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59000f6f0 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.699+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a4ff0 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.700+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe598103120 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.701+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5900051d0 con 0x7fe59810c170 2024-09-12T01:13:33.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.701+0000 7fe58affd640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 0x7fe5840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.702+0000 7fe59da3d640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 0x7fe5840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.702+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8521+0+0 (secure 0 0 0) 0x7fe5900cf800 con 0x7fe59810c170 2024-09-12T01:13:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.702+0000 7fe59da3d640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 0x7fe5840a51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fc0 tx=0x7fe594005f90 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:33.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.705+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe590014030 con 0x7fe59810c170 2024-09-12T01:13:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.917+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe5981997d0 con 0x7fe59810c170 2024-09-12T01:13:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.919+0000 7fe58affd640 1 -- 172.21.15.16:0/4187290408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fe5981997d0 con 0x7fe59810c170 2024-09-12T01:13:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 msgr2=0x7fe5840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 0x7fe5840a51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe594009fc0 tx=0x7fe594005f90 comp rx=0 tx=0).stop 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 msgr2=0x7fe59819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fe59000cad0 tx=0x7fe59000cf90 comp rx=0 tx=0).stop 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 shutdown_connections 2024-09-12T01:13:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe5840a2cf0 0x7fe5840a51b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59810c170 0x7fe59819cd40 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe598103660 0x7fe5981989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 --2- 172.21.15.16:0/4187290408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe598102d40 0x7fe598198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 >> 172.21.15.16:0/4187290408 conn(0x7fe5980fc7a0 msgr2=0x7fe5980fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 shutdown_connections 2024-09-12T01:13:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:33.922+0000 7fe59fca1640 1 -- 172.21.15.16:0/4187290408 wait complete. 2024-09-12T01:13:34.274 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.274 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.274 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-mds[66016]: -- 172.21.15.99:0/3423841216 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c77990d860 con 0x55c7798eb400 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: starting mds.b at 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-mds[65409]: -- 172.21.15.99:0/482824671 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6e5d97c20 con 0x55a6e5d74800 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: starting mds.e at 2024-09-12T01:13:34.275 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-mds[65707]: -- 172.21.15.99:0/4145761472 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556dc98b3860 con 0x556dc9891400 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: starting mds.h at 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-mds[65106]: -- 172.21.15.99:0/408224337 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593997c3c20 con 0x5593997a0800 2024-09-12T01:13:34.276 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:13:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: starting mds.k at 2024-09-12T01:13:34.346 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a[30968]: 2024-09-12T01:13:33.918+0000 7f71b14c3640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:13:34.346 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-mds[71503]: -- 172.21.15.16:0/20460012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649d511d860 con 0x5649d50fb400 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: starting mds.a at 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-mds[71131]: -- 172.21.15.16:0/203075874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559993aabc20 con 0x559993a89000 2024-09-12T01:13:34.347 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:13:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: starting mds.g at 2024-09-12T01:13:34.366 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.366 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-mds[65726]: -- 172.21.15.159:0/1895466732 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562190cd860 con 0x5562190ab400 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: starting mds.c at 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: starting mds.f at 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.367 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-mds[65124]: -- 172.21.15.159:0/3014291910 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f45b5bc680 con 0x55f45b591400 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-mds[65124]: -- 172.21.15.159:0/3014291910 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f45b5b3c20 con 0x55f45b591400 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: starting mds.l at 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: ignoring --setuser ceph since I am not root 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: ignoring --setgroup ceph since I am not root 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-mds[66028]: -- 172.21.15.159:0/2593387666 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0c770cd00 con 0x55b0c76e0800 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-mds[66028]: -- 172.21.15.159:0/2593387666 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0c7703860 con 0x55b0c76e0800 2024-09-12T01:13:34.368 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:13:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: starting mds.i at 2024-09-12T01:13:34.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T01:13:34.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: pgmap v548: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T01:13:35.024 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:34 smithi099 ceph-mon[35654]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:13:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: pgmap v548: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:35.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:13:35.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:13:35.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:13:35.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:13:35.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T01:13:35.115 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:34 smithi016 ceph-mon[30972]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: pgmap v548: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4187290408' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T01:13:35.116 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:34 smithi159 ceph-mon[35566]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.479+0000 7efc3ea06640 1 -- 172.21.15.16:0/3955782122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38106f10 msgr2=0x7efc3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.479+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3955782122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38106f10 0x7efc3810ea90 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7efc30009ca0 tx=0x7efc3002a8c0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 -- 172.21.15.16:0/3955782122 shutdown_connections 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3955782122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38106f10 0x7efc3810ea90 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3955782122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38102790 0x7efc381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3955782122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 -- 172.21.15.16:0/3955782122 >> 172.21.15.16:0/3955782122 conn(0x7efc380fd8c0 msgr2=0x7efc380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 -- 172.21.15.16:0/3955782122 shutdown_connections 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 -- 172.21.15.16:0/3955782122 wait complete. 2024-09-12T01:13:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 Processor -- start 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.480+0000 7efc3ea06640 1 -- start start 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38106f10 0x7efc3819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc38111f70 con 0x7efc38102790 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc38111df0 con 0x7efc38106f10 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc381120f0 con 0x7efc38101de0 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32792/0 (socket says 172.21.15.16:32792) 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 -- 172.21.15.16:0/3731203632 learned_addr learned my addr 172.21.15.16:0/3731203632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3cfa3640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38106f10 0x7efc3819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:35.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc37fff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 -- 172.21.15.16:0/3731203632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 msgr2=0x7efc381983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 -- 172.21.15.16:0/3731203632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38106f10 msgr2=0x7efc3819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38106f10 0x7efc3819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 -- 172.21.15.16:0/3731203632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3819d380 con 0x7efc38102790 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc37fff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc2ffff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7efc2000d950 tx=0x7efc2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20017070 con 0x7efc38102790 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc20009c30 con 0x7efc38102790 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2000f6b0 con 0x7efc38102790 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.481+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3819d670 con 0x7efc38102790 2024-09-12T01:13:35.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.482+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc381a4f50 con 0x7efc38102790 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.483+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc381021c0 con 0x7efc38102790 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.484+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc200109d0 con 0x7efc38102790 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.484+0000 7efc35ffb640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 0x7efc240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.484+0000 7efc37fff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 0x7efc240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.484+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8907+0+0 (secure 0 0 0) 0x7efc200cf160 con 0x7efc38102790 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.485+0000 7efc37fff640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 0x7efc240a51b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc280058c0 tx=0x7efc28005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:35.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.489+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc20009de0 con 0x7efc38102790 2024-09-12T01:13:35.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.701+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efc3802d7c0 con 0x7efc38102790 2024-09-12T01:13:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.927+0000 7efc35ffb640 1 -- 172.21.15.16:0/3731203632 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7efc2009bc00 con 0x7efc38102790 2024-09-12T01:13:35.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 msgr2=0x7efc240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 0x7efc240a51b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efc280058c0 tx=0x7efc28005850 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 msgr2=0x7efc38198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7efc2000d950 tx=0x7efc2000de10 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 shutdown_connections 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7efc240a2cf0 0x7efc240a51b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc38106f10 0x7efc3819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc38102790 0x7efc38198910 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 --2- 172.21.15.16:0/3731203632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc38101de0 0x7efc381983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 >> 172.21.15.16:0/3731203632 conn(0x7efc380fd8c0 msgr2=0x7efc380feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 shutdown_connections 2024-09-12T01:13:35.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:35.930+0000 7efc3ea06640 1 -- 172.21.15.16:0/3731203632 wait complete. 2024-09-12T01:13:36.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6838/2488775388,v1:172.21.15.16:6839/2488775388] up:boot 2024-09-12T01:13:36.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6834/1719252915,v1:172.21.15.159:6838/1719252915] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6832/3776169882,v1:172.21.15.159:6837/3776169882] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6833/3206566327,v1:172.21.15.159:6836/3206566327] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.159:6835/3169708910,v1:172.21.15.159:6839/3169708910] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.16:6840/866209171,v1:172.21.15.16:6841/866209171] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6832/3139548850,v1:172.21.15.99:6834/3139548850] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6833/1084646678,v1:172.21.15.99:6835/1084646678] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6836/1204811313,v1:172.21.15.99:6837/1204811313] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: mds.? [v2:172.21.15.99:6838/3885148081,v1:172.21.15.99:6839/3885148081] up:boot 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:13:36.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:13:36.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:35 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:13:36.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6838/2488775388,v1:172.21.15.16:6839/2488775388] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6834/1719252915,v1:172.21.15.159:6838/1719252915] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6832/3776169882,v1:172.21.15.159:6837/3776169882] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6833/3206566327,v1:172.21.15.159:6836/3206566327] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.159:6835/3169708910,v1:172.21.15.159:6839/3169708910] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.16:6840/866209171,v1:172.21.15.16:6841/866209171] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6832/3139548850,v1:172.21.15.99:6834/3139548850] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6833/1084646678,v1:172.21.15.99:6835/1084646678] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6836/1204811313,v1:172.21.15.99:6837/1204811313] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: mds.? [v2:172.21.15.99:6838/3885148081,v1:172.21.15.99:6839/3885148081] up:boot 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:13:36.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:13:36.276 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:35 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:13:36.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6838/2488775388,v1:172.21.15.16:6839/2488775388] up:boot 2024-09-12T01:13:36.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6834/1719252915,v1:172.21.15.159:6838/1719252915] up:boot 2024-09-12T01:13:36.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6832/3776169882,v1:172.21.15.159:6837/3776169882] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6833/3206566327,v1:172.21.15.159:6836/3206566327] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.159:6835/3169708910,v1:172.21.15.159:6839/3169708910] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.16:6840/866209171,v1:172.21.15.16:6841/866209171] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6832/3139548850,v1:172.21.15.99:6834/3139548850] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6833/1084646678,v1:172.21.15.99:6835/1084646678] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6836/1204811313,v1:172.21.15.99:6837/1204811313] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: mds.? [v2:172.21.15.99:6838/3885148081,v1:172.21.15.99:6839/3885148081] up:boot 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T01:13:36.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='mgr.15213 172.21.15.16:0/330449303' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T01:13:36.368 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:35 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T01:13:36.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T01:13:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: pgmap v550: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: Cluster is now healthy 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:13:37.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:36 smithi016 ceph-mon[30972]: fsmap 12 up:standby 2024-09-12T01:13:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: pgmap v550: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:13:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:13:37.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: Cluster is now healthy 2024-09-12T01:13:37.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:13:37.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:36 smithi099 ceph-mon[35654]: fsmap 12 up:standby 2024-09-12T01:13:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: pgmap v550: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T01:13:37.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T01:13:37.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: Cluster is now healthy 2024-09-12T01:13:37.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/3731203632' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T01:13:37.367 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:36 smithi159 ceph-mon[35566]: fsmap 12 up:standby 2024-09-12T01:13:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.483+0000 7fe2c05a5640 1 -- 172.21.15.16:0/3371508074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81036e0 msgr2=0x7fe2b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.483+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/3371508074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81036e0 0x7fe2b8107b90 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009930 tx=0x7fe2a802a420 comp rx=0 tx=0).stop 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 -- 172.21.15.16:0/3371508074 shutdown_connections 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/3371508074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b81082a0 0x7fe2b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/3371508074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81036e0 0x7fe2b8107b90 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/3371508074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b8102d30 0x7fe2b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 -- 172.21.15.16:0/3371508074 >> 172.21.15.16:0/3371508074 conn(0x7fe2b80fc7d0 msgr2=0x7fe2b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 -- 172.21.15.16:0/3371508074 shutdown_connections 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 -- 172.21.15.16:0/3371508074 wait complete. 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 Processor -- start 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 -- start start 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.484+0000 7fe2c05a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b8102d30 0x7fe2b8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b81036e0 0x7fe2b8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b8112090 con 0x7fe2b81082a0 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b8111f10 con 0x7fe2b8102d30 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b8112210 con 0x7fe2b81036e0 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32816/0 (socket says 172.21.15.16:32816) 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 -- 172.21.15.16:0/4079148813 learned_addr learned my addr 172.21.15.16:0/4079148813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2be341640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b8102d30 0x7fe2b8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2bdb40640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b81036e0 0x7fe2b8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 -- 172.21.15.16:0/4079148813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b81036e0 msgr2=0x7fe2b8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b81036e0 0x7fe2b8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 -- 172.21.15.16:0/4079148813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b8102d30 msgr2=0x7fe2b8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b8102d30 0x7fe2b8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 -- 172.21.15.16:0/4079148813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b819d5c0 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2beb42640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000cad0 tx=0x7fe2b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000ae20 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b819d8b0 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0005020 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.485+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000f6f0 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.486+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b81a5190 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.487+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b8103110 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.488+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe2b00051d0 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.489+0000 7fe2af7fe640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 0x7fe2a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.490+0000 7fe2be341640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 0x7fe2a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.490+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8907+0+0 (secure 0 0 0) 0x7fe2b00cf670 con 0x7fe2b81082a0 2024-09-12T01:13:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.490+0000 7fe2be341640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 0x7fe2a40a51b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40058c0 tx=0x7fe2b400a2d0 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.494+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2b0014030 con 0x7fe2b81082a0 2024-09-12T01:13:37.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.669+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2b810a2b0 con 0x7fe2b81082a0 2024-09-12T01:13:37.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.933+0000 7fe2af7fe640 1 -- 172.21.15.16:0/4079148813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) ==== 181+0+0 (secure 0 0 0) 0x7fe2b009c0b0 con 0x7fe2b81082a0 2024-09-12T01:13:37.933 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-12T01:13:37.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 msgr2=0x7fe2a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 0x7fe2a40a51b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2b40058c0 tx=0x7fe2b400a2d0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 msgr2=0x7fe2b819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000cad0 tx=0x7fe2b000cf90 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 shutdown_connections 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7fe2a40a2cf0 0x7fe2a40a51b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2b81082a0 0x7fe2b819cee0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2b81036e0 0x7fe2b8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 --2- 172.21.15.16:0/4079148813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2b8102d30 0x7fe2b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.937+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 >> 172.21.15.16:0/4079148813 conn(0x7fe2b80fc7d0 msgr2=0x7fe2b8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:37.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.938+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 shutdown_connections 2024-09-12T01:13:37.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:37.938+0000 7fe2c05a5640 1 -- 172.21.15.16:0/4079148813 wait complete. 2024-09-12T01:13:38.275 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:37 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:38.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:37 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:38.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:37 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:38.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 434eab66-70a0-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T01:13:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T01:13:39.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:38 smithi016 ceph-mon[30972]: pgmap v551: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:39.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:38 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:39.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:38 smithi016 ceph-mon[30972]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T01:13:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:38 smithi099 ceph-mon[35654]: pgmap v551: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:38 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:39.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:38 smithi099 ceph-mon[35654]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T01:13:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:38 smithi159 ceph-mon[35566]: pgmap v551: 97 pgs: 97 active+clean; 30 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 716 B/s wr, 11 op/s 2024-09-12T01:13:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:38 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/4079148813' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:39.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:38 smithi159 ceph-mon[35566]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T01:13:39.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 -- 172.21.15.16:0/3442760476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b8410c170 msgr2=0x7f7b8410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:39.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 --2- 172.21.15.16:0/3442760476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b8410c170 0x7f7b8410e610 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ca0 tx=0x7f7b7c02a950 comp rx=0 tx=0).stop 2024-09-12T01:13:39.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 -- 172.21.15.16:0/3442760476 shutdown_connections 2024-09-12T01:13:39.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 --2- 172.21.15.16:0/3442760476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b8410c170 0x7f7b8410e610 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 --2- 172.21.15.16:0/3442760476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b8410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 --2- 172.21.15.16:0/3442760476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84102d40 0x7f7b84103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 -- 172.21.15.16:0/3442760476 >> 172.21.15.16:0/3442760476 conn(0x7f7b840fc7a0 msgr2=0x7f7b840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 -- 172.21.15.16:0/3442760476 shutdown_connections 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.668+0000 7f7b8b538640 1 -- 172.21.15.16:0/3442760476 wait complete. 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 Processor -- start 2024-09-12T01:13:39.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 -- start start 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b84100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b8410c170 0x7f7b84101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111f30 con 0x7f7b84102d40 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111db0 con 0x7f7b8410c170 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b8b538640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841120b0 con 0x7f7b84103660 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51660/0 (socket says 172.21.15.16:51660) 2024-09-12T01:13:39.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 -- 172.21.15.16:0/851793838 learned_addr learned my addr 172.21.15.16:0/851793838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b89ad5640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b8410c170 0x7f7b84101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b88ad3640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b84100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 -- 172.21.15.16:0/851793838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 msgr2=0x7f7b84100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b84100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 -- 172.21.15.16:0/851793838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b8410c170 msgr2=0x7f7b84101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b8410c170 0x7f7b84101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b892d4640 1 -- 172.21.15.16:0/851793838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b841a6900 con 0x7f7b84102d40 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.669+0000 7f7b88ad3640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b892d4640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c890 tx=0x7f7b8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004220 con 0x7f7b84102d40 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b841a6bf0 con 0x7f7b84102d40 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b800043c0 con 0x7f7b84102d40 2024-09-12T01:13:39.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800049e0 con 0x7f7b84102d40 2024-09-12T01:13:39.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.670+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841a7130 con 0x7f7b84102d40 2024-09-12T01:13:39.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.671+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b84103120 con 0x7f7b84102d40 2024-09-12T01:13:39.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.672+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7b800079b0 con 0x7f7b84102d40 2024-09-12T01:13:39.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.673+0000 7f7b7a7fc640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 0x7f7b700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T01:13:39.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.673+0000 7f7b88ad3640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 0x7f7b700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T01:13:39.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.673+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8374+0+0 (secure 0 0 0) 0x7f7b800cf480 con 0x7f7b84102d40 2024-09-12T01:13:39.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.673+0000 7f7b88ad3640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 0x7f7b700a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005fd0 tx=0x7f7b74005850 comp rx=0 tx=0).ready entity=mgr.15213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T01:13:39.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.677+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b80014030 con 0x7f7b84102d40 2024-09-12T01:13:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.856+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b84101ed0 con 0x7f7b84102d40 2024-09-12T01:13:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.940+0000 7f7b7a7fc640 1 -- 172.21.15.16:0/851793838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7f7b8009c0d0 con 0x7f7b84102d40 2024-09-12T01:13:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-12T01:13:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.943+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 msgr2=0x7f7b700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.943+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 0x7f7b700a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005fd0 tx=0x7f7b74005850 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.943+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 msgr2=0x7f7b841009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.943+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000c890 tx=0x7f7b8000cd50 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 shutdown_connections 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:6800/1052477540,v1:172.21.15.16:6801/1052477540] conn(0x7f7b700a2cf0 0x7f7b700a51b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b8410c170 0x7f7b84101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b84103660 0x7f7b84100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 --2- 172.21.15.16:0/851793838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b84102d40 0x7f7b841009a0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 >> 172.21.15.16:0/851793838 conn(0x7f7b840fc7a0 msgr2=0x7f7b840fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T01:13:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 shutdown_connections 2024-09-12T01:13:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-12T01:13:39.944+0000 7f7b8b538640 1 -- 172.21.15.16:0/851793838 wait complete. 2024-09-12T01:13:40.143 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:39 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:40.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:39 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:40.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:39 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T01:13:40.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:13:40.808 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:13:40.842 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:13:40.875 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T01:13:40.875 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:13:40.893 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:13:40.911 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T01:13:40.933 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T01:13:40.934 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T01:13:40.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a 2024-09-12T01:13:41.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:40 smithi016 ceph-mon[30972]: pgmap v553: 65 pgs: 65 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:13:41.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:40 smithi016 ceph-mon[30972]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:41.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:40 smithi016 ceph-mon[30972]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T01:13:41.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:40 smithi016 systemd[1]: Stopping Ceph mon.a for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:40 smithi099 ceph-mon[35654]: pgmap v553: 65 pgs: 65 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:13:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:40 smithi099 ceph-mon[35654]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:41.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:40 smithi099 ceph-mon[35654]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T01:13:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:40 smithi159 ceph-mon[35566]: pgmap v553: 65 pgs: 65 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T01:13:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:40 smithi159 ceph-mon[35566]: from='client.? 172.21.15.16:0/851793838' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T01:13:41.366 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:40 smithi159 ceph-mon[35566]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T01:13:41.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a[30968]: 2024-09-12T01:13:41.250+0000 7f71b6cce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:13:41.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a[30968]: 2024-09-12T01:13:41.250+0000 7f71b6cce640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T01:13:41.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:41 smithi016 podman[133976]: 2024-09-12 01:13:41.342023519 +0000 UTC m=+0.207755428 container died 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T01:13:41.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:41 smithi016 podman[133976]: 2024-09-12 01:13:41.73352637 +0000 UTC m=+0.599258279 container cleanup 77905c72892be146993b2d2f9077ac25a54943f9c3f9df3ff194d02d9e444429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:13:41.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 12 01:13:41 smithi016 bash[133976]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-a 2024-09-12T01:13:42.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T01:13:42.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:42.160 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T01:13:42.160 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T01:13:42.160 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.b 2024-09-12T01:13:42.487 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 systemd[1]: Stopping Ceph mon.b for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:42.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-b[35650]: 2024-09-12T01:13:42.487+0000 7f24e50b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:13:42.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-b[35650]: 2024-09-12T01:13:42.487+0000 7f24e50b5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T01:13:42.774 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 podman[72210]: 2024-09-12 01:13:42.580663188 +0000 UTC m=+0.222918756 container died 1e0b261d1f25f12f7956cda70556ebadf6a3acf2488b5981f0d5b41af64d8ab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-12T01:13:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 podman[72210]: 2024-09-12 01:13:42.930725617 +0000 UTC m=+0.572981184 container cleanup 1e0b261d1f25f12f7956cda70556ebadf6a3acf2488b5981f0d5b41af64d8ab3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T01:13:43.274 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 12 01:13:42 smithi099 bash[72210]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-b 2024-09-12T01:13:43.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T01:13:43.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:43.557 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T01:13:43.557 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T01:13:43.557 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.c 2024-09-12T01:13:43.866 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:43 smithi159 systemd[1]: Stopping Ceph mon.c for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:44.315 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c[35562]: 2024-09-12T01:13:43.905+0000 7fe2cda37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:13:44.315 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c[35562]: 2024-09-12T01:13:43.905+0000 7fe2cda37640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T01:13:44.315 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:43 smithi159 podman[72994]: 2024-09-12 01:13:43.947550797 +0000 UTC m=+0.185610792 container died 125230a9ff3b1ff4d4a7bc709632a1b27d138216384da6ac3cb9f2ff2275f9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T01:13:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 podman[72994]: 2024-09-12 01:13:44.473989549 +0000 UTC m=+0.712049544 container cleanup 125230a9ff3b1ff4d4a7bc709632a1b27d138216384da6ac3cb9f2ff2275f9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T01:13:44.616 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 bash[72994]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c 2024-09-12T01:13:44.904 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 podman[73006]: 2024-09-12 01:13:44.659278236 +0000 UTC m=+0.713706877 container remove 125230a9ff3b1ff4d4a7bc709632a1b27d138216384da6ac3cb9f2ff2275f9a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 systemd[1]: ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-12T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 systemd[1]: Stopped Ceph mon.c for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T01:13:44.928 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 12 01:13:44 smithi159 systemd[1]: ceph-434eab66-70a0-11ef-bcea-c7b262605968@mon.c.service: Consumed 16.260s CPU time. 2024-09-12T01:13:44.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:44.953 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T01:13:44.954 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T01:13:44.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x 2024-09-12T01:13:45.317 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:45 smithi016 systemd[1]: Stopping Ceph mgr.x for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:45.621 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:45 smithi016 podman[134098]: 2024-09-12 01:13:45.318555106 +0000 UTC m=+0.152994734 container died ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T01:13:45.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:45 smithi016 podman[134098]: 2024-09-12 01:13:45.846625854 +0000 UTC m=+0.681065468 container cleanup ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T01:13:45.893 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:45 smithi016 bash[134098]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x 2024-09-12T01:13:46.330 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:46 smithi016 podman[134110]: 2024-09-12 01:13:46.081639172 +0000 UTC m=+0.755523451 container remove ba43bc913f4493b03cdd0da936473a2f2ef7c3bd590611be853a8057ad1d2574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T01:13:46.330 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:46 smithi016 systemd[1]: ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T01:13:46.330 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 12 01:13:46 smithi016 systemd[1]: Stopped Ceph mgr.x for 434eab66-70a0-11ef-bcea-c7b262605968. 2024-09-12T01:13:46.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T01:13:46.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:46.369 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T01:13:46.369 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T01:13:46.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.y 2024-09-12T01:13:46.705 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 01:13:46 smithi099 systemd[1]: Stopping Ceph mgr.y for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:47.002 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 01:13:46 smithi099 podman[72332]: 2024-09-12 01:13:46.706492802 +0000 UTC m=+0.136149004 container died a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T01:13:47.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 01:13:47 smithi099 podman[72332]: 2024-09-12 01:13:47.111514841 +0000 UTC m=+0.541171041 container cleanup a4498af333679d6d19eddac5030083e09431a81c8e0ba27ab8942eeabbcec6a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:13:47.274 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 12 01:13:47 smithi099 bash[72332]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-y 2024-09-12T01:13:47.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T01:13:47.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:47.567 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T01:13:47.567 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T01:13:47.567 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.z 2024-09-12T01:13:47.866 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 01:13:47 smithi159 systemd[1]: Stopping Ceph mgr.z for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:48.212 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 01:13:47 smithi159 podman[73115]: 2024-09-12 01:13:47.908610371 +0000 UTC m=+0.143541032 container died 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:13:48.580 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 01:13:48 smithi159 podman[73115]: 2024-09-12 01:13:48.313121217 +0000 UTC m=+0.548051879 container cleanup 2e923c6eaf7bd1cb9cd6d26dd7217ce30ba1a9a5b53ad9e4e0bf4b48c6f1a9e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T01:13:48.580 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 12 01:13:48 smithi159 bash[73115]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mgr-z 2024-09-12T01:13:48.740 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T01:13:48.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:48.775 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T01:13:48.775 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T01:13:48.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.0 2024-09-12T01:13:49.100 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:48 smithi016 systemd[1]: Stopping Ceph osd.0 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:49.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:49 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T01:13:49.101+0000 7f6a06662640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:13:49.393 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:49 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T01:13:49.101+0000 7f6a06662640 -1 osd.0 108 *** Got signal Terminated *** 2024-09-12T01:13:49.394 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:49 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0[45128]: 2024-09-12T01:13:49.101+0000 7f6a06662640 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:13:54.893 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:54 smithi016 podman[134218]: 2024-09-12 01:13:54.625979423 +0000 UTC m=+5.638873006 container died 66d25cbddf6446e0893392c8fc99eda911ca130f5d561938be1d164ae1d0782f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:13:55.321 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:55 smithi016 podman[134218]: 2024-09-12 01:13:55.143276408 +0000 UTC m=+6.156169991 container cleanup 66d25cbddf6446e0893392c8fc99eda911ca130f5d561938be1d164ae1d0782f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:13:55.321 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:55 smithi016 bash[134218]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0 2024-09-12T01:13:55.643 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:55 smithi016 podman[134234]: 2024-09-12 01:13:55.395163449 +0000 UTC m=+0.770833646 container remove 66d25cbddf6446e0893392c8fc99eda911ca130f5d561938be1d164ae1d0782f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:13:56.080 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:55 smithi016 podman[134331]: 2024-09-12 01:13:55.721427422 +0000 UTC m=+0.021571652 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:13:56.080 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:55 smithi016 podman[134331]: 2024-09-12 01:13:55.829319313 +0000 UTC m=+0.129463527 container create 382b7562cbfe66f3a9d76d29c5fd80e1de985d96ad92751278a6a978fa73e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:13:56.080 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:56 smithi016 podman[134331]: 2024-09-12 01:13:56.081801038 +0000 UTC m=+0.381945262 container init 382b7562cbfe66f3a9d76d29c5fd80e1de985d96ad92751278a6a978fa73e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:13:56.394 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:56 smithi016 podman[134331]: 2024-09-12 01:13:56.085664419 +0000 UTC m=+0.385808643 container start 382b7562cbfe66f3a9d76d29c5fd80e1de985d96ad92751278a6a978fa73e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T01:13:56.394 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:56 smithi016 podman[134331]: 2024-09-12 01:13:56.20711414 +0000 UTC m=+0.507258368 container attach 382b7562cbfe66f3a9d76d29c5fd80e1de985d96ad92751278a6a978fa73e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:13:56.394 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 12 01:13:56 smithi016 podman[134331]: 2024-09-12 01:13:56.282940262 +0000 UTC m=+0.583084477 container died 382b7562cbfe66f3a9d76d29c5fd80e1de985d96ad92751278a6a978fa73e31b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T01:13:56.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T01:13:57.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:13:57.037 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T01:13:57.037 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T01:13:57.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.1 2024-09-12T01:13:57.364 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:13:57 smithi099 systemd[1]: Stopping Ceph osd.1 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:13:57.774 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:13:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T01:13:57.364+0000 7fc0bdb98640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:13:57.774 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:13:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T01:13:57.364+0000 7fc0bdb98640 -1 osd.1 108 *** Got signal Terminated *** 2024-09-12T01:13:57.774 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:13:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1[40141]: 2024-09-12T01:13:57.364+0000 7fc0bdb98640 -1 osd.1 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:03.130 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:02 smithi099 podman[72456]: 2024-09-12 01:14:02.817735072 +0000 UTC m=+5.592115312 container died 8a09476fc8e2d67b68b83a31fd9597da9f7b8623df5d5bdfdff86fb1744d6b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:03.490 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:03 smithi099 podman[72456]: 2024-09-12 01:14:03.256661287 +0000 UTC m=+6.031041524 container cleanup 8a09476fc8e2d67b68b83a31fd9597da9f7b8623df5d5bdfdff86fb1744d6b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T01:14:03.490 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:03 smithi099 bash[72456]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1 2024-09-12T01:14:03.774 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:03 smithi099 podman[72472]: 2024-09-12 01:14:03.491984572 +0000 UTC m=+0.677305277 container remove 8a09476fc8e2d67b68b83a31fd9597da9f7b8623df5d5bdfdff86fb1744d6b40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T01:14:04.210 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:03 smithi099 podman[72562]: 2024-09-12 01:14:03.825709376 +0000 UTC m=+0.016223134 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:04.211 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:03 smithi099 podman[72562]: 2024-09-12 01:14:03.942967144 +0000 UTC m=+0.133480947 container create aa9fa34319f0a4f3fb6fbe98a429312b97fdf9beb8ff8cc2c2f7837843927b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T01:14:04.525 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:04 smithi099 podman[72562]: 2024-09-12 01:14:04.212203395 +0000 UTC m=+0.402717141 container init aa9fa34319f0a4f3fb6fbe98a429312b97fdf9beb8ff8cc2c2f7837843927b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T01:14:04.525 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:04 smithi099 podman[72562]: 2024-09-12 01:14:04.215532749 +0000 UTC m=+0.406046500 container start aa9fa34319f0a4f3fb6fbe98a429312b97fdf9beb8ff8cc2c2f7837843927b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:04.525 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:04 smithi099 podman[72562]: 2024-09-12 01:14:04.320961916 +0000 UTC m=+0.511475681 container attach aa9fa34319f0a4f3fb6fbe98a429312b97fdf9beb8ff8cc2c2f7837843927b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:04.525 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 12 01:14:04 smithi099 podman[72586]: 2024-09-12 01:14:04.425403759 +0000 UTC m=+0.016655093 container died aa9fa34319f0a4f3fb6fbe98a429312b97fdf9beb8ff8cc2c2f7837843927b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-12T01:14:05.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T01:14:05.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:05.182 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T01:14:05.182 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T01:14:05.183 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.2 2024-09-12T01:14:05.591 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:05 smithi159 systemd[1]: Stopping Ceph osd.2 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:05.862 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T01:14:05.592+0000 7f63e4279640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:05.862 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T01:14:05.592+0000 7f63e4279640 -1 osd.2 108 *** Got signal Terminated *** 2024-09-12T01:14:05.862 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2[40506]: 2024-09-12T01:14:05.592+0000 7f63e4279640 -1 osd.2 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:11.366 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:11 smithi159 podman[73240]: 2024-09-12 01:14:11.098661189 +0000 UTC m=+5.718977825 container died d67678fb3499870af555de078a8a1591750a21084a39203ec208eac7140ffe98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T01:14:11.785 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:11 smithi159 podman[73240]: 2024-09-12 01:14:11.492577074 +0000 UTC m=+6.112893714 container cleanup d67678fb3499870af555de078a8a1591750a21084a39203ec208eac7140ffe98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T01:14:11.785 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:11 smithi159 bash[73240]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2 2024-09-12T01:14:12.109 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:11 smithi159 podman[73267]: 2024-09-12 01:14:11.785577747 +0000 UTC m=+0.680791664 container remove d67678fb3499870af555de078a8a1591750a21084a39203ec208eac7140ffe98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:14:12.366 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.013508919 +0000 UTC m=+0.016010210 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:12.367 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.144846026 +0000 UTC m=+0.147347311 container create 18580354ac641a1e9515df23284bea91bfa95ee54a4f865c0088459311f3ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T01:14:12.619 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.42284285 +0000 UTC m=+0.425344136 container init 18580354ac641a1e9515df23284bea91bfa95ee54a4f865c0088459311f3ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-12T01:14:12.619 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.426215385 +0000 UTC m=+0.428716674 container start 18580354ac641a1e9515df23284bea91bfa95ee54a4f865c0088459311f3ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:14:12.619 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.514502099 +0000 UTC m=+0.517003396 container attach 18580354ac641a1e9515df23284bea91bfa95ee54a4f865c0088459311f3ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T01:14:12.975 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 12 01:14:12 smithi159 podman[73361]: 2024-09-12 01:14:12.620698103 +0000 UTC m=+0.623199405 container died 18580354ac641a1e9515df23284bea91bfa95ee54a4f865c0088459311f3ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T01:14:13.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T01:14:13.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:13.358 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T01:14:13.358 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T01:14:13.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.3 2024-09-12T01:14:13.643 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:13 smithi016 systemd[1]: Stopping Ceph osd.3 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:14.143 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:13 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T01:14:13.679+0000 7fb799735640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:14.143 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:13 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T01:14:13.679+0000 7fb799735640 -1 osd.3 108 *** Got signal Terminated *** 2024-09-12T01:14:14.144 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:13 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T01:14:13.679+0000 7fb799735640 -1 osd.3 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:15.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:15 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:15.243+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:16.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:16 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:16.257+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:16.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:16 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:16.511+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:17.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:17 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:17.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:17.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:17.823+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:17.867 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:17.602+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:17.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:17 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:17.502+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:18.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:18 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:18.218+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:18.547 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3[51596]: 2024-09-12T01:14:18.236+0000 7fb795d4b640 -1 osd.3 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.323274+0000 front 2024-09-12T01:13:52.323333+0000 (oldest deadline 2024-09-12T01:14:18.223493+0000) 2024-09-12T01:14:18.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:18.547+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:19.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:18.794+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:19.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:18.629+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:19.374 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:19 smithi016 podman[134435]: 2024-09-12 01:14:19.081384116 +0000 UTC m=+5.520741735 container died 944262d8f0903a591e588425f51f39a70e3ee77a362ca7c603307205318e1348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:14:19.374 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:18 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:18.899+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:19.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:19 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:19.213+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:19.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:19 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:19.549+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:19.643 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:19 smithi016 podman[134435]: 2024-09-12 01:14:19.571954889 +0000 UTC m=+6.011312502 container cleanup 944262d8f0903a591e588425f51f39a70e3ee77a362ca7c603307205318e1348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:19.644 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:19 smithi016 bash[134435]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3 2024-09-12T01:14:19.867 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:19.601+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:19.867 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:19.768+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:20.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:19 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:19.883+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:20.144 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:19 smithi016 podman[134450]: 2024-09-12 01:14:19.848567527 +0000 UTC m=+0.761229267 container remove 944262d8f0903a591e588425f51f39a70e3ee77a362ca7c603307205318e1348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T01:14:20.502 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.101565709 +0000 UTC m=+0.015961293 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:20.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.241054 +0000 UTC m=+0.155449568 container create 5ab8c0da281a3d8dbd42ddea20543c56f63a70b9b8b5c59f15ebba31383c334c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T01:14:20.503 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.504023306 +0000 UTC m=+0.418418874 container init 5ab8c0da281a3d8dbd42ddea20543c56f63a70b9b8b5c59f15ebba31383c334c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T01:14:20.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:20 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:20.215+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:20.866 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:20.602+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:20.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:20.817+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:20.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:20 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:20.542+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:20.894 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.50759823 +0000 UTC m=+0.421993800 container start 5ab8c0da281a3d8dbd42ddea20543c56f63a70b9b8b5c59f15ebba31383c334c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T01:14:20.894 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.595761159 +0000 UTC m=+0.510156750 container attach 5ab8c0da281a3d8dbd42ddea20543c56f63a70b9b8b5c59f15ebba31383c334c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T01:14:20.894 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 12 01:14:20 smithi016 podman[134541]: 2024-09-12 01:14:20.695070928 +0000 UTC m=+0.609466500 container died 5ab8c0da281a3d8dbd42ddea20543c56f63a70b9b8b5c59f15ebba31383c334c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T01:14:21.374 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:20 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:20.926+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:21.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T01:14:21.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:21.449 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T01:14:21.450 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T01:14:21.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.4 2024-09-12T01:14:21.477 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:21 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:21.213+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:21.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:21 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:21.559+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:21.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:21 smithi099 systemd[1]: Stopping Ceph osd.4 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:22.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:21.638+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:22.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:21.805+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:22.234 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:21 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T01:14:21.775+0000 7fbb09795640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:22.234 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:21 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T01:14:21.775+0000 7fbb09795640 -1 osd.4 108 *** Got signal Terminated *** 2024-09-12T01:14:22.234 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:21 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T01:14:21.775+0000 7fbb09795640 -1 osd.4 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:22.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:21 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:21.949+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:22.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:22 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:22.234+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:22.866 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:22 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:22.608+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:22.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:22 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:22.805+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:22.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:22 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:22.519+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:23.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:22 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:22.908+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:23.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:22 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:22.908+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:23.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:23 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:23.192+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:23.872 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:23 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:23.514+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:24.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:23 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:23.821+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:24.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:23 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:23.619+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:24.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:23 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:23.873+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:24.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:23 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:23.873+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:24.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:24 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:24.165+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:24.853 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:24 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:24.555+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:24.866 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:24 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:24.789+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:24.867 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:24 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:24.608+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:24.867 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:24 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:24.608+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:25.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:24 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:24.853+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:25.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:24 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:24.853+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:25.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:25 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:25.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:25.807 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:25 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:25.506+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:26.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:25 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:25.794+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:26.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:25 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:25.646+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:26.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:25 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:25.646+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:26.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:25 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:25.807+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:26.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:25 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:25.807+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:26.502 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:26 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:26.183+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:26.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:26 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:26.304+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:26.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:26 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4[46584]: 2024-09-12T01:14:26.502+0000 7fbb05dab640 -1 osd.4 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.753781+0000 front 2024-09-12T01:14:01.753812+0000 (oldest deadline 2024-09-12T01:14:26.453952+0000) 2024-09-12T01:14:26.774 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:26 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:26.551+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:26.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:26 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:26.556+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:26.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:26 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:26.769+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:26.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:26 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:26.769+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:27.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:26 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:26.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:27.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:26 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:26.664+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:27.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:26 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:26.664+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:27.401 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:27 smithi099 podman[72678]: 2024-09-12 01:14:27.09225523 +0000 UTC m=+5.441970268 container died 11b0f006d48a7a2de5e2086765f0cc77b31cbb955510dbdf259cb1caf83fb23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:27.401 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:27 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:27.216+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:27.402 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:27 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:27.216+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:27.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:27 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:27.261+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:27.743 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:27 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:27.522+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:27.743 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:27 smithi099 podman[72678]: 2024-09-12 01:14:27.509642314 +0000 UTC m=+5.859357341 container cleanup 11b0f006d48a7a2de5e2086765f0cc77b31cbb955510dbdf259cb1caf83fb23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-12T01:14:27.743 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:27 smithi099 bash[72678]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4 2024-09-12T01:14:27.806 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:27 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:27.540+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:27.806 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:27 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:27.540+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:28.024 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:27 smithi099 podman[72693]: 2024-09-12 01:14:27.744710353 +0000 UTC m=+0.648613991 container remove 11b0f006d48a7a2de5e2086765f0cc77b31cbb955510dbdf259cb1caf83fb23f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:28.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:27 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:27.767+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:28.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:27 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:27.682+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:28.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:27 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:27.682+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:28.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:27 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:27.807+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:28.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:27 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:27.807+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:28.472 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.072079562 +0000 UTC m=+0.015971754 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:28.472 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.203903644 +0000 UTC m=+0.147795873 container create 98a6b87893afdf99c21b93d5c95d0bdbab26932431fead01b16ca7c8f70b67db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:28.473 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:28 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:28.167+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:28.473 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:28 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:28.167+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:28.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:28 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:28.253+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:28.774 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:28 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:28.498+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:28.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.473580819 +0000 UTC m=+0.417473019 container init 98a6b87893afdf99c21b93d5c95d0bdbab26932431fead01b16ca7c8f70b67db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:28.774 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.477097555 +0000 UTC m=+0.420989745 container start 98a6b87893afdf99c21b93d5c95d0bdbab26932431fead01b16ca7c8f70b67db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:28.775 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.581718361 +0000 UTC m=+0.525610554 container attach 98a6b87893afdf99c21b93d5c95d0bdbab26932431fead01b16ca7c8f70b67db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T01:14:28.775 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 12 01:14:28 smithi099 podman[72780]: 2024-09-12 01:14:28.666106907 +0000 UTC m=+0.609999097 container died 98a6b87893afdf99c21b93d5c95d0bdbab26932431fead01b16ca7c8f70b67db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T01:14:28.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:28.788+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:28.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:28.788+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:28.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:28.580+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:28.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:28 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:28.580+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:29.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:28 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:28.644+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:29.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:28 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:28.644+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:29.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:28 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:28.755+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:29.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:28 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:28.755+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:29.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:29 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:29.140+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:29.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:29 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:29.140+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:29.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T01:14:29.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:29.427 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T01:14:29.427 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T01:14:29.428 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.5 2024-09-12T01:14:29.616 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 systemd[1]: Stopping Ceph osd.5 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:29.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:29.275+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:29.774 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:29 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:29.500+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:29.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:29.626+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:29.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:29.626+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:29.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:29.794+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:29.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:29 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:29.794+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:30.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:29.749+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:29.749+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:29.672+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:29.672+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:29.737+0000 7f71a6c3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:29.737+0000 7f71a6c3d640 -1 osd.5 108 *** Got signal Terminated *** 2024-09-12T01:14:30.117 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:29 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:29.737+0000 7f71a6c3d640 -1 osd.5 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:30.509 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:30 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:30.186+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:30.509 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:30 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:30.186+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:30.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:30 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:30.292+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:30.774 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:30 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:30.509+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:30.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:30.576+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:30.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:30.576+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:30.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:30.811+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:30.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:30.811+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:31.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:30 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:30.701+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:31.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:30 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:30.701+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:31.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:30 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:30.724+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:31.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:30 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:30.724+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:31.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:31 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:31.170+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:31.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:31 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:31.170+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:31.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:31.297+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:31.820 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:31.561+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:31.820 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:31.561+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:32.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:31 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:31.542+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:32.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:31.716+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:32.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:31.716+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:32.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:31.723+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:32.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:31.723+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:32.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:31 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:31.723+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:32.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:31.821+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:32.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:31.821+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:32.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:31 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:31.821+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:32.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:32 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:32.218+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:32.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:32 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:32.218+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:32.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:32.320+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:32.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:32.792+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:32.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:32.792+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:32.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:32.792+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:32.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:32.585+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:32.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:32.585+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:33.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:32 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:32.560+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:33.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:32.674+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:33.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:32.674+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:33.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:32.674+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.367844+0000 front 2024-09-12T01:14:07.367928+0000 (oldest deadline 2024-09-12T01:14:32.067943+0000) 2024-09-12T01:14:33.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:32.708+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:33.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:32.708+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:33.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:32 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:32.708+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:33.291 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:33.236+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:33.292 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:33.236+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:33.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:33.339+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:33.826 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:33.560+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:33.826 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:33.560+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:33.826 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:33.560+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:34.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:33 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:33.602+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:34.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:33.700+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:34.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:33.700+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:34.116 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:33.700+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.367844+0000 front 2024-09-12T01:14:07.367928+0000 (oldest deadline 2024-09-12T01:14:32.067943+0000) 2024-09-12T01:14:34.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:33.754+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:34.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:33.754+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:34.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:33 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:33.754+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:34.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:33.827+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:34.144 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:33.827+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:34.144 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:33 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:33.827+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:34.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:34 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:34.190+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:34.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:34 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:34.190+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:34.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:34.341+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:34.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:34.341+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:34.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:34.858+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:34.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:34.858+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:34.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:34.858+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:34.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:34.610+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:34.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:34.610+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:34.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:34 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:34.610+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:35.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:34 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:34.639+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:35.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:34 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:34.639+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:34.743+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:34.743+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:34.743+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:34.683+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:50.965464+0000 front 2024-09-12T01:13:50.965595+0000 (oldest deadline 2024-09-12T01:14:16.865686+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:34.683+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.567036+0000 front 2024-09-12T01:14:01.566823+0000 (oldest deadline 2024-09-12T01:14:24.466750+0000) 2024-09-12T01:14:35.102 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:34 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5[47255]: 2024-09-12T01:14:34.683+0000 7f719f24b640 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.367844+0000 front 2024-09-12T01:14:07.367928+0000 (oldest deadline 2024-09-12T01:14:32.067943+0000) 2024-09-12T01:14:35.366 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:35 smithi159 podman[73468]: 2024-09-12 01:14:35.103033219 +0000 UTC m=+5.479334582 container died 223c3d2c7449a977cb6d8863c0313574f81372466c29d36bce290000e43b3be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T01:14:35.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:35 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:35.200+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:35.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:35 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:35.200+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:35.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:35 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:35.200+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:35.721 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:35 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:35.379+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:35.722 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:35 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:35.379+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:35.722 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:35 smithi159 podman[73468]: 2024-09-12 01:14:35.488442107 +0000 UTC m=+5.864743454 container cleanup 223c3d2c7449a977cb6d8863c0313574f81372466c29d36bce290000e43b3be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T01:14:35.722 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:35 smithi159 bash[73468]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5 2024-09-12T01:14:35.861 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:35.608+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:35.861 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:35.608+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:35.861 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:35.608+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:36.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:35 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:35.652+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:36.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:35 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:35.652+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:36.034 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:35 smithi159 podman[73484]: 2024-09-12 01:14:35.723107459 +0000 UTC m=+0.612670279 container remove 223c3d2c7449a977cb6d8863c0313574f81372466c29d36bce290000e43b3be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T01:14:36.035 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:35 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:35.772+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:36.035 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:35 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:35.772+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:36.035 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:35 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:35.772+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:36.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:35.862+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:36.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:35.862+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:36.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:35.862+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:36.325 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:35.93855649 +0000 UTC m=+0.016038047 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:36.325 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:36.065501636 +0000 UTC m=+0.142983193 container create 1db7f9dca5caf33f931d3258570600f94151beca84a983ec48507a18520f8902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:36.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:36 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:36.190+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:36.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:36 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:36.190+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:36.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:36 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:36.190+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:36.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:36.416+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:36.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:36.416+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:36.617 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:36.326560028 +0000 UTC m=+0.404041585 container init 1db7f9dca5caf33f931d3258570600f94151beca84a983ec48507a18520f8902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T01:14:36.617 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:36.330321779 +0000 UTC m=+0.407803337 container start 1db7f9dca5caf33f931d3258570600f94151beca84a983ec48507a18520f8902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T01:14:36.617 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:36.418251941 +0000 UTC m=+0.495733506 container attach 1db7f9dca5caf33f931d3258570600f94151beca84a983ec48507a18520f8902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T01:14:36.617 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 12 01:14:36 smithi159 podman[73571]: 2024-09-12 01:14:36.517506997 +0000 UTC m=+0.594988555 container died 1db7f9dca5caf33f931d3258570600f94151beca84a983ec48507a18520f8902 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:36.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:36.818+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:36.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:36.818+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:36.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:36.818+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:36.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:36.587+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:36.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:36.587+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:36.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:36 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:36.587+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:37.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:36 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:36.626+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:37.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:36 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:36.626+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:37.079 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:36.798+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:37.079 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:36.798+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:37.079 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:36 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:36.798+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:37.105 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T01:14:37.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:37.143 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T01:14:37.143 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T01:14:37.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.6 2024-09-12T01:14:37.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:37 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:37.179+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:37.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:37 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:37.179+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:37.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:37 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:37.179+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:37.584 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 systemd[1]: Stopping Ceph osd.6 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:37.584 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.585+0000 7fbc771c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:37.797 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:37 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:37.450+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:37.797 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:37 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:37.450+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:37.841 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.585+0000 7fbc771c3640 -1 osd.6 108 *** Got signal Terminated *** 2024-09-12T01:14:37.841 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.585+0000 7fbc771c3640 -1 osd.6 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:37.841 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.590+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:37.841 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.590+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:37.841 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:37.590+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:38.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:37 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:37.644+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:38.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:37 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:37.644+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:38.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:37 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:37.797+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:38.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:37 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:37.797+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:38.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:37 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:37.797+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:38.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:37.842+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:38.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:37.842+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:38.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:37 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:37.842+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:38.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:38 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:38.147+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:38.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:38 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:38.147+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:38.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:38 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:38.147+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:38.841 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:38.406+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:38.841 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:38.406+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:38.883 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:38.571+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:38.883 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:38.571+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:38.884 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:38.571+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:39.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:38 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:38.683+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:39.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:38 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:38.683+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:39.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:38.841+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:39.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:38.841+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:39.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:38 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:38.841+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:39.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:38.884+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:39.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:38.884+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:39.144 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:38 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:38.884+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:39.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:39 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:39.157+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:39.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:39 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:39.157+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:39.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:39 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:39.157+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:39.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:39.428+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:39.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:39.428+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:39.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:39.585+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:39.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:39.585+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:39.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:39.585+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:40.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:39 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:39.672+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:40.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:39 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:39.672+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:40.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:39.888+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:40.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:39.888+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:40.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:39 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:39.888+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:40.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:39.911+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:40.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:39.911+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:40.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:39 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:39.911+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:40.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:40.153+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:40.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:40.153+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:40.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:40.153+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:40.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:40.153+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:40.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:40 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:40.429+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:40.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:40 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:40.429+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:40.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:40.607+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:40.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:40.607+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:40.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:40.607+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:41.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:40.657+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:41.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:40.657+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:41.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:40 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:40.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:41.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:40 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:40.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:41.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:40 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:40.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:41.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:40.949+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:41.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:40.949+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:41.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:40 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:40.949+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:41.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:41.126+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:41.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:41.126+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:41.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:41.126+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:41.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:41.126+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:41.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:41.385+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:41.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:41.385+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:41.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:41.385+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:41.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:41.559+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:41.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:41.560+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:41.893 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:41.560+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:42.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:41.666+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:42.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:41.666+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:42.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:41 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:41.666+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:42.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:41.962+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:42.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:41.962+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:42.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:41 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:41.962+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:42.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:41.960+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:42.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:41.960+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:42.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:41 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:41.960+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:42.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:42.089+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:42.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:42.089+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:42.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:42.089+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:42.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:42.089+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:42.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:42.429+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:42.867 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:42.429+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:42.867 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:42.429+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:42.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:42.572+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:53.300376+0000 front 2024-09-12T01:13:53.300406+0000 (oldest deadline 2024-09-12T01:14:16.200569+0000) 2024-09-12T01:14:42.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:42.572+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.002354+0000 front 2024-09-12T01:14:02.002482+0000 (oldest deadline 2024-09-12T01:14:27.302280+0000) 2024-09-12T01:14:42.894 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6[58232]: 2024-09-12T01:14:42.572+0000 7fbc6f7d1640 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:07.302627+0000 front 2024-09-12T01:14:07.302798+0000 (oldest deadline 2024-09-12T01:14:33.202717+0000) 2024-09-12T01:14:43.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:42.694+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:43.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:42.694+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:43.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:42 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:42.694+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:43.218 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:42.942+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:43.218 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:42.942+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:43.219 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:42 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:42.942+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:43.219 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:42 smithi016 podman[134654]: 2024-09-12 01:14:42.932418496 +0000 UTC m=+5.578242182 container died 518c4ba08e62a779fd27d77c05dfa507f19df683c627f84069df9d9c9b2b39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T01:14:43.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:42.987+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:43.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:42.987+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:43.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:42 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:42.987+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:43.474 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:43 smithi016 podman[134654]: 2024-09-12 01:14:43.460784446 +0000 UTC m=+6.106608131 container cleanup 518c4ba08e62a779fd27d77c05dfa507f19df683c627f84069df9d9c9b2b39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T01:14:43.474 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:43 smithi016 bash[134654]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6 2024-09-12T01:14:43.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:43.043+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:43.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:43.043+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:43.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:43.043+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:43.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:43.043+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:43.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:43.393+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:43.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:43.393+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:43.867 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:43.393+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:44.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:43.675+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:44.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:43.675+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:44.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:43.675+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:44.080 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:43 smithi016 podman[134670]: 2024-09-12 01:14:43.670853024 +0000 UTC m=+0.736917826 container remove 518c4ba08e62a779fd27d77c05dfa507f19df683c627f84069df9d9c9b2b39a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:14:44.080 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:43 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:43.914+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:44.080 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:43 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:43.914+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:44.080 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:43 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:43.914+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:44.081 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:43 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:43.914+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:44.340 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:43.984347162 +0000 UTC m=+0.016050256 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:44.340 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:44.088156598 +0000 UTC m=+0.119859686 container create 8723231de2d96419cdc1c9e4b506c67a1c54dff013b4107bea8fa4d436f43069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T01:14:44.340 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:44.341032063 +0000 UTC m=+0.372735150 container init 8723231de2d96419cdc1c9e4b506c67a1c54dff013b4107bea8fa4d436f43069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:44.347 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:43.964+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:44.347 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:43.964+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:44.347 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:43 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:43.964+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:44.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:44.087+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:44.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:44.087+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:44.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:44.087+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:44.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:44.087+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:44.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:44.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:44.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:44.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:44.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:44.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:44.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:44.344765032 +0000 UTC m=+0.376468117 container start 8723231de2d96419cdc1c9e4b506c67a1c54dff013b4107bea8fa4d436f43069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T01:14:44.643 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:44.407713952 +0000 UTC m=+0.439417037 container attach 8723231de2d96419cdc1c9e4b506c67a1c54dff013b4107bea8fa4d436f43069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T01:14:44.644 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 12 01:14:44 smithi016 podman[134758]: 2024-09-12 01:14:44.54590545 +0000 UTC m=+0.577608543 container died 8723231de2d96419cdc1c9e4b506c67a1c54dff013b4107bea8fa4d436f43069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T01:14:45.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:44.672+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:45.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:44.672+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:45.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:44 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:44.672+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:45.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:44 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:44.955+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:45.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:44 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:44.955+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:45.143 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:44 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:44.955+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:45.144 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:44 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:44.955+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:45.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T01:14:45.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:45.249 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T01:14:45.249 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T01:14:45.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.7 2024-09-12T01:14:45.300 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:45.048+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:45.300 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:45.048+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:45.300 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:45.048+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:45.300 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:45.048+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:45.347 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:44.922+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:45.348 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:44.922+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:45.348 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:44 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:44.922+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:45.563 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 systemd[1]: Stopping Ceph osd.7 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:45.563 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.562+0000 7f8e7fd60640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:45.618 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:45.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:45.618 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:45.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:45.618 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:45.348+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:46.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.562+0000 7f8e7fd60640 -1 osd.7 108 *** Got signal Terminated *** 2024-09-12T01:14:46.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.562+0000 7f8e7fd60640 -1 osd.7 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:46.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.697+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:46.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.697+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:46.025 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:45 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:45.697+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:46.299 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:45.927+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:46.299 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:45.927+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:46.299 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:45 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:45.927+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:46.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:45.953+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:46.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:45.953+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:46.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:45.953+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:46.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:45 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:45.953+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:46.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:46.041+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:46.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:46.041+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:46.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:46.041+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:46.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:46.041+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:46.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:46.299+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:46.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:46.299+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:46.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:46.299+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:47.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:46.707+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:47.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:46.707+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:47.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:46 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:46.707+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:47.335 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:46.945+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:47.335 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:46.945+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:47.335 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:46 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:46.945+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:47.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:46.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:47.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:46.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:47.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:46.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:47.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:46 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:46.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:47.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:47.073+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:47.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:47.073+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:47.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:47.073+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:47.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:47.073+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:47.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:47.335+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:47.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:47.335+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:47.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:47.335+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:48.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:47.698+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:48.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:47.698+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:48.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:47.698+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:48.339 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:47.925+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:48.339 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:47.925+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:48.339 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:47 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:47.925+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:48.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:47 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:47.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:48.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:47 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:47.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:48.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:47 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:47.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:48.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:47 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:47.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:48.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:48.094+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:48.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:48.094+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:48.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:48.094+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:48.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:48.094+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:48.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:48.094+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:48.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:48.340+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:48.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:48.340+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:48.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:48.340+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:49.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:48.744+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:49.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:48.744+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:49.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:48 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:48.744+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:49.324 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:48.877+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:49.324 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:48.877+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:49.324 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:48.877+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:49.324 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:48 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:48.877+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:49.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:48.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:49.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:48.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:49.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:48.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:49.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:48.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:49.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:48 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:48.976+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:49.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:49.142+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:49.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:49.142+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:49.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:49.142+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:49.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:49.142+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:49.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:49.142+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:49.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:49.325+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:49.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:49.325+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:49.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:49.325+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:50.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:49.753+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.827773+0000 front 2024-09-12T01:14:01.827798+0000 (oldest deadline 2024-09-12T01:14:26.527873+0000) 2024-09-12T01:14:50.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:49.753+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.329256+0000 front 2024-09-12T01:14:09.329317+0000 (oldest deadline 2024-09-12T01:14:34.029299+0000) 2024-09-12T01:14:50.024 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:49 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7[53509]: 2024-09-12T01:14:49.753+0000 7f8e7836e640 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:16.330387+0000 front 2024-09-12T01:14:16.330488+0000 (oldest deadline 2024-09-12T01:14:41.630288+0000) 2024-09-12T01:14:50.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:49.830+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:50.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:49.830+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:50.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:49.830+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:50.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:49 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:49.830+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:50.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.003+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:50.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.003+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:50.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.003+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:50.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.003+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:50.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.003+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:50.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:50 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:50.181+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:50.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:50 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:50.181+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:50.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:50 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:50.181+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:50.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:50 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:50.181+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:50.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:50 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:50.181+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:50.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:50.296+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:50.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:50.296+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:50.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:50.296+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:50.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:50.296+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:51.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:50.856+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:51.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:50.856+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:51.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:50.856+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:51.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:50 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:50.856+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:50 smithi099 podman[72896]: 2024-09-12 01:14:50.909566846 +0000 UTC m=+5.461903795 container died 2ff099bec90c0185b976a738e1653c48a724cd519a3611fe0d2618b542781754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:51.138+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:51.138+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:51.138+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:51.138+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:51.220 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:51.138+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:51.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.990+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:51.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.990+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:51.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.990+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:51.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.990+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:51.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:50 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:50.990+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:51.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:51 smithi099 podman[72896]: 2024-09-12 01:14:51.320365537 +0000 UTC m=+5.872702491 container cleanup 2ff099bec90c0185b976a738e1653c48a724cd519a3611fe0d2618b542781754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T01:14:51.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:51 smithi099 bash[72896]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7 2024-09-12T01:14:51.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:51.264+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:51.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:51.264+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:51.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:51.264+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:51.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:51.264+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:51.921 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:51 smithi099 podman[72913]: 2024-09-12 01:14:51.580318752 +0000 UTC m=+0.667042782 container remove 2ff099bec90c0185b976a738e1653c48a724cd519a3611fe0d2618b542781754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:52 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:52.162+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:52 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:52.162+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:52 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:52.162+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:52 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:52.162+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:52 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:52.162+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:52.174 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:51 smithi099 podman[73003]: 2024-09-12 01:14:51.922948906 +0000 UTC m=+0.106904751 container create d2a6f45d94787f24a4a722645afee289c12f943c47ec478efe0b25c86844ac19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:14:52.175 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:51 smithi099 podman[73003]: 2024-09-12 01:14:51.831937866 +0000 UTC m=+0.015893680 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:14:52.175 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:52 smithi099 podman[73003]: 2024-09-12 01:14:52.175462025 +0000 UTC m=+0.359417826 container init d2a6f45d94787f24a4a722645afee289c12f943c47ec478efe0b25c86844ac19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T01:14:52.237 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:51.896+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:52.238 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:51.896+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:52.238 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:51.896+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:52.238 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:51 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:51.896+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:52.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:51 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:51.979+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:52.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:51 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:51.979+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:52.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:51 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:51.979+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:52.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:51 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:51.979+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:52.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:51 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:51.979+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:52.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:52 smithi099 podman[73003]: 2024-09-12 01:14:52.178800977 +0000 UTC m=+0.362756781 container start d2a6f45d94787f24a4a722645afee289c12f943c47ec478efe0b25c86844ac19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T01:14:52.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:52 smithi099 podman[73003]: 2024-09-12 01:14:52.267114259 +0000 UTC m=+0.451070079 container attach d2a6f45d94787f24a4a722645afee289c12f943c47ec478efe0b25c86844ac19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:14:52.524 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 12 01:14:52 smithi099 podman[73003]: 2024-09-12 01:14:52.368406189 +0000 UTC m=+0.552362004 container died d2a6f45d94787f24a4a722645afee289c12f943c47ec478efe0b25c86844ac19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T01:14:52.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:52.238+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:52.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:52.238+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:52.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:52.238+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:52.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:52.238+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:53.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T01:14:53.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:14:53.083 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T01:14:53.083 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T01:14:53.083 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.8 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 systemd[1]: Stopping Ceph osd.8 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.235+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.235+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.235+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.235+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:53.235 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:52.941+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:53.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:52.941+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:53.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:52.941+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:53.236 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:52 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:52.941+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:53.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:53.204+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:53.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:53.204+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:53.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:53.204+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:53.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:53.204+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:53.274 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:53 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:53.204+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:53.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:52 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:52.984+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:53.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:52 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:52.984+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:53.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:52 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:52.984+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:53.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:52 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:52.984+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:53.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:52 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:52.984+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:53.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.365+0000 7f5ec5ba3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:14:53.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.365+0000 7f5ec5ba3640 -1 osd.8 108 *** Got signal Terminated *** 2024-09-12T01:14:53.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:53.365+0000 7f5ec5ba3640 -1 osd.8 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:14:54.245 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:53.942+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:54.245 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:53.942+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:54.245 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:53.942+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:54.245 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:53 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:53.942+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:54.295 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:53 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:53.937+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:54.295 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:53 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:53.937+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:54.295 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:53 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:53.937+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:54.295 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:53 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:53.937+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:54.295 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:53 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:53.937+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:54.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:54.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:54.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:54.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:54.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:54.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:54.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:54.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:54.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:54 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:54.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:54.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:54.245+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:54.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:54.245+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:54.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:54.245+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:54.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:54.245+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:55.294 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:54.988+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:55.294 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:54.988+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:55.294 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:54.988+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:55.294 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:54 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:54.988+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:55.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:54 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:54.975+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:55.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:54 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:54.975+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:55.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:54 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:54.975+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:55.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:54 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:54.975+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:55.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:54 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:54.975+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:55.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:55 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:55.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:55.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:55 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:55.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:55.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:55 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:55.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:55.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:55 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:55.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:55.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:55 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:55.184+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:55.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:55.294+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:55.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:55.294+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:55.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:55.294+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:55.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:55.294+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:56.270 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:55.973+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:56.270 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:55.973+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:56.270 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:55.973+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:56.270 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:55.973+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:56.270 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:55.973+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:14:56.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:56.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:56.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:56.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:56.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:56.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:55 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:55.964+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:14:56.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:56.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:56.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:56.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:56.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:56.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:56.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:56.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:56.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:56 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:56.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:56.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:56.269+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:56.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:56.269+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:56.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:56.269+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:56.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:56.269+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:57.315 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:56.972+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:57.315 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:56.972+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:57.315 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:56.972+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:57.315 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:56.972+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:57.316 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:56 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:56.972+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:14:57.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:57.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:57.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:57.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:57.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:57.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:56 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:56.983+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:14:57.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:57.193+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:57.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:57.193+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:57.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:57.193+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:57.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:57.193+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:57.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:57 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:57.193+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:57.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:57.316+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:57.616 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:57.316+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:57.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:57.316+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:57.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:57.316+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:57.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:57.316+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:30.877185+0000 front 2024-09-12T01:14:30.877230+0000 (oldest deadline 2024-09-12T01:14:56.678010+0000) 2024-09-12T01:14:58.322 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:57.957+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:58.322 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:57.957+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:58.322 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:57.957+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:58.322 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:57.957+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:58.322 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:57 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:57.957+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:14:58.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:58.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:58.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:58.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:58.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:58.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:57 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:57.989+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:58.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:58 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:58.228+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:14:58.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:58.322+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:01.272052+0000 front 2024-09-12T01:14:01.272137+0000 (oldest deadline 2024-09-12T01:14:25.371890+0000) 2024-09-12T01:14:58.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:58.322+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.573526+0000 front 2024-09-12T01:14:10.573596+0000 (oldest deadline 2024-09-12T01:14:33.374448+0000) 2024-09-12T01:14:58.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:58.322+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.675345+0000 front 2024-09-12T01:14:18.675382+0000 (oldest deadline 2024-09-12T01:14:40.975454+0000) 2024-09-12T01:14:58.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:58.322+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:20.976054+0000 front 2024-09-12T01:14:20.975947+0000 (oldest deadline 2024-09-12T01:14:49.776435+0000) 2024-09-12T01:14:58.617 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8[53745]: 2024-09-12T01:14:58.322+0000 7f5ec21b9640 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:30.877185+0000 front 2024-09-12T01:14:30.877230+0000 (oldest deadline 2024-09-12T01:14:56.678010+0000) 2024-09-12T01:14:59.190 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:58 smithi159 podman[73673]: 2024-09-12 01:14:58.882655935 +0000 UTC m=+5.602194757 container died 838089491a20f95150197a6638d7c330dfd11db173e2389e23f72743b42aac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:14:59.190 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:58.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:14:59.191 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:58.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:14:59.191 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:58.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:14:59.191 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:58.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:14:59.191 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:58 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:58.912+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:14:59.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:14:59.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:14:59.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:14:59.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:14:59.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:14:59.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:58 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:58.972+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:14:59.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:14:59 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:14:59.268+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:14:59.533 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:59 smithi159 podman[73673]: 2024-09-12 01:14:59.307675697 +0000 UTC m=+6.027214519 container cleanup 838089491a20f95150197a6638d7c330dfd11db173e2389e23f72743b42aac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T01:14:59.533 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:59 smithi159 bash[73673]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8 2024-09-12T01:14:59.866 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:59 smithi159 podman[73688]: 2024-09-12 01:14:59.533937472 +0000 UTC m=+0.640943904 container remove 838089491a20f95150197a6638d7c330dfd11db173e2389e23f72743b42aac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T01:15:00.232 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:14:59 smithi159 podman[73774]: 2024-09-12 01:14:59.879413726 +0000 UTC m=+0.015936723 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:15:00.232 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:15:00 smithi159 podman[73774]: 2024-09-12 01:15:00.001290867 +0000 UTC m=+0.137813845 container create 4066bb275d43bc51705cba94695b2b865fbbd6138da5b5ae95064cb65af72b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:15:00.233 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:59.879+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:00.233 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:59.879+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:00.233 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:59.879+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:00.233 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:59.879+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:00.233 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:14:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:14:59.879+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:00.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:00.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:00.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:00.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:00.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:00.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:14:59 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:14:59.967+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:00.546 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:15:00 smithi159 podman[73774]: 2024-09-12 01:15:00.35374278 +0000 UTC m=+0.490265770 container init 4066bb275d43bc51705cba94695b2b865fbbd6138da5b5ae95064cb65af72b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T01:15:00.546 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:15:00 smithi159 podman[73774]: 2024-09-12 01:15:00.357066711 +0000 UTC m=+0.493589694 container start 4066bb275d43bc51705cba94695b2b865fbbd6138da5b5ae95064cb65af72b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T01:15:00.546 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:15:00 smithi159 podman[73774]: 2024-09-12 01:15:00.420323725 +0000 UTC m=+0.556846704 container attach 4066bb275d43bc51705cba94695b2b865fbbd6138da5b5ae95064cb65af72b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T01:15:00.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:00.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:00.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:00.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:00.775 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:00.775 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:00 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:00.278+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:00.839 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 12 01:15:00 smithi159 podman[73774]: 2024-09-12 01:15:00.548042874 +0000 UTC m=+0.684565853 container died 4066bb275d43bc51705cba94695b2b865fbbd6138da5b5ae95064cb65af72b8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T01:15:01.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:00.867+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:01.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:00.867+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:01.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:00.867+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:01.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:00.867+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:01.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:00 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:00.867+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:01.162 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T01:15:01.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:01.206 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T01:15:01.206 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T01:15:01.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.9 2024-09-12T01:15:01.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:01.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:01.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:01.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:01.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:01.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.015+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:01.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 systemd[1]: Stopping Ceph osd.9 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:01.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:01 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:01.321+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:01.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.531+0000 7f43236e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:01.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.531+0000 7f43236e4640 -1 osd.9 108 *** Got signal Terminated *** 2024-09-12T01:15:01.894 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:01 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:01.531+0000 7f43236e4640 -1 osd.9 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:15:02.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:01.886+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:02.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:01.886+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:02.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:01.886+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:02.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:01.886+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:02.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:01 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:01.886+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:02.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:02.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:02.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:02.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:02.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:02.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:02 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:02.061+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:02.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:02 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:02.332+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:03.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:02.898+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:03.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:02.898+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:03.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:02.898+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:03.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:02.898+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:03.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:02 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:02.898+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:03.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:03.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:03.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:03.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:03.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:03.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.022+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:03.675 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:03.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:03.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:03.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:03.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:03.676 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:03 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:03.289+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:04.349 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:04.349 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:04.349 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:04.350 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:04.350 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:04.350 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:03 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:03.995+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:04.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:04.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:04.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:04.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:04.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:04.367 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:03.882+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:04.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:04.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:04.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:04.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:04.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:04.775 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:04 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:04.319+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:05.268 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:04 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:04.887+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:05.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:05.393 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:05.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:05.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:05.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:05.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:05 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:05.024+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:05.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:05 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:05.288+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:06.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:06.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:06.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:06.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:06.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:06.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:05 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:05.839+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.239104+0000 front 2024-09-12T01:13:52.239153+0000 (oldest deadline 2024-09-12T01:14:18.139279+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:58.139778+0000 front 2024-09-12T01:13:58.139756+0000 (oldest deadline 2024-09-12T01:14:22.839831+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:08.141256+0000 front 2024-09-12T01:14:08.141161+0000 (oldest deadline 2024-09-12T01:14:31.640982+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:18.042305+0000 front 2024-09-12T01:14:18.042337+0000 (oldest deadline 2024-09-12T01:14:43.342513+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:23.343179+0000 front 2024-09-12T01:14:23.343065+0000 (oldest deadline 2024-09-12T01:14:48.642998+0000) 2024-09-12T01:15:06.394 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T01:15:06.010+0000 7f431bcf2640 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.944327+0000 front 2024-09-12T01:14:33.944223+0000 (oldest deadline 2024-09-12T01:14:55.644213+0000) 2024-09-12T01:15:06.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:06.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:06.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:06.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:06.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:06.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:06 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:06.254+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:07.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:07.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:07.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:07.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:07.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:07.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:06 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:06.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:07.171 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:06 smithi016 podman[134873]: 2024-09-12 01:15:06.913104839 +0000 UTC m=+5.503014856 container died 29ef8aa76d989b12356fd1ac229007e4a213afc5c84273ede94ab841fd7ad0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T01:15:07.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:07.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:07.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:07.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:07.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:07.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:07 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:07.242+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:07.539 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:07 smithi016 podman[134873]: 2024-09-12 01:15:07.356150611 +0000 UTC m=+5.946060661 container cleanup 29ef8aa76d989b12356fd1ac229007e4a213afc5c84273ede94ab841fd7ad0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:15:07.539 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:07 smithi016 bash[134873]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9 2024-09-12T01:15:07.893 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:07 smithi016 podman[134889]: 2024-09-12 01:15:07.540731908 +0000 UTC m=+0.624685862 container remove 29ef8aa76d989b12356fd1ac229007e4a213afc5c84273ede94ab841fd7ad0cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T01:15:08.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:08.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:08.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:08.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:08.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:08.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:07.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:08.275 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:07 smithi016 podman[134978]: 2024-09-12 01:15:07.88680028 +0000 UTC m=+0.016073208 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:15:08.276 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:07 smithi016 podman[134978]: 2024-09-12 01:15:07.991475643 +0000 UTC m=+0.120748563 container create 3285c83b7e24f514f1f60911f03b4e5c5fdf53c1744a3e00d94c117c75bf4cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:15:08.276 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:08 smithi016 podman[134978]: 2024-09-12 01:15:08.210523912 +0000 UTC m=+0.339796830 container init 3285c83b7e24f514f1f60911f03b4e5c5fdf53c1744a3e00d94c117c75bf4cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:15:08.276 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:08 smithi016 podman[134978]: 2024-09-12 01:15:08.21405836 +0000 UTC m=+0.343331278 container start 3285c83b7e24f514f1f60911f03b4e5c5fdf53c1744a3e00d94c117c75bf4cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T01:15:08.276 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:08 smithi016 podman[134978]: 2024-09-12 01:15:08.277108336 +0000 UTC m=+0.406381264 container attach 3285c83b7e24f514f1f60911f03b4e5c5fdf53c1744a3e00d94c117c75bf4cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:15:08.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:08.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:08.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:08.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:08.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:08.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:08.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:08 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:08.232+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:08.643 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 12 01:15:08 smithi016 podman[134978]: 2024-09-12 01:15:08.404990563 +0000 UTC m=+0.534263484 container died 3285c83b7e24f514f1f60911f03b4e5c5fdf53c1744a3e00d94c117c75bf4cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:15:09.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T01:15:09.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:09.078 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T01:15:09.079 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T01:15:09.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.10 2024-09-12T01:15:09.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:09.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:09.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:09.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:09.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:09.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:08 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:08.854+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 systemd[1]: Stopping Ceph osd.10 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:09.405 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.203+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:09.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.404+0000 7f3283b89640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:09.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.404+0000 7f3283b89640 -1 osd.10 108 *** Got signal Terminated *** 2024-09-12T01:15:09.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:09 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:09.404+0000 7f3283b89640 -1 osd.10 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:15:10.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:10.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:10.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:10.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:10.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:10.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:09 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:09.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:10.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:10.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:10.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:10.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:10.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:10.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:10.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:10 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:10.217+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:11.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:11.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:11.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:11.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:11.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:11.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:10 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:10.781+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:11.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:11.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:11.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:11.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:11.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:11.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:11.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:11 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:11.221+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:12.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:12.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:12.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:12.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:12.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:12.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:11 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:11.796+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:12.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:12.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:12.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:12.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:12.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:12.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:12.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:12 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:12.209+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:13.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:13.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:13.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:13.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:13.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:13.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:13.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:12 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:12.827+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:13.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:13.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:13.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:13.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:13.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:13.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:13.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:13 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:13.259+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:14.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:14.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:14.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:14.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:14.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:14.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:14.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:13 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:13.795+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:14.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:51.793844+0000 front 2024-09-12T01:13:51.794031+0000 (oldest deadline 2024-09-12T01:14:14.693770+0000) 2024-09-12T01:15:14.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:14:02.295063+0000 front 2024-09-12T01:14:02.295092+0000 (oldest deadline 2024-09-12T01:14:26.995210+0000) 2024-09-12T01:15:14.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:09.196708+0000 front 2024-09-12T01:14:09.196677+0000 (oldest deadline 2024-09-12T01:14:35.096417+0000) 2024-09-12T01:15:14.524 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-12T01:14:15.097030+0000 front 2024-09-12T01:14:15.096951+0000 (oldest deadline 2024-09-12T01:14:39.796788+0000) 2024-09-12T01:15:14.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:26.098453+0000 front 2024-09-12T01:14:26.098410+0000 (oldest deadline 2024-09-12T01:14:47.198560+0000) 2024-09-12T01:15:14.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:33.801546+0000 front 2024-09-12T01:14:33.801596+0000 (oldest deadline 2024-09-12T01:14:57.901483+0000) 2024-09-12T01:15:14.525 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10[60363]: 2024-09-12T01:15:14.273+0000 7f327c197640 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:41.903039+0000 front 2024-09-12T01:14:41.903047+0000 (oldest deadline 2024-09-12T01:15:07.802883+0000) 2024-09-12T01:15:15.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:15.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:14 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:14.825+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:15.162 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:14 smithi099 podman[73117]: 2024-09-12 01:15:14.903009003 +0000 UTC m=+5.623611289 container died 84e5e32db1799d018e3e1c38cc64ed454fc359a34b3bd939783d423b239e72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T01:15:15.513 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:15 smithi099 podman[73117]: 2024-09-12 01:15:15.288385263 +0000 UTC m=+6.008987549 container cleanup 84e5e32db1799d018e3e1c38cc64ed454fc359a34b3bd939783d423b239e72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T01:15:15.513 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:15 smithi099 bash[73117]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10 2024-09-12T01:15:15.774 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:15 smithi099 podman[73135]: 2024-09-12 01:15:15.514567011 +0000 UTC m=+0.610429386 container remove 84e5e32db1799d018e3e1c38cc64ed454fc359a34b3bd939783d423b239e72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T01:15:16.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:16.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:16.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:16.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:16.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:16.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:16.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:15 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:15.785+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:16.166 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:15 smithi099 podman[73225]: 2024-09-12 01:15:15.731815252 +0000 UTC m=+0.015429774 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:15:16.166 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:15 smithi099 podman[73225]: 2024-09-12 01:15:15.831903436 +0000 UTC m=+0.115517993 container create 9eb688a24595b222cae20cc9bd9e1784240ee0076e4ff25e2b633e4e310246e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T01:15:16.166 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:16 smithi099 podman[73225]: 2024-09-12 01:15:16.025894301 +0000 UTC m=+0.309508818 container init 9eb688a24595b222cae20cc9bd9e1784240ee0076e4ff25e2b633e4e310246e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:15:16.166 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:16 smithi099 podman[73225]: 2024-09-12 01:15:16.02920021 +0000 UTC m=+0.312814723 container start 9eb688a24595b222cae20cc9bd9e1784240ee0076e4ff25e2b633e4e310246e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T01:15:16.166 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:16 smithi099 podman[73225]: 2024-09-12 01:15:16.167705195 +0000 UTC m=+0.451319717 container attach 9eb688a24595b222cae20cc9bd9e1784240ee0076e4ff25e2b633e4e310246e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T01:15:16.510 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 12 01:15:16 smithi099 podman[73225]: 2024-09-12 01:15:16.218593104 +0000 UTC m=+0.502207619 container died 9eb688a24595b222cae20cc9bd9e1784240ee0076e4ff25e2b633e4e310246e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T01:15:16.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T01:15:16.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:16.838 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T01:15:16.839 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T01:15:16.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.11 2024-09-12T01:15:17.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:16.829+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:17.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:16 smithi159 systemd[1]: Stopping Ceph osd.11 for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:17.616 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.143+0000 7f45d8589640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:17.616 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.143+0000 7f45d8589640 -1 osd.11 108 *** Got signal Terminated *** 2024-09-12T01:15:17.616 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.143+0000 7f45d8589640 -1 osd.11 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T01:15:18.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:18.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:17 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:17.812+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:19.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:19.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:19.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:19.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:19.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:19.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:19.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:18 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:18.769+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:20.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:20.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:19 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:19.791+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:21.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:21.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:21.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:21.116 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:21.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:21.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:21.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:20 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:20.816+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-12T01:13:52.776324+0000 front 2024-09-12T01:13:52.776276+0000 (oldest deadline 2024-09-12T01:14:16.876117+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-12T01:13:56.876818+0000 front 2024-09-12T01:13:56.876659+0000 (oldest deadline 2024-09-12T01:14:28.677222+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-12T01:14:10.378343+0000 front 2024-09-12T01:14:10.378305+0000 (oldest deadline 2024-09-12T01:14:31.478615+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-12T01:14:25.282079+0000 front 2024-09-12T01:14:25.282192+0000 (oldest deadline 2024-09-12T01:14:48.181984+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-12T01:14:31.583374+0000 front 2024-09-12T01:14:31.583401+0000 (oldest deadline 2024-09-12T01:14:55.683620+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-12T01:14:40.384656+0000 front 2024-09-12T01:14:40.384582+0000 (oldest deadline 2024-09-12T01:15:03.284441+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-12T01:14:47.886139+0000 front 2024-09-12T01:14:47.886222+0000 (oldest deadline 2024-09-12T01:15:11.886754+0000) 2024-09-12T01:15:22.117 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:21 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11[60608]: 2024-09-12T01:15:21.840+0000 7f45d0b97640 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.159:6820 osd.8 since back 2024-09-12T01:14:58.288010+0000 front 2024-09-12T01:14:58.287990+0000 (oldest deadline 2024-09-12T01:15:21.188028+0000) 2024-09-12T01:15:22.972 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:22 smithi159 podman[73888]: 2024-09-12 01:15:22.670970264 +0000 UTC m=+5.628751988 container died 33f966b4fc00389d9e8b1687300ed9363ea2cfef88b78ba8f8787b0b39e70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T01:15:23.300 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73888]: 2024-09-12 01:15:23.059085248 +0000 UTC m=+6.016866969 container cleanup 33f966b4fc00389d9e8b1687300ed9363ea2cfef88b78ba8f8787b0b39e70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T01:15:23.300 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 bash[73888]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11 2024-09-12T01:15:23.616 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73903]: 2024-09-12 01:15:23.301731251 +0000 UTC m=+0.624479274 container remove 33f966b4fc00389d9e8b1687300ed9363ea2cfef88b78ba8f8787b0b39e70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T01:15:23.920 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73992]: 2024-09-12 01:15:23.529623631 +0000 UTC m=+0.016090110 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T01:15:23.920 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73992]: 2024-09-12 01:15:23.64419653 +0000 UTC m=+0.130663043 container create d076fcb79a70e5dabcf3623b6dd85b45f1dbd1020b74a55c5313292179bf5e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T01:15:23.920 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73992]: 2024-09-12 01:15:23.863317681 +0000 UTC m=+0.349784152 container init d076fcb79a70e5dabcf3623b6dd85b45f1dbd1020b74a55c5313292179bf5e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T01:15:23.920 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73992]: 2024-09-12 01:15:23.866536161 +0000 UTC m=+0.353002632 container start d076fcb79a70e5dabcf3623b6dd85b45f1dbd1020b74a55c5313292179bf5e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T01:15:23.920 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:23 smithi159 podman[73992]: 2024-09-12 01:15:23.921537972 +0000 UTC m=+0.408004442 container attach d076fcb79a70e5dabcf3623b6dd85b45f1dbd1020b74a55c5313292179bf5e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T01:15:24.366 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 12 01:15:24 smithi159 podman[73992]: 2024-09-12 01:15:24.053963083 +0000 UTC m=+0.540429554 container died d076fcb79a70e5dabcf3623b6dd85b45f1dbd1020b74a55c5313292179bf5e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T01:15:24.701 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T01:15:24.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:24.739 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T01:15:24.739 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T01:15:24.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.a 2024-09-12T01:15:25.040 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:24 smithi016 systemd[1]: Stopping Ceph mds.a for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:25.313 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:25 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: 2024-09-12T01:15:25.039+0000 7f4262a41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:25.313 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:25 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a[71499]: 2024-09-12T01:15:25.039+0000 7f4262a41640 -1 mds.a *** got signal Terminated *** 2024-09-12T01:15:29.331 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:29 smithi016 podman[135080]: 2024-09-12 01:15:29.051367375 +0000 UTC m=+4.113793106 container died 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-12T01:15:29.643 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:29 smithi016 podman[135080]: 2024-09-12 01:15:29.507248525 +0000 UTC m=+4.569674289 container cleanup 1cbd98c44bc01be69f3fe64c6cf9168e54279949cea2ea22cb1db7f9135250a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:15:29.643 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 12 01:15:29 smithi016 bash[135080]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-a 2024-09-12T01:15:29.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T01:15:29.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:29.987 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T01:15:29.987 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T01:15:29.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.d 2024-09-12T01:15:30.311 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:30 smithi016 systemd[1]: Stopping Ceph mds.d for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:30.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d[70307]: 2024-09-12T01:15:30.310+0000 7f2fa6add640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:30.643 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:30 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d[70307]: 2024-09-12T01:15:30.310+0000 7f2fa6add640 -1 mds.d *** got signal Terminated *** 2024-09-12T01:15:31.761 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:31 smithi016 podman[135211]: 2024-09-12 01:15:31.449944854 +0000 UTC m=+1.238264262 container died dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T01:15:32.094 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:31 smithi016 podman[135211]: 2024-09-12 01:15:31.836615864 +0000 UTC m=+1.624935272 container cleanup dec9a3a84c0347891a7f5a7cd7d85c297b061a5484b29ec9b78f74e2dcce6ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T01:15:32.095 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 12 01:15:31 smithi016 bash[135211]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-d 2024-09-12T01:15:32.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T01:15:32.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:32.302 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T01:15:32.302 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T01:15:32.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.g 2024-09-12T01:15:32.639 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:32 smithi016 systemd[1]: Stopping Ceph mds.g for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:32.893 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: 2024-09-12T01:15:32.639+0000 7f60f0763640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:32.893 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:32 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g[71123]: 2024-09-12T01:15:32.640+0000 7f60f0763640 -1 mds.g *** got signal Terminated *** 2024-09-12T01:15:34.282 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:33 smithi016 podman[135332]: 2024-09-12 01:15:33.984558429 +0000 UTC m=+1.456920016 container died a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:15:34.643 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:34 smithi016 podman[135332]: 2024-09-12 01:15:34.457530268 +0000 UTC m=+1.929891853 container cleanup a2a5f5cc7f0dceedab4e1a506342309ff889d3e9b461d722aac50e8ff7366ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T01:15:34.644 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 12 01:15:34 smithi016 bash[135332]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-g 2024-09-12T01:15:34.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T01:15:34.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:34.924 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T01:15:34.924 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T01:15:34.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.j 2024-09-12T01:15:35.252 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:35 smithi016 systemd[1]: Stopping Ceph mds.j for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:35.643 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j[69909]: 2024-09-12T01:15:35.253+0000 7fc2bf356640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:35.644 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:35 smithi016 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j[69909]: 2024-09-12T01:15:35.253+0000 7fc2bf356640 -1 mds.j *** got signal Terminated *** 2024-09-12T01:15:39.643 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:39 smithi016 podman[135452]: 2024-09-12 01:15:39.263620942 +0000 UTC m=+4.114008001 container died bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T01:15:39.990 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:39 smithi016 podman[135452]: 2024-09-12 01:15:39.740138302 +0000 UTC m=+4.590525363 container cleanup bdbbc108db0e472ce91c1d151c719a3c11cbda7c053acd313a7d96bf0a770bd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-12T01:15:39.990 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 12 01:15:39 smithi016 bash[135452]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-j 2024-09-12T01:15:40.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T01:15:40.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:40.149 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T01:15:40.149 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T01:15:40.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.b 2024-09-12T01:15:40.444 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:40 smithi099 systemd[1]: Stopping Ceph mds.b for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:40.702 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: 2024-09-12T01:15:40.444+0000 7f27fab72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:40.703 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:40 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b[66005]: 2024-09-12T01:15:40.444+0000 7f27fab72640 -1 mds.b *** got signal Terminated *** 2024-09-12T01:15:42.274 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:41 smithi099 podman[73329]: 2024-09-12 01:15:41.991106702 +0000 UTC m=+1.650337705 container died 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T01:15:42.636 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:42 smithi099 podman[73329]: 2024-09-12 01:15:42.436214305 +0000 UTC m=+2.095445308 container cleanup 01164f74bce2905eec1f536cb7f20db09b9a6c7c4267cb56bc7642ca769e3439 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T01:15:42.636 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 12 01:15:42 smithi099 bash[73329]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-b 2024-09-12T01:15:42.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T01:15:42.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:42.889 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T01:15:42.889 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T01:15:42.890 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.e 2024-09-12T01:15:43.206 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:42 smithi099 systemd[1]: Stopping Ceph mds.e for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:43.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: 2024-09-12T01:15:43.206+0000 7f7f043a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:43.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:43 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e[65405]: 2024-09-12T01:15:43.206+0000 7f7f043a7640 -1 mds.e *** got signal Terminated *** 2024-09-12T01:15:46.274 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:45 smithi099 podman[73465]: 2024-09-12 01:15:45.99480014 +0000 UTC m=+2.887097151 container died bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:15:46.598 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:46 smithi099 podman[73465]: 2024-09-12 01:15:46.373228503 +0000 UTC m=+3.265525514 container cleanup bff36bc65523e33dfc2ebe7d6849efb000556fdff94ed97ac13538f21d688b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T01:15:46.598 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 12 01:15:46 smithi099 bash[73465]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-e 2024-09-12T01:15:46.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T01:15:46.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:46.764 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T01:15:46.764 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T01:15:46.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.h 2024-09-12T01:15:47.024 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:46 smithi099 systemd[1]: Stopping Ceph mds.h for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:47.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: 2024-09-12T01:15:47.092+0000 7ff63c39c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:47.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:47 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h[65703]: 2024-09-12T01:15:47.092+0000 7ff63c39c640 -1 mds.h *** got signal Terminated *** 2024-09-12T01:15:50.267 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:49 smithi099 podman[73586]: 2024-09-12 01:15:49.993225011 +0000 UTC m=+3.012011551 container died 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T01:15:50.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:50 smithi099 podman[73586]: 2024-09-12 01:15:50.334877593 +0000 UTC m=+3.353664179 container cleanup 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T01:15:50.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:50 smithi099 bash[73586]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h 2024-09-12T01:15:50.524 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 12 01:15:50 smithi099 podman[73598]: 2024-09-12 01:15:50.496412016 +0000 UTC m=+0.501591729 container remove 0b433f505ac74951850b71706dfd04cc654304ca5dedf11ce133773ea607ade5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T01:15:50.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T01:15:50.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:50.790 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T01:15:50.790 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T01:15:50.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.k 2024-09-12T01:15:51.024 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:50 smithi099 systemd[1]: Stopping Ceph mds.k for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:51.524 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: 2024-09-12T01:15:51.139+0000 7fa1e8897640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:51.524 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:51 smithi099 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k[65102]: 2024-09-12T01:15:51.139+0000 7fa1e8897640 -1 mds.k *** got signal Terminated *** 2024-09-12T01:15:54.274 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:54 smithi099 podman[73709]: 2024-09-12 01:15:54.003662381 +0000 UTC m=+2.999027663 container died 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T01:15:54.631 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:54 smithi099 podman[73709]: 2024-09-12 01:15:54.406758297 +0000 UTC m=+3.402123563 container cleanup 3a2b00e8e30096f8a6215b2952b8d29cdb18182be6d05c37c83dee3e07fa4672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T01:15:54.631 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 12 01:15:54 smithi099 bash[73709]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-k 2024-09-12T01:15:54.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T01:15:54.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:54.799 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T01:15:54.799 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T01:15:54.799 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.c 2024-09-12T01:15:55.115 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:54 smithi159 systemd[1]: Stopping Ceph mds.c for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:55.366 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: 2024-09-12T01:15:55.115+0000 7f04dd5d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:55.366 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:55 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c[65722]: 2024-09-12T01:15:55.115+0000 7f04dd5d6640 -1 mds.c *** got signal Terminated *** 2024-09-12T01:15:58.367 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:57 smithi159 podman[74095]: 2024-09-12 01:15:57.986895549 +0000 UTC m=+2.983708041 container died 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:15:58.823 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:58 smithi159 podman[74095]: 2024-09-12 01:15:58.61528492 +0000 UTC m=+3.612097411 container cleanup 5133935a71e8625654a313e936a84cb075836553fb44f47693010a8de3fd1001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T01:15:58.824 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 12 01:15:58 smithi159 bash[74095]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-c 2024-09-12T01:15:59.041 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T01:15:59.076 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:15:59.077 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T01:15:59.077 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T01:15:59.077 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.f 2024-09-12T01:15:59.366 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:15:59 smithi159 systemd[1]: Stopping Ceph mds.f for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:15:59.866 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:15:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: 2024-09-12T01:15:59.419+0000 7f4486da8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:15:59.866 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:15:59 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f[65422]: 2024-09-12T01:15:59.419+0000 7f4486da8640 -1 mds.f *** got signal Terminated *** 2024-09-12T01:16:02.252 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:16:01 smithi159 podman[74226]: 2024-09-12 01:16:01.987361655 +0000 UTC m=+2.685772377 container died 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T01:16:02.616 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:16:02 smithi159 podman[74226]: 2024-09-12 01:16:02.352263196 +0000 UTC m=+3.050673916 container cleanup 06f3cd5eaea56970e83683561b9668c674dd197287281b01736110f13ef18213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T01:16:02.616 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 12 01:16:02 smithi159 bash[74226]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-f 2024-09-12T01:16:02.802 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T01:16:02.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:16:02.835 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T01:16:02.835 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T01:16:02.836 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.i 2024-09-12T01:16:03.147 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:02 smithi159 systemd[1]: Stopping Ceph mds.i for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:16:03.616 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T01:16:03.147+0000 7f3716172640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:16:03.616 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:03 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i[66023]: 2024-09-12T01:16:03.147+0000 7f3716172640 -1 mds.i *** got signal Terminated *** 2024-09-12T01:16:06.289 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:05 smithi159 podman[74347]: 2024-09-12 01:16:05.99261172 +0000 UTC m=+2.940887190 container died 6b85d3bd13731a44709dc9bfd3f722905d8a95b71a828fb16e8953d240eda472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T01:16:06.616 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:06 smithi159 podman[74347]: 2024-09-12 01:16:06.381007586 +0000 UTC m=+3.329283058 container cleanup 6b85d3bd13731a44709dc9bfd3f722905d8a95b71a828fb16e8953d240eda472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T01:16:06.616 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 12 01:16:06 smithi159 bash[74347]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-i 2024-09-12T01:16:06.852 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T01:16:06.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:16:06.885 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T01:16:06.885 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T01:16:06.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.l 2024-09-12T01:16:07.116 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:06 smithi159 systemd[1]: Stopping Ceph mds.l for 434eab66-70a0-11ef-bcea-c7b262605968... 2024-09-12T01:16:07.616 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: 2024-09-12T01:16:07.228+0000 7fba5f973640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T01:16:07.616 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:07 smithi159 ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l[65120]: 2024-09-12T01:16:07.228+0000 7fba5f973640 -1 mds.l *** got signal Terminated *** 2024-09-12T01:16:10.244 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:09 smithi159 podman[74467]: 2024-09-12 01:16:09.992517332 +0000 UTC m=+2.894411029 container died 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T01:16:10.616 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:10 smithi159 podman[74467]: 2024-09-12 01:16:10.344888843 +0000 UTC m=+3.246782529 container cleanup 1da01add056a00d6a6acc47b0ffea4a1dc3bf225578258d5debc13cdf405fc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T01:16:10.616 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 12 01:16:10 smithi159 bash[74467]: ceph-434eab66-70a0-11ef-bcea-c7b262605968-mds-l 2024-09-12T01:16:10.782 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-434eab66-70a0-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T01:16:10.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T01:16:10.814 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T01:16:10.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:16:11.025 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:13.648 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:16:13.849 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:16.317 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T01:16:16.520 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:18.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:16:18.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:16:18.948 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T01:16:18.974 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T01:16:18.976 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi016/crash 2024-09-12T01:16:18.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:16:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:16:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:16:19.022 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi099/crash 2024-09-12T01:16:19.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:16:19.090 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:16:19.090 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:16:19.092 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi159/crash 2024-09-12T01:16:19.093 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash -- . 2024-09-12T01:16:19.132 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/434eab66-70a0-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T01:16:19.132 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-09-12T01:16:19.133 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T01:16:19.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T01:16:19.195 INFO:tasks.cephadm:Compressing logs... 2024-09-12T01:16:19.196 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:16:19.198 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:16:19.201 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T01:16:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:16:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-09-12T01:16:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T01:16:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T01:16:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:16:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:16:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T01:16:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:16:19.225 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:16:19.225 INFO:teuthology.orchestra.run.smithi159.stderr:: No such file or directory 2024-09-12T01:16:19.225 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:16:19.225 INFO:teuthology.orchestra.run.smithi159.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:16:19.226 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T01:16:19.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T01:16:19.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T01:16:19.226 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.4% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.i.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.11.log: 82.0%gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr: -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.5.log: 90.5% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:16:19.230 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T01:16:19.231 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T01:16:19.231 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:16:19.231 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:16:19.232 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T01:16:19.233 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-09-12T01:16:19.233 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T01:16:19.233 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T01:16:19.233 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T01:16:19.233 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T01:16:19.235 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.l.log: 87.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:16:19.235 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.4.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T01:16:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.0% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T01:16:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T01:16:19.238 INFO:teuthology.orchestra.run.smithi016.stderr: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T01:16:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log: 90.9% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:16:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T01:16:19.239 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:16:19.240 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.d.log: 87.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:16:19.244 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T01:16:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log 2024-09-12T01:16:19.245 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T01:16:19.246 INFO:teuthology.orchestra.run.smithi016.stderr: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T01:16:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T01:16:19.247 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T01:16:19.249 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T01:16:19.251 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log: 87.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T01:16:19.251 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T01:16:19.256 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T01:16:19.256 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T01:16:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.h.log: 92.2% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T01:16:19.257 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.j.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T01:16:19.265 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T01:16:19.315 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T01:16:19.316 INFO:teuthology.orchestra.run.smithi016.stderr: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T01:16:19.316 INFO:teuthology.orchestra.run.smithi016.stderr: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T01:16:19.341 INFO:teuthology.orchestra.run.smithi099.stderr: 92.9% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T01:16:19.367 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.c.log: /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.f.log: 93.0% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T01:16:19.372 INFO:teuthology.orchestra.run.smithi099.stderr: 94.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T01:16:19.393 INFO:teuthology.orchestra.run.smithi099.stderr: 92.7% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T01:16:19.428 INFO:teuthology.orchestra.run.smithi159.stderr: 92.7% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T01:16:19.536 INFO:teuthology.orchestra.run.smithi159.stderr: 93.9% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T01:16:19.697 INFO:teuthology.orchestra.run.smithi099.stderr: 91.8% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T01:16:19.724 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T01:16:19.840 INFO:teuthology.orchestra.run.smithi159.stderr: 91.7% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T01:16:20.106 INFO:teuthology.orchestra.run.smithi016.stderr: 92.5% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T01:16:20.431 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T01:16:20.433 INFO:teuthology.orchestra.run.smithi016.stderr: 92.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T01:16:20.521 INFO:teuthology.orchestra.run.smithi159.stderr: 92.1% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T01:16:20.540 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T01:16:20.651 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T01:16:20.793 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T01:16:20.917 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T01:16:21.027 INFO:teuthology.orchestra.run.smithi159.stderr: 92.9% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T01:16:21.051 INFO:teuthology.orchestra.run.smithi099.stderr: 92.5% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T01:16:21.262 INFO:teuthology.orchestra.run.smithi159.stderr: 92.4% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T01:16:21.283 INFO:teuthology.orchestra.run.smithi159.stderr: 93.1% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T01:16:21.285 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-12T01:16:21.285 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m2.070s 2024-09-12T01:16:21.285 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m9.551s 2024-09-12T01:16:21.285 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m0.317s 2024-09-12T01:16:21.478 INFO:teuthology.orchestra.run.smithi099.stderr: 92.3% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T01:16:21.480 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-12T01:16:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m2.264s 2024-09-12T01:16:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m7.180s 2024-09-12T01:16:21.480 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.285s 2024-09-12T01:16:21.490 INFO:teuthology.orchestra.run.smithi016.stderr: 92.0% -- replaced with /var/log/ceph/434eab66-70a0-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T01:16:21.492 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-12T01:16:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m2.281s 2024-09-12T01:16:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m7.901s 2024-09-12T01:16:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.268s 2024-09-12T01:16:21.493 INFO:tasks.cephadm:Archiving logs... 2024-09-12T01:16:21.493 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi016/log 2024-09-12T01:16:21.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:16:22.653 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi099/log 2024-09-12T01:16:22.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:16:23.790 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi159/log 2024-09-12T01:16:23.791 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T01:16:25.174 INFO:tasks.cephadm:Removing cluster... 2024-09-12T01:16:25.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force 2024-09-12T01:16:25.385 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:25.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force 2024-09-12T01:16:25.854 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:26.121 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 434eab66-70a0-11ef-bcea-c7b262605968 --force 2024-09-12T01:16:26.314 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 434eab66-70a0-11ef-bcea-c7b262605968 2024-09-12T01:16:26.583 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T01:16:26.583 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:16:26.603 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:16:26.620 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T01:16:26.635 INFO:tasks.cephadm:Teardown complete 2024-09-12T01:16:26.636 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T01:16:26.650 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T01:16:26.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:16:26.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:16:26.662 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T01:16:26.794 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T01:16:26.808 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T01:16:26.809 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:16:26.812 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:16:26.814 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T01:16:26.827 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:16:26.827 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:16:26.828 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 -955us[ -959us] +/- 59ms 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -483us[ -483us] +/- 67ms 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +1984us[+1984us] +/- 81ms 2024-09-12T01:16:26.831 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 7 377 108 -832us[ -935us] +/- 58ms 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -497us[ -497us] +/- 67ms 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +1987us[+1987us] +/- 81ms 2024-09-12T01:16:26.833 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 -966us[ -956us] +/- 59ms 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -526us[ -516us] +/- 67ms 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 38 +1985us[+1995us] +/- 81ms 2024-09-12T01:16:26.834 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T01:16:26.835 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T01:16:26.845 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T01:16:26.846 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T01:16:26.872 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:16:26.972 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:16:27.043 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T01:16:27.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:16:27.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:27.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:16:27.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:27.174 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T01:16:27.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:27.203 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T01:16:27.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T01:16:27.273 INFO:teuthology.task.internal:Duration was 2232.693756 seconds 2024-09-12T01:16:27.273 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T01:16:27.306 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T01:16:27.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:16:27.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:16:27.312 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T01:16:27.350 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:16:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:16:27.351 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T01:16:27.820 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T01:16:27.820 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-12T01:16:27.820 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:16:27.844 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-12T01:16:27.845 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:16:27.887 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-09-12T01:16:27.887 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T01:16:27.920 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T01:16:27.920 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:16:27.922 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:16:27.930 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T01:16:27.986 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T01:16:27.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:16:27.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:16:27.998 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T01:16:30.014 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T01:16:30.030 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T01:16:30.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:16:30.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:16:30.086 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T01:16:30.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T01:16:30.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.145 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.163 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-12T01:16:30.168 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-12T01:16:30.186 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-09-12T01:16:30.208 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:30.233 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:30.254 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T01:16:30.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:16:30.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T01:16:30.292 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T01:16:30.294 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi016 2024-09-12T01:16:30.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:16:30.395 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi099 2024-09-12T01:16:30.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:16:30.435 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901507/remote/smithi159 2024-09-12T01:16:30.435 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T01:16:30.473 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T01:16:30.474 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:16:30.476 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:16:30.479 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T01:16:30.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T01:16:30.545 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T01:16:30.545 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T01:16:30.582 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T01:16:30.583 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:16:30.586 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:16:30.588 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T01:16:30.599 INFO:teuthology.orchestra.run.smithi016.stdout: 265341 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:16 /home/ubuntu/cephtest 2024-09-12T01:16:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:16 /home/ubuntu/cephtest 2024-09-12T01:16:30.604 INFO:teuthology.orchestra.run.smithi159.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 01:16 /home/ubuntu/cephtest 2024-09-12T01:16:30.605 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T01:16:30.621 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T01:16:30.686 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2232.693756341934 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T01:16:30.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:16:30.813 INFO:teuthology.run:pass